"no tax at all because it is going out of the country"
That is correct BUT a seller must be careful to be able to prove where the item was delivered in case of tax audit.
In the good old days, to avoid paying PST, or a higher level of GST/HST, many transactions falsely indicated shipping to an address out of province or out of country.
The taxman got wiser and started asking for proof of shipment/delivery where an invoice indicated a local billing address and an out-of-province or out-of-country shipping adress.