You are in Japan.
Your buyer is in the USA.
This is a Board in CANADA, and we have different laws here.
However, if you were a Canadian seller who had a parcel returned from a foreign country, there would be no customs or sales taxes due. Canada Border Services Agency would call this "returned merchandise".
All Canadian duties and taxes would have already been paid before it was shipped out of the country.
I would suggest that you take the parcel, along with the emails exchanged and the original invoice, the payment and the refund paperwork to the Japanese Customs Office and explain the problem.
Ask for a refund of the money charged in error.