You don't.
When you ended the transaction by refunding you should have also sent the buyer a request for a Mutual Cancellation. When that closes (either by his agreement or by his ignoring it) you get your fees back.
As it is, sorry, this was a learning experience.
Hope you don't have another of those.