You can Retract a bid, although there are limited reasons for doing so.
http://offer.ebay.ca/ws/eBayISAPI.dll?RetractBidShow
As a seller, I would rather have a bid retracted than chase a reluctant buyer for a payment on an item he doesn't want.
Bid Retractions do show on your Feedback page.
I would urge you to sign up for a Paypal account, and to use your credit card to back it.
This allows you to pay immediately for the items you buy, with no delays from bank clearances, which can take up to 10 days, during which the seller cannot buy PP shipping labels.
You also have a third line of buyer protection. EBay's policy, Paypal's policy and the card's chargeback policy.
If you do not have a PP account, some of your bids may not be accepted*.
While you can offer to pay by money order or personal cheque, many sellers are reluctant to accept these.
*EBay allows sellers to Block bidders who do not have PP accounts in place.
