The Treasury is changing the rules of the Credit Guarantee Scheme to allow banks to reduce their participation in it ahead of schedule.
The scheme became operational at the height of the credit crunch in October 2008 and closed to new issuance on February 28 2010.
The Treasury charges a fee to a bank or building society in exchange for a government debt guarantee.
Lenders will now be able to buy back and cancel the debt before it was scheduled to mature. To do so they will need to pay 15% of the total fees that would have been payable on the cancelled guarantee.
Chancellor George Osborne says the move demonstrates that the banking sector is returning to a stable footing. It will allow banks to start reducing their reliance on taxpayers while they return to normal market financing without government support.