And such an announcement will push more “floating voters” into choosing Conservative at the next general election, he stressed.
He said: “He’s shot himself in the foot because it is very clear that the majority of the UK do want a vote on the EU.
“If you’re a floating voter and one of the things that is important to you is having a vote on the EU that might be the deciding factor.
“I think that it's a bit of a gift to the Tory party.”
Boulger said that being in the EU inhibits the UK mortgage market going forward, with the UK’s KFI having to switch to the European Standardised Information Sheet by 21 March 2019.
Boulger added: “The EU is forcing lenders to give clients completely useless information and it will cost lenders something to do that.”
He also argued that there is a risk of overseas brokers using the European model to get around tight Financial Conduct Authority regulations.
He said: “The concept of the EU is flawed – for it to work you would have to have all the countries with the same legal practice, so the whole thing is completely mad.
“Pretty much everybody thinks we need a different relationship with Europe but I don’t think there is any way we are going to be able to negotiate enough changes to persuade more to stay.”
Prime Minister David Cameron said: "Our relationship with Europe isn’t working and it needs to change.
"Last year, I set out a clear plan – seek a new settlement for Britain in Europe, and then put it to the British people in an in-out referendum by the end of 2017.
"Today Ed Miliband has made his position crystal clear – there will be no referendum under Labour.
"Only the Conservatives will guarantee and deliver that referendum. It will only happen if I am Prime Minister.
"Both Labour and the Liberal Democrats won’t stand up for Britain, while UKIP simply can’t deliver on anything they promise."
