Despite the requirement for 250,000 new homes a year only around 120,000 are being built.
He said: "How do you control an unruly housing market in the UK?
"Can we ever build enough? Can we build enough to curb prices? Personally I doubt it."
He also discussed how the government is struggling to cope with cash transactions.
Williams said: "What's striking about the UK mortgage market is the rise of cash. There are more cash transactions at the top.
"There's an agenda opening up about how to control the housing market and the mortgage market which is quite difficult."
He said the government has only limited power in controlling the market currently.
He added: "The government has moved from directly funding the housing market to giving guarantees.
"It's got a high political profile but in many ways a relatively low substance behind it.
"Ultimately I think the government ambition is to merge the social and rented sectors and there won't be a distinct social rented sector anymore."
