The Labour leader wants to enable local councils to negotiate lower rents, build homes and cut housing benefit costs rather than paying for the costs of the failure to build.
He said: “We can’t afford to pay billions on ever-rising rents when we should be building homes to bring down the bill. Thirty years ago for every £100 pounds we spent on housing £80 was invested in bricks and mortar and £20 was spent on housing benefit.
“Today for every £100 we spend on housing just £5 is invested in bricks and mortar and £95 goes on housing benefit.”
Miliband said he expects individual families to negotiate with their landlords.
He said it is inevitable that tenants end up paying over the odds as does the state in housing benefit costs.
He said: “It’s time to tackle this problem at source. We can start to bring about the shift from benefits to building. Bringing the housing benefit bill down for the long term too.”
David Brown, commercial director of LSL group, owners of letting agents Your Move and Reeds Rains, said: “The world certainly has changed and the private rented sector has been instrumental in protecting some of the poorest in society.
“While landlords have increased the supply of available homes since the financial crisis rents have actually fallen in real terms compared to other household bills by an average of 1% a year.”
He said rents have gone up below the rate of inflation in four of the last five years.
Brown said any change to the role of councils that could disrupt this success should be treated with trepidation.
But he added: “However we welcome any policy that can support landlords and the rest of the housing industry in their efforts to provide more homes and continue to keep up with tenant demand.
“The government’s recent expansion of the Build to Rent scheme is a step in that direction. A future government – of any stripes – will need to pay more attention to the growing demand for the flexibility and affordability of private renting.”
Richard Lambert, chief executive of the National Landlords Association, said: “The NLA welcomes Mr Miliband's desire to invest in a significant programme of building homes to rent. It’s rare to see a politician who appears to understand the economics of housing supply and rarer still to find one who’s willing to apply them.”
Lambert said the only long-term solution to making housing more affordable is to increase the supply by building more homes in the areas where they are most needed.
He added: “The question is how we get from here to there. Long-term investment in future housing is essential but not at the expense of today's provision.
“The promise of a new home in five years will mean very little to a family if it comes at the cost of struggling to find one today.
“New subsidies must work hand-in-hand with existing support mechanisms to ensure that investment continues in housing across all tenures.”