However, 31% of landlords felt positive about the future, and 59% still said that letting property was a worthwhile investment.
Following the end of the stamp duty holiday, landlords are hesitant about buying more property, according to Simply Business.
The study of 560 landlords from across the UK found that many still have concerns about government regulation, with 35% citing this as a reason not to buy more property.
There was also considerable uncertainty around the viability of the sector in general, with 23% of landlords saying they are worried about the future of the rental market.
Two fifths (43%) of landlords said better compensation should have been offered to those who lost rental income, while 34% believed government should have provided financial support for tenants who were unable to pay their rent.
However, 31% of landlords felt positive about the future, and 59% still said that letting property was a worthwhile investment.
Alan Thomas, chief executive of Simply Business, said: “The impact of the pandemic has been felt across all corners of the country – and that’s no different for the 2.6m landlords who let out residential properties in the UK.
"The extent of the pandemic’s impact is made clear in our latest research, with many landlords still hesitant about investing in new property.
“Contributing over £16bn annually in pre-tax spending, an exodus of smaller landlords from the buy-to-let market could have a devastating impact on the UK economy.
"But more than this, landlords are crucial to our communities, offering much-needed accommodation to over 4.4m households.
“Thankfully, for the most part landlords remain resilient – 59% of landlords still think property is a worthwhile investment, while almost a third are optimistic about their future letting property.
“Insuring over 300,000 landlords has given us a unique insight into the impact of COVID-19 on this audience.
"And now more than ever – as the UK looks to recover from the effects of the pandemic – it's vital that we recognise their significant contribution to the economy, and the important role they play in providing safe and affordable housing in our towns and cities.”