The slight rise calls a halt to a five-month run of reported declines from a peak of 5.2% in September 2011.
The Retail Prices Index measure of inflation, which includes mortgage payments and council tax, fell slightly to 3.6% in March from 3.7% in April.
The ONS said food prices last year had fallen by a record 1.5% between February and March as supermarkets discounted, but that this year they had only fallen by 0.3%.
The inflation figures measure the rate at which prices are rising compared with the same month a year ago.
The ONS cited the price of fruit, bread, cereals and meat as all rising slightly this year, whereas last year they had seen large falls.
Clothing and footwear and computer games also contributed to the rise in CPI inflation.
The Bank of England's target for inflation is 2% on the CPI measure.
Alex Lawson, senior broker at Moneycorp, said: “Tumbling inflation was supposed to be the one silver lining to the dark clouds which still loom large over Britain's economy.
“But far from falling to Earth as hoped, the cost of living is now rising faster than before.
“In February, annual inflation dropped to its lowest rate since November 2010, prompting a spate of predictions that its downward trajectory would continue.
"Those predictions are now looking optimistic. Today's increase in annual CPI, the first since last September, will not have been welcome news at the Bank of England.”
Paul Dixon, a chartered financial planner at Census Financial Planning, said: “Inflation is hitting people on low incomes and in retirement hardest, as the biggest price increases we've seen are with food, petrol and heating.
“High inflation hits the economy hard, as it reduces spending power and impacts confidence. The cycle of misery looks set to continue for a while longer.
“Stubbornly high inflation at least makes more QE less likely, as this has a drastic effect on annuity rates.”
