The deal is expected to go through on 3 November 2015.
Five Arrows has assets of £245.1m financed by £42.9m of shareholders' funds and a £178m loan from parent company Rothschild & Co, while it made a pre-tax profits of £10.2m in the year to 31 March 2015.
The move reflects Paragon's strategy to diversify its funding sources and income streams.
John Heron, managing director of Paragon Mortgages, said: “This is fantastic news for Paragon Bank and in turn, the wider Group.
“The new business has a strong track record and good reputation in operating in specialist markets, which of course is a good fit with Paragon. In addition, the loan book has an excellent credit profile and provides us with a platform to grow in a new market.
“It is expected that the acquisition will also accelerate Paragon Bank’s business plan, allowing the bank to achieve profit in 2016 and further enhance its contribution to the group’s ongoing success.”