I don't, you know, usually give interviews. So if you decide to write anything, I'd rather it wasn't about me.” With this unpromising plea for discretion, my host Jacob Rothschild offers a glass of Château Duhart-Milon 2000, a Rothschild wine – his cellars contain 15,000 bottles, going back to 1870 – and insists that what is interesting is not himself but “what we're doing at Waddesdon”, the National Trust property that houses the Rothschild collection.

Lord Rothschild, 73, is known for tremendous dynamism while staying resolutely behind the scenes. If the tall, thin man standing before me in tweed jacket, grey turtle-neck jumper and black trousers looks familiar, it is not because he has cultivated a public persona; it is because the long oval face, high forehead and arched, hawk-like eyes, as well as the intellectually engaged expression, resemble exactly his 1989 portrait by Lucian Freud.

I comment on this likeness. “When he paints us we think we look awful and horribly old, then 20 years later we're pleased,” he says drily. Freud's painting is in the National Portrait Gallery – “not because of me, but because of him”. A second version hangs at Waddesdon Manor alongside David Hockney's 2003 double portrait of Lord Rothschild and his daughter Hannah, eldest of his four children.

Rothschild has invited me to Eythrope, Buckinghamshire, for Sunday lunch. We are in the 19th-century tea pavilion that is his private home, next to his Waddesdon estate. His wife Serena is skiing in Switzerland, and this is the only free slot in his overloaded schedule as banker, arts philanthropist, collector and country house owner.

After working in and then resigning from the family bank NM Rothschild & Sons, he founded J Rothschild Assurance Group, now St James's Place, with Sir Mark Weinberg in 1991. He is also chairman of his investment trust company RIT Capital Partners and his other business concerns include Spencer House Capital Management and the mini-merchant bank Spencer House Partners.

As a philanthropist, his achievements include restoring the publicly owned Somerset House, one of the neoclassical jewels of London, and establishing it as a centre for the visual arts, and ensuring the future of the Courtauld Institute of Art, which contains an unrivalled collection of impressionist and early modernist masterpieces. As a personal project he bought Spencer House in St James and spent £16m returning it to 18th-century glory.

To all those activities, he has brought individual flair and an instinctive grasp of the innovations needed to uphold historical continuities. Today he says that his “main interest” is the reinvigoration of Waddesdon Manor, the neo-Renaissance chateau built in the 1880s by Ferdinand de Rothschild. The house has just opened for the summer 2010 season with an unexpected twist: the dramatic installation of Jeff Koons's reflective blue 6ft 6in high chromium stainless-steel “Cracked Egg” and, from May 1, chandeliers and furniture by the witty, irreverent Brazilian designers Humberto and Fernando Campana.

Rothschild describes his family as having “operated the first European business, in a way, and, genetically, they got lucky”. Since the mid-19th century its members have tended to offset their astounding wealth and lavish properties with a sober, retiring private demeanour.

We chat in an opulent Eythrope drawing room – deep soft beige sofas, long coffee tables heaped with art books, and superb Chilterns views, though the eye is drawn, above all, to a huge, modernist grey chandelier. It is by Diego Giacometti, brother of the more famous Alberto. “Oh, I was in the Paris studio when Diego was making the chandeliers for the Picasso Museum and he said, ‘Do you want me to make them for you as well?'” he explains. “All the chandeliers in the house are by him.”