Another of Penshurst's splendours is the 11-acre formal walled garden, with records dating back to 1346. One of the oldest gardens in private ownership, it remains much as it was when constructed by Sir Henry Sidney in the Elizabethan era. Central to his ambitious project was the creation of the acclaimed Italian Garden, which involved shifting thousands of tons of earth and building an ingenious system of walls and terraces.

After this era of activity and abundance, there followed a period of neglect when lack of funds meant the garden slipped into decline. On inheriting the house in 1851, the 2nd Lord De L'Isle and Dudley set about restoring the formality of his ancestor's garden. He planted a mile of yew hedge, dividing the garden and orchards into a series of small self-contained garden rooms, each with its own style and character. A further period of decay during the Second World War paved the way for the 1st Viscount De L'Isle, VC KG, to revive and develop the garden rooms further, using some of the leading British designers of the time.