In January 1762 a private society was established at 50 Pall Mall by Messrs. Boothby and James in response to having been blackballed for membership of White's. This society then split to form the predecessors of both Brooks's and Boodle's. The Club that was to become Brooks's was founded in March 1764 by twenty-seven prominent Whig nobles including the Duke of Portland, the Duke of Roxburghe, Lord Crewe and Lord Strathmore. Charles James Fox was elected as a member the following year at the age of sixteen. The Club premises at 49 Pall Mall was a former tavern owned by William Almack as was the neighbouring 50 Pall Mall where the society had previously met and so the Club become simply known as Almack's. These fashionable young men, known as Macaronis, would frequent the premises for the purposes of wining, dining and gambling.
In September 1777 William Brooks, a wine merchant and money lender who acted as Master, or manager, for Almack's, commissioned Henry Holland to design and construct a purpose built clubhouse at a site on neighbouring St James's Street. Paid for at Brooks's own expense, the building was completed in October 1778 and all existing members of Almack's were invited to join. Brooks's gamble paid off as all existing members swiftly moved into the new building and the Club then took on Brooks's name as its own. Brooks himself however would not live long to enjoy this success, dying in poverty in 1782.
The new clubhouse was built of yellow brick and Portland stone in a Palladian style similar to Holland's early country houses. The main suite of rooms on the first floor consisted of the Great Subscription Room, Small Drawing Room and the Card Room. The interiors are in neoclassical style, the Great Subscription Room having a segmental barrel vault ceiling. The interior of the building remained fairly unchanged until 1889 when neighbouring 2 Park Place, which had been purchased a few years before, was converted and adapted as part of Brooks's.