He said the club remained focused on serving the community, describing it as “for the youngsters” and supported by volunteer coaches who gave their time for free.
The gym is funded through memberships and local sponsors.
Oliver said it was “an expensive hobby” into which he had invested significant personal time and money.
Kel Finan-Cooke, Surrey Heath portfolio holder for property and economic development, said the club had “offered boxing training to the local community for decades, helping to keep people fit and active”.
She said the sale would “help preserve the community asset for the long term”.
The site is protected by a restrictive covenant requiring it to be used for physical exercise and leisure.