The site is a former brickworks now used for the storage of waste and treatment to produce soil and aggregate.
The inspector said they were satisfied that removal of the permit was necessary to prevent pollution risks.
The appeal’s conclusion means Mineral Processing Ltd’s permit is now invalid and all activities under its terms must cease.
The company has also been ordered to remove waste from the site by 2029, and if it does not comply, will be committing an offence.
As operator, it remains responsible for managing pollution and odour impacts.
Carly Chambers, area environment manager for the EA in Yorkshire, said:
“Mineral Processing Ltd has shown a total disregard for the regulatory efforts of the Environment Agency and the impact on local residents and the environment.
“We welcome the decision of the Planning Inspectorate to dismiss the company’s appeal, meaning the permit is now revoked.
“We will continue to closely monitor activity at the site and expect the operator to comply with the steps set out in the revocation notice.”