It’s one of the great financial taboos, one based not just on commonsense thrift, but enshrined in law: thou shalt not take so much of a penny of thy pension pot till 55 at the earliest. The only exception is for the terminally ill with a doctor’s certificate.

However, a few policymakers, not to mention lobbyists, are starting to wonder whether the untouchability of pension pots is such a good idea after all. In particular they are exploring whether people should be allowed to access part of their pension savings to buy their first home.

On one level, it appears a no-brainer. Homes are unaffordable for so many. Last week the average home price pushed past the £300,000 mark for the first time, according to Halifax figures. Owner occupation has been diving, especially for the 35-44 age cohort. Why shouldn’t thirtysomethings be allowed to access some of their retirement savings to scrape up a deposit to buy their first home and so liberate them from the tyranny of a lifetime paying rent to private landlords?

For many, having their own place today is an infinitely higher priority than their distant old age. And this is a problem now in danger of feeding on itself: on current trends the proportion of pensioners languishing in expensive, insecure private rented accommodation is set to almost triple from 6 per cent to 16 per cent by 2040.

Steve Webb, the former Lib Dem pensions minister, describes himself as a big fan of loosening the rules. “Having millions of pensioners renting in the private sector in their old age is going to be catastrophic,” he says.

Nikhil Rathi, the chief executive of the Financial Conduct Authority, has also flirted with the idea. He was not “absolutely gung-ho in favour,” he told a recent conference, but “more imaginative thinking” was needed. A few thousand pounds extracted from a pot today for a home deposit could help someone avoid £200,000 in rent in retirement, he pointed out.

Schroders, through its Lifetime Savings Initiative, is also persuaded, suggesting the way we categorise our finances into completely separate pension, mortgage, and rainy-day savings silos is a costly mistake.

For others, however, allowing early access to pensions is absolutely taboo for a very good reason. The very attractive tax benefits of pension saving and the bonus of compulsory employer contributions are designed to ensure fewer people run out of money and end up as a burden on the state. That’s the deal, end of.

Britain takes a firm line. Other countries do allow some access to pension pots for would-be home owners of any age. New Zealand is the most liberal by far, allowing first-time home buyers to make a withdrawal of any size for a home purchase, with the only restriction being that NZ$1,000 (£440) must be left in the pot. The average withdrawal is about NZ$43,000, giving a huge head start to prospective home buyers. The median house price is about NZ$700,000. More than three quarters of first-time buyers in New Zealand use this pension-tapping option.

Singapore is flexible, too, allowing a fixed proportion of pension pots to be drained for housing, medical expenses or education. About 44 per cent of accumulated savings have been withdrawn for housing purchases. Australia and the US are also less strict than the UK. South Africa has a different approach, allowing savers to pledge their pension pots as collateral against a home loan.

All these measures carry risks, not just because they could leave people with less income in retirement, but also because they threaten the smooth running of pension schemes: a scheme faced with having to find cash for big withdrawals in the short-term is much less able to plunge into the world of higher-returning liquid assets.

But perhaps the big reason for caution is that this would be yet another stimulant to home demand rather than home supply. In New Zealand, the new freedom has pushed up house prices, the critics argue, while owner occupation has actually fallen since its introduction.

In the UK in particular, our failure to directly address supply seems especially likely to produce adverse outcomes, lifting prices and putting property further out of reach. The UK’s Help To Buy scheme, now dismantled, was widely seen as a classic case of pushing up prices and dubbed “Help To Sell” by the sceptics.

The issue has not yet reached the popular imagination in the UK. That could change as pension savings increase and their visibility to younger employees becomes more apparent. The public are mildly favourably inclined: 39 per cent would like to see pots used for house deposits, with 33 per cent against, according to a poll last month by the pensions think tank Nest Insight, published alongside a thoughtful report on the subject.

How long before politicians start to notice? So far no party has jumped on this bandwagon. A Labour government or even a populist Reform might see a change to the rules as a potent way of appealing to left-behind voters in their 30s, 40s and increasingly 50s. The Tories in their pomp built their appeal on wider home ownership.

Much will depend on future house prices, of course. Affordability has actually been modestly improving in the past two years as wages outpace house prices. But only today, the estate agency grouping Rics reported a housing market showing signs of an early recovery. That would be good if it boosts deal volumes, but very bad if it presages yet another run-up in prices.

Ultimately, it is going to require a step-up in house-building and in empty-nester downsizing to make UK homes more affordable, not a new pensions freedom day. The other reason for caution is that any proposal that makes the byzantine world of pensions one jot more confusing and complex should be stamped on hard.

Patrick Hosking is Financial Editor of The Times