Police moved residents of about 10 houses to safety during the operation and told other people living nearby to stay inside.
A police spokesperson said armed officers arrived in Aylesbury Road after reports of “a concern for welfare for a man at a property”.
They added that officers were supported by police negotiators, the East of England Ambulance Service, two critical care crews from Magpas Air Ambulance and Bedfordshire Fire and Rescue Service.
Lucy Bywater, a Green councillor for the Castle and Newnham Ward on Bedford Borough Council, said it was “important that all inquiries from residents” were referred to police.
“It’s obviously been a really distressing day for all concerned, including immediate neighbours and our thoughts are with the family and friends of the man who died.”
A cordon remained in place in the evening.