Police said the thieves had used the “quiet Christmas days” to rob the building on Nienhofstrasse in the Buer district.
Initial investigations suggest they gained access to the bank, and escaped, via an adjacent parking garage.
Witnesses have reported seeing several men carrying large bags in the staircase of the garage overnight on Saturday into Sunday.
Police said video footage shows a black Audi RS 6 leaving the garage, on De-La-Chevallerie-Strasse, early on Monday morning.
The hole into the underground vault room was discovered when a fire alarm went off in the early hours of Monday, and police and the fire brigade searched the building.
Affected bank customers have been asked to contact Sparkasse bank, which has set up a hotline. Police secured the entrance of the branch on Tuesday after a large number of customers gathered outside demanding information.