With Millwall forced into a late reshuffle after Casper de Norre was injured in the warm-up, it was the home side who took control early on, as Oli McBurnie and Joseph both went close.
Joseph’s close-range header from John Lundstram’s cross crashed against the bar but the Lions responded by snatching the lead with their first opportunity as Crama headed back Azeez’s free-kick for Cooper to hook in.
Hull struck back almost immediately when Gelhardt scampered through, collecting a return ball from Joseph before slamming beyond Anthony Patterson for his 12th goal of the season.
The Tigers continued to carve out openings and Gelhardt’s deflected drive looped up for Joseph to head home – only for the goal to be chalked off for a foul on the goalkeeper by McBurnie.
The woodwork came to Millwall’s rescue once more before the break, with Hughes’ powerful header from a free-kick rattling the bar and bouncing down but failing to cross the line.
Although the visitors strengthened their midfield at half-time, introducing Luke Cundle and Alfie Doughty, they were under pressure again when McBurnie’s glancing header flew just wide and Regan Slater’s volley was charged down.
However, the Lions held firm and regained the advantage when Azeez danced around his marker and slid an inch-perfect low ball into the box for Ivanovic – sent on after Doughty hobbled off 10 minutes earlier – to walk over the line.
Semi Ajayi’s stray pass then presented Coburn with the opportunity to make it 3-1, taking the ball past Hughes and tucking it low beyond Ivor Pandur to seal his side’s fifth win in six games.