Birmingham made a bright start, with Jhon Solis putting an early chance wide, while it took a fine reaction save from Radek Vítek to keep out a point-blank Luis Vazquez header in the 18th minute.
Vitek went on to deal with an effort from Millar, while Middlesbrough finished the first half with a flurry of chances.
Cozier-Duberry fired over the bar after linking up with Myles Peart-Harris and Will Lankshear had a shot deflected onto a post just minutes before Arfsten broke the deadlock.
While Arfsten came up with the finish, the chance was superbly carved out by Cozier-Duberry as he cut in from right and beat a number of attempted tackles before finding the American.
Millar’s low finish to level up early in the second half added to his already glowing reputation at Birmingham, where he has made a huge impact since arriving on loan from Premier League side Hull City in August.
The goal that put Birmingham ahead four minute later was a messy one, with a Jay Stansfield header at the back post from a corner hitting a number of bodies before McGree directed it into his own net.
It took a mistake from Beadle, although the goalkeeper argued he had been fouled, to allow Malanda to force the ball home and ensure Middlesbrough remain level on points with second-placed West Ham in the table.