Millwall started with real purpose and went close in the early stages when Billy Mitchell had a shot blocked after a cross from Azeez, which was a warning of things to come for the visitors.
Cooper drew a save from Josh Griffiths from close range moments before giving the home side the lead. Azeez’s cross was almost met by Will Smallbone, but the cente-half was there to apply the finish.
Callum Styles’ fizzing cross evaded everyone in the area as West Brom gained a foothold with half-time looming, but Josh Coburn’s physical presence in attack was important for Millwall to relieve the pressure.
The former Middlesbrough man almost touched home from close range six minutes before the break, with Cooper again causing havoc in the West Brom box.
The visitors settled a little better after the interval and had a first effort in the second half when Nat Phillips fired wide from the edge of the box.
Almost instantly, though, Millwall doubled their lead. Mitchell fed Azeez down the right, and he cut inside before unleashing a wonderful strike into the top corner. Griffiths had no chance.
Sturge surged down the left after being found by Thierno Ballo to lash the ball home, as Millwall bounced back from a hefty midweek defeat by Coventry City.
It was a disappointing afternoon for West Brom, who have now lost three of their past five games.