Blackburn were unable to field a striker in their midweek home defeat by Sheffield United, although the return of Iceland international Gudjohnsen from injury eased that situation.
It was further helped by Afolayan’s excellent performance, while Rovers showed some stoic defence as Millwall pressed hard for an equaliser.
Balasz Toth was back in goal for Blackburn and he saved with his legs from Camiel Neghli in a first half which was meandering towards a stalemate until Afolayan took a hand.
A cross from the right found Morsy lurking on the edge of the area and his shot looked too tame to beat Crocombe until Afolayan diverted its path and opened the scoring.
Blackburn suffered another injury worry at half-time as the impressive Ryan Alebiosu was forced off, and his replacement Gady Beyuku was still finding his feet when he unwisely slid in to challenge Tairyk Arconte.
Dykes, who scored twice against Bolton last week, made no mistake from the spot.
Afolayan soon lifted the spirits of the home fans by bursting into the Millwall half and, when Todd Cantwell’s diagonal run drew the defensive cover, he switched the ball onto his left foot and fired in to the bottom corner to restore the lead.
Arconte had a goal disallowed after knocking home Jake Cooper’s flick-on at the near-post, and had another effort brilliantly blocked by Hayden Carter, while Dykes headed narrowly wide as the Lions pushed hard for an equaliser while 2-1 down.
Rovers made sure of the points in the last of three added minutes as substitute Jayden Fevrier led a counter-attack and, when he squared for Beyuku, he allowed it to run to Gudjohnsen who calmly found the bottom corner.