The first real opportunity of the game fell to Portsmouth in a the swirling wind and rain at Fratton Park.
Josh Murphy whipped in a cross from the right and the ball bounced up for Regan Poole who attempted a shot on goal but the defender sliced his effort, which was eventually cleared.
Pompey then started to build wave after wave of attack, forcing Millwall deeper into their own half, and they took the lead on 22 minutes.
Murphy made a run down the left before laying the ball off to Swanson who took one touch to set himself before rifling it into the top left-hand corner from around 25 yards.
It was the full-back’s fifth goal for the Blues since joining from Arsenal in 2022.
Millwall suffered a blow as Zak Sturge took a ball to the face while looking to stop a cross by Florian Bianchini.
The full-back fell spent several minutes on the floor and was then carried off on a stretcher to where an ambulance team was stationed.
Millwall almost got themselves straight back into the game after the break as Macaulay Langstaff’s effort was deflected narrowly wide of the near post.
It was the home side who were able to add to their lead minutes later as Murphy played a low through ball in behind the Lions defence.
Segecic then dinked the ball towards the back post and his effort looped up high and over Crocombe before dropping into the back of the net.
Bryan was shown a straight red card in the 67th minute for taking out Lang but Millwall halved their deficit despite the numerical disadvantage thanks to Ivanovic’s smart right-footed shot.
But Williams ensured Pompey’s victory in style, intercepting a pass on halfway before running all the way to the edge of the Millwall box where he struck a right-footed effort into the back of the net.