In the 16th minute Pompey took the lead through Devlin thanks to a big deflection.
Conor Chaplin whipped in a cross from the left which was cleared only as far as the Blues defender, who struck the ball with his right foot which was diverted off a Hull player and it flew into the top corner.
Just before the half hour mark the Tigers equalised, though, when Coyle dashed down the right wing and slid the ball into the box where Destan finished from close range at the near post.
The hosts then went in front as Gelhardt’s inswinging delivery was headed home by Joseph, but their hopes of going into the interval ahead were dashed when Portsmouth scored to level the game up.
Murphy crossed with his left foot and found the head of unmarked Devlin, who nodded his second of the game.
Just after the hour mark Hull went close when Ryan Giles crossed from the left and found Regan Slater who glanced it over the crossbar.
The hosts then took the lead again with just over 10 minutes remaining, Charlie Hughes drilled a ball over the top which was beautifully taken down by Gelhardt.
The forward – who is on loan from Leeds United – then calmly chipped the ball over onrushing Portsmouth keeper Bursik and into the back of the net.
Pompey have now lost four of their last five matches and last won a Championship game on 4 October.