Sportal.co.nz 10.Sep.2012Getty Images
The Northern Irish province were trailing 13-3 with 56 minutes on the clock, but replacements that included All Blacks prop John Afoa and British Lion Stephen Ferris made a major impact as the Ulstermen made it two wins from two.
The Ospreys looked set to atone for their opening defeat in Treviso as a fine solo try from Hanno Dirksen and eight points from the boot of first five-eighths Matthew Morgan appeared to put them in control.
But they crumbled badly late in the second half with their scrum coming under huge pressure from Ulster's reinforced pack.
The Ospreys had enjoyed the better of an error-strewn first half, lit up by a moment of brilliance from South African-born speedster Dirksen.
There was plenty of intent shown from both sides, but all too often promising attacks were undone by sloppy handling.
It took until 22 minutes for the game to come alive with right wing Dirksen producing a superb individual run to claim the game's first try.
Picking up a loose pass 35 metres out, Dirksen hurtled through a fragile defence, fended off Ulster fullback Jared Payne and then turned on the afterburners to make the line unopposed.
Morgan added the extras, but the Ospreys failed to build on their lead as mistakes continued to plague their game.
First five Niall O'Connor made up for an earlier miss to land a fine long-range penalty after 31 minutes, but failed with an ambitious attempt from inside his own half just before the break as the Ospreys went in at half-time 7-3 to the good.
Two penalties from Morgan after the break extended the home side's advantage as the champions attempted to assert control.
But a clutch of substitutions by Ulster coach Mark Anscombe turned the match on its head.
The arrival of Afoa and Ireland prop Tom Court saw the Ospreys scrum forced into retreat, while Ferris produced some thunderous tackles.
Another replacement, Paddy Jackson, nailed a couple of penalties to make it 13-9 before yet another member of the cavalry, halfback Paul Marshall, claimed the match-winning score with six minutes remaining.
A sharp counter-attack from wing Craig Gilroy set up the opportunity which eventually saw Marshall touch down from close range.
It needed the television match official to confirm the score, but Marshall and his Ulster team-mates were in little doubt.
It left the Ospreys with little time to redeem themselves, but any chance they had of salvaging the win disappeared when they coughed up the ball again, leaving them with just two losing bonus points to show from their opening two matches.