After the siren had sounded at Suncorp Stadium in Brisbane, France had a lineout and the chance to clear the ball to touch.
But they fumbled the ball, and after halfback Tate McDermott grabbed it, the Australians carried the ball forward through a succession of penalties. They couldn't get over the line, but they did gain a penalty chance, and first five-eighths Noah Lolesio kicked the goal for the win.
It was a satisfying moment for the five-eighths who has had his frustrations in his starting debut 43-5 loss to the All Blacks in Sydney last year, and who only gained this start because James O'Connor hadn't recovered from injury.
It was a boost for the Australians in their first outing.
Rennie told the media after the game that he thought the chance to win had gone.
"Hooter has gone, the opposition has got the ball, they win the lineout, you think you're in trouble," he said.
"But there was a lot of character shown by the boys. We had a few opportunities late we didn't nail. I was pleased to get job done in the end."
It had been a long break from Test rugby, with no play since the 2020 Rugby Championship.
"I miss Test footy. We had a win, and we're happy about that.
"But we can be a lot better.
"It looked like a first Test performance, a bit scratchy, some inaccuracy. If we can tidy that up we can apply more pressure and turn that into points," he said.
Lolesio, 21, had missed a dropped goal that would have provided the win moments earlier.
"From my debut to this. I'm so proud of this team, and myself to bounce back like that. Credit to the French, far out man, it's a tough Test," he said.
The sides play their second Test at AAMI Park on Tuesday and their third Test back in Brisbane on Saturday, July 17.