When Japan lock, and New Zealander, Warner Dearns charged down a ruck clearance by halfback Finlay Christie to regather the ball and run unchallenged to get Japan to 24-28 behind, 65,188 Japanese fans had cause to hope a first win over the All Blacks was on the cards.
Then, when lock Brodie Retallick was red-carded for dangerous play in an attempted cleanout in the 65th minute, they had hope again.
Outstanding openside flanker Kazuki Himeno with three minutes left, lifted hopes again, but New Zealand's 14 held them out and ended the game with first five-eighths Richie Mo'unga kicking a penalty goal.
What impressed about Japan's effort was their ability to absorb everything New Zealand threw at them, being forced to make 182 tackles compared to 103 by the All Blacks, while New Zealand won 121 rucks and mauls to 65.
Inspired efforts abounded for the home team as they made the most of the possession they gained. Himeno was ever present in the loose, locks Dearns and Jack Cornelson were athletic and competitive with ball in hand while blindside flanker Michael Leitch contested the loose ball and provided an experienced leadership hand. And in the backs, centre Dylan Reily and left-wing Siosaia Fifita and replacement Gerhard ven der Heever, took chances as they came.
They provided a timely reminder for the All Blacks of the job that lies ahead of them in the remainder of the their tour.
Down 17-21 at the break, New Zealand came out to start the second half at speed, to try to breakdown the Japanese resolve. It was wing Caleb Clarke who ran off a pass from second five-eighths Roger Tuivasa-Sheck to extend the lead. But Japanese defence denied them until they lifted their tempo and power, following Dearns' try, to hammer the Japan line before No8 Hoskins Sotutu scored.
But then losing Retallick gave Japan another chance to apply the pressure, and it was some determined All Blacks defence, and time, that denied them a famous victory.
It was the first time in five victories over Japan that the All Blacks hadn't passed the 50 points mark and represented the advance the host team has made.
Typical of Japan's response was when New Zealand seemed set to pour on the power in the run to halftime, it was Japan who found their second wind and scored two tries in four minutes.
Enjoying a good share of possession throughout the first half, Japan took time to find their continuity, much of the sting in their game being taken out of them by having to contain the All Blacks with their tackling.
Using their set piece to maximise their opportunities during the middle stages of the first half they benefited from the unique skills that last-minute replacement Samisoni Taukei'aho provided, both with his ball play and his lineout throwing.
In New Zealand's first try, Taukei'aho broke the defence in midfield, drew the final defender and then fed Retallick in for a 25-metre run for the try.
New Zealand missed a chance to score from a five-metre scrum in Japan's goalmouth when No8 Hoskins Sotutu was turned and unable to present the ball. Moments later, the All Blacks used a similar position to move the ball right, then left and it was Mo'unga who got a ball away to second five-eighths Roger Tuivasa-Sheck and it was centre Braydon Ennor who scored in the 25th minute.
The TMO Marius Jonker felt there might have been a forward pass by Mo'unga but referee Nika Amashukeli said there wasn't enough evidence to deny the try.
In the 31st minute, Taukei'aho's pinpoint long throw was taken in midfield by Tuivasa-Scheck and his instant in-pass gave left-wing Sevu Reece space with one man to beat and he rounded him with ease for the third try.
Making more attacking play, the All Blacks lost the ball when Himeno turned the ball over. The ball was played down the right flank and kicked ahead. Perofeta almost pulled off a superb catch but lost the ball forward, and then Ennor couldn't hold the ball when going down on it.
It was kicked ahead with talented Japan first five-eighths Takuya Yamasawa kicked it ahead, and then gathered the ball to score.
From the restart, Japan worked the ball down the right flank with Leitch making good ground, the ball went through the line with Reilly getting the ball out wide, and then passed inside to halfback Yutaka Nagare who ran in the try.
Scorers: Japan 31 (Takuya Yamasawa, Yutaka Nagare, Warner Dearns, Kazuki Himeno tries; Yamasawa 2 con, pen; Seungsin Lee 2 con) New Zealand 38 (Brodie Retallick, Braydon Ennor, Sevu Reece, Caleb Clarke, Hoskins Sotutu tries; Richie Mo'unga 5 con, pen). HT: 17-21