We forget one thing, Japan's militarists acted in ways that would be utterly laughable if they hadn't cost the lives of so many innocent people.
Japan's militarists started to believe a delusional fantasy full of magical thinking. Those in charge and who should have known better were blinded by opportunities and early success. When things went completely wrong they were afraid to change course for fear that the "middle party" composed of fanatical true believers would put them before the firing squad for treason …
Even they remained blind to the fact that the allies had fully committed themselves to a total unambiguous victory, they wouldn't settle for anything less, lest it come back and bite them in the butt as it did after 1918.
The militarists dropped the ball in thinking they could still salvage a favourable deal, some even still believed that the allies were so war weary they would allow Japan to keep all their pre-1942 gains 1941.
If anything the Allies were doing everything to precipitate the fall of Japan. They turned to firebombing as the default tactic, in the hope a concentrated air campaign might break Japan's back before an invasion became inevitable. The atom bombs fitted into that category, The bombs wasn't a dramatic step up as some suggested, Japan was already being hammered on a Biblical scale, firebombings and the resulting famines and disruptions killed hundreds of thousands and displaced tens of millions, some cities had 99% destruction rates.
By then the military had finally lost all credibility. The navy was all but gone, their air force spent and the army was being beaten on all fronts. They could not do anything to stop the Allies from destroying Japan simply through massive aerial bombardment.
The opposition, including the old Japanese aristocracy had had enough. They had been pushed aside by the violent military dictatorship, but now they had the backing of the emperor who wasn't a believer in the apocalyptic final stand scenario.
I think we have to see this beyond the scope of "atom bombs preventing a costly invasion". It's the aftermath of many years of brutal warfare across the globe. A massive effort to defeat nazi Germany that took years to achieve and cost millions of lives. It's the many years of fighting in the Far East and Pacific. It's the incredible brutality of the Japanese in their occupied territories. The largely unknown war in China rivals the struggle between the USSR and Germany. The end goal was not pre-empting the invasion, it was the total defeat of Japan.
The firebombing and atom bombs shattered the last illusions that the Japanese could force their enemies to fight for every last inch of Japan and the last defenders would go down in a glorious last stand and prove they were superior to the end. There was no glory in being bombed to pieces even the fanatics realized this soon enough.