No one believes that this just reduces to "voters unhappy = incumbents lose" going forward.

But isn't it usually true?

Sometimes. Voters are often unhappy about something, but still return incumbents to power.

This election cycle was globally unique. Voters turned out incumbent parties not just in countries where that's common (like the U.S.), whether left-leaning (like the U.S.) or right-leaning (like the U.K.). They also roasted incumbents in places like Japan, where the ruling party has been in place since 1955 - through many prior periods of voter unhappiness:

apnews.com - Global elections incumbents defeated

Voters probably won't be "unhappy" if gas prices don't fall below $2, or home prices below $200K. Some of the ones who genuinely believed that Trump would usher in a golden era might be disappointed that their fantasy scenario didn't manifest. But on the whole, voters will probably be okay with a "normal" economy if inflation rates stay low and interest rates fall back down to below 5%.