And if the Apple stock goes to $500, do you get a tax refund?

This is why, from a cash flow perspective, taxing unrealized gains would create a nightmare. A good year in the markets, and taxes paid would run high. A bad year in the markets, and revenue would dry up. To some extent that's probably how things are now, but it would be many times worse.

Let's say someone buys $500k worth of a volatile stock like Coinbase. In a year it's worth $1M, and we tax the $500K unrealized gain so they owe, let's say, $100K in taxes. They sell $100K worth of stock to pay the taxes, so they now have $900K worth of stock. The next year Coinbase falls 50%, so they now have $450K worth of stock, or less than they started with. They also experienced a $450K unrealized loss. How does that get treated? If their salary is $200K, do they get to write off the $450K loss against their salary, so they now owe no taxes on their salary? And still get a refund?

Plus, it's one thing when someone holds a publicly traded stock like Apple. It's easy to know what the value is on any given day. But what about someone who invests in a non-publicly traded partnership that invests venture capital in non-publicly traded businesses, or real estate? Those are not so easy to value year after year. It would be another nightmare. It could certainly create a lot of jobs trying to keep tabs on them all. :-)

I have no solution. It doesn't seem "fair" to not tax unrealized gains, but I think taxing them might create more problems than it solves.