Mac trying to play audio through display port

I have a Dell monitor connected through a thunderbolt port.

Sometimes my Mac for some reason is attempting to play audio through that port. How do I stop this from happening?

It appears in my list of sound outputs in system preferences but I have headphones selected. Unplugging the monitor from the thunderbolt port makes the audio work correctly.