Ignore an AirPlay audio device completely

Is it possible to somehow ignore an AirPlay audio device completely (or at least stop my mac from automatically connecting to one)

We have an AppleTV at the office and every time I open my MBP at work, the AppleTV is automatically selected as sound output. I'm guessing it's because I usually use another AirPlay device at home. So is there any way to prevent the office AppleTV from automatically connecting, but still be able to use AirPlay at home?