Target Display iMac steals Thunderbolt when I unplug and won't give it back

I'm using a 27" iMac in Target Display Mode, daisy-chained though a Thunderbolt display on my MacBook Pro. Both are running Yosemite.

Typically I come into the office in the morning, connect my laptop to the Thunderbolt, then power on the iMac and switch to Target Display Mode. My laptop is now happily running two external displays.

However, when I unplug my laptop to go to a meeting, the iMac grabs the Thunderbolt as a second display. I don't really care about this for now, but when I plug the laptop back in and switch the iMac back to Target Display Mode, the Thunderbolt becomes the iMac's only display, and now the laptop only has a single external display.

Is there any way of preventing the iMac from grabbing the Thunderbolt, or forcing it to relinquish it when I return, short of disconnecting and reconnecting the cable (really don't want to do this, the cable is flaky enough already) or rebooting the iMac? Even putting it to sleep doesn't seem to cut it.