In Screen Sharing, how do I stop the local Mac from grabbing Mission Control keyboard shortcuts meant for the remote Mac?

I have two Macs, each with ^-1 through ^-4 configured to switch between Mission Control desktops in Keyboard/Shortcuts. When I use Screen Sharing, these shortcuts always go to the local Mac, never to the remote Mac, regardless of whether the remote Mac otherwise has focus.

If I go into Keyboard/Shortcuts and disable these on the local Mac, I can then use them on the remote Mac. However, it's a hassle to toggle these every time I start/stop using Screen Sharing. Is there a way to get the local Mac to stop grabbing the shortcuts?