Can't delete Exchange account from Mail or Internet Accounts preferences

The Yosemite upgrade completely hosed Mail—it just won't talk to the Exchange server anymore.

"No problem," I think, "I'll just delete the account and start over as it's all on the server."

If I try and delete the account from Mail.app's preferences, it says "The Mail account “foo@bar.com” is shared by other applications on this Mac. You can remove this account in Internet Accounts."

Ok, so I click on Internet Accounts, it opens Preferences, and the Exchange account isn't there.

(Interestingly enough, I get the exact same problem when I try and remove the account via Contacts and Calendar's preferences—even when "Enable this account" is unchecked in all three.)

So how do I force Yosemite to delete all traces of an Exchange account?