Prevent Mavericks from generating new volume name for external drive

I have an external HDD, formatted exFAT, I believe. Whenever it is re-mounted, it gets assigned a new volume name (e.g., "MyVolume", "MyVolume 1", "MyVolume 2"...). As far as I can tell, this happens whether it is properly ejected, or if it is still mounted when the system is powered down.

Aside from going in to /Volumes/ and deleting all the existing "MyVolume*" files before attempting to mount the drive, is there a way to prevent this from happening?