Is it safe to remove read, write, and execute permissions for 'others' recursively on home directory?

I would like advice on whether it would be safe to block users not in the appropriate group from accessing my home directory. I have some servers running on my Mac (10.9) under a different account, and wish to decrease the consequences of a hacker exploiting one of them by preventing this account from accessing any of my personal files.

I am intending to use file permissions for this, but I am concerned that this may break something, by blocking system processes from doing what they need to. I would ideally like to block all but root and my primary account from accessing my primary account's home directory, with chmod -R o-rwx ~.

Does anyone know whether this would cause any problems, and also whether it would be effective in preventing other accounts (that may not sudo, and are not admin in System Preferences, these may be the same, not sure) from accessing these files, excluding an EoP vulnerability?