SSH via sudo cron on Yosemite

I have a shell script that is supposed to run via cronjob on Yosemite, also when no user is logged in. I discovered the cron has to be edited with sudo crontab -e to do that.

The script connects to a webserver via SSH, which has been set with ssh-copy-id. But the ssh connection only seems to work if I run the script manually.

My first guess would be that the ssh connection from the sudo cron can't access the local key for the user (as this happened to me on another system in combination with ForwardAgent: Yes. But I am not sure and have no idea how to debug this.

Here's an abstract version of that script.sh:

#!/bin/sh
hosts="user@server.com"
for h in $hosts ; do

    # make a local directory: this works every time the cron runs
    rm -r /Users/user/Webserver-Backup/cron-lastrun
    mkdir /Users/user/Webserver-Backup/cron-lastrun

    # make a directory online: this never happens
    ssh $h rm -r /home/${h%%@*}/public_html/cron-lastvisit
    ssh $h mkdir /home/${h%%@*}/public_html/cron-lastvisit

done

Is it even possible to ssh with a key pair via the sudo cron? If yes, how do I have to set up the login?