How can I access the keychain remotely from the command line?

I am storing various passwords (e.g. for remote email servers) in my keychain. From the command line, when logged in locally, I can retrieve these via:

security unlock-keychain ~/Library/Keychains/login.keychain
<enter password>
security find-internet-password -s smtp.gmail.com -a foo@gmail.com
<dumps keychain attributes, not including password>
security find-internet-password -s smtp.gmail.com -a foo@gmail.com -g
<dumps keychain attributes, including password>

However, the results cannot be achieved when run remotely (ssh'ing in to the box from somewhere else):

security unlock-keychain ~/Library/Keychains/login.keychain
<enter password>
security find-internet-password -s smtp.gmail.com -a foo@gmail.com
<dumps keychain attributes, not including password>
security find-internet-password -s smtp.gmail.com -a foo@gmail.com -g
<nothing printed, to stdout or stderr>

Moreover, the return value of the final command (using -g) is 36.

I've dumped the output of set from a local login and compared it to a remote one, and the missing environment variables are:

Apple_PubSub_Socket_Render
GPG_AGENT_INFO
SECURITYSESSIONID
TERM_PROGRAM
TERM_PROGRAM_VERSION
TERM_SESSION_ID
XPC_FLAGS
XPC_SERVICE_NAME

What am I missing? I do have SSH_AUTH_SOCK set to a valid value (returned from ssh-agent).