Deleting an update downloaded from App Store that is not installed

I now run OS X 10.9.5 on my office laptop, which is monitored by my company. App Store alerted me about Yosemite, and I chose to download and install it. After download completed, I got a notification saying that my company does not support Yosemite and the installation was terminated.

Now Yosemite shows up in App Store as "DOWNLOADED" as shown below, and I get notification asking me to install it every day. How can I remove the downloaded file and/or stop receiving this notification?