Making clang always look for libraries in /usr/local/lib

I'm using homebrew and have a lot of libraries in /usr/local/lib as well as include files in /usr/local/include. What is the easiest way to make sure that clang and ld in all circumstances will look into these directories?

My main issue is that some programs seem to have build scripts that filter environment variables, so storing these paths in the environment will not work. Is there a configuration file somewhere that stores the default search paths or is this hard coded into the program?

I realize that I could replace the clang and ld binaries with wrapper scripts that add the -L and -I flags to each call, but this is not very clean and would be overwritten by updates of Xcode.

This question is not about how to create your own project in Xcode and configure the search paths. What I want is to make compilation of 3rd party software easy.