The location of ~/.xsession-errors is, at the time of writing, specified in /etc/X11/Xsession in Debian Squeeze (testing).
To disable logging, change this to
I use several computers with the same, networked, home directory, and having all of them output to the same file was a hopeless mess. However, this solved the problem for me:
ERRFILE=$HOME/.xsession-errors-$(hostname -s) rm -f "$ERRFILE"
This makes a separate .xsession-errors for each of my machines and also removes the file each time X is started so it doesn’t grow too big. If X itself crashes, the errors to to /var/log/messages anyway, so there’s no need to save the .xsession-errors for this.