If Makefile.PL reports that it cannot find X, or chooses a version
you don't like you can specify a version on the command line:

perl Makefile.PL X11=/usr/local/X11R5