I have gotten Mozilla to build using DistCC on Mac. Luckily, everything you need is already installed with XCode. If you can build Mozilla in the machine, you should be good to build using DistCC. DistCC is a wrapper for the compiler which distributes portions of the build process to other machines. Some things which can’t be done in a distributed way like linking are done on the machine in the driver’s seat. In DistCC you have one client and many servers. I found this confusing at first, because I was thinking of the computer driving the build as serving jobs for clients, but in fact, it is the client which is using ‘compile servers’.
This is actually a very simple thing to set up, once you know what you are doing. For this blog post, I tested using two of our school macs. I have spain (142.204.133.122) as a build server and canada (142.204.133.7) as a client. On all the server machines I have started the DistCC daemon by running distccd --daemon. It is possible to run this with only allowing jobs from specified IPs but this is easier. Make sure that port 3632 is reachable by your servers as that is where the jobs are sent and results received. On the client machine (in this case canada) I need to configure distcc to make use of the build servers. With the included version of DistCC on Leopard, you have to specify this using an exported environmental variable. I used export DISTCC_HOSTS='localhost 142.204.133.122' in the shell I ran the build from. Next you are going to need to wrap the compilers to make use of DistCC. I have done this by overriding the default C and C++ compilers in my .mozconfig file (below) with the CC and CXX environment variable in my make flags. You will want to specify a job count (-j6 in this instance) to make use of DistCC. It is hard to pick a good number, but it should definitely be greater than the number of total processor cores and some say an extra two for good measure.
canada:~ jhford$ cat ~/.mozconfig
mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj
ac_add_options --enable-application=browser
mk_add_options MOZ_MAKE_FLAGS="CC='distcc /usr/bin/gcc' CXX='distcc /usr/bin/g++' -j6"
When you start your build with make -f client.mk build you will be able to monitor the status of your distribution using the also included distccmon-text. An important thing to note, which caused me much grief, is that this program is run on the DistCC client (the driver). This program will give you a nice little bit of output like this:
This shows you which jobs are going to which machines. Another thing that indicates that distcc is working is the g++ build steps, which should now look like this (greatly abridged) example:
distcc /usr/bin/g++ -o nsMorkHistoryImporter.o -c -I../../../../dist/include/system_wrappers -include /Users/jhford/mozilla-central/config/gcc_hidden.h -DXPCOM_TRANSLATE_NSGM_ENTRY_POINT=1 -DMOZILLA_INTERNAL_API -D_IMPL_NS_COM -DEXPORT_XPT_API -DEXPORT_XPTC_API -D_IMPL_NS_COM_OBSOLETE -D_IMPL_NS_GFX -D_IMPL_NS_WIDGET -DIMPL_XREAPI -DIMPL_NS_NET -DIMPL_THEBES -DZLIB_INTERNAL -DOSTYPE=\"Darwin9.6.0\" -DOSARCH=Darwin
As far as results go: with an iMac that has 1GB of memory as client I got these times for the DistCC with a Mac Mini as a server (-j6):real 20m41.704s
user 22m5.748s
sys 5m22.026s
With 12 jobs (-j12) I got:real 19m54.374s
user 20m39.539s
sys 5m20.019s
And with just the single iMac with 1GB (-j4):real 20m25.626s
user 27m3.408s
sys 4m22.820s
I watched the output of the distccmon-text and noticed that only about half of the files get distributed, with the other half being done on localhost. I am thinking that it would be good to test this with a more powerful machine as the client and more servers, but as this is right now, there is nearly zero benefit to this configuration.
”. While this only be one user, it occurs in a setting that if anything, encourages this behaviour. This post highlights everything I dislike about ubuntu.
