Don’t Lie To Your Build System

Yesterday, in Make Concepts, I talked about what Make does. Today, I’m going to talk about how Make gets abused.

.PHONY Targets

That last post ended with a short summary of the common features you need to be able to understand the majority of most Makefiles, but it lied a bit. There’s one more thing: .PHONY. It’s a target that tells Make that its dependencies aren’t actually files. It’s also not actually a file. You’ll see this pretty frequently in real Makefiles:

.PHONY: clean all

all: things you actually care about building

# ...

     rm -f things you actually care about building

There are no files actually named .PHONY, clean, or all. Make knows this, so when it goes to build clean, it doesn’t skip the recipe if there happens to be a file named clean. You can read more about why phony targets exist in the GNU Make manual but I’m here to talk about times when they’re not used but should be, or when they are used but shouldn’t be.

That all target is very convenient. Remembering the names of all the things I want to build is hard, particularly when I’m running a complex build system whose ultimate output is a file called out/lib/my_project_v1.7.328.a and maybe another file called out/bin/fancy_program and a bunch of tests whose results are in files called things like out/test_results/test_invert_frob.xml. I just want to write make lib or make exe or make test and have Make build everything it needs for me, rerun tests as appropriate, and put the files where they should be.

This is all good and proper. I can write some rules like

.PHONY: test


out/test_results/test_%.xml: out/tests/test_%
     $^ > $@

out/tests/test_%: tests/test_%.c
     $(CC) $(CFLAGS) -o $@ $^ $(TEST_LFLAGS)

If I build my $(TEST_RESULTS) variable sensibly, I can add tests easily and any time I update my test source code, I can just run make test and I’ll automatically get new values in my test result files for any affected tests. There’s even some weird magic built into GCC that will parse my test .c file and generate a bunch of dependencies for any headers it includes.

Lying With .PHONY

But what if I get lazy, or have the wrong model of what Make is doing, and instead just do something like this:

.PHONY: test

     $(CC) $(CFLAGS) -o test1 test1.c $(TEST_LFLAGS)
     $(CC) $(CFLAGS) -o test2 test2.c $(TEST_LFLAGS)
     $(CC) $(CFLAGS) -o test3 test3.c $(TEST_LFLAGS)
     ./test1 > test1.xml
     ./test2 > test2.xml
     ./test3 > test3.xml

Or maybe instead something like this:

.PHONY: gen all

all: gen final_exe

     generate_code > generated.c

final_exe: $(SOURCE_FILES)
     $(CC) $(CFLAGS) $(IFLAGS) -o $@ $^ $(LFLAGS)

Now, Make will happily run the test recipe for you, and when you run make all you’ll get final_exe out. But it will be a lie. Make’s whole thing is making files from other files, and these examples completely circumvent that thing. These would be better off as shell scripts (or properly written Makefiles) because when someone else comes along to try to extend them, they will fail in frustrating and unexpected ways.

For example, if someone runs make -j all instead of make all in the second example, sometimes it will fail to build. Parallelizing Make is a common tactic for speeding up the build. Its ability to do this is arguably Make’s single killer feature. I once worked at a company whose build took around 30 minutes. Fixing up the Makefiles and flipping on -j 9 dropped that to around 2 minutes for a full build, and incremental rebuilds took 1 to 2 seconds unless they touched widely used and rarely modified header files.

I don’t want to argue that you shouldn’t lie to Make because it will run faster, though. It will, but some builds are either so fast that it doesn’t matter or so slow that it makes no difference. I’m not really even arguing that you shouldn’t lie to Make because it will lead to subtle bugs. It will though, because having a file you modified not get rebuilt is a frustrating problem to have.

You shouldn’t lie to Make because it will simplify your life. Adding tests that depend on your build artifacts is simple when your build artifacts are correctly specified in your Makefile. I sometimes have to do something like the following:

  1. Modify some code.

  2. Rebuild the firmware.

  3. Flash the firmware onto a dev board.

  4. Run a particular test against the recently flashed dev board.

Make can simplify all of this into a single command. If I only care about the one test that’s failing, I can just run make test_that_fails.log and it will do steps 2-4 for me.

Signs That Your Build System Is Being Deceived

Alright, so you’re convinced that you shouldn’t lie to your build system, but you’ve inherited it and you’re not sure whether it needs rejiggering or not. Here are some indicators.

People routinely run make clean before re-building.

If it’s normal to run make clean before building code, you might be lying to your build system. Either that, or you should ask your IT people to get the fileservers synchronized with NTP.

Phony targets have recipes.

If your convenience shortcut targets actually have stuff being done in them, you’re probably lying to your build system. The only thing an all target should generally have is a list of dependencies.

.PHONY: all

all: thing1 thing2
     Big red flag that anything is here

More than one thing is getting built per recipe.

This isn’t a guaranteed indication that something’s wrong; some tools generate lots of outputs (like compile stages where you keep intermediate files to be used by other tooling) but it’s definitely suspicious.

In particular, if you’re running one command to generate one file and another command to generate another file in the same recipe, they should be broken up.

main_thing: $(SOURCE_THINGS)
     $(COMPILE) -o $@ $^
  also do something here > Shiny_Red_Flag


That’s it. Tell your Makefile about the files it’s making and your life will be better. Treat it like a fancy shell script with different options to run different functions and your life will be worse. More importantly, if I work with you, my life will be worse.

Previous: Make Concepts