Back in 1976, when Make first entered the scene, it solved the problem of automating dependency aware software builds. By defining what sources a build target operates on, it could automatically determine what should be compiled based on what files had changed, thus saving on computing time while guaranteeing correct builds.

Command vs Automatic Targets

The ability to define build targets that could be invoked from the command line as make arguments, quickly lead to Make being used for other tasks than just building software, think of make install for example.

This “extracurricular” usage of the Make build tool proves useful to this day. A Makefile at the root of your repository provides a quick way to define commands for; bootstrapping the repository, running code-generation, executing tests, kicking off deployment and last but not least, if your project requires it, building executables. I have been thinking about these targets as “commands”, they are the main entry points into the functionality of your Makefile.

A well equipped makefile becomes a command line tool for your repository.

The beauty of Make is its dependency system. By defining prerequisites (what targets should run before your target, and/or did any of the files-your- target-depends-on change, thus requiring your target to run), you can ensure for example; that code is built before running tests, or that tests re-execute with updated coverage tracking configuration, before opening the test-coverage report in a browser. Let’s call targets in these dependency chains “Automatic” targets. They are like building blocks. You’d rarely execute them individually, but together they enable powerful functionality.

In recent projects, I’ve used Makefiles in this exact way. The Makefile offers a command line interface to the repository. It can be used to:

• bootstrap the repository
• generate code
• run various linters
• run django, both as a server and in shell mode
• run tests, with or without coverage tracking
• open coverage reports in a browser (which of course automatically re-executes tests when necessary)
• build docker containers
• run django shell & server inside the docker container
• run tests inside of the docker container
• deploy
• configure cloud tools to target development, staging or production environments
• forward ports to a kubernetes pod
• etc.

All that, performed with required prerequisite work, by just running make with a simple argument. It has been a joy to use. But as so often in life, you can’t have a silver lining without a cloud. This abundance of functionality in a Makefile brings with it two problems. 1) Silo-ing of potentially re-usable functionality and, 2), poor user experience, due to lack of target discoverability and an epic case of cognitive overload when trying to read through them.

Make help

Most Makefiles contain a boat load of targets, alphabetically-ish sorted if you’re lucky. There is no good way to distinguish command targets from automatic targets, let alone to get a quick idea of what you can do with a Makefile.

Despite that shortcoming, I still thought it was worth writing about this particular way of Makefile usage. In past employment and projects, I’ve seen countless examples of important commands having to be copied from documentation, and I don’t know how I would have done my job without relying heavily on bash history. Makefiles with a variety of command targets would have been very valuable to me in those days.

Then, while doing some research for this post, I came across how to create a self-documenting Makefile, by Victoria Drake. In that article, she not only outlines my repository command line tool use-case for Makefiles, it goes on to introduce the exact thing I had missed in my Makefiles, a quick way to get an overview of the commands contained within them 🤯 🎉.

.PHONY: help
help: ## Show this help
	@egrep -h '\s##\s' $(MAKEFILE_LIST) | \
	sort | \
	awk 'BEGIN {FS = ":.*?## "}; \
		{printf "\033[36m%-20s\033[0m %s\n", $$1, $$2}'

Some more Googling revealed that variants of this help target have been presented a few numerous times over the years, ultimately evolving to the version published by Victoria. Here’s how it works:

• first it loops over MAKEFILE_LIST (a list of all lines in the makefile that define a target) using egrep, and retains the lines that have a ## comment.
• it then sorts the remaining items
• and finally, runs the sorted lines through awk`, with a few tricks
  • BEGIN {FS = ":.*?## "}; BEGIN denotes a block that is to be executed before any input is processed. This block assigns FS, the Field Separator, to a regular expression that matches everything from the semicolon (which follows the target name), including the prerequisite list to the ## that indicates a ‘help’ comment. This causes each Makefile target line to be broken up in the target name ($1) and the help comment ($2).
  • {printf "\033[36m%-20s\033[0m %s\n", $$1, $$2} this one seems a little tricky at first, as there is a lot going on, but simply put, it uses printf to output the target name and the help comment in two columns. Let’s break the format string into its parts:
    • \033[ starts a terminal escape sequence and 36m then sets the text color to “light cyan”
    • %-20s will print a string variable, the target name, padded on the right up to a width of 20 characters
    • \033[ starts another escape sequence in which 0m resets the text color
    • %s\n prints the second string variable, the help text, along with a newline character.
    • $$1, $$2 are the first and second parts of the line, target name and help comment as broken up based on the Field Separator rule defined earlier. $$ because $ needs to be doubled up inside a Makefile.

With the help target defined in the Makefile, and “command” targets annotated with ## help text comments, executing make help will now print out a quick overview of what you can do with the Makefile. Brilliant.

A nice side effect of ## help text comments on command targets is that it visually separates them from their “automatic” siblings when reading through the Makefile.

Makefile target re-use

When using Makefiles as a command interface for your repository, it will likely contain targets that can be shared with other repositories in your organization or other projects you work on. The help target presented above is a prime example. Most Makefiles are self-contained. Their targets are specific to the repository and project they exist in. But Makefiles can include other Makefiles, which offers a way to bring in common targets.

-include common.mk

Makefile includes can live in a different repository, a git submodule, a symlinked directory, etc. The possibilities are endless. The Makefile could even contain bootstrap logic to fetch the included files. If that is the case, ensure the include statements are prefixed with “-”. This will ignore import errors and allow targets in the Makefile to run without the include files being present, thus preventing a bootstrap / include catch-22 situation.

Over the past year, I have created Makefiles for various repositories based on the practices outlined above. Shared logic is brought in through git submodules and using make help, command targets are discoverable and documented. As an added bonus, maintaining and improving this toolkit is a great way to “get in the zone”. I’ll be using this wherever appropriate and I hope you will find it useful too. If so, drop me a line!

-@niels

btw, here are some resources for writing Makefiles I found helpful:

• GNU Make Manual. There is a *lot* to Make. (though I’ve had most luck just searching with Google)
• Your Makefiles are wrong by Jacob Davis-Hansson which introduced me to the concept of sentinel targets. These targets are named after- and touch a “sentinel” file when the work is done. This allows targets that don’t produce anything or produce multiple files to function in a dependency chain. I use this a *lot* for writing “automation” targets.
• How to create a self-documenting Makefile, by Victoria Drake. Make help.
• Checkmake, a Makefile lint tool, conveniently executable from a Docker image, which enables; make lint-make, a self-check for your Makefile 😜

.PHONY: lint-make

lint-make: ## Run Checkmake on Makefile (requires docker)
	@docker run --rm -v "$$(pwd):/data" cytopia/checkmake Makefile