Self-documenting makefiles

README.md

What is it?

A “simple” make rule that allows pretty-printing short documentation for the rules inside a Makefile:

How does it work?

Easy: simply copy everything starting at .DEFAULT_GOAL := show-help to the end of your own Makefile (or include show-help-minified.make, and copy that file into your project). Then document any rules by adding a single line starting with ## immediately before the rule. E.g.:

## Run unit tests
test:
    ./run-tests

Displaying the documentation is done by simply executing make. This overrides any previously set default command — you may not wish to do so; in that case, simply remove the line that sets the .DEFAULT_GOAL. You can then display the help via make show-help. This makes it less discoverable, of course.

Thanks

Based on an idea by @marmelab.

Makefile

# Example makefile with some dummy rules

.PHONY: all
## Make ALL the things; this includes: building the target, testing it, and
## deploying to server.
all: test deploy

.PHONY: build
# No documentation; target will be omitted from help display
build:
	${MAKE} -C build all

.PHONY: test
## Run unit tests
test: build
	./run-tests .

.PHONY: deply
## Deploy to production server
deploy: build
	./upload-to-server . $$SERVER_NAME

.PHONY: clean
## Remove temporary build files
clean:
	${MAKE} -C build clean

# Plonk the following at the end of your Makefile
.DEFAULT_GOAL := show-help

# Inspired by <http://marmelab.com/blog/2016/02/29/auto-documented-makefile.html>
# sed script explained:
# /^##/:
# 	* save line in hold space
# 	* purge line
# 	* Loop:
# 		* append newline + line to hold space
# 		* go to next line
# 		* if line starts with doc comment, strip comment character off and loop
# 	* remove target prerequisites
# 	* append hold space (+ newline) to line
# 	* replace newline plus comments by `---`
# 	* print line
# Separate expressions are necessary because labels cannot be delimited by
# semicolon; see <http://stackoverflow.com/a/11799865/1968>
.PHONY: show-help
show-help:
	@echo "$$(tput bold)Available rules:$$(tput sgr0)"
	@echo
	@sed -n -e "/^## / { \
		h; \
		s/.*//; \
		:doc" \
		-e "H; \
		n; \
		s/^## //; \
		t doc" \
		-e "s/:.*//; \
		G; \
		s/\\n## /---/; \
		s/\\n/ /g; \
		p; \
	}" ${MAKEFILE_LIST} \
	| LC_ALL='C' sort --ignore-case \
	| awk -F '---' \
		-v ncol=$$(tput cols) \
		-v indent=19 \
		-v col_on="$$(tput setaf 6)" \
		-v col_off="$$(tput sgr0)" \
	'{ \
		printf "%s%*s%s ", col_on, -indent, $$1, col_off; \
		n = split($$2, words, " "); \
		line_length = ncol - indent; \
		for (i = 1; i <= n; i++) { \
			line_length -= length(words[i]) + 1; \
			if (line_length <= 0) { \
				line_length = ncol - indent - length(words[i]) - 1; \
				printf "\n%*s ", -indent, " "; \
			} \
			printf "%s ", words[i]; \
		} \
		printf "\n"; \
	}' \
	| more $(shell test $(shell uname) == Darwin && echo '--no-init --raw-control-chars')

show-help-minified.make

.DEFAULT_GOAL := show-help
# See <https://gist.github.com/klmr/575726c7e05d8780505a> for explanation.
.PHONY: show-help
show-help:
	@echo "$$(tput bold)Available rules:$$(tput sgr0)";echo;sed -ne"/^## /{h;s/.*//;:d" -e"H;n;s/^## //;td" -e"s/:.*//;G;s/\\n## /---/;s/\\n/ /g;p;}" ${MAKEFILE_LIST}|LC_ALL='C' sort -f|awk -F --- -v n=$$(tput cols) -v i=19 -v a="$$(tput setaf 6)" -v z="$$(tput sgr0)" '{printf"%s%*s%s ",a,-i,$$1,z;m=split($$2,w," ");l=n-i;for(j=1;j<=m;j++){l-=length(w[j])+1;if(l<= 0){l=n-i-length(w[j])-1;printf"\n%*s ",-i," ";}printf"%s ",w[j];}printf"\n";}'|more $(shell test $(shell uname) == Darwin && echo '-Xr')

Hi there,

thanks a bunch for this gist.

Automatically folds long lines in output with proper indentation.

I don't know whether this works for me, even with the updated show-help rule. Would it be possible to solve line wrapping using the fine fold program?

man fold

NAME
       fold - wrap each input line to fit in specified width

AUTHOR
       Written by David MacKenzie.

With kind regards,
Andreas.

What we are actually looking at would be something like that. A multiline comment which gets rendered 1:1.

## Load the MiniNet module to the device
## and start a WiFi STA connection.
##
## Synopsis::
##
##   make connect-wifi SSID=<YourNetwork> PASSWORD=<YourPassword>
##
connect-wifi:
	@$(rshell) $(rshell_options) --quiet cp lib/mininet.py /flash/lib
	@$(rshell) $(rshell_options) --quiet repl "~ from mininet import MiniNet ~ MiniNet().connect_wifi('$(SSID)', '$(PASSWORD)')"

-- https://github.com/hiveeyes/hiveeyes-micropython-firmware/blob/master/Makefile

@amotl I considered using fold but couldn’t get it to work satisfactorily (IIRC due to lack of support for hanging indents).

Rendering comments as-is (including multiline comments) is, in a way, much easier since you don’t need to care about line breaks etc., but it’s fundamentally incompatible with the automatic line breaking approach: You’d need to choose either one or the other.

If someone is interested, here is an adaptation of this help which supports multi-line documentation comments for both targets and variables (either defined globally, via define, or via a target), and categories (by adding a @category The category annotation in a doc comment).

It uses two seds and a sort. The first sed is used to retrieve and parse the documentation into a 'tag' like format. The second sed is here to render the documentation properly. It might be possible to change the sort order by changing the names of the tags. I used two sed instead of one sed and one awk because it seems that the sed command is faster that the awk on the os I use.

# fancy colors
RULE_COLOR := "$$(tput setaf 6)"
VARIABLE_COLOR = "$$(tput setaf 2)"
VALUE_COLOR = "$$(tput setaf 1)"
CLEAR_STYLE := "$$(tput sgr0)"
TARGET_STYLED_HELP_NAME = "$(RULE_COLOR)TARGET$(CLEAR_STYLE)"
ARGUMENTS_HELP_NAME = "$(VARIABLE_COLOR)ARGUMENT$(CLEAR_STYLE)=$(VALUE_COLOR)VALUE$(CLEAR_STYLE)"

# search regex
target_regex = [a-zA-Z0-9%_\/%-]+
variable_regex = [^:=\s ]+
variable_assignment_regex = \s*:?[+:!\?]?=\s*
value_regex = .*
category_annotation_regex = @category\s+
category_regex = [^<]+

# tags used to delimit each parts
target_tag_start = "\<target-definition\>"
target_tag_end = "\<\\\/target-definition\>"
target_variable_tag_start = "\<target-variable\>"
target_variable_tag_end = "\<\\\/target-variable\>"
variable_tag_start = "\<variable\>"
variable_tag_end = "\<\\\/variable\>"
global_variable_tag_start = "\<global-variable\>"
global_variable_tag_end = "\<\\\/global-variable\>"
value_tag_start = "\<value\>"
value_tag_end = "\<\\\/value\>"
prerequisites_tag_start = "\<prerequisites\>"
prerequisites_tag_end = "\<\\\/prerequisites\>"
doc_tag_start = "\<doc\>"
doc_tag_end = "\<\\\/doc\>"
category_tag_start = "\<category-other\>"
category_tag_end = "\<\\\/category-other\>"
default_category_tag_start = "\<category-default\>"
default_category_tag_end = "\<\\\/category-default\>"

DEFAULT_CATEGORY = Targets and Arguments

## show the help
help:
	@echo "Usage: make [$(TARGET_STYLED_HELP_NAME) [$(TARGET_STYLED_HELP_NAME) ...]] [$(ARGUMENTS_HELP_NAME) [$(ARGUMENTS_HELP_NAME) ...]]"
	@sed -n -e "/^## / { \
		h; \
		s/.*/##/; \
		:doc" \
		-E -e "H; \
		n; \
		s/^##\s*(.*)/$(doc_tag_start)\1$(doc_tag_end)/; \
		t doc" \
		-e "s/\s*#[^#].*//; " \
		-E -e "s/^(define\s*)?($(variable_regex))$(variable_assignment_regex)($(value_regex))/$(global_variable_tag_start)\2$(global_variable_tag_end)$(value_tag_start)\3$(value_tag_end)/;" \
		-E -e "s/^($(target_regex))\s*:?:\s*(($(variable_regex))$(variable_assignment_regex)($(value_regex)))/$(target_variable_tag_start)\1$(target_variable_tag_end)$(variable_tag_start)\3$(variable_tag_end)$(value_tag_start)\4$(value_tag_end)/;" \
		-E -e "s/^($(target_regex))\s*:?:\s*($(target_regex)(\s*$(target_regex))*)?/$(target_tag_start)\1$(target_tag_end)$(prerequisites_tag_start)\2$(prerequisites_tag_end)/;" \
		-E -e " \
		G; \
		s/##\s*(.*)\s*##/$(doc_tag_start)\1$(doc_tag_end)/; \
		s/\\n//g;" \
		-E -e "/$(category_annotation_regex)/!s/.*/$(default_category_tag_start)$(DEFAULT_CATEGORY)$(default_category_tag_end)&/" \
		-E -e "s/^(.*)$(doc_tag_start)$(category_annotation_regex)($(category_regex))$(doc_tag_end)/$(category_tag_start)\2$(category_tag_end)\1/" \
		-e "p; \
	}"  ${MAKEFILE_LIST} \
	| sort  \
	| sed -n \
		-e "s/$(default_category_tag_start)/$(category_tag_start)/" \
		-e "s/$(default_category_tag_end)/$(category_tag_end)/" \
		-E -e "{G; s/($(category_tag_start)$(category_regex)$(category_tag_end))(.*)\n\1/\2/; s/\n.*//; H; }" \
		-e "s/$(category_tag_start)//" \
		-e "s/$(category_tag_end)/:\n/" \
		-e "s/$(target_variable_tag_start)/$(target_tag_start)/" \
		-e "s/$(target_variable_tag_end)/$(target_tag_end)/" \
		-e "s/$(target_tag_start)/    $(RULE_COLOR)/" \
		-e "s/$(target_tag_end)/$(CLEAR_STYLE) /" \
		-e "s/$(prerequisites_tag_start)$(prerequisites_tag_end)//" \
		-e "s/$(prerequisites_tag_start)/[/" \
		-e "s/$(prerequisites_tag_end)/]/" \
		-E -e "s/$(variable_tag_start)/$(VARIABLE_COLOR)/g" \
		-E -e "s/$(variable_tag_end)/$(CLEAR_STYLE)/" \
		-E -e "s/$(global_variable_tag_start)/    $(VARIABLE_COLOR)/g" \
		-E -e "s/$(global_variable_tag_end)/$(CLEAR_STYLE)/" \
		-e "s/$(value_tag_start)/ (default: $(VALUE_COLOR)/" \
		-e "s/$(value_tag_end)/$(CLEAR_STYLE))/" \
		-e "s/$(doc_tag_start)/\n        /g" \
		-e "s/$(doc_tag_end)//g" \
		-e "p"

Which whould print something lite this

$ make
Usage: make [TARGET [TARGET ...]] [ARGUMENT=VALUE [ARGUMENT=VALUE ...]]
Targets and Arguments:
    cache-clear
        delete the cache
    help
        show the help
Tests:
    tests
        run the tests
    tests CACHE_RESULT (default: )
        cache the test results
    tests FILTERS (default: )
        execute only the tests matching the given filter
    tests STOP_ON_DEFECT (default: )
        stop testing at first defect / failure 

Hope this helps.
If someone has any improvements, I'm all ears.

A comprehensive and easy to use C++ Makefile example can also be found here:

https://www.partow.net/programming/makefile/index.html

Hello
I really liked @syffer 's solution (looks awesome !), however I had trouble editing it to make the adjustments I wanted (parameter docs, sections not sorted alphabetically) because it's quite complex for me to understand what these sed commands are doing 😆 so I went for a much simpler bash script which procedurally generates the docs. The script is not super well written I admit (help doc not generated from the actual help target for example, duplicate regexes with escaping, etc.), however it should be fairly simple to edit & extend !
Also, I wanted this 'help' to be available anywhere, from any makefile, so what I did was create two files

my-folder/tools/generate-makefile-help
my-folder/tools/Makefile-help.mk

and added /path/to/my/folder to a $ROOT_PATH environment variable loaded by my bashrc (note there is also a make install command that adds that variable to the bashrc if it's not yet added),
So that anywhere I want to add help target to a makefile, I can just add include $ROOT_PATH/tools/Makefile-helper.mk
here's the code:
Makefile-help.mk:

help:
	@FILE=Makefile ${ROOT_PATH}/tools/generate-makefile-help

generate-makefile-help:

#!/bin/bash

RULE_COLOR="$(tput setaf 6)"
SECTION_COLOR="$(tput setaf 3)"
VARIABLE_COLOR="$(tput setaf 2)"
VALUE_COLOR="$(tput setaf 1)"
CLEAR_STYLE="$(tput sgr0)"
TARGET_STYLED_HELP_NAME="${RULE_COLOR}TARGET${CLEAR_STYLE}"
ARGUMENTS_HELP_NAME="${VARIABLE_COLOR}ARGUMENT${CLEAR_STYLE}=${VALUE_COLOR}VALUE${CLEAR_STYLE}"

echo "Usage: make [$TARGET_STYLED_HELP_NAME [$TARGET_STYLED_HELP_NAME ...]] [$ARGUMENTS_HELP_NAME [$ARGUMENTS_HELP_NAME ...]]"
echo "${SECTION_COLOR}Targets:${CLEAR_STYLE}"
echo "    ${RULE_COLOR}help${CLEAR_STYLE}"
echo "        Get help for commands in this folder"
echo ""

TARGET_REGEX="^[a-zA-Z0-9%_\/%-]+:"
SECTION_REGEX="^##\s*@section\s*(.*)$"
DOCBLOCK_REGEX="^##\s*(.*)$"
PARAM_REGEX="@param\s+([a-zA-Z_]+)(=([^\s]+))?\s*(.*$)?"

COMMENT=""
PARAMS=""
PARAMS_DOC=""
cat $FILE | while read line
do
    # do something with $line here
    if [[ ! -z $line ]]
    then
        if [[ $line =~ $SECTION_REGEX ]]
        then
            SECTION_NAME=$(echo $line | sed -e "s/^##\s*@section\s*\(.*\)$/\1/g")
            echo "$SECTION_COLOR$SECTION_NAME$CLEAR_STYLE:"
        elif [[ $line =~ $TARGET_REGEX ]]
        then
            # if there is no comment for this target, we don't display it in the docs to keep private targets hidden
            if [[ ! -z $COMMENT ]]
            then
                TARGET=$(echo $line | sed -e "s/^\([a-zA-Z0-9%_\/%-]\+\):.*/\1/g")
                echo "    $RULE_COLOR$TARGET$CLEAR_STYLE $PARAMS"
                echo -e "$COMMENT"
                if [[ ! -z $PARAMS_DOC ]]
                then
                    echo "        Params:"
                    echo -e "$PARAMS_DOC"
                fi
            fi
            COMMENT=""
            PARAMS=""
            PARAMS_DOC=""
        elif [[ $line =~ $PARAM_REGEX ]]
        then
            PARAM=$(echo $line | sed -e "s/##\s*@param\s\+\([a-zA-Z_]\+\)\(=\([^[:space:]]\+\)\)\?\s*\(.*\)\?$/${VARIABLE_COLOR}\1${CLEAR_STYLE}=${VALUE_COLOR}\3${CLEAR_STYLE}/g")
            PARAM_DOC=$(echo $line | sed -e "s/##\s*@param\s\+\([a-zA-Z_]\+\)\(=\([^[:space:]]\+\)\)\?\s*\(.*\)\?$/- \1 (ex: \3) \4/g")
            PARAMS="${PARAMS}${PARAM} "
            PARAMS_DOC="${PARAMS_DOC}         ${PARAM_DOC}\n"
        elif [[ $line =~ $DOCBLOCK_REGEX ]]
        then
            # echo "doc : $line"
            # echo $line | sed -e "s/^##\s*\(.*\)$/\1/g"
            LINE_CLEANED=$(echo $line | sed -e "s/^##\s*\(.*\)$/\1/g")
            COMMENT="${COMMENT}        $LINE_CLEANED\n"
        fi
    fi
done

And here's an example Makefile:

# Line to add to any Supermood Makefile - generates the 'help' target from the Makefile comments
include ${SUPERMOOD_ROOT}/tools/Makefile-help.mk

## @section Installation

## Install the Supermood environment (adds $SUPERMOOD_ROOT to your shell)
install:
	@echo Setting SUPERMOOD_ROOT environment
	@./tools/bashrc-setup

## @section Supermood Release log generation

## Generates the production release log from git history.
## Post the output to the #releases slack channel
## @param FROM=1.234.0 Git reference from which we show the log
## @param TO=1.235.0 Git reference to which we show the log
releaselog:
	@./internal/releaselog/display-release-log $$FROM $$TO

## Generates the full release log (including [no-releaselog] commits) from git history.
## Post the output to your squad's channel or #tech-team
## @param FROM=1.234.0 Git reference from which we show the log
## @param TO=1.235.0 Git reference to which we show the log
full-releaselog:
	@./internal/releaselog/display-full-release-log $$FROM $$TO

# This is just hidden, probably a target we don't want people to call
sometarget: any other file
	echo "hello"

.PHONY: install releaselog full-releaselog

Which generates this:

This has been extremely useful to me. Thanks a lot, everyone! @hilnius I noticed that your solution is currently lacking support for targets defined/documented in included makefiles or if the makefile does not use the default name (i.e. make invoked with -f or with -C). A quick and dirty solution I came up with is to modify your bash script as follows:

#!/bin/bash
RULE_COLOR="$(tput setaf 6)"
SECTION_COLOR="$(tput setaf 3)"
VARIABLE_COLOR="$(tput setaf 2)"
VALUE_COLOR="$(tput setaf 1)"
CLEAR_STYLE="$(tput sgr0)"
TARGET_STYLED_HELP_NAME="${RULE_COLOR}TARGET${CLEAR_STYLE}"
ARGUMENTS_HELP_NAME="${VARIABLE_COLOR}ARGUMENT${CLEAR_STYLE}=${VALUE_COLOR}VALUE${CLEAR_STYLE}"

echo "Usage: make [$TARGET_STYLED_HELP_NAME [$TARGET_STYLED_HELP_NAME ...]] [$ARGUMENTS_HELP_NAME [$ARGUMENTS_HELP_NAME ...]]"
echo "${SECTION_COLOR}Targets:${CLEAR_STYLE}"
echo "    ${RULE_COLOR}help${CLEAR_STYLE}"
echo "        Get help for commands in this folder"
echo ""

TARGET_REGEX="^[a-zA-Z0-9%_\/%-]+:"
SECTION_REGEX="^##\s*@section\s*(.*)$"
DOCBLOCK_REGEX="^##\s*(.*)$"
PARAM_REGEX="@param\s+([a-zA-Z_]+)(=([^\s]+))?\s*(.*$)?"

COMMENT=""
PARAMS=""
PARAMS_DOC=""

for FILE in $MAKEFILES
do
cat $FILE | while read line
  do
    # do something with $line here
    if [[ ! -z $line ]]
    then
        if [[ $line =~ $SECTION_REGEX ]]
        then
            SECTION_NAME=$(echo $line | sed -e "s/^##\s*@section\s*\(.*\)$/\1/g")
            echo "$SECTION_COLOR$SECTION_NAME$CLEAR_STYLE:"
        elif [[ $line =~ $TARGET_REGEX ]]
        then
            # if there is no comment for this target, we don't display it in the docs to keep private targets hidden
            if [[ ! -z $COMMENT ]]
            then
                TARGET=$(echo $line | sed -e "s/^\([a-zA-Z0-9%_\/%-]\+\):.*/\1/g")
                echo "    $RULE_COLOR$TARGET$CLEAR_STYLE $PARAMS"
                echo -e "$COMMENT"
                if [[ ! -z $PARAMS_DOC ]]
                then
                    echo "        Params:"
                    echo -e "$PARAMS_DOC"
                fi
            fi
            COMMENT=""
            PARAMS=""
            PARAMS_DOC=""
        elif [[ $line =~ $PARAM_REGEX ]]
        then
            PARAM=$(echo $line | sed -e "s/##\s*@param\s\+\([a-zA-Z_]\+\)\(=\([^[:space:]]\+\)\)\?\s*\(.*\)\?$/${VARIABLE_COLOR}\1${CLEAR_STYLE}=${VALUE_COLOR}\3${CLEAR_STYLE}/g")
            PARAM_DOC=$(echo $line | sed -e "s/##\s*@param\s\+\([a-zA-Z_]\+\)\(=\([^[:space:]]\+\)\)\?\s*\(.*\)\?$/- \1 (example: \3) \4/g")
            PARAMS="${PARAMS}${PARAM} "
            PARAMS_DOC="${PARAMS_DOC}         ${PARAM_DOC}\n"
        elif [[ $line =~ $DOCBLOCK_REGEX ]]
        then
            # echo "doc : $line"
            # echo $line | sed -e "s/^##\s*\(.*\)$/\1/g"
            LINE_CLEANED=$(echo $line | sed -e "s/^##\s*\(.*\)$/\1/g")
            COMMENT="${COMMENT}        $LINE_CLEANED\n"
        fi
    fi
  done
done

It would then have to be invoked (after all includes) with:

.PHONY: help
help:
	@MAKEFILES="$(MAKEFILE_LIST)" $(mkfile_dir)generate-makefile-help.sh

I.e. if you save this snippet in a Makefile-help.mk as suggested by @hilnius, make sure to include it at the very and of your makefile (or at least after all other makefile includes) in order to catch documentation in all the relevant makefiles. This approach, however, does not work for conditional file inclusion.

@hilnius
To handle export VAR ?= something this works, i guess?

		-e "s|^\(export *\)\($(variable_regex)\)$(variable_assignment_regex)\($(value_regex)\)|$(global_variable_tag_start)\2$(global_variable_tag_end)$(value_tag_start)\3$(value_tag_end)|;" \

i also found improved: https://github.com/cert-manager/cert-manager/blob/7ce1f9cffb70eae4d3dd3572564a90f2553d3b52/make/help.mk

Inspired by deno task:

@echo "$$(tput setaf 2)Available rules:$$(tput sgr0)";sed -ne"/^## /{h;s/.*//;:d" -e"H;n;s/^## /---/;td" -e"s/:.*//;G;s/\\n## /===/;s/\\n//g;p;}" ${MAKEFILE_LIST}|awk -F === -v n=$$(tput cols) -v i=4 -v a="$$(tput setaf 6)" -v z="$$(tput sgr0)" '{printf"- %s%s%s\n",a,$$1,z;m=split($$2,w,"---");l=n-i;for(j=1;j<=m;j++){l-=length(w[j])+1;if(l<= 0){l=n-i-length(w[j])-1;}printf"%*s%s\n",-i," ",w[j];}}'