递归 `make` 和 `-j`
Recursive `make` and `-j`

原始链接: https://quuxplusone.github.io/blog/2026/09/30/recursive-make-j/

文章探讨了递归调用 `make` 时,GNU Make 3.81 至 4.3 如何处理并行执行。如果配方中直接运行 `make`,父进程的作业服务器(jobserver)可能不会传递给子进程,从而出现类似 `jobserver unavailable: using -j1` 的警告,导致并行能力丧失,甚至可能调用 `$PATH` 中另一个 `make` 可执行文件。 在配方前加上 `+`(例如 `+make -C subdir`),或使用惯用的 `$(MAKE)` 变量,可以解决这些问题,并确保子进程继承父进程的作业服务器设置。 显式指定 `make -j2` 或 `make -j4` 等选项会覆盖继承的作业数限制,并禁用或重置作业服务器模式;不同 GNU Make 版本产生的警告措辞可能不同。单独使用 `make -j` 会请求无限并行,但在未标记的递归配方中,其行为因版本而异,因此不可移植。总体而言,递归配方应使用 `$(MAKE)`,除非有意覆盖父进程的作业服务器配置,否则应避免指定 `-j`。

《The Hacker News》上的一篇讨论涉及递归式 `make` 和使用 `-j` 的并行构建。一位评论者指出,在 GNU Make 4.4 中,传统用于向子 `make` 传递作业服务器信息的 `+` 前缀或 `$(MAKE)` 约定已不再必要,因为 GNU Make 4.4 默认使用基于 FIFO 的作业服务器。GNU Make 的 `-j` 参数作用于顶层调用,由顶层调用统一管理和限制包括子进程在内的并行任务。 多位参与者援引了“递归式 Make 有害”这一观点,认为嵌套构建可能掩盖依赖关系,并使正确性难以保证。另一些人则坚决为递归式 make 辩护,尤其是对于基于 autoconf 的项目,只要整体构建经过精心设计,形成无环依赖图即可。他们建议明确划分项目边界、进行充分测试,并严格管理依赖关系;还有人指出,对于规模非常大的系统,现代构建工具或许具有更好的扩展能力。一处更正补充说明,Apple 并未提供 GNU Make;macOS 附带的是 BSD Make。
相关文章

原文

Today I looked into “recursive” invocations of make; that is, cases in which a Makefile recipe itself invokes make. The official documentation is quite good, but since I’m working with a range of different GNU Make versions, I looked at how the behavior has changed over time, from GNU Make 3.81 to 4.3.

Basically, the issue is that if you have a recipe in your Makefile like this:

and then you run make -j4 ser, recent versions of GNU Make will give a warning (jobserver unavailable: using -j1). And you’ll lose the parallelism you wanted, because the “jobserver” (basically, the -j option) isn’t forwarded to the child make. And, incidentally, it won’t necessarily use the same make binary that you invoked; it’ll run whatever make happens to be in your path.

You can fix all three of those issues by prefixing + to the recipe line, or (since that feels uncomfortably magical) replacing make with $(MAKE). More on this at the end of the post. Now that we’re forwarding the “jobserver” properly, I wondered, how exactly do the various -j options interact, when you specify one degree of parallelism on the command line and another in the recursive recipe?

Here’s the pair of Makefiles I used for my test:

detab -E -4 - >Makefile <<EOF
ser:
    make -C subdir
par2:
    make -j2 -C subdir
par4:
    make -j4 -C subdir
par:
    make -j -C subdir
serplus:
    +make -C subdir
par2plus:
    +make -j2 -C subdir
par4plus:
    +make -j4 -C subdir
parplus:
    +make -j -C subdir
sermake:
    $(MAKE) -C subdir
par2make:
    $(MAKE) -j2 -C subdir
par4make:
    $(MAKE) -j4 -C subdir
parmake:
    $(MAKE) -j -C subdir
EOF
mkdir subdir
detab -E -4 - >subdir/Makefile <<EOF
all: one two three four five
one:
    echo 'one' && sleep 1 && echo 'ONE'
two:
    echo 'two' && sleep 1 && echo 'TWO'
three:
    echo 'thr' && sleep 1 && echo 'THR'
four:
    echo 'fou' && sleep 1 && echo 'FOU'
five:
    echo 'fiv' && sleep 1 && echo 'FIV'
EOF

Then I tested running make ser, make -j2 ser, etc., on various versions of Make available to me.

Here’s GNU Make 3.81 (on OS X). The number in each cell indicates the number of recipes running in parallel. The four rows marked with a dagger † differ among the versions of Make I tested; those idioms are “non-portable” and IMHO should be avoided in practice.

make…   -j2 -j4 -j
ser† 1 2 4 ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ 2 4 ∞
parplus† ∞ 2 4 ∞
parmake† ∞ 2 4 ∞

In the table above, “b” indicates this warning:

$ make -j2 par2
make[1]: warning: -jN forced in submake: disabling jobserver mode.

GNU Make 4.0 (on Debian 8.11):

make…   -j2 -j4 -j
ser† 1 1a 1a ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ 1a 1a ∞
parplus† ∞ 2 4 ∞
parmake† ∞ 2 4 ∞

In the table above, “a” and “b” indicate these two warnings, respectively:

$ make -j2 ser
make[1]: warning: jobserver unavailable: using -j1.  Add '+' to parent make rule.

$ make -j2 par2
make[1]: warning: -jN forced in submake: disabling jobserver mode.

GNU Make 4.2.1 (on Rocky 8.10) and GNU Make 4.3 (on Red Hat 9 or Ubuntu 22.04) tweak the latter error message to include the value of N:

$ make -j2 par
make[1]: warning: -j0 forced in submake: resetting jobserver mode.
$ make -j2 par2
make[1]: warning: -j2 forced in submake: resetting jobserver mode.

and also change the behavior of rows par, parplus, and parmake:

make…   -j2 -j4 -j
ser† 1 1a 1a ∞
serplus 1 2 4 ∞
sermake 1 2 4 ∞
par2 2 2b 2b 2
par2plus 2 2b 2b 2
par2make 2 2b 2b 2
par4 4 4b 4b 4
par4plus 4 4b 4b 4
par4make 4 4b 4b 4
par† ∞ ∞b ∞b ∞
parplus† ∞ ∞b ∞b ∞
parmake† ∞ ∞b ∞b ∞

Warning “a,” which explicitly tells you to “Add '+' to parent make rule,” will indeed always vanish if you prefix the make-containing line with +, like this:

I find that solution as uncomfortably magical as prefix @ or prefix -. (Personally, I’d rather see “|| true” at the end of a recipe line than a magic “-” at the beginning.) A solution that’s less magic-looking (although equally magic under the hood, apparently) is to use $(MAKE), like this:

sermake:
    $(MAKE) -C subdir

Neither method silences warning “b,” about “resetting jobserver mode,” though.

What we really want is a row that would look like fantasy below. But at the moment, it seems we have to choose our preferred tradeoff from one of the quoted rows, instead.

make…   -j2 -j4 -j
fantasy nprocs 2 4 ∞
sermake 1 2 4 ∞
par4make 4 4b 4b 4
联系我们 contact @ memedata.com