makefile thoughts
Andy Dougherty
doughera at lafayette.edu
Mon Jan 4 19:00:13 UTC 2010
On Mon, 4 Jan 2010, Nicholas Clark wrote:
> On Mon, Jan 04, 2010 at 09:55:50AM -0500, Andy Dougherty wrote:
[on the parrot-dev mailing list about multiple targets and parallel make]
> That would mean that this rule in perl 5 is wrong:
>
> lib/Config_git.pl git_version.h: $(MINIPERL_EXE) make_patchnum.pl
> $(MINIPERL) make_patchnum.pl
>
Yes, that rule is wrong.
[Long response not addressed to Nick but to p5p, which hasn't seen any of
the background.]
A rule of the form
target1 target2: ...
action that updates both target1 and target2
is equivalent to two rules
target1: ...
action
target2: ...
action
A parallel make could, in principle, run them both simultaneously.
This results in a race condition, since two separate instances of 'action'
could be simultaneously trying to update the target files.
You can verify this with perl 5 with GNU make by running
sh Configure -Dusedevel -des
make -j 2 perl 2>&1 | tee make.log
grep make_patchnum.pl make.log
You should find two entries for make_patchnum.pl.
If the writes are short and quick (as they are in this perl 5 example)
then a collision is highly unlikely. If the writes are longer and
slower (as they are in the parrot example that spawned this thread)
then a collision becomes more likely.
There are a variety of non-portable ways to tell make not to run these
in parallel. With Sun's dmake, you could write the rule as
target1 + target2: ...
action that updates both target1 and target2
where the '+' sign indicates that the multiple targets are built by a
single invocation of the rule. GNU make doesn't support that notation.
There are also various special targets, such as .WAIT, .NOTPARALLEL, and
.NO_PARALLEL, but the names and meanings vary among different versions of
make, so there's no simple portable invocation.
The workaround I proposed for parrot (assuming target1 is written first by
'action' and then target2) is to simply write
target1: [... target 1 dependencies . . . ]
action that updates both target1 and target2
target2: target1 [... target 1 dependencies . . . ]
This isn't strictly correct, since if you manually update target1, target2
won't get fixed at all. In the case in parrot, target1 has a big 'Do not
edit!!!' header, so I'm not too worried about it.
Another alternative is to rewrite 'action' to use lockfiles or other
race-condition-avoiding techniques.
A third alternative is to rewrite 'action' into two separate actions. For
perl 5, generating lib/Config_git.pl looks to be rather complicated.
However, I'd think that if you can assume lib/Config_git.pl has correctly
been built, then writing git_version.h based on it probably isn't too
hard. Perhaps someone who understands those two files could help split up
the make_patchnum.pl accordingly.
Meanwhile, perhaps I'll try to split up the target. (It'll probably
actually take me longer to remember how to do anything in git than it will
to actually make the actual patch!)
Disclaimer: I am in no way a "parallel make" expert. I just read the
fine manuals and experiment with variants of make debugging flags.
--
Andy Dougherty doughera at lafayette.edu
More information about the parrot-dev
mailing list