makefile thoughts

Andy Dougherty doughera at lafayette.edu
Mon Jan 4 19:00:13 UTC 2010


On Mon, 4 Jan 2010, Nicholas Clark wrote:

> On Mon, Jan 04, 2010 at 09:55:50AM -0500, Andy Dougherty wrote:

[on the parrot-dev mailing list about multiple targets and parallel make]

> That would mean that this rule in perl 5 is wrong:
> 
> lib/Config_git.pl git_version.h: $(MINIPERL_EXE) make_patchnum.pl
> 	$(MINIPERL) make_patchnum.pl
> 

Yes, that rule is wrong.

[Long response not addressed to Nick but to p5p, which hasn't seen any of 
the background.]

A rule of the form

    target1 target2: ...
	action that updates both target1 and target2

is equivalent to two rules

    target1: ...
	action

    target2: ...
	action

A parallel make could, in principle, run them both simultaneously.
This results in a race condition, since two separate instances of 'action'
could be simultaneously trying to update the target files.

You can verify this with perl 5 with GNU make by running

	sh Configure -Dusedevel -des
	make -j 2 perl 2>&1 | tee make.log
	grep make_patchnum.pl make.log

You should find two entries for make_patchnum.pl.

If the writes are short and quick (as they are in this perl 5 example)
then a collision is highly unlikely.  If the writes are longer and
slower (as they are in the parrot example that spawned this thread)
then a collision becomes more likely.

There are a variety of non-portable ways to tell make not to run these
in parallel.  With Sun's dmake, you could write the rule as

    target1 + target2: ...
	action that updates both target1 and target2

where the '+' sign indicates that the multiple targets are built by a
single invocation of the rule.  GNU make doesn't support that notation.

There are also various special targets, such as .WAIT, .NOTPARALLEL, and 
.NO_PARALLEL, but the names and meanings vary among different versions of 
make, so there's no simple portable invocation.

The workaround I proposed for parrot (assuming target1 is written first by 
'action' and then target2) is to simply write

    target1:  [... target 1 dependencies . . . ]
	action that updates both target1 and target2
    
    target2: target1 [... target 1 dependencies . . . ]

This isn't strictly correct, since if you manually update target1, target2 
won't get fixed at all.  In the case in parrot, target1 has a big 'Do not 
edit!!!' header, so I'm not too worried about it.

Another alternative is to rewrite 'action' to use lockfiles or other 
race-condition-avoiding techniques.

A third alternative is to rewrite 'action' into two separate actions. For 
perl 5, generating lib/Config_git.pl looks to be rather complicated. 
However, I'd think that if you can assume lib/Config_git.pl has correctly 
been built, then writing git_version.h based on it probably isn't too 
hard. Perhaps someone who understands those two files could help split up 
the make_patchnum.pl accordingly.

Meanwhile, perhaps I'll try to split up the target.  (It'll probably 
actually take me longer to remember how to do anything in git than it will 
to actually make the actual patch!)

Disclaimer:  I am in no way a "parallel make" expert.  I just read the 
fine manuals and experiment with variants of make debugging flags.

-- 
    Andy Dougherty		doughera at lafayette.edu


More information about the parrot-dev mailing list