Experimental Physics and
Industrial Control System

Benjamin Franksen <[email protected]> · Thu, 28 Feb 2013 12:11:51 +0100

In our control system we have many scripts that generate multiple targets in 
one go. With sequential make this is now problem, but when using parallel make 
(make -j) we observed strange errors: target files were sometimes corrupt, or 
missing, or just empty.

This message explains the issue with a simple example and show hoe to fix the 
build rules so that these errors don't happen.

Suppose you have a script, say 'create_some_files.pl', that creates files 
named 'one, 'two, 'three'. Your Makefile says:

TARGETS=one two three

all: $(TARGETS)

$(TARGETS): create_some_files.pl /usr/share/dict/words
one two three: create_some_files.pl
	./create_some_files.pl

Now, with parallel make, this may not work reliably, because the parallel make 
will try to execute the script three times in parallel:

> make -j
./create_some_files.pl
./create_some_files.pl
./create_some_files.pl

Of course, your script may be written in such a way that executing it mroe 
than once in parallel does no harm. But this is typically not the case, as 
this requires extra effort. Furthermore, calling the script multiple times is 
redundant (one run already creates all the files) and thus inefficient, even 
if the script is re-entrant.

The generic solution is to split the make rule with an intermediate target:

$(TARGETS): files_created

files_created: create_some_files.pl
	./create_some_files.pl
	touch files_created

This means the multiple targets depend on one single file (named 
"files_created", here), so make knows that it has to create this file first 
(before the targets). The rule for the targets is empty, so nothing is 
executed. This indeed executes the script only once:

> make -j
./create_some_files.pl
touch files_created

Cheers
-- 
Ben Franksen
()  ascii ribbon campaign - against html e-mail 
/\  www.asciiribbon.org   - against proprietary attachments

Experimental Physics and Industrial Control System

Experimental Physics and
Industrial Control System