KISS

Keep It Simple Stupid

xargs with multiple argument substitutions

| comments

One of the build steps of the iOS app I’m working on at the moment is compressing JavaScript files. Some time ago I set up compiling all the JS files into a big one, but then the app changed and required all the files being separate. Obviously, it’s a task for a Run Script build step in Xcode.

The most straightforward way to write the script is this:

1
2
find . -type f -name '*.js' | egrep -v 'lib/' | \
    xargs -I{} -n1 sh -c 'echo {}; java -jar ~/bin/js_compiler.jar --charset UTF-8 --jscomp_off internetExplorerChecks --js_output_file {}.1 {} && mv {}.1 {};'

The line finds all the .js files, except for those in lib/ directory, and for each one calls the command: echo {}; java -jar ~/bin/js_compiler.jar --charset UTF-8 --jscomp_off internetExplorerChecks --js_output_file {}.1 {} && mv {}.1 {};, replacing {} with the filename. It worked great, but for one thing: it didn’t want to replace the last pair of brackets, thus couldn’t rename the file.

I tried running different shell commands like echo app/component/AppComponent.js | xargs -I {} -n 1 -t -p sh -c "echo {} {} {} {}; echo {} {}; echo {}.1; {}" to figure out what was wrong. A good idea was consulting with man xargs:

1
2
3
4
-I replstr
             Execute utility for each input line, replacing one or more
             occurrences of replstr in up to replacements (or 5 if no -R flag
             is specified) arguments to utility with the entire line of input. …

Aha, the default limit is 5 replacements! No prob, but specifying -I {} -R -1 didn’t help. After another half an hour I contemplated on the rest of the -I description:

1
2
3
4
5
6
7
         The resulting arguments, after replacement is done, will not
         be allowed to grow beyond 255 bytes; this is implemented by
         concatenating as much of the argument containing replstr as
         possible, to the constructed arguments to utility, up to 255
         bytes.  The 255 byte limit does not apply to arguments to
         utility which do not contain replstr, and furthermore, no
         replacement will be done on utility itself.

I didn’t believe my command was growing to 255 chars, but it did! It was quite a subtle pitfall. So my workaround is the following command:

1
2
find . -type f -name '*.js' | egrep -v 'lib/' | \
    xargs -I {} -n 1 sh -c 'export f="{}"; echo $f; java -jar ~/bin/js_compiler.jar --charset UTF-8 --jscomp_off internetExplorerChecks --js_output_file "${f}1" "$f" && mv -f "${f}1" "${f}"'

Here I assign the filename to an environment variable of the subshell, and then use it. It works!

Comments