Linux Ask!

Linux Ask! is a Q & A web site specific for Linux related questions. Questions are collected, answered and audited by experienced Linux users.

Dec 282013

Parallel the execution of a job that read from stdin


Assume you have a huge list of file, e.g.


You want to "feed every line" of this file to an external program that read from stdin, to parallel it, you can use the gnu parallel, e.g.

cat input.txt | parallel -j16 -N1 --pipe myprogram

 Leave a Reply



You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>