Welcome to GNU Parallel's documentation!
parallel
- NAME
- SYNOPSIS
- DESCRIPTION
- OPTIONS
- EXAMPLES
- SPREADING BLOCKS OF DATA
- TIME POSTFIXES
- UNIT PREFIX
- QUOTING
- LIST RUNNING JOBS
- COMPLETE RUNNING JOBS BUT DO NOT START NEW JOBS
- ENVIRONMENT VARIABLES
- DEFAULT PROFILE (CONFIG FILE)
- PROFILE FILES
- EXIT STATUS
- DIFFERENCES BETWEEN GNU Parallel AND ALTERNATIVES
- BUGS
- REPORTING BUGS
- AUTHOR
- LICENSE
- DEPENDENCIES
- SEE ALSO
parallel_examples
- GNU PARALLEL EXAMPLES
- EXAMPLE: Working as xargs -n1. Argument appending
- EXAMPLE: Simple network scanner
- EXAMPLE: Reading arguments from command line
- EXAMPLE: Inserting multiple arguments
- EXAMPLE: Context replace
- EXAMPLE: Compute intensive jobs and substitution
- EXAMPLE: Substitution and redirection
- EXAMPLE: Composed commands
- EXAMPLE: Composed command with perl replacement string
- EXAMPLE: Composed command with multiple input sources
- EXAMPLE: Match parts of input source
- EXAMPLE: Replacement fields from CSV file with headers
- EXAMPLE: Calling Bash functions
- EXAMPLE: Function tester
- EXAMPLE: Identify few failing jobs
- EXAMPLE: Continously show the latest line of output
- EXAMPLE: Log rotate
- EXAMPLE: Removing file extension when processing files
- EXAMPLE: Replacing parts of file names
- EXAMPLE: Removing strings from the argument
- EXAMPLE: Download 24 images for each of the past 30 days
- EXAMPLE: Download world map from NASA
- EXAMPLE: Download Apollo-11 images from NASA using jq
- EXAMPLE: Download video playlist in parallel
- EXAMPLE: Prepend last modified date (ISO8601) to file name
- EXAMPLE: Save output in ISO8601 dirs
- EXAMPLE: Digital clock with "blinking" :
- EXAMPLE: Aggregating content of files
- EXAMPLE: Breadth first parallel web crawler/mirrorer
- EXAMPLE: Process files from a tar file while unpacking
- EXAMPLE: Rewriting a for-loop and a while-read-loop
- EXAMPLE: Rewriting nested for-loops
- EXAMPLE: Finding the lowest difference between files
- EXAMPLE: for-loops with column names
- EXAMPLE: All combinations in a list
- EXAMPLE: From a to b and b to c
- EXAMPLE: Count the differences between all files in a dir
- EXAMPLE: Speeding up fast jobs
- EXAMPLE: Using shell variables
- EXAMPLE: Group output lines
- EXAMPLE: Tag output lines
- EXAMPLE: Colorize output
- EXAMPLE: Keep order of output same as order of input
- EXAMPLE: Keep order, but make job 1 output fast
- EXAMPLE: Parallel grep
- EXAMPLE: Grepping n lines for m regular expressions.
- EXAMPLE: Using remote computers
- EXAMPLE: Transferring of files
- EXAMPLE: Advanced file transfer
- EXAMPLE: Distributing work to local and remote computers
- EXAMPLE: Running the same command on remote computers
- EXAMPLE: Running 'sudo' on remote computers
- EXAMPLE: Using remote computers behind NAT wall
- EXAMPLE: Use sshpass with ssh
- EXAMPLE: Use outrun instead of ssh
- EXAMPLE: Slurm cluster
- EXAMPLE: Parallelizing rsync
- EXAMPLE: Use multiple inputs in one command
- EXAMPLE: Use a table as input
- EXAMPLE: Output to database
- EXAMPLE: Output to CSV-file for R
- EXAMPLE: Use XML as input
- EXAMPLE: Run the same command 10 times
- EXAMPLE: Working as cat | sh. Resource inexpensive jobs and evaluation
- EXAMPLE: Call program with FASTA sequence
- EXAMPLE: Call program with interleaved FASTQ records
- EXAMPLE: Processing a big file using more CPUs
- EXAMPLE: Grouping input lines
- EXAMPLE: Running more than 250 jobs workaround
- EXAMPLE: Working as mutex and counting semaphore
- EXAMPLE: Mutex for a script
- EXAMPLE: Start editor with file names from stdin (standard input)
- EXAMPLE: Running sudo
- EXAMPLE: Run ping in parallel
- EXAMPLE: GNU Parallel as queue system/batch manager
- EXAMPLE: GNU Parallel as dir processor
- EXAMPLE: Locate the missing package
- AUTHOR
- LICENSE
- SEE ALSO
sem
env_parallel
parset
parallel_design
- Design of GNU Parallel
- One file program
- Choice of programming language
- Old Perl style
- Scalability up and down
- Exponentially back off
- Shell compatibility
- env_parallel
- parset (supported in sh, ash, dash, bash, zsh, ksh, mksh)
- Job slots
- Rsync protocol version
- Compression
- Wrapping
- Convenience options --nice --basefile --transfer --return --cleanup --tmux --group --compress --cat --fifo --workdir --tag --tagstring
- --shard
- Shell shock
- The remote system wrapper
- Transferring of variables and functions
- Base64 encoded bzip2
- Which shell to use
- Always running commands in a shell
- Quoting
- --pipepart vs. --pipe
- --block-size adjustment
- Automatic --block-size computation
- --jobs and --onall
- --shuf
- --csv
- Buffering on disk
- Disk full
- Memory usage
- Argument separators ::: :::: :::+ ::::+
- Perl replacement strings, {= =}, and --rpl
- Test suite
- Median run time
- Error messages and warnings
- Determine number of CPUs
- Computation of load
- Killing jobs
- SQL interface
- Logo
- Citation notice
- Ideas for new design
- Historical decisions
parallel_tutorial
- GNU Parallel Tutorial
- Prerequisites
- Input sources
- Building the command line
- No command means arguments are commands
- Replacement strings
- The 7 predefined replacement strings
- Changing the replacement strings
- Perl expression replacement string
- Positional replacement strings
- Positional perl expression replacement string
- Input from columns
- Header defined replacement strings
- More pre-defined replacement strings with --plus
- Dynamic replacement strings with --plus
- More than one argument
- Quoting
- Trimming space
- Respecting the shell
- Controlling the output
- Controlling the execution
- Remote execution
- Saving output to shell variables (advanced)
- Saving to an SQL base (advanced)
- --pipe
- Shebang
- Semaphore
- Informational
- Profiles
- Spread the word
parallel_alternatives
- NAME
- DIFFERENCES BETWEEN GNU Parallel AND ALTERNATIVES
- SUMMARY LEGEND
- DIFFERENCES BETWEEN xargs AND GNU Parallel
- DIFFERENCES BETWEEN find -exec AND GNU Parallel
- DIFFERENCES BETWEEN make -j AND GNU Parallel
- DIFFERENCES BETWEEN ppss AND GNU Parallel
- DIFFERENCES BETWEEN pexec AND GNU Parallel
- DIFFERENCES BETWEEN xjobs AND GNU Parallel
- DIFFERENCES BETWEEN prll AND GNU Parallel
- DIFFERENCES BETWEEN dxargs AND GNU Parallel
- DIFFERENCES BETWEEN mdm/middleman AND GNU Parallel
- DIFFERENCES BETWEEN xapply AND GNU Parallel
- DIFFERENCES BETWEEN AIX apply AND GNU Parallel
- DIFFERENCES BETWEEN paexec AND GNU Parallel
- DIFFERENCES BETWEEN map(sitaramc) AND GNU Parallel
- DIFFERENCES BETWEEN ladon AND GNU Parallel
- DIFFERENCES BETWEEN jobflow AND GNU Parallel
- DIFFERENCES BETWEEN gargs AND GNU Parallel
- DIFFERENCES BETWEEN orgalorg AND GNU Parallel
- DIFFERENCES BETWEEN Rust parallel(mmstick) AND GNU Parallel
- DIFFERENCES BETWEEN parallelion AND GNU Parallel
- DIFFERENCES BETWEEN Rush AND GNU Parallel
- DIFFERENCES BETWEEN ClusterSSH AND GNU Parallel
- DIFFERENCES BETWEEN coshell AND GNU Parallel
- DIFFERENCES BETWEEN spread AND GNU Parallel
- DIFFERENCES BETWEEN pyargs AND GNU Parallel
- DIFFERENCES BETWEEN concurrently AND GNU Parallel
- DIFFERENCES BETWEEN map(soveran) AND GNU Parallel
- DIFFERENCES BETWEEN loop AND GNU Parallel
- DIFFERENCES BETWEEN lorikeet AND GNU Parallel
- DIFFERENCES BETWEEN spp AND GNU Parallel
- DIFFERENCES BETWEEN paral AND GNU Parallel
- DIFFERENCES BETWEEN concurr AND GNU Parallel
- DIFFERENCES BETWEEN lesser-parallel AND GNU Parallel
- DIFFERENCES BETWEEN npm-parallel AND GNU Parallel
- DIFFERENCES BETWEEN machma AND GNU Parallel
- DIFFERENCES BETWEEN interlace AND GNU Parallel
- DIFFERENCES BETWEEN otonvm Parallel AND GNU Parallel
- DIFFERENCES BETWEEN k-bx par AND GNU Parallel
- DIFFERENCES BETWEEN parallelshell AND GNU Parallel
- DIFFERENCES BETWEEN shell-executor AND GNU Parallel
- DIFFERENCES BETWEEN non-GNU par AND GNU Parallel
- DIFFERENCES BETWEEN fd AND GNU Parallel
- DIFFERENCES BETWEEN lateral AND GNU Parallel
- DIFFERENCES BETWEEN with-this AND GNU Parallel
- DIFFERENCES BETWEEN Tollef's parallel (moreutils) AND GNU Parallel
- DIFFERENCES BETWEEN rargs AND GNU Parallel
- DIFFERENCES BETWEEN threader AND GNU Parallel
- DIFFERENCES BETWEEN runp AND GNU Parallel
- DIFFERENCES BETWEEN papply AND GNU Parallel
- DIFFERENCES BETWEEN async AND GNU Parallel
- DIFFERENCES BETWEEN pardi AND GNU Parallel
- DIFFERENCES BETWEEN bthread AND GNU Parallel
- DIFFERENCES BETWEEN simple_gpu_scheduler AND GNU Parallel
- DIFFERENCES BETWEEN parasweep AND GNU Parallel
- DIFFERENCES BETWEEN parallel-bash(2021) AND GNU Parallel
- DIFFERENCES BETWEEN parallel-bash(2024) AND GNU Parallel
- DIFFERENCES BETWEEN bash-concurrent AND GNU Parallel
- DIFFERENCES BETWEEN spawntool AND GNU Parallel
- DIFFERENCES BETWEEN go-pssh AND GNU Parallel
- DIFFERENCES BETWEEN go-parallel AND GNU Parallel
- DIFFERENCES BETWEEN p AND GNU Parallel
- DIFFERENCES BETWEEN senechal AND GNU Parallel
- DIFFERENCES BETWEEN async AND GNU Parallel
- DIFFERENCES BETWEEN tandem AND GNU Parallel
- DIFFERENCES BETWEEN rust-parallel(aaronriekenberg) AND GNU Parallel
- DIFFERENCES BETWEEN parallelium AND GNU Parallel
- DIFFERENCES BETWEEN forkrun AND GNU Parallel
- DIFFERENCES BETWEEN parallel-sh AND GNU Parallel
- DIFFERENCES BETWEEN bash-parallel AND GNU Parallel
- DIFFERENCES BETWEEN PaSH AND GNU Parallel
- DIFFERENCES BETWEEN korovkin-parallel AND GNU Parallel
- DIFFERENCES BETWEEN xe AND GNU Parallel
- DIFFERENCES BETWEEN sp AND GNU Parallel
- DIFFERENCES BETWEEN repeater AND GNU Parallel
- DIFFERENCES BETWEEN parallelize AND GNU Parallel
- Todo
- TESTING OTHER TOOLS
- MIX: Output mixes
- STDERRMERGE: Stderr is merged with stdout
- RAM: Output limited by RAM
- DISKFULL: Incomplete data if /tmp runs full
- CLEANUP: Leaving tmp files at unexpected death
- SPCCHAR: Dealing badly with special file names.
- COMPOSED: Composed commands do not work
- ONEREP: Only one replacement string allowed
- INPUTSIZE: Length of input should not be limited
- NUMWORDS: Speed depends on number of words
- 4GB: Output with a line > 4GB should be OK
- AUTHOR
- LICENSE
- DEPENDENCIES
- SEE ALSO
parcat
niceload