Python Running multiple bash scripts simultaneously

[member 428835]

Hi PF!

How can I run two bash scripts run1.sh run2.sh simultaneously (having two separate terminals, one for each run.sh, is fine but I'd like to execute the python script from a single terminal)? Won't the below execute run1.sh and then when it's finished execute run2.sh?

subprocess.call(['./run1.sh'])
subprocess.call(['./run2.sh'])

 

[PeterDonis]

Won't the below execute run1.sh and then when it's finished execute run2.sh?

Yes. However, there are other ways to run a subprocess in Python that don't wait for the process to complete. Look at the documentation for subprocess.Popen.

Note, however, that none of the subprocess module functions in Python will give the subprocess a terminal you can interact with like the normal shell terminal you are used to. (There are ways to do this in a limited fashion using Popen.communicate, but it's pretty complicated.) So if by "each one has a separate terminal", you mean that you need to interact with each one separately in a terminal window, you have to actually open two terminal windows and run each script in one of them. You can't do it by running a single Python script.

 

[jedishrfu]

You could also have your python script write a bash script of scripts to run in parallel using "&" ending characters and then run it.

#!/bin/bash
echo "start of launcher script"

run_script_a &
run_script_b &
run_script_c &

echo "end of launcher script"

admittedly you lose the control of tracking the three scripts but its overall a simpler solution than having your python run them without blocking.

 

[phyzguy]

I typically do it like this. This will run run1, and wait for it to finish before launching run2.

step1 = subprocess.Popen('./run1.sh',shell=True)
subprocess.Popen.wait(step1)
step2 =subprocess.Popen('./run2.sh',shell=True)
subprocess.Popen.wait(step2)

 

[jedishrfu]

I've had issues when scripts generate output and you are reading it back. Basically the scripts would appear to freeze. This occurred with an awk script that launched commands and scanned their output.

 

[PeterDonis]

I've had issues when scripts generate output and you are reading it back. Basically the scripts would appear to freeze.

The Python documentation on the subprocess module discusses a number of potential pitfalls when doing this and how to avoid them.

 

[jedishrfu]

Yes I know, I brought it up to make the OP be aware of what might happen when spawning processes that generate terminal output.