# Shell Scripting or AWK?

by mar2194
Tags: awk, geophysics, scripting, shell-script, unix
 P: 10 Hey everyone, I'm currently running a program which accepts an input file. The input file has various parameters, 3 of which I need to change, run again, etc. The range of those 3 parameters are 15 values for one, 12 values for another and 10 for another, so I have 15*12*10 (=1800) possible input files I need to make for the program. The output filename will need to match the parameters for that run as well. Obviously creating 3375 input files by hand would be impractical, not only because it would be time consuming but the risk of human error would be great. I need help with the following: 1. Varying the parameters of my input file. I know this can be done with AWK, or perhaps using shell script. The input file is plain text (ascii) 2. recursively running the script each time a parameter is changed. Suppose my input file looks like: #input file mockup #Value for A 1 #Value for B 2 #Value for C 4 #Value for D 12 Suppose the three parameters I want to change are A, B and D. Suppose each value of A is spaced 5 units apart, and I want to run over 3 values. My possible values of A would then be {1, 6, 11} My possible values for B are spaced 3 units apart and there are 4 values {2, 5, 8, 11} and my Values for D are spaced 1 apart and there are two of them {12, 13} Allowing for repeated values, the sets I would want process are: {1, 2, 12} {1, 5, 12} {1, 8, 12} {1, 11, 12} {1, 2, 13} {1, 5, 13} {1, 8, 13} {1, 11, 13} {6, 2, 12} {6, 5, 12} {6, 8, 12} {6, 11, 12} {6, 2, 13} {6, 5, 13} {6, 8, 13} {6, 11, 13} {11, 2, 12} {11, 5, 12} {11, 8, 12} {11, 11, 12} {11, 2, 13} {11, 5, 13} {11, 8, 13} {11, 11, 13} where each place in the brackets corresponds to the values for {A, B, D} So my two problems are essentially: 1 Make a wrapper to run the program recursively with the changed input file 2 Change the three variables so I get every combination and pass the corresponding input file to the program. here's what I think it should look like in pseudo-code (I know this is wrong need some help): A = 1 B = 2 D = 12 for B<11, for A<11, for D<13, B+3, A+5, D++, program input file #input file mockup #Value for A Some Expression for {A} #Value for B Some Expression for {B} #Value for C 4 #Value for D Some Expression for {D} outputdirectory: mkdir Run$A$$B$$D$ or something like this.... NOTE: I've posted this question on a different site to see if anyone else could answer it there. I am not trolling or crowd-sourcing; looking for genuine help. Also, if anyone's interested, I'm running a co-seismic displacement model for which I need to vary the depth, length and displacement each run through. I'm assuming linear slip and uniform viscosity throughout the medium. The program is fine and ready to go I just need to figure out a way to alter my input file with variable parameters.
 P: 749 Hi, you don't need to use logic. you can simply list the numbers you want to loop over. for example, in BASH (a shell) you might have something like for A in 1 2 3; do for B in 6,8,10; do for C in 4,7,11; do # write the input file to your binary exec echo $A$B $C > file # run the exec ./exec done done done P: 10 Shell Scripting or AWK?  Quote by billiards Hi, you don't need to use logic. you can simply list the numbers you want to loop over. for example, in BASH (a shell) you might have something like for A in 1 2 3; do for B in 6,8,10; do for C in 4,7,11; do # write the input file to your binary exec echo$A $B$C > file # run the exec ./exec done done done
 Sci Advisor PF Gold P: 1,386 There is a lot of confusion here. Please show us EXACTLY what one "run" with a single set of parameters look like. This would be the awful case of you typing the command by hand: We think is it like: some_program_name 11 3 9 where some_program_name is some kind of compiled program. And 11 3 9 are parameters you feed the program. ( I made up the numbers so that does not confuse you) Please show us if you want useful help.
 P: 10 Here is the input file (run1.input): # grid size (sx1, sx2, sx3) 256 256 256 # sampling size, smoothing & nyquist (dx1,dx2,dx3,beta,nq) 0.5 0.5 0.5 0.5 2 # origin position & rotation 0 0 0 # geographic origin (longitude and latitude), UTM zone and real length unit (usually m or km) # displacements and stress are converted to lon/lat geographic coordinates # unit corresponds to a scaling from dx1,dx2,dx3 to real unit # use unit = 1 if dimensions are described in units of m # use unit = 1e3 if dimensions are described in units of km 120 22 51 1e3 # observation depth (displacements and stress) (stress is only exported in GRD) 0 5 # output directory (all output files are stored in the following directory) output1 # the elastic parameters (Lambda and mu) are the Lame parameter and the shear modulus # gamma=(1-nu)*rho*g/mu is a wavelength relevant to buoyancy. # elastic parameters and gamma = (1-nu) rho g / mu = 8.33e-7 /m = 8.33e-4 /km 4.26E+04 2.6E+04 8.48e-4 # integration time, time step and scaling 0 -1 1 # number of viscous observation slice 0 # number of observation points 0 # number of Coulomb patches 0 # number of prestress interfaces 0 # number of linear viscous interfaces 0 # number of power-law viscous interfaces 0 # number of friction faults 0 # number of interseismic loading strike-slip and opening 0 0 # number of coseismic events (when slip distribution is prescribed) 1 # number of shear dislocations (strike-slip and dip-slip faults) 1 # no slip xs ys zs length width strike dip rake 1 2 0 0 40 20 10 0 60 0 # number of tensile cracks 0 # number of dilatation sources 0 # number of surface traction 0 Here is the shell-script I currently use to run over that input file using the program(run1.sh): #!/bin/sh WDIR=./output1 if [ ! -e $WDIR ]; then echo adding directory$WDIR mkdir $WDIR fi OMP_NUM_THREADS=4 relax$* < run1.input | tee $WDIR/in.param As you can see, it copies the output of the program to an in.param file in the directory output1 after running the program relax on 4 threads using run1.input I need to change 3 parameters from my input file and create a new directory for every run of new variable combination.  P: 166 #!/bin/sh WDIR=./output1 if [ ! -d$WDIR ]; then if [ -e $WDIR ]; then echo${WDIR} exists and is not a directory. exit fi echo adding directory $WDIR mkdir$WDIR fi for I in 1 6 11 do for J in 12 13 do for K in 2 5 8 11 do # OMP_NUM_THREADS=4 relax ${I}${K} ${J} < run1.input | tee$WDIR/in.param echo ${I}${K} \${J} done done done Will do what you want with your example. I kept the actual relax bit commented out and just have it echoing instead. If you want to change the loop arguments, it should be obvious where to look. Also changed the -e to a -d, so it will exit out if 'output1' exists but is not a directory.