Bash script for moving data from one directory to another

[member 428835]

Hi PF!

I'm a new Linux user (please be patient :) ). I would like to read data stored as a .dat file from a different directory so I can reference it; do you know how?

Specifically, I want to replace a line from different C files (controlDict and U), located here ./system/controlDict and here ./0/U (lines 33 and 26 respectively).

I also want to access two different .dat files from a directory here /home/josh/Documents/NASA/PSI_DATA/Double_Drain/ICF1-9 where the file names are VEL.dat and VEL_t.dat. VEL.dat is an $n \times 1$ column vector and VEL_t is a single scalar number.

Through a TON of effort, below is what I have.

#    LOOP THROUGH ALL OF VEL
for (( j = 0; j < ${#VEL[@]}; j++ ));  # THIS SHOULD LOOP THROUGH ALL OF VEL, THAT IS, ${#VEL[@]} IS THE LENGTH OF VEL, RIGHT?

    do

    #    SET LINE NUMBERS
    lineU             = 33
    lineContDict = 26

    #    REPLACE lineU (line 33) in ./0/U WITH value uniform (0 $VEL(j) 0)
    sed -i "${lineU}s/.*/value uniform (0 ${VEL[j]} 0);/" ./0/U                      # THIS SHOULD REPLACE ALL OF LINE 33 WITH: value uniform (0 ${VEL[j]} 0); WHERE j IS THE jTH COMPONENT OF VEL

    #    UPDATE endTime (line 26) IN ./system/controlDict WITH VEL_t
    sed -i "${lineContDict}s/.*/endTime j*$VEL_t;/" ./system/controlDict # THIS SHOULD REPLACE LINE 26 WITH: endtime j*VEL_t; WHICH IS j MULTIPLIED BY VEL_t

done

I've tried googling this solution, but I really have no clue what I'm looking for since I'm so new at bash scripting. Honestly, just getting here was a miracle.

 

[phyzguy]

I used to use bash scripts to do these types of things, but they are so hard to edit and read that I no longer use them. I do all of these kinds of things in Python, which is much easier to read and understand. I recommend you drop the bash script approach and move to Python. I don't understand exactly what you are trying to do, but here is a simple Python program which would replace line 33 from one file with line 33 from a second file. Hopefully you can understand what the program is doing and make modifications. If not, come back and ask more questions.

filename1 = "/home/josh/Documents/NASA/PSI_DATA/Double_Drain/ICF1-9/VEL.dat"
filename2 = "./test.dat"

# Read filename1
file1 = open(filename1,'r')
lines1 = file1.readlines()
file1.close

# Read filename2
file2 = open(filename2,'r')
lines2 = file2.readlines()
file2.close

# Replace line 33 in filename2 with line 33 in filename1
lines2[33] = lines1[33]

# Write out the new filename2.  This will erase to old filename2
# and replace it with the new one.  If you don't want this, you could use
# a new name (filename3, for example)
newfile = open(filename2,'w')
for line in lines2:
        newfile.write(line)
newfile.close()

 

[PeterDonis]

I want to replace a line from different C files (controlDict and U), located here ./system/controlDict and here ./0/U (lines 33 and 26 respectively).

I also want to access two different .dat files from a directory here /home/josh/Documents/NASA/PSI_DATA/Double_Drain/ICF1-9 where the file names are VEL.dat and VEL_t.dat. VEL.dat is an n×1 n \times 1 column vector and VEL_t is a single scalar number.

Why do you want to do these things? What is the higher level task you are trying to accomplish?

 

[member 428835]

I recommend you drop the bash script approach and move to Python.

I don't use python but am happy to learn it. From the tutorials I've seen for this particular software, everyone seems to use bash scripting, so I thought I would do the same.

Why do you want to do these things? What is the higher level task you are trying to accomplish?

I don't understand exactly what you are trying to do

This should address both points: I am using a CFD (computational fluid dynamics) software called OpenFOAM. Rather than go into the directory each time and modify boundary conditions, I would like to have one script to do it for me. VEL.dat is a file that contains numbers, i.e. [1 -3 4 5], which are velocity values at a boundary. VEL_t.dat is a file that contains a single number i.e. 3. What I'm trying to do is change a line in the velocity boundary condition folder called U, located at ./0/U to its successive value every j*VEL_t time.

So following the example above, from 0 to 3 seconds the simulation imposes 1 mm/s velocity. Then 3 to 6 seconds the simulation imposes -3 mm/s velocity at that boundary. This process continues until all entries of VEL.dat are used.

So the actual script I run is this:

#!/bin/sh

rm -r postProcessing
rm -r processor*
rm 0/alpha.water
rm constant/dynamicMeshDict
cp -r ../mesh/constant/polyMesh constant
setFields
decomposePar

#    LOOP THROUGH ALL OF VEL
for (( i = 0; j < ${#VEL[@]}; j++ ));
  
    do

    #    SET LINE NUMBERS
    lineU        = 33
    lineContDict = 26

    #    REPLACE lineU (LINE 33) in 0/U WITH value uniform (0 $VEL(j) 0)
    sed -i "${lineU}s/.*/value uniform (0 ${VEL[j]} 0);/" ./0/U

    #    UPDATE endTime (LINE 26) IN system/controlDict WITH VEL_T
    sed -i "${lineContDict}s/.*/endTime j*$VEL_t;/" ./system/controlDict

    #    RUN SOLVER
    mpirun -np 16 interFoam -parallel # > log.interFoam & # COMMENT TO VIEW OUTPUT

done

 

[phyzguy]

Well, all of these things can be done in Python as well. I've used Python to control the running of astrophysics code, just like you are trying to do, where the Python program would modify the input deck, launch new runs, look at the output of the runs, make new modifications, launch new runs, etc. But it sounds like you don't want to go that way so I'll bow out.

 

[member 428835]

Well, all of these things can be done in Python as well. I've used Python to control the running of astrophysics code, just like you are trying to do, where the Python program would modify the input deck, launch new runs, look at the output of the runs, make new modifications, launch new runs, etc. But it sounds like you don't want to go that way so I'll bow out.

Hmmm this is not a bad suggestion though! I'm talking to a few other people about it, but this might be better than using bash.

Did you elect python because it's simpler?

 

[member 428835]

CASE CLOSED: verdict is I'm switching to python for scripting language, thanks to phyzguy! Might post something similar to this if I can't get this working, but I'll try solo first.

 

[phyzguy]

Did you elect python because it's simpler?

Yes! It's much simpler and easier to understand. As you found out, it is very hard to understand what lines like the following line are supposed to do:

sed -i "${lineU}s/.*/value uniform (0 ${VEL[j]} 0);/" ./0/U

 

[member 428835]

Yes! It's much simpler and easier to understand. As you found out, it is very hard to understand what lines like the following line are supposed to do:

sed -i "${lineU}s/.*/value uniform (0 ${VEL[j]} 0);/" ./0/U

Agreed. So now a simple question: python 2 or 3? Which do you recommend and why?

 

[phyzguy]

I switched to Python3 a year or so ago. It just took me a day or so. All new code seems to be written in Python3, so I would go that way. But either one will probably work for you.

 

[member 428835]

Thanks a lot! Be safe out there (coronavirus in case anyone references this years from now)

 

[PeterDonis]

I am using a CFD (computational fluid dynamics) software called OpenFOAM.

This is C++ software, I believe. There are also CFD solver packages in Python, based on the NumPy and SciPy libraries. You might find those worth looking into.

python 2 or 3?

Python 2 is at end of life. Any new Python code should use Python 3.

 

[member 428835]

This is C++ software, I believe. There are also CFD solver packages in Python, based on the NumPy and SciPy libraries. You might find those worth looking into.
Python 2 is at end of life. Any new Python code should use Python 3.

Thanks for this!

Final question (I hope): what IDE or code compiler do you recommend? Looking for anything good that's free.

 

[PeterDonis]

what IDE or code compiler do you recommend?

Python is an interpreted language, there is no compiler. (Technically the interpreter does something called "compiling" your source code to Python bytecode, but that happens automatically, you don't have to do anything.)

I don't use an IDE so I don't have any recommendations for one.

 

[lomidrevo]

Final question (I hope): what IDE or code compiler do you recommend? Looking for anything good that's fr

As already explained, Python is interpreted language.
Regarding IDEs, if you like to experiment, you might want to check quite popular Python platform called anaconda:
https://www.anaconda.com/distribution/
It comes preloaded with many useful libraries and environments, and simplified package management.
Personally I really like jupyter notebooks for rapid script development and testing.

 

[phyzguy]

I don't use an IDE either. I just use a text editor to write the Python code (emacs in my case, but any text editor will work). Your Unix distribution should run Python out of the box.

 

[bigfooted]

I am also using jupyter a lot lately, a very convenient python environment that runs in your browser. There is also pyfoam, a python module to work with openfoam, but that's where my knowledge ends. I don't know how useful it is.

 

[Nick-stg]

Regarding IDEs, if you like to experiment, you might want to check quite popular Python platform called anaconda:

I'll second the anaconda recommendation. I use Anaconda's Spyder IDE on my Windows box and I like it a lot. It is where I do most of my more creative work. On my Linux box I tend to keep things more minimalist so like @phyzguy I only use the text editor gedit.

 

[lomidrevo]

I use Spyder too, when I work on bigger projects and when maintaining the jupyter cells (keep them consistent) is getting harder. Spyder is also very helpful for debugging!

 

[Dr Transport]

Agreed. So now a simple question: python 2 or 3? Which do you recommend and why?

Python 3.x only. Version 2.x is unsupported and no longer maintained.