Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Editing a file with a script (using vi?)

  1. Mar 7, 2014 #1


    User Avatar
    Gold Member

    Hello guys!
    There's an open source chess engine (Stockfish) that I really like and I always want the latest version. The project is so active that there's approximately a new version every day. I decided to make a script that auto downloads the code, auto compile Stockfish, auto move the executable to a folder I want, auto rename another file (for the settings of the engine) and auto remove the unnecessary files and folders. I have succeeded in doing this.
    However I wish to include a new part in my script: I want to edit the code of Stockfish (a file more precisely, called endgame.cpp) prior to compiling it, but I don't know how to use "vi" nor a similar program to do so. In fact I don't even know programming.

    Because the project is very active, they may modify endgame.cpp and so the lines I want to modify may not be "constant". I.e. maybe today I want to modify lines 25 to 32 but tomorrow this would be lines 37 to 44.

    More precisely, I want to comment the lines
    Code (Text):

    so that the new/modified code would look like:
    Code (Text):
    //  add<KPK>("KPK");
    //  add<KNNK>("KNNK");
    //  add<KBNK>("KBNK");
    //  add<KRKP>("KRKP");
    //  add<KRKB>("KRKB");
    //  add<KRKN>("KRKN");
    //  add<KQKP>("KQKP");
    //  add<KQKR>("KQKR");

    //  add<KNPK>("KNPK");
    //  add<KNPKB>("KNPKB");
    //  add<KRPKR>("KRPKR");
    //  add<KRPKB>("KRPKB");
    //  add<KBPKB>("KBPKB");
    //  add<KBPKN>("KBPKN");
    I have done some google searches but I don't really know how to do it. I guess I'd have to type a command to "search" a line, comment it, "search" another line, comment it, etc. until the last line and then "save" the file.
    Any help is appreciated, thank you!
  2. jcsd
  3. Mar 7, 2014 #2


    Staff: Mentor

    I'm not familiar with vi, but I would check to see if it supports regular expressions. If so, what you want it to do is to search for any line of code that starts with "add<K" and ends with ";" and replace it with the same string with "//" appended to the front. That's where I would start.
  4. Mar 7, 2014 #3
    Do you want or need to use vi?
    In vi you can do search and replace by going into command mode by typing : (so shift and the ":" key)
    you can then type %s/search/replace
    where search is for example add< and replace is \/\/ add<

    note that the for-slash and backslash sign is special. They are used for instance to separate the search string from the replace string . To make them unspecial, just put a backslash in front of it.

    However, if you are making a script, you are probably better off doing this with a sed, awk or perl command in your script.
  5. Mar 8, 2014 #4


    User Avatar

    Staff: Mentor

    You should be able to do this with a single sed command in your script. Try a Google search on "sed global search and replace". You'll get dozens of hits.
  6. Mar 8, 2014 #5


    User Avatar
    Science Advisor
    Homework Helper

    vi is just a visual interface (hence the name) to a command line based editor called ex, which should also be on any unix system.

    Any command preceded by a : in vi is actually running ex.

    Running ex in a script, using input that is part of the script, is no problem - that's what it was written for.

    Alternatives like sed or awk are not necessarily better or worse - just different, IMO.
  7. Mar 8, 2014 #6

    I like Serena

    User Avatar
    Homework Helper

    Alternatively, you can use the unix tool "diff" to create a difference file and "patch" to merge the difference file into a new version.
  8. Mar 8, 2014 #7


    User Avatar
    Gold Member

    Thanks a lot guys.
    No, I am not forced at all to use vi, I thought about vi but now that I've read about sed, it looks like much easier. I have tried some commands like
    Code (Text):
    sed s/add<KPK>("KPK");/ / <old >new
    and some variants of that but I always got errors, for example due to the "(".
    Anyway I am done with this task, not because I have have/not succeeded in doing it but because the changes I wanted to do in the code are likely not improving the strength of the program (noticeably at least). I've tested it against the normal non modified version and it scored 53 wins, 57 losses and 191 draws. This doesn't mean that it's weaker, what it means is that the change in elo is so small that I'd need around 50k games to determine which version is stronger. So in the end my modification is not worth it.
    Actually the result I got translate into -4.6 elo for my version (I don't have the error bars). And the likelihood of superiority (i.e. the chance that my version is stronger than the original one) is 0.35.
  9. Mar 9, 2014 #8


    User Avatar
    Science Advisor
    Homework Helper

    fluidistic: vi and ex editors are generally better for interactive editing, whereas the sed editor and awk command are better for noninteractive editing. In your case, you want noninteractive editing. Even though you do not need it now, your sed command in post 7 was close to correct, and instead should be,

    sed 's/add<KPK>("KPK");/ /' <old >new​

    I probably would use the following, instead, if I were going to use sed. Here I used at-sign (@) as the delimiter, but you can use any character you prefer for the delimiter character. The input file is in01.

    Code (Text):
    sed -e '\@add<KPK>("KPK");@s@^@// @' \
       -e '\@add<KNNK>("KNNK");@s@^@// @' in01
    Here is another way to do it, using awk. Type the following code into a file named, e.g., myprog01.sh.

    Code (Text):
    # myprog01.sh - Bourne shell program to comment out certain records. The
    #    given string can begin in any column.
    # Created:  2014-03-09, nvn.
    # Usage:  sh myprog01.sh infile outfile
    awk '
       /add<K/ {
          if(index($0,"add<KPK>(\"KPK\");")>0)line01="// "$0;
          else if(index($0,"add<KNNK>(\"KNNK\");")>0)line01="// "$0;
          else if(index($0,"add<KBNK>(\"KBNK\");")>0)line01="// "$0;

          else if(index($0,"add<KNPK>(\"KNPK\");")>0)line01="// "$0;
          else if(index($0,"add<KNPKB>(\"KNPKB\");")>0)line01="// "$0;
          else if(index($0,"add<KRPKR>(\"KRPKR\");")>0)line01="// "$0;
       ' $1 > $2
    Next, if your input is in a file named in01, then issue the following command at the linux bash shell prompt. The output from this command will be written (overwritten, caution) to file out01.

    sh myprog01.sh in01 out01​
    Last edited: Mar 9, 2014
  10. Mar 9, 2014 #9

    I like Serena

    User Avatar
    Homework Helper

    Using diff and patch it is:

    Code (Text):
    diff -u orig/endgame.cpp mod/endgame.cpp > endgame.patch
    patch new/endgame.cpp < endgame.patch
    In particular, patch will make sure the change succeeds completely without surprises.
    If the new endgame.cpp does not match any more due to more structural changes, patch will give an informational error, meaning you will have to do some manual work.

    Using perl (which I favor over sed and awk) it is for instance:
    Code (Text):
    perl -pi -e '
      s#(add<KPK>("KPK");)#// $1#;
      s#(add<KNNK>("KNNK");)#// $1#;
      ...' endgame.cpp
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook