Editing a file with a script (using vi?)

  • Thread starter Thread starter fluidistic
  • Start date Start date
  • Tags Tags
    File
Click For Summary

Discussion Overview

The discussion revolves around editing a specific file (endgame.cpp) in the Stockfish chess engine using scripting techniques. Participants explore various methods to automate the commenting out of specific lines of code before compilation, focusing on tools like vi, sed, awk, and perl.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant describes their goal of automating the editing of endgame.cpp but lacks familiarity with vi or similar programs.
  • Another suggests using regular expressions in vi to search for lines starting with "add
  • A participant explains how to perform search and replace in vi using command mode, mentioning that using sed or awk might be more suitable for scripting.
  • It is proposed that a single sed command could accomplish the task of commenting out the lines in the script.
  • One participant clarifies that vi is a visual interface for the ex editor, which can be used in scripts, while noting that alternatives like sed and awk are different tools for similar tasks.
  • Another participant introduces the use of diff and patch for managing changes in files, suggesting it as an alternative method.
  • A participant expresses that they are not required to use vi and finds sed easier, but encountered errors with their attempts at using sed commands.
  • One participant provides a corrected sed command and an awk script as potential solutions for the editing task.
  • Another participant shares a perl command as a preferred method for commenting out the lines in the file.

Areas of Agreement / Disagreement

Participants do not reach a consensus on the best tool to use for the task, as multiple methods (vi, sed, awk, perl) are discussed with varying opinions on their suitability for noninteractive editing.

Contextual Notes

Some participants mention the complexity of the task due to the changing nature of the code in endgame.cpp, which may affect the lines to be modified. There are also references to potential errors in sed commands related to special characters.

Who May Find This Useful

This discussion may be useful for individuals interested in automating code editing tasks, particularly those working with open-source projects or using scripting for software development.

fluidistic
Gold Member
Messages
3,934
Reaction score
286
Hello guys!
There's an open source chess engine (Stockfish) that I really like and I always want the latest version. The project is so active that there's approximately a new version every day. I decided to make a script that auto downloads the code, auto compile Stockfish, auto move the executable to a folder I want, auto rename another file (for the settings of the engine) and auto remove the unnecessary files and folders. I have succeeded in doing this.
However I wish to include a new part in my script: I want to edit the code of Stockfish (a file more precisely, called endgame.cpp) prior to compiling it, but I don't know how to use "vi" nor a similar program to do so. In fact I don't even know programming.

Because the project is very active, they may modify endgame.cpp and so the lines I want to modify may not be "constant". I.e. maybe today I want to modify lines 25 to 32 but tomorrow this would be lines 37 to 44.

More precisely, I want to comment the lines
Code:
add<KPK>("KPK");
add<KNNK>("KNNK");
add<KBNK>("KBNK");
add<KRKP>("KRKP");
add<KRKB>("KRKB");
add<KRKN>("KRKN");
add<KQKP>("KQKP");
add<KQKR>("KQKR");

add<KNPK>("KNPK");
add<KNPKB>("KNPKB");
add<KRPKR>("KRPKR");
add<KRPKB>("KRPKB");
add<KBPKB>("KBPKB");
add<KBPKN>("KBPKN");
so that the new/modified code would look like:
Code:
//  add<KPK>("KPK");
//  add<KNNK>("KNNK");
//  add<KBNK>("KBNK");
//  add<KRKP>("KRKP");
//  add<KRKB>("KRKB");
//  add<KRKN>("KRKN");
//  add<KQKP>("KQKP");
//  add<KQKR>("KQKR");

//  add<KNPK>("KNPK");
//  add<KNPKB>("KNPKB");
//  add<KRPKR>("KRPKR");
//  add<KRPKB>("KRPKB");
//  add<KBPKB>("KBPKB");
//  add<KBPKN>("KBPKN");

I have done some google searches but I don't really know how to do it. I guess I'd have to type a command to "search" a line, comment it, "search" another line, comment it, etc. until the last line and then "save" the file.
Any help is appreciated, thank you!
 
Technology news on Phys.org
I'm not familiar with vi, but I would check to see if it supports regular expressions. If so, what you want it to do is to search for any line of code that starts with "add<K" and ends with ";" and replace it with the same string with "//" appended to the front. That's where I would start.
 
Do you want or need to use vi?
In vi you can do search and replace by going into command mode by typing : (so shift and the ":" key)
you can then type %s/search/replace
where search is for example add< and replace is \/\/ add<

note that the for-slash and backslash sign is special. They are used for instance to separate the search string from the replace string . To make them unspecial, just put a backslash in front of it. However, if you are making a script, you are probably better off doing this with a sed, awk or perl command in your script.
 
You should be able to do this with a single sed command in your script. Try a Google search on "sed global search and replace". You'll get dozens of hits.
 
vi is just a visual interface (hence the name) to a command line based editor called ex, which should also be on any unix system.

Any command preceded by a : in vi is actually running ex.

Running ex in a script, using input that is part of the script, is no problem - that's what it was written for.

Alternatives like sed or awk are not necessarily better or worse - just different, IMO.
 
Alternatively, you can use the unix tool "diff" to create a difference file and "patch" to merge the difference file into a new version.
 
Thanks a lot guys.
No, I am not forced at all to use vi, I thought about vi but now that I've read about sed, it looks like much easier. I have tried some commands like
Code:
sed s/add<KPK>("KPK");/ / <old >new
and some variants of that but I always got errors, for example due to the "(".
Anyway I am done with this task, not because I have have/not succeeded in doing it but because the changes I wanted to do in the code are likely not improving the strength of the program (noticeably at least). I've tested it against the normal non modified version and it scored 53 wins, 57 losses and 191 draws. This doesn't mean that it's weaker, what it means is that the change in elo is so small that I'd need around 50k games to determine which version is stronger. So in the end my modification is not worth it.
Actually the result I got translate into -4.6 elo for my version (I don't have the error bars). And the likelihood of superiority (i.e. the chance that my version is stronger than the original one) is 0.35.
 
fluidistic: vi and ex editors are generally better for interactive editing, whereas the sed editor and awk command are better for noninteractive editing. In your case, you want noninteractive editing. Even though you do not need it now, your sed command in post 7 was close to correct, and instead should be,

sed 's/add<KPK>("KPK");/ /' <old >new​

I probably would use the following, instead, if I were going to use sed. Here I used at-sign (@) as the delimiter, but you can use any character you prefer for the delimiter character. The input file is in01.

Code:
sed -e '\@add<KPK>("KPK");@s@^@// @' \
   -e '\@add<KNNK>("KNNK");@s@^@// @' in01

Here is another way to do it, using awk. Type the following code into a file named, e.g., myprog01.sh.

Code:
#!/bin/sh
# myprog01.sh - Bourne shell program to comment out certain records. The
#    given string can begin in any column.
# Created:  2014-03-09, nvn.
# Usage:  sh myprog01.sh infile outfile
awk '
   {line01=$0;}
   /add<K/ {
      if(index($0,"add<KPK>(\"KPK\");")>0)line01="// "$0;
      else if(index($0,"add<KNNK>(\"KNNK\");")>0)line01="// "$0;
      else if(index($0,"add<KBNK>(\"KBNK\");")>0)line01="// "$0;

      else if(index($0,"add<KNPK>(\"KNPK\");")>0)line01="// "$0;
      else if(index($0,"add<KNPKB>(\"KNPKB\");")>0)line01="// "$0;
      else if(index($0,"add<KRPKR>(\"KRPKR\");")>0)line01="// "$0;
      }
   {print(line01);}
   ' $1 > $2
exit

Next, if your input is in a file named in01, then issue the following command at the linux bash shell prompt. The output from this command will be written (overwritten, caution) to file out01.

sh myprog01.sh in01 out01​
 
Last edited:
Using diff and patch it is:

Code:
diff -u orig/endgame.cpp mod/endgame.cpp > endgame.patch
patch new/endgame.cpp < endgame.patch

In particular, patch will make sure the change succeeds completely without surprises.
If the new endgame.cpp does not match any more due to more structural changes, patch will give an informational error, meaning you will have to do some manual work.Using perl (which I favor over sed and awk) it is for instance:
Code:
perl -pi -e '
  s#(add<KPK>("KPK");)#// $1#;
  s#(add<KNNK>("KNNK");)#// $1#;
  ...' endgame.cpp
 

Similar threads

  • · Replies 20 ·
Replies
20
Views
5K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 18 ·
Replies
18
Views
2K
  • · Replies 3 ·
Replies
3
Views
7K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 19 ·
Replies
19
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 75 ·
3
Replies
75
Views
7K