# Merging columns from two text files

1. May 25, 2016

### ChrisVer

Suppose I have two .txt files:
Code (Python):

>>text1.txt
X    Y    Z
x1  y1   z1
x2  y2   z2
...
xN  yN  zN

>>text2.txt
W   R
w1   r1
w2   r2
...
wN   rN

Is it possible to merge the collumns in a single txt file like:
Code (Python):

>>merged.txt
X    Y    Z   W   R
x1  y1   z1   w1  r1
x2  y2   z2   w2  r2
...
xN  yN  zN   wN  rN

?
Either with some c++ or python code, or with bash or vim.

2. May 25, 2016

### Guneykan Ozgul

Yes, the first but maybe inefficient solution that comes to my mind is as follows,
You read the first file and put the values into a 2D array(think of it as a matrix) and then you can read the second file and put the values into the corresponding index of the 2D array and then you can write a new file from this array.

3. May 25, 2016

### BvU

I hate excel, but for this kind of thing I would be inclined to use it.

4. May 25, 2016

### MarneMath

A simple way to do this in python is to use pandas.

Code (Text):

>>> import pandas as pd

>>> text1 = pd.read_csv("text1.txt", sep = "\t")

>>> text1

X   Y   Z

0  x1  y1  z1

1  x2  y2  z2
>>> text2 = pd.read_csv("text2.txt", sep = "\t")

>>> text2

W   R

0  w1  r1

1  w2  r2
>>> result = pd.concat([text1,text2],axis = 1)

>>> result

X   Y   Z   W   R

0  x1  y1  z1  w1  r1

1  x2  y2  z2  w2  r2

result.to_csv("save_me.txt",sep = "\t")

Also if you want to use bash then this works too:

Code (Text):

paste text1.txt text2.txt | awk '{print $1,$2,$3,$4,$5}' or if you want to write it out to another file. paste text1.txt text2.txt | awk '{print$1,$2,$3,$4,$5}' > result.txt

Last edited: May 25, 2016
5. May 25, 2016

### Staff: Mentor

Assuming that the two files are laid out as you show, with three columns of data in text1.txt and two columns of data in text2.txt, and both files have N lines of data, here is an algorithm:

Code (Text):
Do
Read x, y, z from text1.txt
Write x, y, z, w, r to merged.txt
While (text1.txt contains data AND text2.txt contains data)

6. May 25, 2016

### gsal

Did you say vim? As in one time shot with an editor? I don't know vim, but good editors like Notepad++ (on Windows), Nedit (on Linux) and others will allow you to select the two columns of the one file as a rectangular selection; then, you simply paste on the other file.

7. May 26, 2016

### ChrisVer

I said vim because I know that vim allows a visual block selection (I guess it's the same as the rectangular selection you mentioned)...
So I was guessing that if it's able to select blocks, it should also be able to paste them in some particular position (like an appending of collumn).

8. May 26, 2016

### Staff: Mentor

bash means Linux. There are a lot of excellent text manipulation utilities in Linux. My point is do not write code when a command does it.

Try paste. Example shell script - you can simply run the one line "paste .." command on the command line.
Code (Text):

paste test1.txt text2.txt > newfile.txt

IMO writing code for this is counterproductive.

9. May 27, 2016

### Pepper Mint

Parallel processing library of C# I really love works like a charm. You should give it a try!
PHP:

private static void TextFileMerge(string fPath1, string fPath2, string fOutPath)
{
try {
if (fileContent1.Length != fileContent2.Length)
{
throw new ArgumentException("Lengths of two files are different");
}
string[] fileMerge = new string[fileContent1.Length];
using (StreamWriter sw = new StreamWriter(fOutPath))
{
Parallel.For(0, fileContent1.Length, x =>
{
fileContent1[x] += " " + fileContent2[x];

});
foreach (string s in fileContent1)
{
sw.WriteLine(s);
}
}
}
catch (Exception)
{
//handle
}
}

10. May 29, 2016

### harborsparrow

I would import to Excel, then copy and paste. Might require only minutes.

11. May 29, 2016

### rbelli1

Is this something you need to automate or just an occasional need? If it is an occasional need find a text editor that does rectangular select. You can just copy and paste the data. For Windows I recommend Notepad++.

BoB

12. May 30, 2016

### ChrisVer

1. needs excel and too much manual work (txt->excel->txt).
2. excell is not able to accept columns. I think it would put everything into a single cell.

yes... I want the program to do the calculations and stuff, the result is saved into text files in a form that can be immediately be compiled into a latex table.
Because the code I am running has problems with closing files I get a system error of too many opened files. In order to overcome this problem I run the code several times with less file inputs. SO, the results are saved into different txt files which I want to merge to compile.

Last edited: May 30, 2016
13. May 30, 2016

### Staff: Mentor

When you are finished with a file, are you closing it? Since you started this thread asking about C++ and python, my answer is in that context. If you use either of those languages (and others besides those two), good housekeeping dictates that you close a file once you're done with it.

14. May 30, 2016

### ChrisVer

No, there was generally an issue with how the code handled the open files. I know they solved the problem in the new version of their FW but my analysis is based on the old one [when I tried to update funny things happened].

15. May 30, 2016

### Staff: Mentor

Which code are you talking about?
What is FW?