How to Efficiently Merge ROOT Files in Multiple Directories?

  • Thread starter Thread starter ChrisVer
  • Start date Start date
  • Tags Tags
    Root
Click For Summary
SUMMARY

The optimal method for merging multiple ROOT files across 40 directories involves using a shell script with a for loop to automate the process. The user can utilize the "hadd" command to combine ROOT files efficiently, avoiding the tedious manual entry of commands for each directory. The final solution includes a script that iterates through the directories and merges the files into a single output file per directory, ensuring both histograms and trees are included. This approach significantly streamlines the merging process and enhances productivity.

PREREQUISITES
  • Familiarity with ROOT framework and .root file structure
  • Basic knowledge of shell scripting and command line operations
  • Understanding of the "hadd" command for merging ROOT files
  • Experience with using loops in shell scripts
NEXT STEPS
  • Learn advanced shell scripting techniques for file manipulation
  • Explore the ROOT framework documentation for additional merging options
  • Investigate the use of TChain for merging different types of ROOT objects
  • Practice writing and executing shell scripts in a Linux environment
USEFUL FOR

Data scientists, physicists, and researchers working with ROOT files who need to efficiently merge large datasets across multiple directories.

ChrisVer
Science Advisor
Messages
3,372
Reaction score
465
Suppose I have a collection of 40 directories, each containing 1, 2 ,3 ,4 or 5 .root files with histograms and trees.
What is an optimal way to merge those .root files so I would end up with 40 directories and 40 root files with the histos added together?
One way I thought of was to start typing for the 40 different directories an "hadd" command in the terminal.

"hadd sum1.root dir1/*.root"
"hadd sum2.root dir2/*.root"
...
"hadd sum40.root dir40/*.root"


However I find this kind of tiring and stupid. Do you have any better solution?
My problem with a macro is that I am not sure if it understands the *.root notation (so I cannot think of iterating).
Finally I am not sure if a TChain can, apart from Trees in different root files, merge TH1s (histograms) too.

THanks :)
 
Technology news on Phys.org
You could write a shell script that uses iteration to generate a text file containing a single line of the form:

hadd sum1.root dir1/*.root & ; hadd sum1.root dir2/*.root & ; hadd sum3.root dir1/*.root & ; ... ; hadd sum1.root dir40/*.root​

then just copy the line to the command prompt and execute it
 
  • Like
Likes   Reactions: ChrisVer
andrewkirk said:
You could write a shell script that uses iteration to generate a text file containing a single line of the form:

hadd sum1.root dir1/*.root & ; hadd sum1.root dir2/*.root & ; hadd sum3.root dir1/*.root & ; ... ; hadd sum1.root dir40/*.root​

then just copy the line to the command prompt and execute it

Thanks for the reply...
It helped me to try to do it manually... after the 3rd try, I figured out what "for loop" I had to use in the shell script...That's what I had to do:
Bash:
i=0
for folder in directory_path_of_folders/*; do
    hadd -f target_directory/sumfile$((i++)).root $folder/*.root
done
 

Similar threads

  • · Replies 9 ·
Replies
9
Views
3K
  • · Replies 1 ·
Replies
1
Views
5K
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
11K
  • · Replies 13 ·
Replies
13
Views
4K