How to build an interface to my data?

Click For Summary
SUMMARY

The discussion centers on creating an interface for managing and visualizing simulation data in numerical relativity. The user, identified as "atoasttoall," seeks alternatives to SVN for handling input files that frequently branch but rarely merge. Suggestions include using Perl for processing ASCII data, highlighting its capabilities in data extraction and reporting. The user expresses a desire for existing projects that address similar challenges in data lineage and overview.

PREREQUISITES
  • Understanding of numerical relativity and simulation data management
  • Familiarity with version control systems, specifically SVN
  • Basic programming skills, particularly in Perl
  • Knowledge of data formats, especially ASCII
NEXT STEPS
  • Research existing projects for data management in numerical relativity
  • Learn Perl for data processing and report generation
  • Explore alternatives to SVN for handling branching data, such as Git or Mercurial
  • Investigate tools for visualizing simulation data and lineage tracking
USEFUL FOR

This discussion is beneficial for numerical relativists, simulation researchers, and developers looking to optimize data management and visualization in scientific computing.

atoasttoall
Messages
3
Reaction score
0
I've long been planning to create an interface which provides an overview of my data. I'm not sure how to get started so I'm looking for advice.

I'm a numerical relativist and I do a lot of simulations. I organize these by running in directories which I label by date. These directories contain the input files that setup the run and the output that results.

The code that we use is in an svn repository and that seems to work pretty well. It's the input files that are the problem. I don't think svn is the right approach for dealing with them. They tend to branch a lot, but rarely merge.

I'd love to have an interface that gave me an overview of the runs I've done and the lineage of the input files for each run. It's hard to imagine that I'm the first numerical physicist to have this problem so there must be projects out there that deal with this, but I don't know where to look. Any suggestions?

Thanks,
atoasttoall
 
Technology news on Phys.org
If you are a numerical relativist, you would certainly know how to programme.
You can take a look at the language Perl, or
PERL - Practical Extraction and Report Language
if your data are in ASCII format, Perl can do a lot for you.
 

Similar threads

  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 19 ·
Replies
19
Views
2K
  • · Replies 7 ·
Replies
7
Views
2K
  • · Replies 5 ·
Replies
5
Views
1K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 21 ·
Replies
21
Views
4K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 6 ·
Replies
6
Views
2K
Replies
4
Views
4K