Question about input format file (for creating a net)

[ORF]

Hello

I am not sure about what should be the best format for a input text file. I think before start shooting in the dark, it's better to ask experienced people.

The (C++) program has to create a net. The nodes will be defined in a input text file.

For convenience, the net will be a bunch of std::vector (so each path inside the net, starting from the first node, will correspond to a single std::vector). I know a bunch of std::vector is not the best way of storing a net, but it's the best way for the function which will use the net.

The nodes may be connected up to 4 nodes.

I thought in three different input format, but I don't know what could be the best option, and if the input format can be optimized:

-JSON: the nodes can be defined one-by-one. In addition to the node information, the next nodes should be defined. The problem of defining the nodes individually is that you should create the net at runtime. The net is not defined in a visual/intuitive-way (that means the probability of writing a wrong input is high).

-XML: implementing hierarchy is relatively easy, but for such number of connections, the problem is similar to the previous option: the net is not defined in a visual/intuitive-way.

-CSV: as a primitive version of JSON format.

To sum up, I think JSON would be the best option, but a strict control section is needed. I don't know if there are other ways of defining a net/tree from a text file.

Thank you for your time.

Regards,
ORF

 

[newjerseyrunner]

CSV is not a primitive JSON, it's just comma separated values.

I would use JSON personally but that may be because I simply have the most experience with it. I use it all the time for nets, I have hundreds of neural networks whose state is saved in JSON files. You can create a net by adding a list of linked items in your object:

[
   "obj1" :{
      "links": []
   },
   "obj2" :{
      "links": [ "obj1" ]
   }
]

I use libjson to work with JSON in C++. It's fast and has an easy interface. I also wrote it.

 

[jedishrfu]

A net can be defined as structured data or as tabular data.

JSON and XML are pretty much two ways to store structured data in a text file format.

CSV format is good for tabular data.

I would choose CSV if I only needed to edit specific node contents but not its connections. I also would only use CSV if each node had no more than n connections with being 6 or less like north south east west up down directions.

Otherwise I’d go with JSON as it is less of a hassle to parse when reading in.

One advantage XML brings is that you can label each node with a name and ID that you can use to locate its connecting nodes in your file. XML parsing is sensitive to proper begin and tagging and may get when you make a mistake during an edit.

Property files are another way you could consider where you use property keys that identify the node and it’s attributes.

Node1.text=xxxxxxxxx
Node1.east=Node5

...

In each of these cases you don’t really want to store more than 10,000 lines of text as that becomes cumbersome for a text editor.

The last option would be an application specific sql database using a table to hold your node information and using sql to traverse your net. This could free you from potential size limitations that the other schemes impose for in memory storage.

The last option would be to use a serialization scheme like in Java and store node info in binary. Your program builds and maintains the net and you never look at the binary serialization. You also need to mark nodes as you save them be sure you don’t get caught in a loop.

 

[harborsparrow]

Personally, I haven't found an easy way to parse JSON input files, whereas there are generally builltin libraries to handle either of the other two options. If there is a good JSON input library, that it is fine.