How to import .CSV table in C++

[member 428835]

Hi PF!

Given a .csv file with the first row a name (string) and successive rows doubles, how can I read and manipulate this data in C++? Typically I just google these sorts of questions, but I'm seeing so many links. Do you have one that you recommend, or do you have a function you've written that does the job?

Thanks so much.

 

[Vanadium 50]

You can use C's scanf and it's variations. std::format is also something you could look up.

 

[berkeman]

Hi PF!

Given a .csv file with the first row a name (string) and successive rows doubles, how can I read and manipulate this data in C++? Typically I just google these sorts of questions, but I'm seeing so many links. Do you have one that you recommend, or do you have a function you've written that does the job?

Thanks so much.

Um, what I/O have you done in C++ programs so far? Reading in a CSV file is about as basic as you can get in programming. Please show links to your reading and learning of basic C++ I/O so far.

And have you at least done this in C?

 

[jedishrfu]

There’s a book resource the C++ Cookbook which covers this in chapter 4 reading comma separated files.

https://www.amazon.com/dp/0596007612/?tag=pfamazon01-20

 

[FactChecker]

CSV files can be surprisingly difficult because there can be commas within strings that do not separate data. If you are sure that your data does not have that, then you can try reading it and parsing it yourself. Otherwise, I would recommend finding a program that already has dealt with that and using it. It might require preprocessing your data with another program or tool.

 

[member 428835]

Um, what I/O have you done in C++ programs so far? Reading in a CSV file is about as basic as you can get in programming. Please show links to your reading and learning of basic C++ I/O so far.

And have you at least done this in C?

I have done zero I/O. Just started learning it last week. I have not done anything in C, but after googling lots (too many links to post) I managed to put together something that does the basics, which is reading the sample.csv file.

// READ .CSV DATA
#include <iostream>
#include <fstream>
using namespace std;
int main()
{
    ifstream datas; // A CLASS OF THE ifstream LIB ALLOWING FILES TO BE READ
    datas.open("sample.csv"); // OPEN THE CSV FILE 
    while (datas.good()) { // WHILE SOMETHING IS IN THE FILE KEEP READING IT
        string line; // DEFINE A STRING
        getline(datas, line, ','); // GRAB THE ELEMENT UNTIL THE TEXT SEES A COMMA
        cout << line << endl; // PRINT THE ELEMENT
    }
}

However, the .csv file has data categorized by columns. But, the above code reads row-wise. Is there a way to save each column of the CSV file as a vector of numbers, say doubles, creating a matrix of data?

And I'll check out the book, but I've kinda been given several to read, so right now it's a lot.

 

[Mayhem]

You can use C's scanf and it's variations. std::format is also something you could look up.

IMO it's better to use fgetc() or fgets() but don't know if it's a part of C++.

 

[Mark44]

IMO it's better to use fgetc() or fgets() but don't know if it's a part of C++.

You can do I/O in C++ in either of two ways: using stream-based features declared in the iostream header or using the C standard library features in the stdio.h header. fgetc() and fgets() are part of the stdio.h header.

 

[Mark44]

// READ .CSV DATA
#include <iostream>
#include <fstream>
using namespace std;                     

int main()
{
    ifstream datas; // A CLASS OF THE ifstream LIB ALLOWING FILES TO BE READ
    datas.open("sample.csv"); // OPEN THE CSV FILE
    while (datas.good()) { // WHILE SOMETHING IS IN THE FILE KEEP READING IT
        string line; // DEFINE A STRING
        getline(datas, line, ','); // GRAB THE ELEMENT UNTIL THE TEXT SEES A COMMA
        cout << line << endl; // PRINT THE ELEMENT
    }
}

A few comments on your code.
1. It's very unusual to have comments in all caps.
2. Many experts warn against "using namespace std:" as it brings in everything in this very large namespace. A better choice is to limit the identifiers that you import, such as by prefacing these identifiers with the namespace name ( std::cout and std::endl). Alternatively, instead of "using namespace std;" you can write "using std::cout;" and "using std::endl;" and so on.
3. Your variable datas is of type ifstream, so a better comment would be "input file stream".
4. Your comment // OPEN THE CSV FILE is not very helpful, as it restates what the code is doing. Because ifstream is an input file stream, calling open() opens this stream for reading.
5. The line string line; doesn't really need a comment. It should be obvious to nearly all readers that you are declaring a C++ style string object whose identifier is line.
6. The line cout << line << endl; is also obvious to most readers with at least a smattering of knowledge about C++. What it's doing is to insert the contents of the string variable + a new line character into the standard output stream, cout. What is less obvious to new C++ programmers is that the actual work is being performed by the stream insertion operator <<.

 

[jedishrfu]

In addition to @Mark44, you may need to become familiar with the string parsing functions to convert a string into an array of strings with the comma (CSV) or tab (TSV) or whatever character (eg "|") is chosen as the separator.

Here are some examples from Rosettacode. They may not be the best examples:

https://rosettacode.org/wiki/CSV_data_manipulation#C

https://rosettacode.org/wiki/CSV_data_manipulation#C.2B.2B

 

[pbuk]

Personally I'd do it in

import csv

columns = {}
# Read from a CSV file with column headings in the first line.
with open('./path/to/my/file.csv') as csvfile:
    file = csv.DictReader(csvfile)
    for name in file.fieldnames:
        columns[name] = []
    for row in file:
        for field in row:
            columns[field].append(float(row[field]))

for name in columns:
    print(name, columns[name], '\n')

 

[pbuk]

Personally I'd do it in

import csv

columns = {}
# Read from a CSV file with column headings in the first line.
with open('./path/to/my/file.csv') as csvfile:
    file = csv.DictReader(csvfile)
    for name in file.fieldnames:
        columns[name] = []
    for row in file:
        for field in row:
            columns[field].append(float(row[field]))

for name in columns:
    print(name, columns[name], '\n')

Python is very suited for dealing with things like this and I would need to have a very strong reason to want to do it in C++.

 

[Mayhem]

You can do I/O in C++ in either of two ways: using stream-based features declared in the iostream header or using the C standard library features in the stdio.h header. fgetc() and fgets() are part of the stdio.h header.

IIRC scanf can cause some overflow issues, but it's been a while since I reviewed this.

 

[Mark44]

IIRC scanf can cause some overflow issues, but it's been a while since I reviewed this.

Yes, scanf() and other C standard library routines can cause buffer overrun problems. For this reason Microsoft provided more secure versions in Visual Studio with an additional parameter that specifies the maximum size of the buffer that will receive the input characters. These MSFT-specific versions include scanf_s(), fscanf_s(), gets_s(), and others. In each of these, the appended _s indicates that these are secure versions. Visual Studio issues a compiler error if you use the deprecated versions scanf(), fscanf(), gets(), and so on.

Regarding @pbuk's comment that Python would be a better choice, I agree, but would also add that C# also includes functions that work with .CSV files.

 

[jtbell]

    while (datas.good()) { // WHILE SOMETHING IS IN THE FILE KEEP READING IT
        string line; // DEFINE A STRING
        getline(datas, line, ','); // GRAB THE ELEMENT UNTIL THE TEXT SEES A COMMA
        cout << line << endl; // PRINT THE ELEMENT

There is a very nice idiom for detecting the end of a file which can simplify and shorten your input code a bit. All C++ input operations (AFAIK) can be used as the condition in an if- or while-statement, giving "true" if the input operation succeeded, and "false" if it failed (because of end of file, or some input error).

string line;
while (getline (datas, line, ','))
{
    cout << line << endl;
}

Both C-style arrays and C++-style vectors can be used to construct matrices. I prefer vectors myself, in which a matrix is a "vector of vectors". Google "c++ vector of vectors" and you'll get examples.

How will your program "know" how many rows and columns your data comes in? Is it simply going to assume a fixed size such as 5 rows and 10 columns, or will it have to "size" the matrix automatically according to the number of lines of data and how many items are in each line?

However, the .csv file has data categorized by columns. But, the above code reads row-wise. Is there a way to save each column of the CSV file as a vector of numbers, say doubles, creating a matrix of data?

If the program has to "size" the matrix automatically, having rows correspond to vectors (with the matrix being a vector of rows) is much more natural than having rows correspond to columns (with the matrix being a vector of columns). If you want to end up with a vector of columns, I think the simplest way would be to read it initially into a vector of rows, then copy the data into a new matrix which is intended to be interpreted as a vector of columns, in effect transposing rows and columns.

To see how to make a vector "grow" so it accommodates your data, look up vector::push_back() (the push_back() member function of the standard vector class).

I'll let others comment on doing this with C-style arrays.