Python How to Write Python Output into an Excel CSV File?

Click For Summary
The discussion revolves around generating a CSV file in Python from a set of random numbers associated with letters. The original code outputs a text file, but the user seeks to format it as a CSV for better compatibility with Excel. Key points include the importance of using the correct delimiter (commas vs. semicolons) based on regional settings, as Excel may interpret these differently. Users suggest using the pandas library for easier CSV and Excel file handling, emphasizing its efficiency in managing data. There are also discussions about potential issues with Excel's handling of long text lines and character encoding, which can lead to data loss. The conversation highlights the need for proper file opening methods in Excel to ensure data is displayed correctly, and it suggests using text-to-columns functionality for better data organization. Overall, the thread provides insights into generating and managing CSV files in Python while addressing common pitfalls when interfacing with Excel.
  • #31
FactChecker said:
I would be surprised if Python did not have similar methods.

Pandas, which is a Python library that has precisely such methods, has already been referred to several times in this thread.
 
  • Like
Likes FactChecker
Technology news on Phys.org
  • #32
PeterDonis said:
Pandas, which is a Python library that has precisely such methods, has already been referred to several times in this thread.
Oh, sorry! I stand corrected. I did not realize that it was a Python library. I thought that an entirely different language was being recommended.
 
  • #33
Speaking from extensive experience in exporting data into .csv files, I would not use a library or try to automate Excel. Versions of Excel change, but a .csv file (always openable by Excel) is plain text. You'll want to be careful about encoding--use at least UTF-8 and make it explicit. You'll want to check each cell's worth of data for commas, and if you surround each cell with ", you'll want to watch for and handle that in each data item also. It's not that hard to do--takes a bit of tinkering--and if you treat it like a plain text file and handle it yourself, it will continue to work no matter what version of libraries or Excel comes along in a few years.

I could give you C# code that does this in one instance (try the link here--it's a list of algae taxa):

http://diatom.ansp.org/taxaservice/ExcelExportNawqaTaxaWithFlags.ashx?taxonomy_id=21&diatom=0

The above file comes from a REST web service (ASP.NET) written in C#; I just looked at the code. It pulls data from a database, converts everything to text and stuffs it into a "StringBuilder" object (I imagine Python may have something similar), thus assembling the entire file in memory, then converts it to a String and writes the text to the output.

Once someone downloads the file, they can open it in Excel (whatever version) and save it as any version of Excel they want. They will not be able to edit the .csv file in Excel, but it should always open. You can look at its contents in detail using any text editor.
 
Last edited:
  • #34
harborsparrow said:
You'll want to check each cell's worth of data for commas, and if you surround each cell with ", you'll want to watch for and handle that in each data item also. It's not that hard to do--takes a bit of tinkering--and if you treat it like a plain text file and handle it yourself, it will continue to work no matter what version of libraries or Excel comes along in a few years.
If the cell might contain general text that may include commas and quotes, it gets messy.
 
  • #35
As noted, for comma-delimited quoted data cells, you'll have to do something about , and " in the actual data. There should be a way to "escape" those characters in Excel (as it is a database first and foremost). Once you know the escape sequence, it's a simple string substitution before writing data into a cell.

"cell data", "cell2 data", "cell3 data", "data with , is fine", "data with a quote "" requires that the quote be doubled", "another data item"

It's not that bad--just check each data item and escape any double quotes as "" first, THEN add it to the current line of text. Delimit as always. Each line starts and ends with " and "," goes between each data item.
 
Last edited:
  • Like
Likes FactChecker

Similar threads

  • · Replies 9 ·
Replies
9
Views
2K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 23 ·
Replies
23
Views
3K
  • · Replies 4 ·
Replies
4
Views
1K
  • · Replies 17 ·
Replies
17
Views
3K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 1 ·
Replies
1
Views
1K
Replies
6
Views
2K
Replies
8
Views
2K
  • · Replies 4 ·
Replies
4
Views
1K