Python Interesting article how to use Pandas with Excel sheets

Click For Summary
The discussion centers on using Pandas in Python to manage multi-tab spreadsheets, highlighting an article that outlines steps for reading, manipulating, and writing Excel data. The user finds the process intriguing but notes that it may involve complex steps where data integrity must be checked. A concern is raised about potential limitations on the number of rows that can be pivoted, referencing Excel's maximum column limit of 16,384. The user typically utilizes MATLAB for data manipulation but questions how to handle mixed data types in Python. Additionally, there is curiosity about the possibility of pivoting data across multiple sheets, suggesting a need for further exploration of Pandas' capabilities in handling 3D data structures.
scottdave
Science Advisor
Homework Helper
Insights Author
Messages
2,009
Reaction score
974
I came across this article about using Pandas in Python to read in a multi-tab spreadsheet to Python, and work with the data then write back to an Excel spreadsheet:
https://towardsdatascience.com/seve...w-i-use-python-where-excel-fails-62061f86ef9c
I have not yet tried it, but it looks pretty cool. The link was posted on a Slack channel for an online course I'm taking through edX/Georgia Tech. (Introduction to Computing for Data Analysis). It is part of their MicroMasters program in Analytics.
 
  • Like
Likes Asymptotic and QuantumQuest
Technology news on Phys.org
Yes, that's pretty cool. The steps are somewhat involved and you'd have to check to make sure something didn't get lost.

Also is there some limit as to how many rows can get pivoted in this way? I'm guessing the max cols of the spreadsheet program which is 16,384:

https://support.office.com/en-us/ar...d-limits-1672b34d-7043-467e-8e27-269d656771c3

Normally for the data we work with, we'd use MATLAB functions to transpose the matrix of data but I'm not sure how that would work in this case with mixed datatypes of strings and numbers.

I also wonder if you could pivot across sheets (ala 3D matrix of data) as it seems that its possible.
 
  • Like
Likes scottdave and QuantumQuest
Learn If you want to write code for Python Machine learning, AI Statistics/data analysis Scientific research Web application servers Some microcontrollers JavaScript/Node JS/TypeScript Web sites Web application servers C# Games (Unity) Consumer applications (Windows) Business applications C++ Games (Unreal Engine) Operating systems, device drivers Microcontrollers/embedded systems Consumer applications (Linux) Some more tips: Do not learn C++ (or any other dialect of C) as a...

Similar threads

  • · Replies 5 ·
Replies
5
Views
4K
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
10
Views
5K