Choice of Pipelines for Data Analysis

Click For Summary
SUMMARY

The discussion centers on the comparison between using Python Pandas and TensorFlow (TF) for performing Linear and Multilinear Regression. It concludes that while TensorFlow can achieve similar outputs to traditional methods, the introduction of hidden layers in neural networks allows for more complex, non-linear fits that often yield better performance. The conversation also highlights the importance of activation functions and model structure in achieving convergence. Resources such as "Hands On ML with Scikit-Learn, TF, and Keras" and "The 100 Page ML Book" are recommended for further learning.

PREREQUISITES
  • Familiarity with Python Pandas for data manipulation
  • Understanding of TensorFlow (TF) and Keras for machine learning
  • Knowledge of Linear and Multilinear Regression concepts
  • Basic understanding of activation functions and neural network architecture
NEXT STEPS
  • Learn how to implement Linear Regression using TensorFlow and Keras
  • Explore the impact of different activation functions in neural networks
  • Study the Keras tutorial on the fuel efficiency dataset for practical application
  • Read "Hands On ML with Scikit-Learn, TF, and Keras" for advanced pipeline techniques
USEFUL FOR

This discussion is beneficial for data analysts, machine learning practitioners, and anyone interested in comparing traditional regression methods with neural network approaches using TensorFlow and Keras.

WWGD
Science Advisor
Homework Helper
Messages
7,777
Reaction score
13,013
TL;DR
What kind of rules of thumb are there to decide choice of pipeline?
Hi,
So say I have some data to process. I am trying, say, Linear/Multilinear Regression. I know how to do this within Python Pandas. I can learn how with Tensorflow (TF). Would TF produce the same output given the "right" choice of Activation Functions *? Or would it output a model that is somehow "More General"?

* I assume this is the only/main variable affecting this choice and not other variables such as choice of metrics, sessions, etc.
 
Physics news on Phys.org
Given the same model structure (choice of metrics, properly normalized data, loss function) I don't see why TF should not converge on the same coefficients as a traditional calculation. However with a neural network we introduce hidden layers that create a non-linear fit which in most cases will perform better.

Have you worked through the Keras tutorial on the fuel efficiency dataset?
 
  • Like
Likes   Reactions: WWGD
Does TF stand for TensorFlow or The f$%*? ;). Thanks for your answer. Will look up the link; thanks.
 
  • Haha
Likes   Reactions: pbuk
WWGD said:
Does TF stand for TensorFlow or The f$%*? ;).
I must admit to having used the words "why won't you converge you f$%*?" or similar on a number of occasions.
 
Last edited:
  • Like
Likes   Reactions: PhDeezNutz and WWGD

Similar threads

  • · Replies 1 ·
Replies
1
Views
1K
  • · Replies 13 ·
Replies
13
Views
4K
  • · Replies 24 ·
Replies
24
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 26 ·
Replies
26
Views
3K
  • · Replies 6 ·
Replies
6
Views
3K
  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 5 ·
Replies
5
Views
4K