Multilayer perceptron questions

  • Thread starter Thread starter SotirisD
  • Start date Start date
AI Thread Summary
In the discussion, the focus is on optimizing multilayer perceptrons in Matlab for image classification using adaptive learning with momentum backpropagation (traingdx). It is noted that lower initial learning rates yield better accuracy, while higher rates lead to a significant drop in model performance. This behavior can be attributed to the sensitivity of the model to learning rates, where excessive values may cause instability in training. Additionally, the impact of different output activation functions on model performance is explored. The user observes that certain configurations, such as {'tansig', 'tansig', 'purelin'} and {'tansig', 'tansig', 'tansig'}, perform well, while {'tansig', 'tansig', 'logsig'} fails, likely due to the logsig function zeroing out negative values. The discussion highlights the importance of both learning rate selection and the choice of activation functions, suggesting that experimentation may be necessary to find optimal configurations.
SotirisD
Messages
1
Reaction score
0
So I am experimenting with different configurations of multilayer perceptrons in Matlab and my training data are extracted from images which I want to classify.

-I am currently using adaptive learning with momentum backpropagation (traingdx) setting different initial learning rates.What I get is that for low values I have a pretty good results but when the initial rate gets bigger the accuracy of my model drops dramatically.How can this be explained?

-Another question I have is how different output activation functions can affect your model.Are there some heuristics for this or just trial and error? For example I get good results with {'tansig', 'tansig', 'purelin'}, {'tansig', 'tansig', 'tansig'} but {'tansig', 'tansig', 'logsig'} fails, I suspect it has to do with negative values getting zeroed by logsig.
 
There is a discussion of learning rates in Sutskever's guest post on Yisong Yue's blog: http://yyue.blogspot.sg/2015/01/a-brief-overview-of-deep-learning.html
 

Thread 'Learning Assembly and computer architecture for x86'

Dear Peeps I have posted a few questions about programing on this sectio of the PF forum. I want to ask you veterans how you folks learn program in assembly and about computer architecture for the x86 family. In addition to finish learning C, I am also reading the book From bits to Gates to C and Beyond. In the book, it uses the mini LC3 assembly language. I also have books on assembly programming and computer architecture. The few famous ones i have are Computer Organization and...
View full post »

Thread 'Learning data structures and algorithms in different programming languages'

I have a quick questions. I am going through a book on C programming on my own. Afterwards, I plan to go through something call data structures and algorithms on my own also in C. I also need to learn C++, Matlab and for personal interest Haskell. For the two topic of data structures and algorithms, I understand there are standard ones across all programming languages. After learning it through C, what would be the biggest issue when trying to implement the same data...
View full post »

Similar threads

Getting Started with Application Development
Replies
6
Views
2K
Resources for the rectangular segmentation of an image (ML)
Replies
2
Views
1K
Inductance of Multilayer Solenoid
Replies
2
Views
6K
MHB What is the difference between GET vs POST vs PUT? Mainly POST vs PUT?
Replies
1
Views
1K
Reliability of LLM for generating examples in mathematics
Replies
25
Views
163
C/C++ Questions about C++, pointers, OS and compiler memory allocation
Replies
17
Views
3K
Calculation of electrical mean free path in nanomaterials?
Replies
0
Views
3K
Microsoft Technical Interview question
Replies
27
Views
3K
Python Applying Accelerated Raymarching to Reduce Rendering Time
Replies
0
Views
1K
Signal classification and processing for a gesture sensor module
Replies
7
Views
1K

Hot Threads

Recent Insights

Back
Top