What is meant when $\sigma$ is said to be discriminatory?

  • Context: Graduate 
  • Thread starter Thread starter jamesb1
  • Start date Start date
  • Tags Tags
    Measure theory
Click For Summary
SUMMARY

The discussion centers on the interpretation of the function $\sigma$ in the context of neural networks, specifically regarding its role as a discriminatory function. The user references the paper by Cybenko, which discusses the approximating power of neural networks, and expresses confusion over the relationship between the Lebesgue integral being zero and the measure $\mu$ also being zero, leading to the conclusion that $\sigma$ must be non-zero. The user seeks clarification on how $\sigma$ correlates with these definitions, particularly in relation to its potential characterization as a sigmoidal function.

PREREQUISITES
  • Understanding of Borel measures and their properties
  • Familiarity with Lebesgue integration and its implications
  • Knowledge of neural networks and their approximating capabilities
  • Basic concepts of sigmoidal functions and their applications in machine learning
NEXT STEPS
  • Research the properties of Borel measures in the context of functional analysis
  • Study the implications of the Lebesgue integral in measure theory
  • Explore the role of sigmoidal functions in neural network architectures
  • Examine the paper by Cybenko on the approximating power of neural networks for deeper insights
USEFUL FOR

Mathematicians, machine learning practitioners, and researchers interested in the theoretical foundations of neural networks and measure theory.

jamesb1
Messages
22
Reaction score
0
I am reviewing this http://deeplearning.cs.cmu.edu/pdfs/Cybenko.pdf on the approximating power of neural networks and I came across a definition that I could not quite understand. The definition reads:
0OFMv.png


where $I_n$ is the n-dimensional unit hypercube and $M(I_n)$ is the space of finite, signed, regular Borel measures on In.

The only thing that I could get from this definition which again does not seem plausible enough is: since whenever the integral is 0 it must imply that the measure μ is 0, then σ is non-zero. I am not sure if this right though.

I literally could not find any other literature or similar definitions on this and I've looked in a number of textbooks such as Kreyzig, Rudin and Stein & Shakarchi.

Any insight/help?
 
Last edited by a moderator:
Physics news on Phys.org
WWGD said:
I understand that, what I mean is I don't understand how it correlates with this definition; where the Lebesgue integral being zero implies that the measure μ is also 0.
 
Well, if you tell us up front what you understand and the main definitions/results you are working with, we may be better able to help you.
 
jamesb1 said:
where $I_n$ is the n-dimensional unit hypercube and $M(I_n)$ is the space of finite, signed, regular Borel measures on I_n.

The only thing that I could get from this definition which again does not seem plausible enough is: since whenever the integral is 0 it must imply that the measure μ is 0, then σ is non-zero. I am not sure if this right though.

WWGD said:
Well, if you tell us up front what you understand and the main definitions/results you are working with, we may be better able to help you.

The other definitions in the paper I linked are not really necessary as this definition does not take any special insight from them. I gave the information needed specifically for I_n and M(I_n) because they're mentioned but other than that, it can pretty much be taken as a standalone definition in my opinion. I just cannot understand what the correlation is between the integral being zero and the measure being zero. What does that tell us about the function sigma and how is it the case? I said what I think but I do not think it is correct.

*** EDIT ***
What MAY be of use is the fact sigma in this definition is possibly being taken as sigmoidal ONLY, but then again if that is the case then my understanding does not make sense since a sigmoid function f(t) can tend to zero as t -> -∞
 
Last edited:
All I know is the definition I read, in which I understood that ##\sigma ## is intended to approximate a characteristic function of a set S, i.e., to be 1 on S and 0 otherwise.
 

Similar threads

  • · Replies 7 ·
Replies
7
Views
7K
Replies
0
Views
5K
  • · Replies 7 ·
Replies
7
Views
3K
Replies
24
Views
8K