What is Regularization: Definition and 70 Discussions

In mathematics, statistics, finance, computer science, particularly in machine learning and inverse problems, regularization is the process of adding information in order to solve an ill-posed problem or to prevent overfitting.Regularization can be applied to objective functions in ill-posed optimization problems. The regularization term, or penalty, imposes a cost on the optimization function to make the optimal solution unique.
Independent of the problem or model, there is always a data term, that corresponds to a likelihood of the measurement and a regularization term that corresponds to a prior. By combining both using Bayesian statistics, one can compute a posterior, that includes both information sources and therefore stabilizes the estimation process. By trading off both objectives, one choses to be more addictive to the data or to enforce generalization (to prevent overfitting). There is a whole research branch dealing with all possible regularizations. The work flow usually is, that one tries a specific regularization and then figures out the probability density that corresponds to that regularization to justify the choice. It can also be physically motivated by common sense or intuition, which is more difficult.
In machine learning, the data term corresponds to the training data and the regularization is either the choice of the model or modifications to the algorithm. It is always intended to reduce the generalization error, i.e. the error score with the trained model on the evaluation set and not the training data.One of the earliest uses of regularization is related to the method of least squares. The figured out probability density is the gaussian distribution, which is now known under the name "Tikhonov regularization".

View More On Wikipedia.org
Recent contents Top users
  1. Ramtin123

    A Renormalisation of the Fermionic triangle loop

    I am trying to renormalise the following loop diagram in the Standard Model: Using the Feynman rules, we can write the amplitude as follows: $$ \Gamma_f \sim - tr \int \frac{i}{\displaystyle{\not}\ell -m_f} \frac{i^2}{(\displaystyle{\not}\ell+ \displaystyle{\not}k -m_f)^2} \frac{d^4 \ell}{(2...
  2. E

    A The μ in dimensional regularization

    I have a question about the ##\mu## in dimensional regularization and how it is related to renormalization conditions. I follow the same notation and conventions as in Schwartz. Take QED as an example: $$\mathcal{L} =-\frac{1}{4}\left( F_{0}^{\mu \nu }\right)^{2} +\overline{\psi }_{0}\left(...
  3. S

    I Expansion at first order in QCD counterterm

    What is the meaning of the expansion at first order in ##\delta_2## and ##\delta_3## at the second step in the last line? These quantities are not "small" - on the contrary, the entire point is to then take the ##\epsilon \to 0## limit and the counterterms blow up
  4. Diracobama2181

    A Pauli-Villars Regularization

    For the below integral in Euclidean space, $$\int d^4k_E k_E^2 \frac{1}{(k_E^2+M(\Delta))^2}= 2\pi^2\int dk_E k_E^5 \frac{1}{(k_E^2+M(\Delta))^2}$$ we find $$2\pi^2\int_0^{\infty} dk_E k_E^5 \frac{1}{(k_E^2+M(\Delta))^2}\xrightarrow{}2\pi^2\int_0^{\infty} dk_E...
  5. K

    I Pauli-Villars regularization for Vacuum Polarization

    Hello! I am currently reading Itzykson Zuber QFT book and on Chapter 7 where for the first time loops are considered. Particular method of dealing with divergences namely Pauli-Villars regularization is considered in section 7-1-1 considering vacuum polarization diagram. I do understand physics...
  6. J

    A Connection between QFT renormalization and field regularization?

    While in QFT we remove infinite energy problem with renormalization procedure, asking e.g. "what is mean energy density in given distance from charged particle", electric filed alone would say $$\rho \propto |E|^2 \propto 1/r^4 $$ But such energy density would integrate to infinity due to...
  7. A

    A Algebra of divergent integrals

    Hello, guys! I would like to know your opinion and discuss this extension of real numbers: https://mathoverflow.net/questions/115743/an-algebra-of-integrals/342651#342651 In essence, it extends real numbers with entities that correspond to divergent integrals and series. By adding the rules...
  8. Q

    A Why do different regularization schemes give different answers?

    My question is why is it okay that two different regularizations of a one loop contribution to the full propagator give two different answers? Are the finite parts for all regularization schemes the same? If that's the case, do the divergent parts only contain information about high energy...
  9. SchroedingersLion

    A Regression: Regularization parameter 0

    Hi guys, I am using ScikitLearn's Elastic Net implementation to perform regression on a data set where number of data points is larger than number of features. The routine uses crossvalidation to find the two hyperparameters: ElasticNetCV The elastic net minimizes ##\frac {1}{2N} ||y-Xw||^2 +...
  10. topsquark

    MHB Regularization of an Non-conergent Integral

    The equations here come from calculating the amplitude of a Feynman diagram. I can set up the problem if you really want me to but here I am just interested in why and how the regularization process is supposed to work Mathematically. The generalized meaning of this is if we are given a...
  11. A

    I What is the point of regularization?

    Take for example dimensional regularization. Is it correct to say that the main point of the dimensional regularization of divergent momentum integrals in QFT is to express the divergence of these integrals in such a way that they can be absorbed into the counterterms? Can someone tell me what...
  12. D

    A Renormalisation: what are the physical observables?

    I'm trying to understand renormalisation properly, however, I've run into a few stumbling blocks. To set the scene, I've been reading Matthew Schwartz's "Quantum Field Theory & the Standard Model", in particular the section on mass renormalisation in QED. As I understand it, in order to tame the...
  13. N

    A Dimensional Regularization of Feynman Integrals

    I am looking at Appendix A Equation 52 (Loop Integrals and Dimensional Regularization) in Peskin and Schroeder's book. ∫ddk/(2π)d1/(k2 - Δ)2 = Γ(2-d/2)/(4π)2(1/Δ)2-d/2 = (1/4π)2(2/ε - logΔ - γ + log4π) Can somebody explain how this equation is derived? I would also like to know what the...
  14. F

    I Momentum cut-off regularisation & Lorentz invariance

    Why is it that introducing a hard cut-off ##p^{2}=\Lambda^{2}## breaks Lorentz invariance? Is it simply that it introduces an energy scale and energy is not a Lorentz invariant quantity? Sorry if this is a trivial question, but I just want to make sure I understand the reasoning as I've...
  15. Luca_Mantani

    A Why is the Integral of Dimensional Regularization Equal to 0?

    Hi everyone, I read this property of dimensional regularization, but i do not understand why it is so. $$\int d^dp =0$$. Actually looking for an answer i also saw that a general property of dim. reg. is $$\int d^dp \, (p^2)^\alpha=0$$ for any value of ##\alpha##, so also for ##\alpha=0##...
  16. Anchovy

    A Understanding renormalization / regularization basics

    I've been trying to get a rough understanding of what renormalization involves (in a particle physics context; I'm aware it has many other applications eg. condensed matter) but it hasn't quite clicked yet. The things I have in my head so far are as follows: - A particle will be surrounded by a...
  17. nomadreid

    I Riemann zeta: regularization and universality

    I am not sure which is the appropriate rubric to put this under, so I am putting it in General Math. If anyone wants to move it, that is fine. Two questions, unrelated except both have to do with the Riemann zeta function (and are not about the Riemann Hypothesis). First, in...
  18. V

    I Tikhonov regularization and SVD to compute condition number

    I am solving linear least squares problems with generalized Tikhonov regularization, minimizing the function: \chi^2 = || b - A x ||^2 + \lambda^2 || L x || where L is a diagonal regularization matrix and \lambda is the regularization parameter. I am solving this system using the singular...
  19. B

    Relation between dimensional regularization and high-energy modes

    I state that I am a beginner in QFT, but it seems to me that the methods to regularize the integrals of the perturbation series before renormalize serve to cut off the high-energy modes that are responsable for the UV divergences. This ( the cut off of high-energy modes ) nevertheless is not so...
  20. C

    Cuts of a Feynman diagram and the massless limit

    Consider a ##j## point all massive leg one loop polygonal Feynman diagram ##P## representing some scattering process cut on a particular mass channel ##s_i##. Invoking the relevant Feynman rules and proceeding with the integration via dimensional regularisation for example gives me an expression...
  21. K

    Issue with integral in dimensional regularization

    So I am calculating the renormalization group rquations for some exotic new particles and used dimensional regularization for all the calculations up to this point. Now I am look at the vertex corrections in the massless limit in which all external momentum are equal to 0. The issue here is I...
  22. G

    Dimensional Regularization in Peskin

    Homework Statement I'm trying to understand dimensional regularization with Peskin. There is a transitions that is not clear. Homework Equations On page 250, the general expression for the d-dimensional integral is given: ##\int \frac{d^d...
  23. quantatanu0

    Cut-off Regularization of Chiral Perturbation Theory

    I was trying to learn renormalization in the context of ChPT using momentum-space cut-off regularization procedure at one-loop order using order of p^2 Lagrangian. So, 1. There are counter terms in ChPT of order of p^4 when calculating in one-loop order using Lagrangian of order p^2 . 2...
  24. C

    Pauli-villars regularization in simple phi^4 case

    I'm trying to work through the one-loop, one-vertex diagram in \phi^4 theory using Pauli-Villars regularization, and I'm having trouble. Specifically, I can't get the momentum dependence to fall out after integrating, which I think it should. In computing the "seagull" diagram (two external...
  25. alyafey22

    MHB Analytic continuation and Regularization simplified.

    Hello MHB members In this set of lectures we are going to explore the nice idea of analytic continuation and regularization of divergent series and integrals. Don't get panic ,the idea is so simple that you are actually using it without knowing. I'll try to make the tutorials as simple as...
  26. V

    Tikhonov regularization

    The least-squares solution of A x = b using Tikhonov regularization with a matrix \mu^2 I has the solution: x = \sum_i \left( \frac{\sigma_i^2}{\sigma_i^2 + \mu^2} \right) \left( \frac{u_i^T b}{\sigma_i} \right) v_i where A = U S V^T is the SVD of A and u_i,v_i are the columns of U,V...
  27. I

    On QFT, dimensionality and regularization

    Hello everyone ! First of all, Quantum Field Theory is not my field of research. However, I have to investigate on some problems in QFT and I'm trying to get familiar with it again. I'm basically working with scalar fields and I encounter some problems in dealing with renormalization...
  28. C

    Why does dimensional regularization respect the Ward identity?

    It is often stated that this is the case, but I have often wondered if it is a general statement or just something that we observe to be the case when calculating the relevant loop corrections. Can it be proven generally? Is it somehow easy to see?
  29. C

    Dimensional regularization and renormalization scale.

    Hi. I have observed that Ryder in his book on QFT before doing dimensional regularization introduces a scale ##\mu## in order to keep the coupling constant dimensionless in the lagragnian. However in two other books; Weinberg and Peskin and Schroeder, they do not introduce this scale in the same...
  30. G

    Regularization scheme independence?

    How can two different regularization schemes give the same physical results? This seems unbelievable. If you impose the same renormalization conditions, then in all regularization schemes, the cutoff, or dimension, or the heavy masses get absorbed into constants in such a way that the...
  31. N

    L1 Regularization Question

    I am trying to understand the difference between L1 vs. L2 regularization in OLS. I understand the concept of center of ellipsoid being the optimal solution and ellipse itself being contours of constant squared errors. And when we use L2 regularization we introduce a spherical constraint on...
  32. P

    Parameter μ dimensional regularization qed

    in the dimensional regularization of the QED we introduct an arbitrary parameter μ with the dimension of a mass... so there are finite terms that are function of μ... so they are arbitrary? how we can fix this parameter? what is the physical meaning of μ?
  33. N

    Dimensional regularization vs momentum cutoff

    Hi everyone, I have a 2D sigma model with supersymmetry on the worldsheet. It has both cubic and quartic interactions and I'm interested in the one loop correction to the worldsheet masses. When I calculate this with dimensional regularization I find that everything is zero as expected. In...
  34. B

    Problem with Dimensional Regularization

    Good day to everyone. I am trying to apply dimensional regularization to divergent integral \int\frac{d^{4}l}{\left(2\pi\right)^{4}}\frac{4\, l_{\mu}l_{\nu}}{\left[l^{2}-\triangle+i\epsilon\right]^{3}}. I am very new to these thing. The first question is how should i apply Wicks rotation to...
  35. K

    The ln(m^2) term in dimensional regularization

    Using dimensional regularization we frequently end up with a term 2/(4-d)+ ln(somthing with mass dimension), c.f. Peskin page 250~251, and Peskin said the scale of the logarithm is hidden in the 2/(4-d) term. How is this so? No matter how hard I try to look at 2/(4-d), I see a purely...
  36. P

    QFT books that discuss Infra-red dimensional regularization?

    I learned QFT by reading Peskin & Schroeder, and now find myself unfamiliar with dimensional regularization of IR divergences which is prevalent in the literature. Are there good QFT books which discusses IR dim. reg.? I understand the general idea of going to more than 4 dimensions as opposed...
  37. W

    How to deal with gama5 in dimensional regularization?

    Note that the anti_symmetric tensor εμνρλis only defined in four-dimension space.
  38. K

    Ultraviolet via dimensional regularization

    The integral: \intd^{3}k\frac{1}{k^{2}+m^{2}} is linearly divergent i.e. ultraviolet divergent. However, If one performs dimensional regularization to the above integral: \frac{1}{(2\pi)^d}\intd^{d}k\frac{1}{k^{2}+m^{2}}=\frac{(m^{2})^{d/2-1}}{(4\pi)^{d/2}}\Gamma(1-d/2) As you can notice...
  39. K

    Using Dimensional Regularization to Tame Divergent Momentum Integrals

    Dear all, Dimensional regularization is a very important technique to remove the divergence from momentum integrals. Suppose that you have to calculate a quantity composed of three integrals over k_1, k_2 and k_3 (each one is three dimensional). the integral over k_3 gives ultra violet...
  40. Z

    Zeta regularization and product of dirac delta distribution

    using the convolution theorem with power functions x^{m} we may define via the convolution theorem the product of 2 dirac delta distribution then main idea is to consider the convolution integral \int_{R}dt(x-t)^{m}t^{n} and then apply the Fourier transform with respect to variable 'x'...
  41. Z

    Iterated dimensional regularization

    let be the 2-loop integral \iint d^{k}pd^{k}qF(q,p) where k is the dimension so we regularize it by dimensional regularization the my idea is the following i integrate oper ''q' considering p is constant to get F(p,e^{-1} ) here e is the parameter inside k=4-e after...
  42. Z

    My teachers say this trick (regularization) is not valid

    i want to compute the integral \iint_{D} f(x,y)dxdy here f(x,y) is a Rational function and the integral is DIVERGENT. in order to regularize i had the following idea , i introduce a regulator \int_{D} \frac{f(x,y)}{(x^{2}+y^{2}+1)^{s}}dxdy so for big 's' the integral is convergent i make...
  43. P

    Dimensional regularization and Fractals - Is it a crackpot idea?

    I came across a curious site on this topic: http://e-infinity-energy.blogspot.com/2011/01/t-hooft-veltman-dimensional.html On one hand, the blog history is filled with non-mainstream ideas. (They invented a new subfield called E-infinity.) On the other hand, the people there seem to be tenured...
  44. Z

    Regularization by differentiation respect to a parameter

    let be the integrals \int_{0}^{\infty}dxx^{2} (x-a)^{1/2}=I1 and \int_{0}^{\infty}dxx^{2} (x-a)^{-1/2}=I2 is then correct that I2= 2\frac{dI1}{da} whenever applying a regularization scheme , is it correct to differentiate with respct to external parameters ??
  45. Z

    Zeta regularization of divergent integrals

    From the model used in the zeta regularization procedure to give a meaning to divergent series in the form 1+2+3+4+... , we propose a similar method to give a finite meaning to divergent integrals in the form \int_{0}^{\infty}dx x^{m} for positive 'm' in terms of the negative values of the...
  46. E

    Dimensional Regularization and epsilon

    Hi! Quick question: Does it make a difference if i choose my dim reg. to be D=4-2epsilon or D=4+2epsilon (i suppose in both cases epsilon >0). I mean opinion i should not matter but standard qft books normally don't touch this question very deeply... Cheers, earth2
  47. Z

    Regularization of a divergent integral in several variables

    i've got the following problem let be the integral \int_{R^{3}} dxdydz \frac{R(x,y,z)}{Q(x,y,z)} here R(x,y,z) and Q(x,y,z) are Polynomials on several variable. Let us suppose this integral is divergent, in order to regularize it i have thought about substracting several terms so...
  48. Z

    Regularization of integral by substraction

    given the divergent integral in n-variables \int_{V} f(q1,q2,...,qn)dq1,dq2,...dqn my question is if in general one can substract a Polynomial K in the variables q1,q2,...,qn so the integral \int_{V} (f(q1,q2,...,qn)-K(q1,q2,...,qn))dq1,dq2,...dqn is FINITE , then it would...
  49. Z

    Can ZETA REGULARIZATION avoid all the UV divergences ?

    can ZETA REGULARIZATION avoid all the UV divergences ?? i found a paper http://vixra.org/pdf/1005.0071v1.pdf so using Zeta regularization i can regularize all the integrals by replacing divergent integrals by divergent sums and applying zeta regularization algorithm to get only FINITE results...
  50. R

    Why do some papers use binomial expansion in dimensional regularization?

    I was looking at a paper that used dimensional regularization and the following expression was derived: \int dx \mbox{ }[p^2(1-x)^2-\lambda^2(1-x)]^{\epsilon} Factoring out p^2(1-x)^2 : \int dx \mbox{ }[p^2(1-x)^2]^{\epsilon}[1-\frac{\lambda^2}{p^2(1-x)}]^{\epsilon} The part that I...
Back
Top