# What is Regularization: Definition and 70 Discussions

In mathematics, statistics, finance, computer science, particularly in machine learning and inverse problems, regularization is the process of adding information in order to solve an ill-posed problem or to prevent overfitting.Regularization can be applied to objective functions in ill-posed optimization problems. The regularization term, or penalty, imposes a cost on the optimization function to make the optimal solution unique.
Independent of the problem or model, there is always a data term, that corresponds to a likelihood of the measurement and a regularization term that corresponds to a prior. By combining both using Bayesian statistics, one can compute a posterior, that includes both information sources and therefore stabilizes the estimation process. By trading off both objectives, one choses to be more addictive to the data or to enforce generalization (to prevent overfitting). There is a whole research branch dealing with all possible regularizations. The work flow usually is, that one tries a specific regularization and then figures out the probability density that corresponds to that regularization to justify the choice. It can also be physically motivated by common sense or intuition, which is more difficult.
In machine learning, the data term corresponds to the training data and the regularization is either the choice of the model or modifications to the algorithm. It is always intended to reduce the generalization error, i.e. the error score with the trained model on the evaluation set and not the training data.One of the earliest uses of regularization is related to the method of least squares. The figured out probability density is the gaussian distribution, which is now known under the name "Tikhonov regularization".

View More On Wikipedia.org

3. ### I Expansion at first order in QCD counterterm

What is the meaning of the expansion at first order in ##\delta_2## and ##\delta_3## at the second step in the last line? These quantities are not "small" - on the contrary, the entire point is to then take the ##\epsilon \to 0## limit and the counterterms blow up

16. ### A Understanding renormalization / regularization basics

I've been trying to get a rough understanding of what renormalization involves (in a particle physics context; I'm aware it has many other applications eg. condensed matter) but it hasn't quite clicked yet. The things I have in my head so far are as follows: - A particle will be surrounded by a...
17. ### I Riemann zeta: regularization and universality

I am not sure which is the appropriate rubric to put this under, so I am putting it in General Math. If anyone wants to move it, that is fine. Two questions, unrelated except both have to do with the Riemann zeta function (and are not about the Riemann Hypothesis). First, in...
18. ### I Tikhonov regularization and SVD to compute condition number

I am solving linear least squares problems with generalized Tikhonov regularization, minimizing the function: \chi^2 = || b - A x ||^2 + \lambda^2 || L x || where L is a diagonal regularization matrix and \lambda is the regularization parameter. I am solving this system using the singular...
19. ### Relation between dimensional regularization and high-energy modes

I state that I am a beginner in QFT, but it seems to me that the methods to regularize the integrals of the perturbation series before renormalize serve to cut off the high-energy modes that are responsable for the UV divergences. This ( the cut off of high-energy modes ) nevertheless is not so...
20. ### Cuts of a Feynman diagram and the massless limit

Consider a ##j## point all massive leg one loop polygonal Feynman diagram ##P## representing some scattering process cut on a particular mass channel ##s_i##. Invoking the relevant Feynman rules and proceeding with the integration via dimensional regularisation for example gives me an expression...
21. ### Issue with integral in dimensional regularization

So I am calculating the renormalization group rquations for some exotic new particles and used dimensional regularization for all the calculations up to this point. Now I am look at the vertex corrections in the massless limit in which all external momentum are equal to 0. The issue here is I...
22. ### Dimensional Regularization in Peskin

Homework Statement I'm trying to understand dimensional regularization with Peskin. There is a transitions that is not clear. Homework Equations On page 250, the general expression for the d-dimensional integral is given: ##\int \frac{d^d...
23. ### Cut-off Regularization of Chiral Perturbation Theory

I was trying to learn renormalization in the context of ChPT using momentum-space cut-off regularization procedure at one-loop order using order of p^2 Lagrangian. So, 1. There are counter terms in ChPT of order of p^4 when calculating in one-loop order using Lagrangian of order p^2 . 2...
24. ### Pauli-villars regularization in simple phi^4 case

I'm trying to work through the one-loop, one-vertex diagram in \phi^4 theory using Pauli-Villars regularization, and I'm having trouble. Specifically, I can't get the momentum dependence to fall out after integrating, which I think it should. In computing the "seagull" diagram (two external...
25. ### MHB Analytic continuation and Regularization simplified.

Hello MHB members In this set of lectures we are going to explore the nice idea of analytic continuation and regularization of divergent series and integrals. Don't get panic ,the idea is so simple that you are actually using it without knowing. I'll try to make the tutorials as simple as...
26. ### Tikhonov regularization

The least-squares solution of A x = b using Tikhonov regularization with a matrix \mu^2 I has the solution: x = \sum_i \left( \frac{\sigma_i^2}{\sigma_i^2 + \mu^2} \right) \left( \frac{u_i^T b}{\sigma_i} \right) v_i where A = U S V^T is the SVD of A and u_i,v_i are the columns of U,V...
27. ### On QFT, dimensionality and regularization

Hello everyone ! First of all, Quantum Field Theory is not my field of research. However, I have to investigate on some problems in QFT and I'm trying to get familiar with it again. I'm basically working with scalar fields and I encounter some problems in dealing with renormalization...
28. ### Why does dimensional regularization respect the Ward identity?

It is often stated that this is the case, but I have often wondered if it is a general statement or just something that we observe to be the case when calculating the relevant loop corrections. Can it be proven generally? Is it somehow easy to see?
29. ### Dimensional regularization and renormalization scale.

Hi. I have observed that Ryder in his book on QFT before doing dimensional regularization introduces a scale ##\mu## in order to keep the coupling constant dimensionless in the lagragnian. However in two other books; Weinberg and Peskin and Schroeder, they do not introduce this scale in the same...
30. ### Regularization scheme independence?

How can two different regularization schemes give the same physical results? This seems unbelievable. If you impose the same renormalization conditions, then in all regularization schemes, the cutoff, or dimension, or the heavy masses get absorbed into constants in such a way that the...
31. ### L1 Regularization Question

I am trying to understand the difference between L1 vs. L2 regularization in OLS. I understand the concept of center of ellipsoid being the optimal solution and ellipse itself being contours of constant squared errors. And when we use L2 regularization we introduce a spherical constraint on...
32. ### Parameter μ dimensional regularization qed

in the dimensional regularization of the QED we introduct an arbitrary parameter μ with the dimension of a mass... so there are finite terms that are function of μ... so they are arbitrary? how we can fix this parameter? what is the physical meaning of μ?
33. ### Dimensional regularization vs momentum cutoff

Hi everyone, I have a 2D sigma model with supersymmetry on the worldsheet. It has both cubic and quartic interactions and I'm interested in the one loop correction to the worldsheet masses. When I calculate this with dimensional regularization I find that everything is zero as expected. In...
34. ### Problem with Dimensional Regularization

Good day to everyone. I am trying to apply dimensional regularization to divergent integral \int\frac{d^{4}l}{\left(2\pi\right)^{4}}\frac{4\, l_{\mu}l_{\nu}}{\left[l^{2}-\triangle+i\epsilon\right]^{3}}. I am very new to these thing. The first question is how should i apply Wicks rotation to...
35. ### The ln(m^2) term in dimensional regularization

Using dimensional regularization we frequently end up with a term 2/(4-d)+ ln(somthing with mass dimension), c.f. Peskin page 250~251, and Peskin said the scale of the logarithm is hidden in the 2/(4-d) term. How is this so? No matter how hard I try to look at 2/(4-d), I see a purely...
36. ### QFT books that discuss Infra-red dimensional regularization?

I learned QFT by reading Peskin & Schroeder, and now find myself unfamiliar with dimensional regularization of IR divergences which is prevalent in the literature. Are there good QFT books which discusses IR dim. reg.? I understand the general idea of going to more than 4 dimensions as opposed...
37. ### How to deal with gama5 in dimensional regularization?

Note that the anti_symmetric tensor εμνρλis only defined in four-dimension space.
38. ### Ultraviolet via dimensional regularization

The integral: \intd^{3}k\frac{1}{k^{2}+m^{2}} is linearly divergent i.e. ultraviolet divergent. However, If one performs dimensional regularization to the above integral: \frac{1}{(2\pi)^d}\intd^{d}k\frac{1}{k^{2}+m^{2}}=\frac{(m^{2})^{d/2-1}}{(4\pi)^{d/2}}\Gamma(1-d/2) As you can notice...
39. ### Using Dimensional Regularization to Tame Divergent Momentum Integrals

Dear all, Dimensional regularization is a very important technique to remove the divergence from momentum integrals. Suppose that you have to calculate a quantity composed of three integrals over k_1, k_2 and k_3 (each one is three dimensional). the integral over k_3 gives ultra violet...
40. ### Zeta regularization and product of dirac delta distribution

using the convolution theorem with power functions x^{m} we may define via the convolution theorem the product of 2 dirac delta distribution then main idea is to consider the convolution integral \int_{R}dt(x-t)^{m}t^{n} and then apply the Fourier transform with respect to variable 'x'...
41. ### Iterated dimensional regularization

let be the 2-loop integral \iint d^{k}pd^{k}qF(q,p) where k is the dimension so we regularize it by dimensional regularization the my idea is the following i integrate oper ''q' considering p is constant to get F(p,e^{-1} ) here e is the parameter inside k=4-e after...
42. ### My teachers say this trick (regularization) is not valid

i want to compute the integral \iint_{D} f(x,y)dxdy here f(x,y) is a Rational function and the integral is DIVERGENT. in order to regularize i had the following idea , i introduce a regulator \int_{D} \frac{f(x,y)}{(x^{2}+y^{2}+1)^{s}}dxdy so for big 's' the integral is convergent i make...
43. ### Dimensional regularization and Fractals - Is it a crackpot idea?

I came across a curious site on this topic: http://e-infinity-energy.blogspot.com/2011/01/t-hooft-veltman-dimensional.html On one hand, the blog history is filled with non-mainstream ideas. (They invented a new subfield called E-infinity.) On the other hand, the people there seem to be tenured...
44. ### Regularization by differentiation respect to a parameter

let be the integrals \int_{0}^{\infty}dxx^{2} (x-a)^{1/2}=I1 and \int_{0}^{\infty}dxx^{2} (x-a)^{-1/2}=I2 is then correct that I2= 2\frac{dI1}{da} whenever applying a regularization scheme , is it correct to differentiate with respct to external parameters ??
45. ### Zeta regularization of divergent integrals

From the model used in the zeta regularization procedure to give a meaning to divergent series in the form 1+2+3+4+... , we propose a similar method to give a finite meaning to divergent integrals in the form \int_{0}^{\infty}dx x^{m} for positive 'm' in terms of the negative values of the...
46. ### Dimensional Regularization and epsilon

Hi! Quick question: Does it make a difference if i choose my dim reg. to be D=4-2epsilon or D=4+2epsilon (i suppose in both cases epsilon >0). I mean opinion i should not matter but standard qft books normally don't touch this question very deeply... Cheers, earth2
47. ### Regularization of a divergent integral in several variables

i've got the following problem let be the integral \int_{R^{3}} dxdydz \frac{R(x,y,z)}{Q(x,y,z)} here R(x,y,z) and Q(x,y,z) are Polynomials on several variable. Let us suppose this integral is divergent, in order to regularize it i have thought about substracting several terms so...
48. ### Regularization of integral by substraction

given the divergent integral in n-variables \int_{V} f(q1,q2,...,qn)dq1,dq2,...dqn my question is if in general one can substract a Polynomial K in the variables q1,q2,...,qn so the integral \int_{V} (f(q1,q2,...,qn)-K(q1,q2,...,qn))dq1,dq2,...dqn is FINITE , then it would...
49. ### Can ZETA REGULARIZATION avoid all the UV divergences ?

can ZETA REGULARIZATION avoid all the UV divergences ?? i found a paper http://vixra.org/pdf/1005.0071v1.pdf so using Zeta regularization i can regularize all the integrals by replacing divergent integrals by divergent sums and applying zeta regularization algorithm to get only FINITE results...
50. ### Why do some papers use binomial expansion in dimensional regularization?

I was looking at a paper that used dimensional regularization and the following expression was derived: \int dx \mbox{ }[p^2(1-x)^2-\lambda^2(1-x)]^{\epsilon} Factoring out p^2(1-x)^2 : \int dx \mbox{ }[p^2(1-x)^2]^{\epsilon}[1-\frac{\lambda^2}{p^2(1-x)}]^{\epsilon} The part that I...