Constrained Least Square Optimization

  • Context: Graduate 
  • Thread starter Thread starter ahmadnajeeb
  • Start date Start date
  • Tags Tags
    Optimization Square
Click For Summary
SUMMARY

The discussion centers on constrained least squares optimization, specifically the equation a = argmin_{a}[\sum{||a^Tx_i - y_i||^2}+\alpha ||a||^2]. The participants clarify that the term α ||a||^2 introduces a regularization constraint, which alters the optimization model compared to standard least squares. The use of Matlab's inverse operator is noted as a method for solving the unconstrained problem, but the impact of the constraint on the solution requires further exploration. The mathematical concepts involved include matrix norms and optimization techniques relevant to linear algebra.

PREREQUISITES
  • Understanding of constrained optimization techniques
  • Familiarity with matrix algebra and vector calculus
  • Knowledge of regularization methods in machine learning
  • Proficiency in using Matlab for numerical computations
NEXT STEPS
  • Research "Constrained Least Squares Optimization" techniques
  • Learn about "Ridge Regression" as a form of regularization
  • Explore "Matlab optimization toolbox" for practical implementations
  • Study "Matrix norms" and their applications in optimization problems
USEFUL FOR

Mathematicians, data scientists, and engineers involved in optimization problems, particularly those utilizing Matlab for constrained optimization tasks.

ahmadnajeeb
Messages
1
Reaction score
0
Hi,
I want to know the solution of the following equation.
[tex] a = argmin_{a}[\sum{||a^Tx_i - y_i||^2}+\alpha ||a||^2] \\[/tex]
where [tex]x_i, y_i[/tex] are column vectors of dimensions m and n respectively where [tex]m>n[/tex]. [tex]\alpha[/tex] is a scalar and
[tex]Y = a^T X[/tex] where [tex]X=[x_1 x_2 ... x_k], Y = [y_1 y_2 ... y_k][/tex]

I know that without this constraint [tex]\alpha ||a||^2[/tex], its a simple least square optimization problem and I can solve it using Matlab's inverse operator. I want to use the same inverse operator but don't know how this constraint changes my original model.
 
Physics news on Phys.org
Sorry, but I'm curious- What is this? I don't recognize the aT thing or the double absolute value marks. Could you tell me what type of math is this? Is it Calculus III?
 
What matrix norm are you using for your constraint?
 

Similar threads

  • · Replies 3 ·
Replies
3
Views
4K
  • · Replies 18 ·
Replies
18
Views
3K
  • · Replies 5 ·
Replies
5
Views
2K
  • · Replies 6 ·
Replies
6
Views
4K
  • · Replies 3 ·
Replies
3
Views
1K
  • · Replies 5 ·
Replies
5
Views
3K
  • · Replies 11 ·
Replies
11
Views
1K
  • · Replies 3 ·
Replies
3
Views
2K
  • · Replies 6 ·
Replies
6
Views
2K
  • · Replies 2 ·
Replies
2
Views
1K