How can the l1 norm of a linear function be maximized?

  • Thread starter humbug
  • Start date
  • Tags
    Norm
In summary, the person is looking for an algorithm or theorem that can maximize the L1 norm of a linear function, given certain constraints. They mention that the problem scales badly and the simplex algorithm does not work. They also discuss the idea of using the separating hyperplane theorem and converting the problem into a binomial choice problem. They ask for any help or ideas on how to deal with the problem.
  • #1
humbug
2
0
Hi all,

Sorry if this is in the wrong section...first time and i couldn't see a convex analysis section.

I'm trying to find a good algorithm/theorem that will maximise the l1 norm (sum of absolute values) of a linear function. Namely, given a function z = c + Ax where z is (nx1), A is (nxm) and x is (mx1), what is the optimal way to choose x such that the sum of the absolute values of z are maximised (obviously x is bounded).

My problem scales really badly so its not feasible to just compute at the bounds of x and choose the biggest outcome. I've also found that the simplex algorithm cannot do it (it only works for minimising the l1 norm). I think the separating hyperplane theorem might be able to help but i can't really see how.

Any help/ideas would be greatly appreciated
 
Mathematics news on Phys.org
  • #2
I think this will work - just make all but one element of x equal to zero. With the position of the one nonzero element corresponding to the column of A which has the maximum L1 norm.

Edit : Ok scrub that, it's only correct for c=0.

BTW c is a vector (nx1) isn't it?
 
Last edited:
  • #3
In general c is a vector (nx1), however one of my cases deals with c being a constant vector so any thoughts on how to deal with that would also be helpful. As I understand it, if each element of x is in the interval [a,b], then shouldn't x be populated with a and b only? I thought that since the abs function is convex then its maximum will lie at the bounds always (or in other words, the solution lies at the corner of some hypercube). You can use this argument to turn the problem into a binomial choice problem if that makes things easier (thats what I'm playing with at the moment but it hasn't made it easier for me yet)
 

1. What is the l1 norm and why is it used in maximisation?

The l1 norm, also known as the Manhattan norm, is a mathematical measure of the magnitude of a vector. It is used in maximisation because it penalizes large values more heavily than small values, making it useful for selecting the most relevant features in a dataset.

2. How does maximizing the l1 norm relate to feature selection?

Maximizing the l1 norm is a common method for feature selection in machine learning. By setting a constraint on the l1 norm, only the most important features will have non-zero coefficients, effectively selecting the most relevant features for a particular task or model.

3. Can the l1 norm be used for both linear and non-linear models?

Yes, the l1 norm can be used for both linear and non-linear models. It is a versatile optimization technique that can be applied to a wide range of models, including linear regression, logistic regression, and support vector machines.

4. How is the l1 norm different from the l2 norm?

The l2 norm, also known as the Euclidean norm, is another mathematical measure of vector magnitude. The main difference between the l1 and l2 norms is that the l1 norm is more robust to outliers and tends to produce sparse solutions, whereas the l2 norm is more sensitive to outliers and tends to produce dense solutions.

5. Are there any disadvantages to using the l1 norm for maximization?

One disadvantage of using the l1 norm for maximization is that it can be computationally expensive, especially for large datasets. Additionally, the l1 norm only considers individual features and does not take into account the interactions between features, which may be important in some models.

Similar threads

  • Calculus and Beyond Homework Help
Replies
8
Views
468
Replies
13
Views
3K
  • Topology and Analysis
Replies
8
Views
3K
Replies
6
Views
1K
  • Classical Physics
Replies
5
Views
1K
Replies
4
Views
2K
Replies
1
Views
3K
  • Linear and Abstract Algebra
Replies
5
Views
7K
Replies
2
Views
1K
  • Math Proof Training and Practice
2
Replies
46
Views
4K
Back
Top