Yet another convex quadratic programMinimize a complex quadratic subject to two convex quadratic...

Hotkey (or other quick way) to insert a keyframe for only one component of a vector-valued property?

How do I express some one as a black person?

Can you reject a postdoc offer after the PI has paid a large sum for flights/accommodation for your visit?

Does the nature of the Apocalypse in The Umbrella Academy change from the first to the last episode?

Find longest word in a string: are any of these algorithms good?

UART pins to unpowered MCU?

Are babies of evil humanoid species inherently evil?

Why the color red for the Republican Party

weren't playing vs didn't play

How to secure an aircraft at a transient parking space?

NASA's RS-25 Engines shut down time

Was Luke Skywalker the leader of the Rebel forces on Hoth?

PTIJ: Should I kill my computer after installing software?

Is "history" a male-biased word ("his+story")?

Difference on montgomery curve equation between EFD and RFC7748

Does a warlock using the Darkness/Devil's Sight combo still have advantage on ranged attacks against a target outside the Darkness?

Why does liquid water form when we exhale on a mirror?

Filtering SOQL results with optional conditionals

Should I tell my boss the work he did was worthless

Counting all the hearts

How to write ı (i without dot) character in pgf-pie

Warn me when the equation number is pushed to the next line

Is it "Vierergruppe" or "Viergruppe", or is there a distinction?

What are actual Tesla M60 models used by AWS?



Yet another convex quadratic program


Minimize a complex quadratic subject to two convex quadratic constraintsMaximize Trace as standard semidefinite optimizationLinear programming with non-convex quadratic constraintQuadratic Program ReformulationHow to determine the optimal step size in a quadratic function optimizationComputational complexity of the following quadratic program (QP)quadratic programming positive semidefinite matrix proofConvex matrix optimization with a linear constraint on the inverseQuadratic program reformulation maximum to minimumDoes this special case of convex quadratic programming have a partially-unique solution?













1












$begingroup$


I want to find the minimum of the following optimization problem but don't even understand the problem in the first place.



$$minlimits_{AX=b}frac{1}{2}X^{T}QX+C^TX+alpha$$



where $Q, A in M_{n times n} (Bbb{R})$, $X, b, C in Bbb{R^n}$ and $alpha in Bbb{R}$. Also, matrix $Q$ is symmetric and positive definite.



This is all I have about the problem. So, can anyone please, explain the problem to me? Thanks for your time.










share|cite|improve this question











$endgroup$








  • 1




    $begingroup$
    This kind of optimization problem is called a quadratic program.
    $endgroup$
    – David M.
    2 days ago










  • $begingroup$
    Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
    $endgroup$
    – Alex Shtof
    2 days ago










  • $begingroup$
    @Alex Shtof: Yes, it is positive definite and symmetric.
    $endgroup$
    – Omojola Micheal
    2 days ago
















1












$begingroup$


I want to find the minimum of the following optimization problem but don't even understand the problem in the first place.



$$minlimits_{AX=b}frac{1}{2}X^{T}QX+C^TX+alpha$$



where $Q, A in M_{n times n} (Bbb{R})$, $X, b, C in Bbb{R^n}$ and $alpha in Bbb{R}$. Also, matrix $Q$ is symmetric and positive definite.



This is all I have about the problem. So, can anyone please, explain the problem to me? Thanks for your time.










share|cite|improve this question











$endgroup$








  • 1




    $begingroup$
    This kind of optimization problem is called a quadratic program.
    $endgroup$
    – David M.
    2 days ago










  • $begingroup$
    Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
    $endgroup$
    – Alex Shtof
    2 days ago










  • $begingroup$
    @Alex Shtof: Yes, it is positive definite and symmetric.
    $endgroup$
    – Omojola Micheal
    2 days ago














1












1








1


1



$begingroup$


I want to find the minimum of the following optimization problem but don't even understand the problem in the first place.



$$minlimits_{AX=b}frac{1}{2}X^{T}QX+C^TX+alpha$$



where $Q, A in M_{n times n} (Bbb{R})$, $X, b, C in Bbb{R^n}$ and $alpha in Bbb{R}$. Also, matrix $Q$ is symmetric and positive definite.



This is all I have about the problem. So, can anyone please, explain the problem to me? Thanks for your time.










share|cite|improve this question











$endgroup$




I want to find the minimum of the following optimization problem but don't even understand the problem in the first place.



$$minlimits_{AX=b}frac{1}{2}X^{T}QX+C^TX+alpha$$



where $Q, A in M_{n times n} (Bbb{R})$, $X, b, C in Bbb{R^n}$ and $alpha in Bbb{R}$. Also, matrix $Q$ is symmetric and positive definite.



This is all I have about the problem. So, can anyone please, explain the problem to me? Thanks for your time.







optimization definition convex-optimization quadratic-programming






share|cite|improve this question















share|cite|improve this question













share|cite|improve this question




share|cite|improve this question








edited 2 days ago







Omojola Micheal

















asked 2 days ago









Omojola MichealOmojola Micheal

1,969324




1,969324








  • 1




    $begingroup$
    This kind of optimization problem is called a quadratic program.
    $endgroup$
    – David M.
    2 days ago










  • $begingroup$
    Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
    $endgroup$
    – Alex Shtof
    2 days ago










  • $begingroup$
    @Alex Shtof: Yes, it is positive definite and symmetric.
    $endgroup$
    – Omojola Micheal
    2 days ago














  • 1




    $begingroup$
    This kind of optimization problem is called a quadratic program.
    $endgroup$
    – David M.
    2 days ago










  • $begingroup$
    Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
    $endgroup$
    – Alex Shtof
    2 days ago










  • $begingroup$
    @Alex Shtof: Yes, it is positive definite and symmetric.
    $endgroup$
    – Omojola Micheal
    2 days ago








1




1




$begingroup$
This kind of optimization problem is called a quadratic program.
$endgroup$
– David M.
2 days ago




$begingroup$
This kind of optimization problem is called a quadratic program.
$endgroup$
– David M.
2 days ago












$begingroup$
Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
$endgroup$
– Alex Shtof
2 days ago




$begingroup$
Do you know anything about the matrix Q? Is it positive semidefinite, or is it something else?
$endgroup$
– Alex Shtof
2 days ago












$begingroup$
@Alex Shtof: Yes, it is positive definite and symmetric.
$endgroup$
– Omojola Micheal
2 days ago




$begingroup$
@Alex Shtof: Yes, it is positive definite and symmetric.
$endgroup$
– Omojola Micheal
2 days ago










2 Answers
2






active

oldest

votes


















2












$begingroup$

You are given two fixed $n times n$ matrices $Q$ and $A$, two fixed n-dimensional vectors $B$ and $C$, and a fixed real number $alpha$. You are supposed to miminimize the value of the objective function $f(X)=tfrac12 X^T Q X + B^T X + alpha$ by varying $X$, subject to the constraint $AX=B$.



So, if we define $S = {X in mathbb{R}^n : AX=B}$, then you need to find $bar X in S$ Such that $f(bar X) le f(X)$ for all $X in S$.



So, this is a quadratic programming problem with a set of linear equality constraints.






share|cite|improve this answer











$endgroup$













  • $begingroup$
    +1. Thanks for the answer. So, is there a natural way I can solve this problem?
    $endgroup$
    – Omojola Micheal
    2 days ago






  • 1




    $begingroup$
    Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
    $endgroup$
    – bubba
    2 days ago










  • $begingroup$
    Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
    $endgroup$
    – Omojola Micheal
    2 days ago












  • $begingroup$
    I don't think that $S$ is bounded, in general.
    $endgroup$
    – bubba
    2 days ago



















1












$begingroup$

You were already provided an accepted answer, but I will still attempt to provide more information. I assume that $A in mathbb{R}^{m times n}$ where $m < n$, that is, the set $S = { X in mathbb{R}^n : A X = B }$ is infinite (otherwise, the problem is trivial). Note, that the set $S$ is affine, and therefore convex.



First, observe that since $Q$ is positive definite, the quadratic objective function is coercive, and therefore a minimizer exists. Since it is strictly convex (the Hessian is $2Q$, which is positive definite), the minimizer is unique.



Now, you have several strategies. The first one, is feeding your problem into any Quadratic Programming solver software like the first answer suggests, and it will give you the (unique) solution. The major drawback of this approach is that such solvers are not always fast. But if it is good enough for you, you do not need to do anything else. Below are two other strategies.



Via KKT conditions



The convex programming problem you seek to solve trivially satisfies Slater's condition, since there are no inequality constraints. Thus, any point is optimal if and only if it satisfies the KKT conditions.



In your case, the Lagrangian is
$$L(X; lambda) = X^T Q X + C^T X + alpha + lambda^T (A X - B),$$
and therefore the KKT conditions are
$$
2 Q X + C + A^T lambda = 0, quad A X = B,
$$

or equivalently written
$$
begin{bmatrix} 2 Q & A^T \ A & 0 end{bmatrix} begin{bmatrix} X \ lambda end{bmatrix} = begin{bmatrix} -C \ B end{bmatrix}.
$$

This is a linear system of equations. Any solution of this system using your favorite linear solver gives you an optimal solution of your desired optimization problem. If you can exploit the special structure of $Q$ or $A$ to solve it quickly, you can solve the optimization problem quickly.



By constraint elimination



Recall, that any rectangular matrix $V$ with more rows than columns admists a QR decomposition into $V = Q R$, where $Q$ is orthogonal and $R$ is upper-triangular. There are also plenty of numerical libraries which can compute it. Since you already used the name $Q$, I will rename your $Q$ matrix into $P$.



By computing the decomposition of $A^T$, you have $A^T = Q R$, and the problem becomes
$$
min_X X^T P X + C^T X + alpha quad text{s.t.} quad R^T Q^T X = B
$$

Substituting $Q^T X = Y$, or $X = Q Y$ (since $Q$ is orthogonal), we obtain
$$
min_Y Y^T (Q^T P Q) Y + (Q^T C) Y + alpha quad text{s.t.} quad R^T X = B
$$

Finally, $R$ is upper triangular. Meaning that it has the following structure
$$
R = begin{bmatrix} tilde{R} \ 0 end{bmatrix},
$$

where $tilde{R}$ is upper triangular and invertible, which is followed by rows full of zeros. Therefore, by decomposing $Y = begin{bmatrix}Y_1 \ Y_2 end{bmatrix}$, where $Y_1$ are the columns which correspond to $tilde{R}$, the constraint $R^T Y = B$ can be written as $tilde{R}^T Y_1 = B$. You can solve this system, find the value of $Y_1$, substitute it into the objective, and obtain an unconstrained problem with $Y_1$, which can be solved any off-the-shelf least-squares solver.



This approach is useful if you need to solve many problems which various matrices $P$, but all share the same matrix $A$. Moreover, it is useful if the dimension of the remaining variable $Y_2$ is much smaller than the dimension of $X$. Constraint elimination allows you to reduce the dimension of the given problem substantially in that case.






share|cite|improve this answer









$endgroup$













    Your Answer





    StackExchange.ifUsing("editor", function () {
    return StackExchange.using("mathjaxEditing", function () {
    StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
    StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
    });
    });
    }, "mathjax-editing");

    StackExchange.ready(function() {
    var channelOptions = {
    tags: "".split(" "),
    id: "69"
    };
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function() {
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled) {
    StackExchange.using("snippets", function() {
    createEditor();
    });
    }
    else {
    createEditor();
    }
    });

    function createEditor() {
    StackExchange.prepareEditor({
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader: {
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    },
    noCode: true, onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    });


    }
    });














    draft saved

    draft discarded


















    StackExchange.ready(
    function () {
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3140869%2fyet-another-convex-quadratic-program%23new-answer', 'question_page');
    }
    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    2












    $begingroup$

    You are given two fixed $n times n$ matrices $Q$ and $A$, two fixed n-dimensional vectors $B$ and $C$, and a fixed real number $alpha$. You are supposed to miminimize the value of the objective function $f(X)=tfrac12 X^T Q X + B^T X + alpha$ by varying $X$, subject to the constraint $AX=B$.



    So, if we define $S = {X in mathbb{R}^n : AX=B}$, then you need to find $bar X in S$ Such that $f(bar X) le f(X)$ for all $X in S$.



    So, this is a quadratic programming problem with a set of linear equality constraints.






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      +1. Thanks for the answer. So, is there a natural way I can solve this problem?
      $endgroup$
      – Omojola Micheal
      2 days ago






    • 1




      $begingroup$
      Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
      $endgroup$
      – bubba
      2 days ago










    • $begingroup$
      Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
      $endgroup$
      – Omojola Micheal
      2 days ago












    • $begingroup$
      I don't think that $S$ is bounded, in general.
      $endgroup$
      – bubba
      2 days ago
















    2












    $begingroup$

    You are given two fixed $n times n$ matrices $Q$ and $A$, two fixed n-dimensional vectors $B$ and $C$, and a fixed real number $alpha$. You are supposed to miminimize the value of the objective function $f(X)=tfrac12 X^T Q X + B^T X + alpha$ by varying $X$, subject to the constraint $AX=B$.



    So, if we define $S = {X in mathbb{R}^n : AX=B}$, then you need to find $bar X in S$ Such that $f(bar X) le f(X)$ for all $X in S$.



    So, this is a quadratic programming problem with a set of linear equality constraints.






    share|cite|improve this answer











    $endgroup$













    • $begingroup$
      +1. Thanks for the answer. So, is there a natural way I can solve this problem?
      $endgroup$
      – Omojola Micheal
      2 days ago






    • 1




      $begingroup$
      Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
      $endgroup$
      – bubba
      2 days ago










    • $begingroup$
      Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
      $endgroup$
      – Omojola Micheal
      2 days ago












    • $begingroup$
      I don't think that $S$ is bounded, in general.
      $endgroup$
      – bubba
      2 days ago














    2












    2








    2





    $begingroup$

    You are given two fixed $n times n$ matrices $Q$ and $A$, two fixed n-dimensional vectors $B$ and $C$, and a fixed real number $alpha$. You are supposed to miminimize the value of the objective function $f(X)=tfrac12 X^T Q X + B^T X + alpha$ by varying $X$, subject to the constraint $AX=B$.



    So, if we define $S = {X in mathbb{R}^n : AX=B}$, then you need to find $bar X in S$ Such that $f(bar X) le f(X)$ for all $X in S$.



    So, this is a quadratic programming problem with a set of linear equality constraints.






    share|cite|improve this answer











    $endgroup$



    You are given two fixed $n times n$ matrices $Q$ and $A$, two fixed n-dimensional vectors $B$ and $C$, and a fixed real number $alpha$. You are supposed to miminimize the value of the objective function $f(X)=tfrac12 X^T Q X + B^T X + alpha$ by varying $X$, subject to the constraint $AX=B$.



    So, if we define $S = {X in mathbb{R}^n : AX=B}$, then you need to find $bar X in S$ Such that $f(bar X) le f(X)$ for all $X in S$.



    So, this is a quadratic programming problem with a set of linear equality constraints.







    share|cite|improve this answer














    share|cite|improve this answer



    share|cite|improve this answer








    edited 2 days ago

























    answered 2 days ago









    bubbabubba

    30.7k33188




    30.7k33188












    • $begingroup$
      +1. Thanks for the answer. So, is there a natural way I can solve this problem?
      $endgroup$
      – Omojola Micheal
      2 days ago






    • 1




      $begingroup$
      Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
      $endgroup$
      – bubba
      2 days ago










    • $begingroup$
      Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
      $endgroup$
      – Omojola Micheal
      2 days ago












    • $begingroup$
      I don't think that $S$ is bounded, in general.
      $endgroup$
      – bubba
      2 days ago


















    • $begingroup$
      +1. Thanks for the answer. So, is there a natural way I can solve this problem?
      $endgroup$
      – Omojola Micheal
      2 days ago






    • 1




      $begingroup$
      Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
      $endgroup$
      – bubba
      2 days ago










    • $begingroup$
      Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
      $endgroup$
      – Omojola Micheal
      2 days ago












    • $begingroup$
      I don't think that $S$ is bounded, in general.
      $endgroup$
      – bubba
      2 days ago
















    $begingroup$
    +1. Thanks for the answer. So, is there a natural way I can solve this problem?
    $endgroup$
    – Omojola Micheal
    2 days ago




    $begingroup$
    +1. Thanks for the answer. So, is there a natural way I can solve this problem?
    $endgroup$
    – Omojola Micheal
    2 days ago




    1




    1




    $begingroup$
    Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
    $endgroup$
    – bubba
    2 days ago




    $begingroup$
    Look up Lagrange multipliers. Or, presumably your course material tells you how to solve such problems, otherwise it's an inappropriate homework problem.
    $endgroup$
    – bubba
    2 days ago












    $begingroup$
    Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
    $endgroup$
    – Omojola Micheal
    2 days ago






    $begingroup$
    Thanks, I now know what Theorem to apply. However, how do I show that $S$ is bounded? I have shown it is closed.
    $endgroup$
    – Omojola Micheal
    2 days ago














    $begingroup$
    I don't think that $S$ is bounded, in general.
    $endgroup$
    – bubba
    2 days ago




    $begingroup$
    I don't think that $S$ is bounded, in general.
    $endgroup$
    – bubba
    2 days ago











    1












    $begingroup$

    You were already provided an accepted answer, but I will still attempt to provide more information. I assume that $A in mathbb{R}^{m times n}$ where $m < n$, that is, the set $S = { X in mathbb{R}^n : A X = B }$ is infinite (otherwise, the problem is trivial). Note, that the set $S$ is affine, and therefore convex.



    First, observe that since $Q$ is positive definite, the quadratic objective function is coercive, and therefore a minimizer exists. Since it is strictly convex (the Hessian is $2Q$, which is positive definite), the minimizer is unique.



    Now, you have several strategies. The first one, is feeding your problem into any Quadratic Programming solver software like the first answer suggests, and it will give you the (unique) solution. The major drawback of this approach is that such solvers are not always fast. But if it is good enough for you, you do not need to do anything else. Below are two other strategies.



    Via KKT conditions



    The convex programming problem you seek to solve trivially satisfies Slater's condition, since there are no inequality constraints. Thus, any point is optimal if and only if it satisfies the KKT conditions.



    In your case, the Lagrangian is
    $$L(X; lambda) = X^T Q X + C^T X + alpha + lambda^T (A X - B),$$
    and therefore the KKT conditions are
    $$
    2 Q X + C + A^T lambda = 0, quad A X = B,
    $$

    or equivalently written
    $$
    begin{bmatrix} 2 Q & A^T \ A & 0 end{bmatrix} begin{bmatrix} X \ lambda end{bmatrix} = begin{bmatrix} -C \ B end{bmatrix}.
    $$

    This is a linear system of equations. Any solution of this system using your favorite linear solver gives you an optimal solution of your desired optimization problem. If you can exploit the special structure of $Q$ or $A$ to solve it quickly, you can solve the optimization problem quickly.



    By constraint elimination



    Recall, that any rectangular matrix $V$ with more rows than columns admists a QR decomposition into $V = Q R$, where $Q$ is orthogonal and $R$ is upper-triangular. There are also plenty of numerical libraries which can compute it. Since you already used the name $Q$, I will rename your $Q$ matrix into $P$.



    By computing the decomposition of $A^T$, you have $A^T = Q R$, and the problem becomes
    $$
    min_X X^T P X + C^T X + alpha quad text{s.t.} quad R^T Q^T X = B
    $$

    Substituting $Q^T X = Y$, or $X = Q Y$ (since $Q$ is orthogonal), we obtain
    $$
    min_Y Y^T (Q^T P Q) Y + (Q^T C) Y + alpha quad text{s.t.} quad R^T X = B
    $$

    Finally, $R$ is upper triangular. Meaning that it has the following structure
    $$
    R = begin{bmatrix} tilde{R} \ 0 end{bmatrix},
    $$

    where $tilde{R}$ is upper triangular and invertible, which is followed by rows full of zeros. Therefore, by decomposing $Y = begin{bmatrix}Y_1 \ Y_2 end{bmatrix}$, where $Y_1$ are the columns which correspond to $tilde{R}$, the constraint $R^T Y = B$ can be written as $tilde{R}^T Y_1 = B$. You can solve this system, find the value of $Y_1$, substitute it into the objective, and obtain an unconstrained problem with $Y_1$, which can be solved any off-the-shelf least-squares solver.



    This approach is useful if you need to solve many problems which various matrices $P$, but all share the same matrix $A$. Moreover, it is useful if the dimension of the remaining variable $Y_2$ is much smaller than the dimension of $X$. Constraint elimination allows you to reduce the dimension of the given problem substantially in that case.






    share|cite|improve this answer









    $endgroup$


















      1












      $begingroup$

      You were already provided an accepted answer, but I will still attempt to provide more information. I assume that $A in mathbb{R}^{m times n}$ where $m < n$, that is, the set $S = { X in mathbb{R}^n : A X = B }$ is infinite (otherwise, the problem is trivial). Note, that the set $S$ is affine, and therefore convex.



      First, observe that since $Q$ is positive definite, the quadratic objective function is coercive, and therefore a minimizer exists. Since it is strictly convex (the Hessian is $2Q$, which is positive definite), the minimizer is unique.



      Now, you have several strategies. The first one, is feeding your problem into any Quadratic Programming solver software like the first answer suggests, and it will give you the (unique) solution. The major drawback of this approach is that such solvers are not always fast. But if it is good enough for you, you do not need to do anything else. Below are two other strategies.



      Via KKT conditions



      The convex programming problem you seek to solve trivially satisfies Slater's condition, since there are no inequality constraints. Thus, any point is optimal if and only if it satisfies the KKT conditions.



      In your case, the Lagrangian is
      $$L(X; lambda) = X^T Q X + C^T X + alpha + lambda^T (A X - B),$$
      and therefore the KKT conditions are
      $$
      2 Q X + C + A^T lambda = 0, quad A X = B,
      $$

      or equivalently written
      $$
      begin{bmatrix} 2 Q & A^T \ A & 0 end{bmatrix} begin{bmatrix} X \ lambda end{bmatrix} = begin{bmatrix} -C \ B end{bmatrix}.
      $$

      This is a linear system of equations. Any solution of this system using your favorite linear solver gives you an optimal solution of your desired optimization problem. If you can exploit the special structure of $Q$ or $A$ to solve it quickly, you can solve the optimization problem quickly.



      By constraint elimination



      Recall, that any rectangular matrix $V$ with more rows than columns admists a QR decomposition into $V = Q R$, where $Q$ is orthogonal and $R$ is upper-triangular. There are also plenty of numerical libraries which can compute it. Since you already used the name $Q$, I will rename your $Q$ matrix into $P$.



      By computing the decomposition of $A^T$, you have $A^T = Q R$, and the problem becomes
      $$
      min_X X^T P X + C^T X + alpha quad text{s.t.} quad R^T Q^T X = B
      $$

      Substituting $Q^T X = Y$, or $X = Q Y$ (since $Q$ is orthogonal), we obtain
      $$
      min_Y Y^T (Q^T P Q) Y + (Q^T C) Y + alpha quad text{s.t.} quad R^T X = B
      $$

      Finally, $R$ is upper triangular. Meaning that it has the following structure
      $$
      R = begin{bmatrix} tilde{R} \ 0 end{bmatrix},
      $$

      where $tilde{R}$ is upper triangular and invertible, which is followed by rows full of zeros. Therefore, by decomposing $Y = begin{bmatrix}Y_1 \ Y_2 end{bmatrix}$, where $Y_1$ are the columns which correspond to $tilde{R}$, the constraint $R^T Y = B$ can be written as $tilde{R}^T Y_1 = B$. You can solve this system, find the value of $Y_1$, substitute it into the objective, and obtain an unconstrained problem with $Y_1$, which can be solved any off-the-shelf least-squares solver.



      This approach is useful if you need to solve many problems which various matrices $P$, but all share the same matrix $A$. Moreover, it is useful if the dimension of the remaining variable $Y_2$ is much smaller than the dimension of $X$. Constraint elimination allows you to reduce the dimension of the given problem substantially in that case.






      share|cite|improve this answer









      $endgroup$
















        1












        1








        1





        $begingroup$

        You were already provided an accepted answer, but I will still attempt to provide more information. I assume that $A in mathbb{R}^{m times n}$ where $m < n$, that is, the set $S = { X in mathbb{R}^n : A X = B }$ is infinite (otherwise, the problem is trivial). Note, that the set $S$ is affine, and therefore convex.



        First, observe that since $Q$ is positive definite, the quadratic objective function is coercive, and therefore a minimizer exists. Since it is strictly convex (the Hessian is $2Q$, which is positive definite), the minimizer is unique.



        Now, you have several strategies. The first one, is feeding your problem into any Quadratic Programming solver software like the first answer suggests, and it will give you the (unique) solution. The major drawback of this approach is that such solvers are not always fast. But if it is good enough for you, you do not need to do anything else. Below are two other strategies.



        Via KKT conditions



        The convex programming problem you seek to solve trivially satisfies Slater's condition, since there are no inequality constraints. Thus, any point is optimal if and only if it satisfies the KKT conditions.



        In your case, the Lagrangian is
        $$L(X; lambda) = X^T Q X + C^T X + alpha + lambda^T (A X - B),$$
        and therefore the KKT conditions are
        $$
        2 Q X + C + A^T lambda = 0, quad A X = B,
        $$

        or equivalently written
        $$
        begin{bmatrix} 2 Q & A^T \ A & 0 end{bmatrix} begin{bmatrix} X \ lambda end{bmatrix} = begin{bmatrix} -C \ B end{bmatrix}.
        $$

        This is a linear system of equations. Any solution of this system using your favorite linear solver gives you an optimal solution of your desired optimization problem. If you can exploit the special structure of $Q$ or $A$ to solve it quickly, you can solve the optimization problem quickly.



        By constraint elimination



        Recall, that any rectangular matrix $V$ with more rows than columns admists a QR decomposition into $V = Q R$, where $Q$ is orthogonal and $R$ is upper-triangular. There are also plenty of numerical libraries which can compute it. Since you already used the name $Q$, I will rename your $Q$ matrix into $P$.



        By computing the decomposition of $A^T$, you have $A^T = Q R$, and the problem becomes
        $$
        min_X X^T P X + C^T X + alpha quad text{s.t.} quad R^T Q^T X = B
        $$

        Substituting $Q^T X = Y$, or $X = Q Y$ (since $Q$ is orthogonal), we obtain
        $$
        min_Y Y^T (Q^T P Q) Y + (Q^T C) Y + alpha quad text{s.t.} quad R^T X = B
        $$

        Finally, $R$ is upper triangular. Meaning that it has the following structure
        $$
        R = begin{bmatrix} tilde{R} \ 0 end{bmatrix},
        $$

        where $tilde{R}$ is upper triangular and invertible, which is followed by rows full of zeros. Therefore, by decomposing $Y = begin{bmatrix}Y_1 \ Y_2 end{bmatrix}$, where $Y_1$ are the columns which correspond to $tilde{R}$, the constraint $R^T Y = B$ can be written as $tilde{R}^T Y_1 = B$. You can solve this system, find the value of $Y_1$, substitute it into the objective, and obtain an unconstrained problem with $Y_1$, which can be solved any off-the-shelf least-squares solver.



        This approach is useful if you need to solve many problems which various matrices $P$, but all share the same matrix $A$. Moreover, it is useful if the dimension of the remaining variable $Y_2$ is much smaller than the dimension of $X$. Constraint elimination allows you to reduce the dimension of the given problem substantially in that case.






        share|cite|improve this answer









        $endgroup$



        You were already provided an accepted answer, but I will still attempt to provide more information. I assume that $A in mathbb{R}^{m times n}$ where $m < n$, that is, the set $S = { X in mathbb{R}^n : A X = B }$ is infinite (otherwise, the problem is trivial). Note, that the set $S$ is affine, and therefore convex.



        First, observe that since $Q$ is positive definite, the quadratic objective function is coercive, and therefore a minimizer exists. Since it is strictly convex (the Hessian is $2Q$, which is positive definite), the minimizer is unique.



        Now, you have several strategies. The first one, is feeding your problem into any Quadratic Programming solver software like the first answer suggests, and it will give you the (unique) solution. The major drawback of this approach is that such solvers are not always fast. But if it is good enough for you, you do not need to do anything else. Below are two other strategies.



        Via KKT conditions



        The convex programming problem you seek to solve trivially satisfies Slater's condition, since there are no inequality constraints. Thus, any point is optimal if and only if it satisfies the KKT conditions.



        In your case, the Lagrangian is
        $$L(X; lambda) = X^T Q X + C^T X + alpha + lambda^T (A X - B),$$
        and therefore the KKT conditions are
        $$
        2 Q X + C + A^T lambda = 0, quad A X = B,
        $$

        or equivalently written
        $$
        begin{bmatrix} 2 Q & A^T \ A & 0 end{bmatrix} begin{bmatrix} X \ lambda end{bmatrix} = begin{bmatrix} -C \ B end{bmatrix}.
        $$

        This is a linear system of equations. Any solution of this system using your favorite linear solver gives you an optimal solution of your desired optimization problem. If you can exploit the special structure of $Q$ or $A$ to solve it quickly, you can solve the optimization problem quickly.



        By constraint elimination



        Recall, that any rectangular matrix $V$ with more rows than columns admists a QR decomposition into $V = Q R$, where $Q$ is orthogonal and $R$ is upper-triangular. There are also plenty of numerical libraries which can compute it. Since you already used the name $Q$, I will rename your $Q$ matrix into $P$.



        By computing the decomposition of $A^T$, you have $A^T = Q R$, and the problem becomes
        $$
        min_X X^T P X + C^T X + alpha quad text{s.t.} quad R^T Q^T X = B
        $$

        Substituting $Q^T X = Y$, or $X = Q Y$ (since $Q$ is orthogonal), we obtain
        $$
        min_Y Y^T (Q^T P Q) Y + (Q^T C) Y + alpha quad text{s.t.} quad R^T X = B
        $$

        Finally, $R$ is upper triangular. Meaning that it has the following structure
        $$
        R = begin{bmatrix} tilde{R} \ 0 end{bmatrix},
        $$

        where $tilde{R}$ is upper triangular and invertible, which is followed by rows full of zeros. Therefore, by decomposing $Y = begin{bmatrix}Y_1 \ Y_2 end{bmatrix}$, where $Y_1$ are the columns which correspond to $tilde{R}$, the constraint $R^T Y = B$ can be written as $tilde{R}^T Y_1 = B$. You can solve this system, find the value of $Y_1$, substitute it into the objective, and obtain an unconstrained problem with $Y_1$, which can be solved any off-the-shelf least-squares solver.



        This approach is useful if you need to solve many problems which various matrices $P$, but all share the same matrix $A$. Moreover, it is useful if the dimension of the remaining variable $Y_2$ is much smaller than the dimension of $X$. Constraint elimination allows you to reduce the dimension of the given problem substantially in that case.







        share|cite|improve this answer












        share|cite|improve this answer



        share|cite|improve this answer










        answered yesterday









        Alex ShtofAlex Shtof

        539515




        539515






























            draft saved

            draft discarded




















































            Thanks for contributing an answer to Mathematics Stack Exchange!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid



            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.


            Use MathJax to format equations. MathJax reference.


            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function () {
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3140869%2fyet-another-convex-quadratic-program%23new-answer', 'question_page');
            }
            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Nidaros erkebispedøme

            Birsay

            Where did Arya get these scars? Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara Favourite questions and answers from the 1st quarter of 2019Why did Arya refuse to end it?Has the pronunciation of Arya Stark's name changed?Has Arya forgiven people?Why did Arya Stark lose her vision?Why can Arya still use the faces?Has the Narrow Sea become narrower?Does Arya Stark know how to make poisons outside of the House of Black and White?Why did Nymeria leave Arya?Why did Arya not kill the Lannister soldiers she encountered in the Riverlands?What is the current canonical age of Sansa, Bran and Arya Stark?