Why Cost Function for Linear Regression Is Always a Convex Shaped Function? The Next CEO of...
Implement the Thanos sorting algorithm
What is the point of a new vote on May's deal when the indicative votes suggest she will not win?
Trouble understanding the speech of overseas colleagues
Was a professor correct to chastise me for writing "Prof. X" rather than "Professor X"?
What happens if you roll doubles 3 times then land on "Go to jail?"
Why do remote companies require working in the US?
Does it take more energy to get to Venus or to Mars?
When airplanes disconnect from a tanker during air to air refueling, why do they bank so sharply to the right?
Return the Closest Prime Number
What is the difference between "behavior" and "behaviour"?
Should I tutor a student who I know has cheated on their homework?
How to safely derail a train during transit?
Are there languages with no euphemisms?
Why does standard notation not preserve intervals (visually)
Why do professional authors make "consistency" mistakes? And how to avoid them?
How easy is it to start Magic from scratch?
Inappropriate reference requests from Journal reviewers
How to write the block matrix in LaTex?
Does the Brexit deal have to be agreed by both Houses?
Solution of this Diophantine Equation
Unreliable Magic - Is it worth it?
How to use tikz in fbox?
How do I solve this limit?
How do I construct this japanese bowl?
Why Cost Function for Linear Regression Is Always a Convex Shaped Function?
The Next CEO of Stack OverflowGradient Descent on Non-Convex Function Works But How?Machine learning Linear regression cost functionLogistic regression: Prove that the cost function is convexLinear Regression: linear or reciprocal function?Why using squared distances in the cost function (Linear Regression)?Machine Learning Linear Regression Cost Function NotationLogistic Regression: When can the cost function be non-convex?Minimum sample sizes needed for a series of allowed errors, given a required accuracy and required confidencewhy is the least square cost function for linear regression convexFunky change in correlation
$begingroup$
This diagram is from Andrew Ng course for ML/DL:
enter image description here
But isn't the cost function (least squares function) shape depends on scatter of the data ?
For example below, the minimum will be at (0,1):
enter image description here
that doesn't correspond to convex shape (if you will imagine it in 3d plot), that Andrew Ng showed above.
UPDATE
Oh, i think I understand... my example is a convex shape too, but simply shifted by coordinates, relatively to the Andrew's example.
Am i right?
statistics statistical-inference machine-learning least-squares linear-regression
edited May 31 '18 at 21:12
Royi
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
$endgroup$
add a comment |
$begingroup$
This diagram is from Andrew Ng course for ML/DL:
enter image description here
But isn't the cost function (least squares function) shape depends on scatter of the data ?
For example below, the minimum will be at (0,1):
enter image description here
that doesn't correspond to convex shape (if you will imagine it in 3d plot), that Andrew Ng showed above.
UPDATE
Oh, i think I understand... my example is a convex shape too, but simply shifted by coordinates, relatively to the Andrew's example.
Am i right?
statistics statistical-inference machine-learning least-squares linear-regression
edited May 31 '18 at 21:12
Royi
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
$endgroup$
1
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39
add a comment |
$begingroup$
This diagram is from Andrew Ng course for ML/DL:
enter image description here
But isn't the cost function (least squares function) shape depends on scatter of the data ?
For example below, the minimum will be at (0,1):
enter image description here
that doesn't correspond to convex shape (if you will imagine it in 3d plot), that Andrew Ng showed above.
UPDATE
Oh, i think I understand... my example is a convex shape too, but simply shifted by coordinates, relatively to the Andrew's example.
Am i right?
statistics statistical-inference machine-learning least-squares linear-regression
edited May 31 '18 at 21:12
Royi
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
$endgroup$
This diagram is from Andrew Ng course for ML/DL:
enter image description here
But isn't the cost function (least squares function) shape depends on scatter of the data ?
For example below, the minimum will be at (0,1):
enter image description here
that doesn't correspond to convex shape (if you will imagine it in 3d plot), that Andrew Ng showed above.
UPDATE
Oh, i think I understand... my example is a convex shape too, but simply shifted by coordinates, relatively to the Andrew's example.
Am i right?
statistics statistical-inference machine-learning least-squares linear-regression
statistics statistical-inference machine-learning least-squares linear-regression
edited May 31 '18 at 21:12
Royi
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
edited May 31 '18 at 21:12
Royi
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
edited May 31 '18 at 21:12
Royi
3,59012354
edited May 31 '18 at 21:12
Royi
3,59012354
edited May 31 '18 at 21:12
Royi
3,59012354
3,59012354
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
asked Mar 8 '18 at 17:31
up-to-youup-to-you
84
84
1
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39
add a comment |
1
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39
1
1
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Because the cost function is given by:
$$ frac{1}{2} left| X theta - y right|_{2}^{2} $$
Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ theta $.
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
$endgroup$
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation||is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!
$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2682688%2fwhy-cost-function-for-linear-regression-is-always-a-convex-shaped-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Because the cost function is given by:
$$ frac{1}{2} left| X theta - y right|_{2}^{2} $$
Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ theta $.
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
$endgroup$
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation||is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!
$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
add a comment |
$begingroup$
Because the cost function is given by:
$$ frac{1}{2} left| X theta - y right|_{2}^{2} $$
Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ theta $.
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
$endgroup$
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation||is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!
$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
add a comment |
$begingroup$
Because the cost function is given by:
$$ frac{1}{2} left| X theta - y right|_{2}^{2} $$
Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ theta $.
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
$endgroup$
Because the cost function is given by:
$$ frac{1}{2} left| X theta - y right|_{2}^{2} $$
Which is a Linear Regression Problem with the Least Squares cost function which is a Convex Function of $ theta $.
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
answered Mar 16 '18 at 15:43
RoyiRoyi
3,59012354
3,59012354
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation||is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!
$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
add a comment |
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation||is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!
$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Can you please add more details to your answer? Thank you!
$endgroup$
– Alex Yursha
Oct 10 '18 at 1:58
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
Could you point me what would you like to be extended?
$endgroup$
– Royi
Oct 10 '18 at 14:32
$begingroup$
The notation
|| is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation
|| is not familiar to me. Perhaps, at least provide links to supplemental resources of your choice, which will help to interpret the formula you wrote. Thank you!$endgroup$
– Alex Yursha
Oct 14 '18 at 5:52
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
$begingroup$
The notation $ left| cdot right| $ stands for norm. So the $ {left| cdot right|}_{2} $ stands for the $ {L}_{2} $ norm and $ {left| cdot right|}_{2}^{2} $ is the $ {L}_{2} $ norm squared.
$endgroup$
– Royi
Oct 14 '18 at 6:04
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f2682688%2fwhy-cost-function-for-linear-regression-is-always-a-convex-shaped-function%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
$begingroup$
the cost function should be convex, which corresponds roughly to a "bow" shape
$endgroup$
– Tony S.F.
Mar 8 '18 at 17:34
$begingroup$
sorry, updating question
$endgroup$
– up-to-you
Mar 8 '18 at 17:39