How is $(x_1,x_2)$ normal to $x_1w_1 + x_2w_2 = y$? Announcing the arrival of Valued Associate...
How to compare two different files line by line in unix?
What do you call the main part of a joke?
Is there a kind of relay only consumes power when switching?
Is there any way for the UK Prime Minister to make a motion directly dependent on Government confidence?
Is it cost-effective to upgrade an old-ish Giant Escape R3 commuter bike with entry-level branded parts (wheels, drivetrain)?
How to answer "Have you ever been terminated?"
Circuit to "zoom in" on mV fluctuations of a DC signal?
How to Make a Beautiful Stacked 3D Plot
When a candle burns, why does the top of wick glow if bottom of flame is hottest?
Chinese Seal on silk painting - what does it mean?
Do square wave exist?
Wu formula for manifolds with boundary
Can a party unilaterally change candidates in preparation for a General election?
Should I use a zero-interest credit card for a large one-time purchase?
Irreducible of finite Krull dimension implies quasi-compact?
Can anything be seen from the center of the Boötes void? How dark would it be?
For a new assistant professor in CS, how to build/manage a publication pipeline
How to find all the available tools in mac terminal?
Compare a given version number in the form major.minor.build.patch and see if one is less than the other
Is grep documentation wrong?
How to react to hostile behavior from a senior developer?
Do I really need recursive chmod to restrict access to a folder?
Trademark violation for app?
Has negative voting ever been officially implemented in elections, or seriously proposed, or even studied?
How is $(x_1,x_2)$ normal to $x_1w_1 + x_2w_2 = y$?
Announcing the arrival of Valued Associate #679: Cesar Manara
Planned maintenance scheduled April 17/18, 2019 at 00:00UTC (8:00pm US/Eastern)Normal vector from equation questionNormal Vector to from a line equationIf ${x_1, x_2}$ are orthogonal vectors in $mathbb R^3$, how to prove that we can always find the third vector so that they become orthogonal basis?How to check if these vectors are normal or orthogonal?Vector that is orthogonal to one vector in a plane, automatically the normal?For $xin E$ and $x = x_1 + x_2$, where $x_1 in E_1$, shouldn't the distance vector of x from the subspace $E_1$ be unique?Find a basis for $W^perp$ for $W={(x_1,x_2,x_3)inmathbb{R}^3:x_1-x_2-x_3=0}$Gradient method: convergence in finite number of steps for $f(x_1,x_2) = x_1^2 + 4x_2^2 - 4x_1 - 8x_2$Let $S$ be the subspace of $Bbb R^3$ spanned by the vector $x= (x_1,x_2,x_3)^T$ and $y= (y_1,y_2,y_3)^T$Let $S$ be the subspace of $mathbb{R}^4$ spanned by $x_1=(1,0,-2,1)^T$ and $x_2=(0,1,3,-2)^T$. Find a basis for $S_perp$
$begingroup$
Note: this question is related to the maths of Neural Nets, if you need clarification about the question do comment.
Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the intersection of two lines.
What I can't understand is how the iterations of on-line backpropagation are perpendicular to the (current) constraint. More specifically, how is $frac12(x_1w_1 + x_2w_2 + y)^2$'s gradient, $(x_1,x_2)$, normal to the constraint $x_1w_1 + x_2w_2 = y$?
orthogonality gradient-descent
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
$endgroup$
add a comment |
$begingroup$
Note: this question is related to the maths of Neural Nets, if you need clarification about the question do comment.
Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the intersection of two lines.
What I can't understand is how the iterations of on-line backpropagation are perpendicular to the (current) constraint. More specifically, how is $frac12(x_1w_1 + x_2w_2 + y)^2$'s gradient, $(x_1,x_2)$, normal to the constraint $x_1w_1 + x_2w_2 = y$?
orthogonality gradient-descent
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
$endgroup$
add a comment |
$begingroup$
Note: this question is related to the maths of Neural Nets, if you need clarification about the question do comment.
Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the intersection of two lines.
What I can't understand is how the iterations of on-line backpropagation are perpendicular to the (current) constraint. More specifically, how is $frac12(x_1w_1 + x_2w_2 + y)^2$'s gradient, $(x_1,x_2)$, normal to the constraint $x_1w_1 + x_2w_2 = y$?
orthogonality gradient-descent
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
$endgroup$
Note: this question is related to the maths of Neural Nets, if you need clarification about the question do comment.
Raul Rojas' Neural Networks A Systematic Introduction, section 8.1.2 relates off-line backpropagation and on-line backpropagation with Gauss-Jacobi and Gauss-Seidel methods for finding the intersection of two lines.
What I can't understand is how the iterations of on-line backpropagation are perpendicular to the (current) constraint. More specifically, how is $frac12(x_1w_1 + x_2w_2 + y)^2$'s gradient, $(x_1,x_2)$, normal to the constraint $x_1w_1 + x_2w_2 = y$?
orthogonality gradient-descent
orthogonality gradient-descent
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
asked Mar 24 at 19:59
EmmanuelMessEmmanuelMess
33
33
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
$begingroup$
If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
$$(x_1, x_2) cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
$endgroup$
add a comment |
$begingroup$
Suppose you earmark two points in the plane $xcdot w=y$. The path between them is a vector $dw$ satisfying $xcdot dw=0$. Therefore, $x$ is normal to any such path.
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
$endgroup$
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3160982%2fhow-is-x-1-x-2-normal-to-x-1w-1-x-2w-2-y%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
$$(x_1, x_2) cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
$endgroup$
add a comment |
$begingroup$
If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
$$(x_1, x_2) cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
$endgroup$
add a comment |
$begingroup$
If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
$$(x_1, x_2) cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
$endgroup$
If you choose two points $(w_1, w_2), (v_1, v_2)$ along this line, then
$$(x_1, x_2) cdot ((w_1, w_2) - (v_1, v_2)) = x_1 w_1 + x_2 w_2 - (x_1 v_1 + x_2 v_2) = y - y = 0.$$
That is, the direction $(x_1, x_2)$ is perpendicular to any vector lying along the line, i.e. $(x_1, x_2)$ is normal to the line.
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
answered Mar 24 at 20:07
Theo BenditTheo Bendit
21.3k12355
21.3k12355
add a comment |
add a comment |
$begingroup$
Suppose you earmark two points in the plane $xcdot w=y$. The path between them is a vector $dw$ satisfying $xcdot dw=0$. Therefore, $x$ is normal to any such path.
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
$endgroup$
add a comment |
$begingroup$
Suppose you earmark two points in the plane $xcdot w=y$. The path between them is a vector $dw$ satisfying $xcdot dw=0$. Therefore, $x$ is normal to any such path.
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
$endgroup$
add a comment |
$begingroup$
Suppose you earmark two points in the plane $xcdot w=y$. The path between them is a vector $dw$ satisfying $xcdot dw=0$. Therefore, $x$ is normal to any such path.
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
$endgroup$
Suppose you earmark two points in the plane $xcdot w=y$. The path between them is a vector $dw$ satisfying $xcdot dw=0$. Therefore, $x$ is normal to any such path.
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
answered Mar 24 at 20:08
J.G.J.G.
33.7k23252
33.7k23252
add a comment |
add a comment |
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3160982%2fhow-is-x-1-x-2-normal-to-x-1w-1-x-2w-2-y%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
