Matrix Differentiation (involving Hadamard products)How do you expand this frobenius form?Inequality...
How can I get players to stop ignoring or overlooking the plot hooks I'm giving them?
Does this video of collapsing warehouse shelves show a real incident?
What are actual Tesla M60 models used by AWS?
Can one live in the U.S. and not use a credit card?
Recommendation letter by significant other if you worked with them professionally?
Are all players supposed to be able to see each others' character sheets?
Hotkey (or other quick way) to insert a keyframe for only one component of a vector-valued property?
Accountant/ lawyer will not return my call
weren't playing vs didn't play
How did Alan Turing break the enigma code using the hint given by the lady in the bar?
How to write ı (i without dot) character in pgf-pie
When traveling to Europe from North America, do I need to purchase a different power strip?
Conservation of Mass and Energy
Could you please stop shuffling the deck and play already?
Meaning of ちはース as an exclamation
Do recommendation systems necessarily use machine learning algorithms?
Reversed Sudoku
Doesn't allowing a user mode program to access kernel space memory and execute the IN and OUT instructions defeat the purpose of having CPU modes?
Bash script should only kill those instances of another script's that it has launched
NASA's RS-25 Engines shut down time
Good for you! in Russian
How to draw cubes in a 3 dimensional plane
How can The Temple of Elementary Evil reliably protect itself against kinetic bombardment?
What problems would a superhuman have whose skin is constantly hot?
Matrix Differentiation (involving Hadamard products)
How do you expand this frobenius form?Inequality involving traces and matrix inversionsDerivative of determinant of symmetric matrix wrt a scalarPartial derivative of weighted matrix factorization?Derivatives involving inner product and Hadamard productDerivative of trace functions using chain ruleDerivative of Frobenius norm of Hadamard ProductHelp with matrix derivativeFrobenius Norm of Hadamard Product and TraceDerivative of Hadamard product with respect to matrix
$begingroup$
I am trying to differentiate over the following Frobenius Norm: $$Phi =||A-(Bcirc C)D ||^2_F$$
with respect to B, C, D respectively, i.e.:
$$frac{partial Phi}{partial B}, frac{partial Phi}{partial C},frac{partial Phi}{partial D}$$
After expending the norm and writing it in the trace form, it becomes:
$$Phi = tr(A^TA) -tr(A^T(Bcirc C)D )-tr(D^T(B^Tcirc C^T)A)+tr(D^T(B^T circ C^T)(B circ C)D) $$
(please correct me if I did the expansion wrong)
I've tried to write them elements by elements and checked out the whole matrix cookbook.
Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.
Could anyone help me out please? I appreciate a lot.
linear-algebra matrices derivatives matrix-calculus hadamard-product
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am trying to differentiate over the following Frobenius Norm: $$Phi =||A-(Bcirc C)D ||^2_F$$
with respect to B, C, D respectively, i.e.:
$$frac{partial Phi}{partial B}, frac{partial Phi}{partial C},frac{partial Phi}{partial D}$$
After expending the norm and writing it in the trace form, it becomes:
$$Phi = tr(A^TA) -tr(A^T(Bcirc C)D )-tr(D^T(B^Tcirc C^T)A)+tr(D^T(B^T circ C^T)(B circ C)D) $$
(please correct me if I did the expansion wrong)
I've tried to write them elements by elements and checked out the whole matrix cookbook.
Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.
Could anyone help me out please? I appreciate a lot.
linear-algebra matrices derivatives matrix-calculus hadamard-product
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
add a comment |
$begingroup$
I am trying to differentiate over the following Frobenius Norm: $$Phi =||A-(Bcirc C)D ||^2_F$$
with respect to B, C, D respectively, i.e.:
$$frac{partial Phi}{partial B}, frac{partial Phi}{partial C},frac{partial Phi}{partial D}$$
After expending the norm and writing it in the trace form, it becomes:
$$Phi = tr(A^TA) -tr(A^T(Bcirc C)D )-tr(D^T(B^Tcirc C^T)A)+tr(D^T(B^T circ C^T)(B circ C)D) $$
(please correct me if I did the expansion wrong)
I've tried to write them elements by elements and checked out the whole matrix cookbook.
Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.
Could anyone help me out please? I appreciate a lot.
linear-algebra matrices derivatives matrix-calculus hadamard-product
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
$endgroup$
I am trying to differentiate over the following Frobenius Norm: $$Phi =||A-(Bcirc C)D ||^2_F$$
with respect to B, C, D respectively, i.e.:
$$frac{partial Phi}{partial B}, frac{partial Phi}{partial C},frac{partial Phi}{partial D}$$
After expending the norm and writing it in the trace form, it becomes:
$$Phi = tr(A^TA) -tr(A^T(Bcirc C)D )-tr(D^T(B^Tcirc C^T)A)+tr(D^T(B^T circ C^T)(B circ C)D) $$
(please correct me if I did the expansion wrong)
I've tried to write them elements by elements and checked out the whole matrix cookbook.
Since there's a hadamard product involving in the equation, I got completely confused of differentiating the last three terms.
Could anyone help me out please? I appreciate a lot.
linear-algebra matrices derivatives matrix-calculus hadamard-product
linear-algebra matrices derivatives matrix-calculus hadamard-product
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
Rio LiRio Li
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 2 days ago
Rio LiRio Li
1
asked 2 days ago
Rio LiRio Li
1
1
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Rio Li is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
$begingroup$
Define the matrices
$$eqalign{
X &= Bcirc C,quad Y=XD-A cr
}$$
Write the function in terms of these new variables.
Then find its differentials and gradients (in alphabetical order).
$$eqalign{
phi &= |Y|_F^2 = Y:Y cr
dphi &= 2Y:dY cr
&= 2Y:dX,D cr
&= 2YD^T:Ccirc dB cr
&= 2Ccirc(YD^T):dB cr
frac{partialphi}{partial B} &= 2Ccirc(YD^T) cr
}$$
Since $B$ and $C$ are indistinguishable
$$eqalign{
frac{partialphi}{partial C} &= 2Bcirc(YD^T) cr
}$$
Finally, the gradient wrt $D$
$$eqalign{
dphi &= 2Y:X,dD cr
&= 2X^TY:dD cr
frac{partialphi}{partial D} &= 2X^TY cr
}$$
NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {rm Tr}(B^TC)$$
answered 2 days ago
greggreg
8,7751824
$endgroup$
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
add a comment |
Your Answer
StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Rio Li is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3140877%2fmatrix-differentiation-involving-hadamard-products%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
$begingroup$
Define the matrices
$$eqalign{
X &= Bcirc C,quad Y=XD-A cr
}$$
Write the function in terms of these new variables.
Then find its differentials and gradients (in alphabetical order).
$$eqalign{
phi &= |Y|_F^2 = Y:Y cr
dphi &= 2Y:dY cr
&= 2Y:dX,D cr
&= 2YD^T:Ccirc dB cr
&= 2Ccirc(YD^T):dB cr
frac{partialphi}{partial B} &= 2Ccirc(YD^T) cr
}$$
Since $B$ and $C$ are indistinguishable
$$eqalign{
frac{partialphi}{partial C} &= 2Bcirc(YD^T) cr
}$$
Finally, the gradient wrt $D$
$$eqalign{
dphi &= 2Y:X,dD cr
&= 2X^TY:dD cr
frac{partialphi}{partial D} &= 2X^TY cr
}$$
NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {rm Tr}(B^TC)$$
answered 2 days ago
greggreg
8,7751824
$endgroup$
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
add a comment |
$begingroup$
Define the matrices
$$eqalign{
X &= Bcirc C,quad Y=XD-A cr
}$$
Write the function in terms of these new variables.
Then find its differentials and gradients (in alphabetical order).
$$eqalign{
phi &= |Y|_F^2 = Y:Y cr
dphi &= 2Y:dY cr
&= 2Y:dX,D cr
&= 2YD^T:Ccirc dB cr
&= 2Ccirc(YD^T):dB cr
frac{partialphi}{partial B} &= 2Ccirc(YD^T) cr
}$$
Since $B$ and $C$ are indistinguishable
$$eqalign{
frac{partialphi}{partial C} &= 2Bcirc(YD^T) cr
}$$
Finally, the gradient wrt $D$
$$eqalign{
dphi &= 2Y:X,dD cr
&= 2X^TY:dD cr
frac{partialphi}{partial D} &= 2X^TY cr
}$$
NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {rm Tr}(B^TC)$$
answered 2 days ago
greggreg
8,7751824
$endgroup$
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
add a comment |
$begingroup$
Define the matrices
$$eqalign{
X &= Bcirc C,quad Y=XD-A cr
}$$
Write the function in terms of these new variables.
Then find its differentials and gradients (in alphabetical order).
$$eqalign{
phi &= |Y|_F^2 = Y:Y cr
dphi &= 2Y:dY cr
&= 2Y:dX,D cr
&= 2YD^T:Ccirc dB cr
&= 2Ccirc(YD^T):dB cr
frac{partialphi}{partial B} &= 2Ccirc(YD^T) cr
}$$
Since $B$ and $C$ are indistinguishable
$$eqalign{
frac{partialphi}{partial C} &= 2Bcirc(YD^T) cr
}$$
Finally, the gradient wrt $D$
$$eqalign{
dphi &= 2Y:X,dD cr
&= 2X^TY:dD cr
frac{partialphi}{partial D} &= 2X^TY cr
}$$
NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {rm Tr}(B^TC)$$
answered 2 days ago
greggreg
8,7751824
$endgroup$
Define the matrices
$$eqalign{
X &= Bcirc C,quad Y=XD-A cr
}$$
Write the function in terms of these new variables.
Then find its differentials and gradients (in alphabetical order).
$$eqalign{
phi &= |Y|_F^2 = Y:Y cr
dphi &= 2Y:dY cr
&= 2Y:dX,D cr
&= 2YD^T:Ccirc dB cr
&= 2Ccirc(YD^T):dB cr
frac{partialphi}{partial B} &= 2Ccirc(YD^T) cr
}$$
Since $B$ and $C$ are indistinguishable
$$eqalign{
frac{partialphi}{partial C} &= 2Bcirc(YD^T) cr
}$$
Finally, the gradient wrt $D$
$$eqalign{
dphi &= 2Y:X,dD cr
&= 2X^TY:dD cr
frac{partialphi}{partial D} &= 2X^TY cr
}$$
NB: In several of the steps above, a colon is used as a convenient product notation for the trace, i.e. $$B:C = {rm Tr}(B^TC)$$
answered 2 days ago
greggreg
8,7751824
answered 2 days ago
greggreg
8,7751824
answered 2 days ago
greggreg
8,7751824
answered 2 days ago
greggreg
8,7751824
8,7751824
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
add a comment |
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
$begingroup$
Thank you so much for your answer. It is very clear and understandable. This helps me a lot.
$endgroup$
– Rio Li
yesterday
add a comment |
Rio Li is a new contributor. Be nice, and check out our Code of Conduct.
Rio Li is a new contributor. Be nice, and check out our Code of Conduct.
Rio Li is a new contributor. Be nice, and check out our Code of Conduct.
Rio Li is a new contributor. Be nice, and check out our Code of Conduct.
Thanks for contributing an answer to Mathematics Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
Use MathJax to format equations. MathJax reference.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3140877%2fmatrix-differentiation-involving-hadamard-products%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
