Dtjytyk

Would a high gravity rocky planet be guaranteed to have an atmosphere?

How to run a prison with the smallest amount of guards?

Roman Numeral Treatment of Suspensions

Increase performance creating Mandelbrot set in python

Hostile work environment after whistle-blowing on coworker and our boss. What do I do?

Term for the "extreme-extension" version of a straw man fallacy?

Type int? vs type int

Is oxalic acid dihydrate considered a primary acid standard in analytical chemistry?

Method to test if a number is a perfect power?

Why didn't Theresa May consult with Parliament before negotiating a deal with the EU?

What is the best translation for "slot" in the context of multiplayer video games?

How do we know the LHC results are robust?

How to pronounce the slash sign

How to Reset Passwords on Multiple Websites Easily?

Detecting if an element is found inside a container

when is out of tune ok?

How do scammers retract money, while you can’t?

Is this apparent Class Action settlement a spam message?

What is the opposite of 'gravitas'?

Trouble understanding the speech of overseas colleagues

Lay out the Carpet

Class Action - which options I have?

Anatomically Correct Strange Women In Ponds Distributing Swords

Avoiding estate tax by giving multiple gifts

Confused about chain rule in Frechet derivative

Matrix derivative of $Tr(Alog(X))$Frechet derivative of shift operator in $l_2$?Derivative of the inverse of a symmetric matrixFrechet Derivative of operatorChain rule for 3rd derivative, multiple variablesChain Rule: Derivative of Squared Mahalanobis DistanceDoes the chain rule need to account for derivatives as test functions?Derivative of pseudoinverse with respect to original matrixChain rule for $f(x_3(w, x_2(w, x_1(w, x_0))))$Derivative of von Neumann entropyDerivative of log marginal likelihood

0

$begingroup$

I've recently started to learn about Frechet derivatives and now have a simple example which I'm not sure if I've solved correctly. To be honest, I've only got a poor understanding of how it works so please assume I am starting at zero. Given $delta = sum_i lambda_iX_i$ for some matrices $X_i$ and real numbers $lambda_i$ and $N(X)$ being a linear transformation on $X$, find the Frechet derivative

$$frac{partial}{partial lambda_l}Tr(Alog(N(delta)),$$

for some $l$.

---

Redoing my attempted solution after reading greg's answer. Would appreciate if anyone can comment on the correctness of it!

We have
begin{align}
d Tr(Alog(N(delta)) &= Tr(dAlog(N(delta)) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)dN(delta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(ddelta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(X_k dlambda_k)right) \
end{align}

I have used the fact that $Tr$ and $N()$ are both linear operators and the solution posted here to write the function $d (Alog X)$. I believe that this cannot proceed further unless I can say something about $N()$ to rewrite $N(X_k dlambda_k)$ as $ M(X_k) dlambda_k$. If and only if I could do that step, then I have

begin{align}
d Tr(Alog(N(delta)) &= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k) dlambda_kright)\
frac{partial}{partiallambda_k}Tr(Alog(N(delta)) &= left(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k)right)^T
end{align}

matrix-calculus chain-rule frechet-derivative

share|cite|improve this question

edited Mar 18 at 10:58

user1936752

asked Mar 15 at 16:13

user1936752user1936752

5841515

$endgroup$

add a comment | 

0

$begingroup$

I've recently started to learn about Frechet derivatives and now have a simple example which I'm not sure if I've solved correctly. To be honest, I've only got a poor understanding of how it works so please assume I am starting at zero. Given $delta = sum_i lambda_iX_i$ for some matrices $X_i$ and real numbers $lambda_i$ and $N(X)$ being a linear transformation on $X$, find the Frechet derivative

$$frac{partial}{partial lambda_l}Tr(Alog(N(delta)),$$

for some $l$.

---

Redoing my attempted solution after reading greg's answer. Would appreciate if anyone can comment on the correctness of it!

We have
begin{align}
d Tr(Alog(N(delta)) &= Tr(dAlog(N(delta)) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)dN(delta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(ddelta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(X_k dlambda_k)right) \
end{align}

I have used the fact that $Tr$ and $N()$ are both linear operators and the solution posted here to write the function $d (Alog X)$. I believe that this cannot proceed further unless I can say something about $N()$ to rewrite $N(X_k dlambda_k)$ as $ M(X_k) dlambda_k$. If and only if I could do that step, then I have

begin{align}
d Tr(Alog(N(delta)) &= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k) dlambda_kright)\
frac{partial}{partiallambda_k}Tr(Alog(N(delta)) &= left(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k)right)^T
end{align}

matrix-calculus chain-rule frechet-derivative

share|cite|improve this question

edited Mar 18 at 10:58

user1936752

asked Mar 15 at 16:13

user1936752user1936752

5841515

$endgroup$

add a comment | 

$begingroup$

I've recently started to learn about Frechet derivatives and now have a simple example which I'm not sure if I've solved correctly. To be honest, I've only got a poor understanding of how it works so please assume I am starting at zero. Given $delta = sum_i lambda_iX_i$ for some matrices $X_i$ and real numbers $lambda_i$ and $N(X)$ being a linear transformation on $X$, find the Frechet derivative

$$frac{partial}{partial lambda_l}Tr(Alog(N(delta)),$$

for some $l$.

---

Redoing my attempted solution after reading greg's answer. Would appreciate if anyone can comment on the correctness of it!

We have
begin{align}
d Tr(Alog(N(delta)) &= Tr(dAlog(N(delta)) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)dN(delta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(ddelta)right) \
&= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)N(X_k dlambda_k)right) \
end{align}

I have used the fact that $Tr$ and $N()$ are both linear operators and the solution posted here to write the function $d (Alog X)$. I believe that this cannot proceed further unless I can say something about $N()$ to rewrite $N(X_k dlambda_k)$ as $ M(X_k) dlambda_k$. If and only if I could do that step, then I have

begin{align}
d Tr(Alog(N(delta)) &= Trleft(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k) dlambda_kright)\
frac{partial}{partiallambda_k}Tr(Alog(N(delta)) &= left(left(int_0^{infty}frac{1}{1+tN(delta)}Amathrm{d}tfrac{1}{1+tN(delta)}right)M(X_k)right)^T
end{align}

matrix-calculus chain-rule frechet-derivative

share|cite|improve this question

edited Mar 18 at 10:58

user1936752

asked Mar 15 at 16:13

user1936752user1936752

5841515

$endgroup$

share|cite|improve this question

edited Mar 18 at 10:58

user1936752

asked Mar 15 at 16:13

user1936752user1936752

5841515

share|cite|improve this question

edited Mar 18 at 10:58

user1936752

asked Mar 15 at 16:13

user1936752user1936752

5841515

asked Mar 15 at 16:13

user1936752user1936752

5841515

asked Mar 15 at 16:13

user1936752user1936752

5841515

1 Answer
1

active

oldest

votes

1

$begingroup$

Define the matrix
$$eqalign{
Y &= sum_{i=k}^N lambda_kX_k = lambda_kX_k cr
}$$
where the expression on the far RHS uses the index summation convention.

Now calculate the differential and derivative for the trace of a simple nonlinear function.
$$eqalign{
phi &= {rm Tr}Big(Y^3Big) cr
dphi
&= {rm Tr}Big(3Y^2,dYBig) = {rm Tr}Big(3Y^2X_k,dlambda_kBig) cr
frac{dphi}{dlambda_k} &= {rm Tr}Big(3Y^2X_kBig) cr
}$$
A general function $f(Y)$ follows the same pattern.
$$eqalign{
psi &= {rm Tr}Big(f(Y)Big) cr
frac{dpsi}{dlambda_k} &= {rm Tr}Big(f'(Y),X_kBig) cr
}$$
where $f'$ denotes the ordinary derivative of the function.

share|cite|improve this answer

answered Mar 15 at 20:51

greggreg

8,9951824

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for this answer. Using this and your comments on my other question, I've redone the solution. I believe it can only proceed if I make an assumption about $N()$, as I have clarified in the edited question. If you wouldn't mind taking a moment to read it, can I check that this is the correct solution to the original problem I posed?
  $endgroup$
  – user1936752
  Mar 17 at 22:23
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  It looks correct, I'm just not sure how practical it will prove to be. Evaluating an improper integral of a matrix-valued function is extremely tricky. If you just need a closed-form result, is an integral expression more useful than the original derivative expression?
  $endgroup$
  – greg
  Mar 19 at 0:32
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yeah, that's a good point. Thanks for the help :)
  $endgroup$
  – user1936752
  Mar 19 at 10:56
  

  
  
  
  

add a comment | 

Your Answer

StackExchange.ifUsing("editor", function () {
return StackExchange.using("mathjaxEditing", function () {
StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
});
});
}, "mathjax-editing");

StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "69"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
noCode: true, onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});

draft saved

draft discarded

Sign up or log in

StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});

Sign up using Google

Sign up using Facebook

Sign up using Email and Password

Post as a guest

Name

Email

Required, but never shown

StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f3149484%2fconfused-about-chain-rule-in-frechet-derivative%23new-answer', 'question_page');
}
);

Post as a guest

Name

Email

Required, but never shown

1

$begingroup$

Define the matrix
$$eqalign{
Y &= sum_{i=k}^N lambda_kX_k = lambda_kX_k cr
}$$
where the expression on the far RHS uses the index summation convention.

Now calculate the differential and derivative for the trace of a simple nonlinear function.
$$eqalign{
phi &= {rm Tr}Big(Y^3Big) cr
dphi
&= {rm Tr}Big(3Y^2,dYBig) = {rm Tr}Big(3Y^2X_k,dlambda_kBig) cr
frac{dphi}{dlambda_k} &= {rm Tr}Big(3Y^2X_kBig) cr
}$$
A general function $f(Y)$ follows the same pattern.
$$eqalign{
psi &= {rm Tr}Big(f(Y)Big) cr
frac{dpsi}{dlambda_k} &= {rm Tr}Big(f'(Y),X_kBig) cr
}$$
where $f'$ denotes the ordinary derivative of the function.

share|cite|improve this answer

answered Mar 15 at 20:51

greggreg

8,9951824

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for this answer. Using this and your comments on my other question, I've redone the solution. I believe it can only proceed if I make an assumption about $N()$, as I have clarified in the edited question. If you wouldn't mind taking a moment to read it, can I check that this is the correct solution to the original problem I posed?
  $endgroup$
  – user1936752
  Mar 17 at 22:23
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  It looks correct, I'm just not sure how practical it will prove to be. Evaluating an improper integral of a matrix-valued function is extremely tricky. If you just need a closed-form result, is an integral expression more useful than the original derivative expression?
  $endgroup$
  – greg
  Mar 19 at 0:32
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yeah, that's a good point. Thanks for the help :)
  $endgroup$
  – user1936752
  Mar 19 at 10:56
  

  
  
  
  

add a comment | 

1

$begingroup$

Define the matrix
$$eqalign{
Y &= sum_{i=k}^N lambda_kX_k = lambda_kX_k cr
}$$
where the expression on the far RHS uses the index summation convention.

Now calculate the differential and derivative for the trace of a simple nonlinear function.
$$eqalign{
phi &= {rm Tr}Big(Y^3Big) cr
dphi
&= {rm Tr}Big(3Y^2,dYBig) = {rm Tr}Big(3Y^2X_k,dlambda_kBig) cr
frac{dphi}{dlambda_k} &= {rm Tr}Big(3Y^2X_kBig) cr
}$$
A general function $f(Y)$ follows the same pattern.
$$eqalign{
psi &= {rm Tr}Big(f(Y)Big) cr
frac{dpsi}{dlambda_k} &= {rm Tr}Big(f'(Y),X_kBig) cr
}$$
where $f'$ denotes the ordinary derivative of the function.

share|cite|improve this answer

answered Mar 15 at 20:51

greggreg

8,9951824

$endgroup$

• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Thank you for this answer. Using this and your comments on my other question, I've redone the solution. I believe it can only proceed if I make an assumption about $N()$, as I have clarified in the edited question. If you wouldn't mind taking a moment to read it, can I check that this is the correct solution to the original problem I posed?
  $endgroup$
  – user1936752
  Mar 17 at 22:23
  

  
  
  
  


• 
  
  
  

  
  1
  

  
  
  
  
  
  

  
  $begingroup$
  It looks correct, I'm just not sure how practical it will prove to be. Evaluating an improper integral of a matrix-valued function is extremely tricky. If you just need a closed-form result, is an integral expression more useful than the original derivative expression?
  $endgroup$
  – greg
  Mar 19 at 0:32
  

  
  
  
  


• 
  
  
  

  
  

  
  
  
  
  
  

  
  $begingroup$
  Yeah, that's a good point. Thanks for the help :)
  $endgroup$
  – user1936752
  Mar 19 at 10:56
  

  
  
  
  

add a comment | 

$begingroup$

Define the matrix
$$eqalign{
Y &= sum_{i=k}^N lambda_kX_k = lambda_kX_k cr
}$$
where the expression on the far RHS uses the index summation convention.

Now calculate the differential and derivative for the trace of a simple nonlinear function.
$$eqalign{
phi &= {rm Tr}Big(Y^3Big) cr
dphi
&= {rm Tr}Big(3Y^2,dYBig) = {rm Tr}Big(3Y^2X_k,dlambda_kBig) cr
frac{dphi}{dlambda_k} &= {rm Tr}Big(3Y^2X_kBig) cr
}$$
A general function $f(Y)$ follows the same pattern.
$$eqalign{
psi &= {rm Tr}Big(f(Y)Big) cr
frac{dpsi}{dlambda_k} &= {rm Tr}Big(f'(Y),X_kBig) cr
}$$
where $f'$ denotes the ordinary derivative of the function.

share|cite|improve this answer

answered Mar 15 at 20:51

greggreg

8,9951824

$endgroup$

share|cite|improve this answer

answered Mar 15 at 20:51

greggreg

8,9951824

answered Mar 15 at 20:51

greggreg

8,9951824

answered Mar 15 at 20:51

greggreg

8,9951824