Reference for matrix calculusWhere can I find a broad set of exercises on Matrix calculus?Good introductory...

How to find program name(s) of an installed package?

Can I make popcorn with any corn?

"to be prejudice towards/against someone" vs "to be prejudiced against/towards someone"

Fencing style for blades that can attack from a distance

How can bays and straits be determined in a procedurally generated map?

What are the differences between the usage of 'it' and 'they'?

Mathematical cryptic clues

Has the BBC provided arguments for saying Brexit being cancelled is unlikely?

Can divisibility rules for digits be generalized to sum of digits

Why are electrically insulating heatsinks so rare? Is it just cost?

What does it mean to describe someone as a butt steak?

Example of a continuous function that don't have a continuous extension

In Japanese, what’s the difference between “Tonari ni” (となりに) and “Tsugi” (つぎ)? When would you use one over the other?

Maximum likelihood parameters deviate from posterior distributions

Do I have a twin with permutated remainders?

Why not use SQL instead of GraphQL?

What typically incentivizes a professor to change jobs to a lower ranking university?

Why Is Death Allowed In the Matrix?

Why are 150k or 200k jobs considered good when there are 300k+ births a month?

Risk of getting Chronic Wasting Disease (CWD) in the United States?

How old can references or sources in a thesis be?

TGV timetables / schedules?

What do you call a Matrix-like slowdown and camera movement effect?

Why do I get two different answers for this counting problem?



Reference for matrix calculus


Where can I find a broad set of exercises on Matrix calculus?Good introductory book for matrix calculusCorrect method to differentiate a first order vector/matrixBook Reference for Calculus and Linear Algebra :: EngineerFormal Variational Calculus Reference RequestReference for Hasse-Witt invariantJoseph Kitchen's Calculus (reference)Reference Request for CalculusInteresting calculus problems for beginnerAdvanced / In-Depth Calculus Book for Self-EdificationReference Book for CalculusSuitable reference for learning symplectic geometryWhich course for deep learning?













3












$begingroup$


Could someone provide a good reference for learning matrix calculus? I've recently moved to a more engineering-oriented field where it's commonly used and don't have much experience with it.






calculus matrices reference-request







share|cite|improve this question









$endgroup$








  • 1




    $begingroup$
    Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
    $endgroup$
    – Marek
    Nov 11 '10 at 19:46






  • 1




    $begingroup$
    Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
    $endgroup$
    – Simon
    Nov 11 '10 at 19:55


















3












$begingroup$


Could someone provide a good reference for learning matrix calculus? I've recently moved to a more engineering-oriented field where it's commonly used and don't have much experience with it.






calculus matrices reference-request







share|cite|improve this question









$endgroup$








  • 1




    $begingroup$
    Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
    $endgroup$
    – Marek
    Nov 11 '10 at 19:46






  • 1




    $begingroup$
    Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
    $endgroup$
    – Simon
    Nov 11 '10 at 19:55
















3












3








3


6



$begingroup$


Could someone provide a good reference for learning matrix calculus? I've recently moved to a more engineering-oriented field where it's commonly used and don't have much experience with it.






calculus matrices reference-request







share|cite|improve this question









$endgroup$




Could someone provide a good reference for learning matrix calculus? I've recently moved to a more engineering-oriented field where it's commonly used and don't have much experience with it.






calculus matrices reference-request




calculus matrices reference-request






share|cite|improve this question













share|cite|improve this question











share|cite|improve this question




share|cite|improve this question










asked Nov 11 '10 at 19:34









SimonSimon

599414




599414








  • 1




    $begingroup$
    Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
    $endgroup$
    – Marek
    Nov 11 '10 at 19:46






  • 1




    $begingroup$
    Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
    $endgroup$
    – Simon
    Nov 11 '10 at 19:55
















  • 1




    $begingroup$
    Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
    $endgroup$
    – Marek
    Nov 11 '10 at 19:46






  • 1




    $begingroup$
    Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
    $endgroup$
    – Simon
    Nov 11 '10 at 19:55










1




1




$begingroup$
Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
$endgroup$
– Marek
Nov 11 '10 at 19:46




$begingroup$
Could you be a bit more specific? What exactly are you trying to learn and what do you already know? Do you want to understand things such as A = exp(Ct) solves dA/dt = CA for A, C matrices? Or perhaps even study matrix Lie groups and their differential properties (I can imagine this could have many engineering applications)?
$endgroup$
– Marek
Nov 11 '10 at 19:46




1




1




$begingroup$
Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
$endgroup$
– Simon
Nov 11 '10 at 19:55






$begingroup$
Well, in linear regression, one can derive a closed-form solution in the form of the normal equations: en.wikipedia.org/wiki/Normal_equations#General_linear_model . The derivation uses results about the gradient of the trace of a matrix product. I guess I just wanted to feel comfortable with these ideas - maybe build up a geometric picture and get some insight as to how everything works.
$endgroup$
– Simon
Nov 11 '10 at 19:55












2 Answers
2






active

oldest

votes


















6












$begingroup$

Actually the books cited above by Sivaram are excellent for numerical stuff. If you want "matrix calculus" then the following books might be helpful:




  1. Matrix Differential Calculus with Applications in Statistics and Econometrics by Magnus and Neudecker


  2. Functions of Matrices by N. Higham


  3. Calculus on Manifolds by Spivak



Some classic, but very useful material can also be found in




  1. Introduction to Matrix Analysis by Bellman.


As a simple example, the books will teach (unless you already know it) how to compute, say, the derivative of $f(X) = logdet(X)$ for an invertible matrix $X$.






share|cite|improve this answer









$endgroup$





















    1












    $begingroup$

    Gene Howard Golub, Charles F. Van Loan book on "Matrix Computations" is regarded as the "Bhagavad Gita" for Matrix Algorithms.



    http://books.google.com/books?id=mlOa7wPX6OYC&printsec=frontcover



    There is also another book by "Gene Howard Golub, Gerard Meurant" on "Matrices, moments, and quadrature with applications".



    http://books.google.com/books?id=IZvkFET3LlwC&printsec=frontcover



    Also, "Numerical Linear Algebra" by Trefethen and Bau is well-written and easy to read.



    http://books.google.com/books?id=bj-Lu6zjWbEC&printsec=frontcover



    I would highly recommend Trefethen and Bau since I have read it completely. I feel it is ideal for self-study or for a one quarter course. Once you are done with this you can take a look at Golubs' book. Golubs' book is really good for reference.






    share|cite|improve this answer











    $endgroup$














      Your Answer





      StackExchange.ifUsing("editor", function () {
      return StackExchange.using("mathjaxEditing", function () {
      StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix) {
      StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["$", "$"], ["\\(","\\)"]]);
      });
      });
      }, "mathjax-editing");

      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "69"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      noCode: true, onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f9891%2freference-for-matrix-calculus%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      6












      $begingroup$

      Actually the books cited above by Sivaram are excellent for numerical stuff. If you want "matrix calculus" then the following books might be helpful:




      1. Matrix Differential Calculus with Applications in Statistics and Econometrics by Magnus and Neudecker


      2. Functions of Matrices by N. Higham


      3. Calculus on Manifolds by Spivak



      Some classic, but very useful material can also be found in




      1. Introduction to Matrix Analysis by Bellman.


      As a simple example, the books will teach (unless you already know it) how to compute, say, the derivative of $f(X) = logdet(X)$ for an invertible matrix $X$.






      share|cite|improve this answer









      $endgroup$


















        6












        $begingroup$

        Actually the books cited above by Sivaram are excellent for numerical stuff. If you want "matrix calculus" then the following books might be helpful:




        1. Matrix Differential Calculus with Applications in Statistics and Econometrics by Magnus and Neudecker


        2. Functions of Matrices by N. Higham


        3. Calculus on Manifolds by Spivak



        Some classic, but very useful material can also be found in




        1. Introduction to Matrix Analysis by Bellman.


        As a simple example, the books will teach (unless you already know it) how to compute, say, the derivative of $f(X) = logdet(X)$ for an invertible matrix $X$.






        share|cite|improve this answer









        $endgroup$
















          6












          6








          6





          $begingroup$

          Actually the books cited above by Sivaram are excellent for numerical stuff. If you want "matrix calculus" then the following books might be helpful:




          1. Matrix Differential Calculus with Applications in Statistics and Econometrics by Magnus and Neudecker


          2. Functions of Matrices by N. Higham


          3. Calculus on Manifolds by Spivak



          Some classic, but very useful material can also be found in




          1. Introduction to Matrix Analysis by Bellman.


          As a simple example, the books will teach (unless you already know it) how to compute, say, the derivative of $f(X) = logdet(X)$ for an invertible matrix $X$.






          share|cite|improve this answer









          $endgroup$



          Actually the books cited above by Sivaram are excellent for numerical stuff. If you want "matrix calculus" then the following books might be helpful:




          1. Matrix Differential Calculus with Applications in Statistics and Econometrics by Magnus and Neudecker


          2. Functions of Matrices by N. Higham


          3. Calculus on Manifolds by Spivak



          Some classic, but very useful material can also be found in




          1. Introduction to Matrix Analysis by Bellman.


          As a simple example, the books will teach (unless you already know it) how to compute, say, the derivative of $f(X) = logdet(X)$ for an invertible matrix $X$.







          share|cite|improve this answer












          share|cite|improve this answer



          share|cite|improve this answer










          answered Nov 12 '10 at 9:53







          user1709






























              1












              $begingroup$

              Gene Howard Golub, Charles F. Van Loan book on "Matrix Computations" is regarded as the "Bhagavad Gita" for Matrix Algorithms.



              http://books.google.com/books?id=mlOa7wPX6OYC&printsec=frontcover



              There is also another book by "Gene Howard Golub, Gerard Meurant" on "Matrices, moments, and quadrature with applications".



              http://books.google.com/books?id=IZvkFET3LlwC&printsec=frontcover



              Also, "Numerical Linear Algebra" by Trefethen and Bau is well-written and easy to read.



              http://books.google.com/books?id=bj-Lu6zjWbEC&printsec=frontcover



              I would highly recommend Trefethen and Bau since I have read it completely. I feel it is ideal for self-study or for a one quarter course. Once you are done with this you can take a look at Golubs' book. Golubs' book is really good for reference.






              share|cite|improve this answer











              $endgroup$


















                1












                $begingroup$

                Gene Howard Golub, Charles F. Van Loan book on "Matrix Computations" is regarded as the "Bhagavad Gita" for Matrix Algorithms.



                http://books.google.com/books?id=mlOa7wPX6OYC&printsec=frontcover



                There is also another book by "Gene Howard Golub, Gerard Meurant" on "Matrices, moments, and quadrature with applications".



                http://books.google.com/books?id=IZvkFET3LlwC&printsec=frontcover



                Also, "Numerical Linear Algebra" by Trefethen and Bau is well-written and easy to read.



                http://books.google.com/books?id=bj-Lu6zjWbEC&printsec=frontcover



                I would highly recommend Trefethen and Bau since I have read it completely. I feel it is ideal for self-study or for a one quarter course. Once you are done with this you can take a look at Golubs' book. Golubs' book is really good for reference.






                share|cite|improve this answer











                $endgroup$
















                  1












                  1








                  1





                  $begingroup$

                  Gene Howard Golub, Charles F. Van Loan book on "Matrix Computations" is regarded as the "Bhagavad Gita" for Matrix Algorithms.



                  http://books.google.com/books?id=mlOa7wPX6OYC&printsec=frontcover



                  There is also another book by "Gene Howard Golub, Gerard Meurant" on "Matrices, moments, and quadrature with applications".



                  http://books.google.com/books?id=IZvkFET3LlwC&printsec=frontcover



                  Also, "Numerical Linear Algebra" by Trefethen and Bau is well-written and easy to read.



                  http://books.google.com/books?id=bj-Lu6zjWbEC&printsec=frontcover



                  I would highly recommend Trefethen and Bau since I have read it completely. I feel it is ideal for self-study or for a one quarter course. Once you are done with this you can take a look at Golubs' book. Golubs' book is really good for reference.






                  share|cite|improve this answer











                  $endgroup$



                  Gene Howard Golub, Charles F. Van Loan book on "Matrix Computations" is regarded as the "Bhagavad Gita" for Matrix Algorithms.



                  http://books.google.com/books?id=mlOa7wPX6OYC&printsec=frontcover



                  There is also another book by "Gene Howard Golub, Gerard Meurant" on "Matrices, moments, and quadrature with applications".



                  http://books.google.com/books?id=IZvkFET3LlwC&printsec=frontcover



                  Also, "Numerical Linear Algebra" by Trefethen and Bau is well-written and easy to read.



                  http://books.google.com/books?id=bj-Lu6zjWbEC&printsec=frontcover



                  I would highly recommend Trefethen and Bau since I have read it completely. I feel it is ideal for self-study or for a one quarter course. Once you are done with this you can take a look at Golubs' book. Golubs' book is really good for reference.







                  share|cite|improve this answer














                  share|cite|improve this answer



                  share|cite|improve this answer








                  edited Nov 11 '10 at 19:44









                  Aryabhata

                  70.2k6157247




                  70.2k6157247










                  answered Nov 11 '10 at 19:41







                  user17762





































                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Mathematics Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid …



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      Use MathJax to format equations. MathJax reference.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fmath.stackexchange.com%2fquestions%2f9891%2freference-for-matrix-calculus%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      -

                      Popular posts from this blog

                      Magento 2 - Add success message with knockout Planned maintenance scheduled April 23, 2019 at 23:30 UTC (7:30pm US/Eastern) Announcing the arrival of Valued Associate #679: Cesar Manara Unicorn Meta Zoo #1: Why another podcast?Success / Error message on ajax request$.widget is not a function when loading a homepage after add custom jQuery on custom themeHow can bind jQuery to current document in Magento 2 When template load by ajaxRedirect page using plugin in Magento 2Magento 2 - Update quantity and totals of cart page without page reload?Magento 2: Quote data not loaded on knockout checkoutMagento 2 : I need to change add to cart success message after adding product into cart through pluginMagento 2.2.5 How to add additional products to cart from new checkout step?Magento 2 Add error/success message with knockoutCan't validate Post Code on checkout page

                      Image
                      What were wait-states, and why was it only an issue for PCs? “Since the train was delayed for more than an hour, passengers were given a full refund.” – Why is there no article before “passengers”? Will I be more secure with my own router behind my ISP's router? Short story about an alien named Ushtu(?) coming from a future Earth, when ours was destroyed by a nuclear explosion When does Bran Stark remember Jamie pushing him? How to charge percentage of transaction cost? Pointing to problems without suggesting solutions false 'Security alert' from Google - every login generates mails from 'no-reply@accounts.google.com' Why doesn't the university give past final exams' answers? Like totally amazing interchangeable sister outfit accessory swapping or whatever Does traveling In The United States require a passport or can I use my green card if not a US citizen? Who's this lady in the war room? Who can become a wight? "Destructi
                      Read more

                      Fil:Tokke komm.svg

                      Image
                      Skildring Skildring Tokke komm.svg Norsk nynorsk: Våpen for Tokke kommune i Telemark fylke, godkjent ved kongeleg resolusjon 6. februar 1987. Motiv: På gull grunn ein gåande svart bjørn. [1] Tokke ligg i eit gamalt bjørneområde og bjørnen er eit viktig innslag i fleire lokale historier. Bjørnen kan òg symbolisere naturen og dyrelivet i kommunen. English: Coat of arms of the Norwegian municipality of Tokke, Telemark. Dato 6. februar 1987 Kjelde Original by Stein Davidsen, based upon image from caplex.no. Opphavsperson Eige arbeid Vektorgrafikken vart laga med Inkscape. Løyve (Gjenbruk av denne fila) Public domain Public domain false false This official Norwegian coat of arms is ineligible for copyright, but its usage is restricted according to Norwegian penal law, independent of the copyright status of the depiction shown here. Dette biletet er sett saman av eit flagg, eit våpen, eit sigill eller eit anna offisielt insignium . Bruken av slike symbol er avgrensa i fleire land. Desse r
                      Read more

                      Where did Arya get these scars? Unicorn Meta Zoo #1: Why another podcast? Announcing the arrival of Valued Associate #679: Cesar Manara Favourite questions and answers from the 1st quarter of 2019Why did Arya refuse to end it?Has the pronunciation of Arya Stark's name changed?Has Arya forgiven people?Why did Arya Stark lose her vision?Why can Arya still use the faces?Has the Narrow Sea become narrower?Does Arya Stark know how to make poisons outside of the House of Black and White?Why did Nymeria leave Arya?Why did Arya not kill the Lannister soldiers she encountered in the Riverlands?What is the current canonical age of Sansa, Bran and Arya Stark?

                      Image
                      How to translate "red flag" into Spanish? France's Public Holidays' Puzzle How did Elite on the NES work? Where can I find how to tex symbols for different fonts? In search of the origins of term censor, I hit a dead end stuck with the greek term, to censor, λογοκρίνω How to check if string is entirely made of same substring? What's called a person who works as someone who puts products on shelves in stores? What helicopter has the most rotor blades? Did war bonds have better investment alternatives during WWII? Page Layouts : 1 column , 2 columns-left , 2 columns-right , 3 column Marquee sign letters Why did Israel vote against lifting the American embargo on Cuba? What *exactly* is electrical current, voltage, and resistance? Mechanism of the formation of peracetic acid Does Prince Arnaud cause someone holding the Princess to lose? Bright yellow or light yellow? What is /etc/mtab in Linux? What is the ongoing value of the Kanban
                      Read more