Useful Identities of Computing Gradients

∂∂Xf(X)T=(∂f(X)∂X)T\frac{\partial}{\partial X} f(X)^T =(\frac{\partial f(X)}{\partial X})^T ∂∂Xtr(f(X))=tr(∂f(X)∂X)\frac{\partial}{\partial X} tr(f(X)) =tr (\frac{\partial f(X)}{\partial X}) ∂∂Xdet⁡(f(X))=det⁡(f(X))tr(f(X)−1∂f(X)∂X)\frac{\partial}{\partial X} \det (f(X)) =\det(f(X)) tr(f(X) ^{-1}\frac{\partial f(X)}{\partial X}) ∂∂Xf(X)−1=−f(X)−1∂f(X)∂Xf(X)−1\frac{\partial}{\partial X} f(X)^{-1} =-f(X)^{-1} \frac{\partial f(X)}{\partial X} f(X)^-1 ∂aTX−1b∂X=−(X−1)TabT(X−1)T\frac{\partial a^TX^{-1}b}{\partial X} = - (X^{-1})^Tab^T(X^{-1})^T ∂xTa∂x=aT\frac{\partial x^T a}{\partial x} =a^T ∂aTx∂x=aT\frac{\partial a^Tx}{\partial x} =a^T ∂aTXb∂X=aTb\frac{\partial a^TXb}{\partial X} = a^Tb ∂xTBx∂X=xT(BT+B)\frac{\partial x^T B x}{\partial X} = x^T(B^T + B) ∂∂s(x−As)TW(x−As)=−2(x−As)TWA for symmetric matrix W\frac{\partial}{\partial s}(x - As)^T W (x - As) = - 2(x - As)^T W A \text{ for symmetric matrix W}