主要是关于矩阵的求导。

1. ∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y​ [ y y y是标量， x \mathbf{x} x是列向量，导数是行向量]
   x = [ x 1 x 2 ⋮ x n ] , ∂ y ∂ x = [ ∂ y ∂ x 1 , ∂ y ∂ x 2 , ⋯   , ∂ y ∂ x n ] \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \quad \frac{\partial y}{\partial \mathbf{x}} = [\frac{\partial y}{\partial x_1}, \frac{\partial y}{\partial x_2}, \cdots, \frac{\partial y}{\partial x_n}] x=⎣⎢⎢⎢⎡​x1​x2​⋮xn​​⎦⎥⎥⎥⎤​,∂x∂y​=[∂x1​∂y​,∂x2​∂y​,⋯,∂xn​∂y​]
   例如： ∂ ∂ x x 1 2 + 2 x 2 2 = [ 2 x 1 , 4 x 2 ] \frac{\partial}{\partial \mathbf{x}}x_1^2 + 2x_2^2 = [2x_1, 4x_2 ] ∂x∂​x12​+2x22​=[2x1​,4x2​]

   一些样例

   y y y	a a a	a u au au	sum ( x ) \text{sum}(x) sum(x)	∥ x ∥ 2 \|\mathbf{x}\|^2 ∥x∥2
   ∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y​	0 T \mathbf{0}^T 0T	a ∂ u ∂ x a\frac{\partial u}{\partial \mathbf{x}} a∂x∂u​	1 T \mathbf{1}^T 1T	2 x T 2\mathbf{x}^T 2xT

   y y y	u + v u+v u+v	u v uv uv	< u , v > <\mathbf{u}, \mathbf{v}> <u,v>
   ∂ y ∂ x \frac{\partial y}{\partial \mathbf{x}} ∂x∂y​	∂ u ∂ x + ∂ v ∂ x \frac{\partial u}{\partial \mathbf{x}} + \frac{\partial v}{\partial \mathbf{x}} ∂x∂u​+∂x∂v​	∂ u ∂ x v + ∂ v ∂ x u \frac{\partial u}{\partial \mathbf{x}} v+ \frac{\partial v}{\partial \mathbf{x}} u ∂x∂u​v+∂x∂v​u	u T ∂ v ∂ x + v T ∂ u ∂ x \mathbf{u}^T \frac{\partial \mathbf{v}}{\partial \mathbf{x}} + \mathbf{v}^T \frac{\partial \mathbf{u}}{\partial \mathbf{x}} uT∂x∂v​+vT∂x∂u​

2. ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial x} ∂x∂y​ [ y \mathbf{y} y是列向量， x x x是标量，导数是列向量]
   y = [ y 1 y 2 ⋮ y m ] , ∂ y ∂ x = [ ∂ y 1 x ∂ y 2 x ⋮ ∂ y m x ] \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{bmatrix}, \quad \frac{\partial \mathbf{y}}{\partial x} = \begin{bmatrix} \frac{\partial y_1}{x} \\ \frac{\partial y_2}{x} \\ \vdots \\ \frac{\partial y_m}{x} \end{bmatrix} y=⎣⎢⎢⎢⎡​y1​y2​⋮ym​​⎦⎥⎥⎥⎤​,∂x∂y​=⎣⎢⎢⎢⎡​x∂y1​​x∂y2​​⋮x∂ym​​​⎦⎥⎥⎥⎤​

3. ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y​ [ y \mathbf{y} y是列向量， x \mathbf{x} x是列向量，导数是矩阵]
   x = [ x 1 x 2 ⋮ x n ] , y = [ y 1 y 2 ⋮ y m ] , ∂ y ∂ x = [ ∂ y 1 x ∂ y 2 x ⋮ ∂ y m x ] = [ ∂ y 1 ∂ x 1 , ∂ y 1 ∂ x 2 , ⋯   , ∂ y 1 ∂ x n ∂ y 2 ∂ x 1 , ∂ y 2 ∂ x 2 , ⋯   , ∂ y 2 ∂ x n ⋮ ∂ y m ∂ x 1 , ∂ y m ∂ x 2 , ⋯   , ∂ y m ∂ x n ] \mathbf{x} = \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_n \end{bmatrix}, \mathbf{y} = \begin{bmatrix} y_1 \\ y_2 \\ \vdots \\ y_m \end{bmatrix}, \frac{\partial \mathbf{y}}{\partial \mathbf{x}} = \begin{bmatrix} \frac{\partial y_1}{\mathbf{x}} \\ \frac{\partial y_2}{\mathbf{x}} \\ \vdots \\ \frac{\partial y_m}{\mathbf{x}} \end{bmatrix} = \begin{bmatrix} \frac{\partial y_1}{\partial x_1}, & \frac{\partial y_1}{\partial x_2}, & \cdots, & \frac{\partial y_1}{\partial x_n} \\ \frac{\partial y_2}{\partial x_1}, & \frac{\partial y_2}{\partial x_2}, & \cdots, & \frac{\partial y_2}{\partial x_n} \\ \vdots \\ \frac{\partial y_m}{\partial x_1}, & \frac{\partial y_m}{\partial x_2}, & \cdots, &\frac{\partial y_m}{\partial x_n} \\ \end{bmatrix} x=⎣⎢⎢⎢⎡​x1​x2​⋮xn​​⎦⎥⎥⎥⎤​,y=⎣⎢⎢⎢⎡​y1​y2​⋮ym​​⎦⎥⎥⎥⎤​,∂x∂y​=⎣⎢⎢⎢⎡​x∂y1​​x∂y2​​⋮x∂ym​​​⎦⎥⎥⎥⎤​=⎣⎢⎢⎢⎢⎡​∂x1​∂y1​​,∂x1​∂y2​​,⋮∂x1​∂ym​​,​∂x2​∂y1​​,∂x2​∂y2​​,∂x2​∂ym​​,​⋯,⋯,⋯,​∂xn​∂y1​​∂xn​∂y2​​∂xn​∂ym​​​⎦⎥⎥⎥⎥⎤​
   一些样例

   y \mathbf{y} y	a \mathbf{a} a	x \mathbf{x} x	A x \mathbf{Ax} Ax	x T A \mathbf{x^TA} xTA
   ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y​	0 \mathbf{0} 0	I \mathbf{I} I	A \mathbf{A} A	A T \mathbf{A}^T AT

   y \mathbf{y} y	a u a\mathbf{u} au	A u \mathbf{Au} Au	u + v \mathbf{u}+ \mathbf{v} u+v
   ∂ y ∂ x \frac{\partial \mathbf{y}}{\partial \mathbf{x}} ∂x∂y​	a ∂ u ∂ x a\frac{\partial \mathbf{u}}{\partial \mathbf{x}} a∂x∂u​	A ∂ u ∂ x \mathbf{A}\frac{\partial \mathbf{u}}{\partial \mathbf{x}} A∂x∂u​	∂ u ∂ x + ∂ v ∂ x \frac{\partial \mathbf{u}}{\partial \mathbf{x}} + \frac{\partial \mathbf{v}}{\partial \mathbf{x}} ∂x∂u​+∂x∂v​