February 5 Information

Let ${\displaystyle f:\mathbb {R} ^{n}\to \mathbb {R} }$ be a smooth function. Let ${\displaystyle x\in \mathbb {R} ^{n}}$, such that the gradient of f at x is zero. Let H be the Hessian matrix of f at the point x. Let V be the vector space spanned by the eigenvectors corresponding to negative eigenvalues of H. Let ${\displaystyle y\in V}$. Then, f(x)>f(y)? or maybe f(x)>f(x+y)?

In other words, does negative eigenvalue imply maximum point at the direction of the corresponding eigenvector, or maybe this is a maximum in another direction, and not in the direction of the eigenvector? עברית ( talk) 06:45, 5 February 2016 (UTC) reply

See Morse lemma. Sławomir
Biały 12:23, 5 February 2016 (UTC) reply

( edit conflict)You're kind of circling around the second derivative test for functions of several variables. The Taylor expansion for f at x is

${\displaystyle f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+\mathbf {y} ^{\mathrm {T} }\mathrm {D} f(\mathbf {x} )+{\frac {1}{2!}}\mathbf {y} ^{\mathrm {T} }\mathrm {D} ^{2}f(\mathbf {x} )\mathbf {y} +\cdots }$

where Df is the gradient and D²f is the Hessian. In this case the gradient is 0 at x so this reduces to

${\displaystyle f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}\mathbf {y} ^{\mathrm {T} }\mathrm {D} ^{2}f(\mathbf {x} )\mathbf {y} +\cdots .}$

Let e be an eigenvector with eigenvalue λ, and wlog take e to be length 1. If y = te, then

${\displaystyle f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}\lambda t^{2}+\cdots }$

so f has a local minimum or maximum along the line parallel to e though x, depending on whether λ is positive or negative. If e, f ... are several linearly independent eigenvectors, with eigenvalues λ, μ, ... , and y = te + uf + ... , then

${\displaystyle f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}(\lambda t^{2}+\mu u^{2}+\cdots )+\cdots }$

so f has a local minimum or maximum in the relevant space though x provided λ, μ, ... have the same sign. (The eigenvectors may be taken to be orthogonal since D²f is symmetric.) Note, this is only valid for y sufficiently small, otherwise the higher order terms in the Taylor series become significant and the approximation is no longer valid. -- RDBury ( talk) 12:46, 5 February 2016 (UTC) reply

Oh, great! Thank you! :) עברית ( talk) 08:38, 6 February 2016 (UTC) reply