Questions tagged [hessian-matrix]
The Hessian matrix of function is used to second derivative test when $f$ has a critical point $x$. If the Hessian is positive definite at $x$, then $f$ attains a local minimum at $x$. If the Hessian is negative definite at $x$, then $f$ attains a local maximum at $x$. If the Hessian has both positive and negative eigenvalues then $x$ is a saddle point for $f$.
879 questions
1
vote
0
answers
40
views
Critical points of the squared distance function on surfaces [closed]
Let S be a surface, $p_0 \in \mathbb{R}^3$, we define the squared distance function $f: S \rightarrow \mathbb{R}$ given by
$$f(p)= \| p - p_0\|^2 , p \in S$$
prove that $p_0$ is a critical point of $f$...
0
votes
1
answer
46
views
Determinant of matrix, that has non-zero entries only in the first row, first column and the diagonal
Given are constants $\alpha_1,\ldots,\alpha_n\in\mathbb{R}$, $\beta_1,\ldots,\beta_n>0$ and the matrix
$$
M:=\begin{pmatrix}
0 & \alpha_1 & \ldots & \alpha_n \\
\alpha_1 & \beta_1 &...
- 3,238
0
votes
0
answers
28
views
Can a point be a local extrema under constraints if Hessian matrix is indefinite?
So, I know that if I have some open subset which contains stationary point I can use Sylvester's criterion to check if a Hessian matrix is positive-definite. If it's indefinite, that point for sure ...
- 329
0
votes
0
answers
68
views
Second order PDE with Hessian
I am wondering if there is a existence/uniqueness result for the solution to PDE
$$
D^2 u = F (x, u, Du)
$$
with appropriate initial value conditions.
(Just to clarify, $u : \mathbb R^d \to \mathbb R$ ...
- 157
0
votes
0
answers
18
views
How can we get $\frac{\partial^2 u}{ \partial \nu\partial y_i}=0$
Let $\Omega$ is bounded domain in $\mathbb{R}^n$ with a smooth boundary $\partial \Omega$, $u \in C^2(\bar{\Omega})$, $u > 0$ in $\Omega$, $u = 0$ in $\partial\Omega$, if $\exists x_0 \in \partial \...
- 115
1
vote
0
answers
101
views
Checking Positive-Definiteness of Hessian in Matrix Calculus
Let $X$ be a $2 \times N$ matrix and $\mathbf{v}$ an $N \times 1$ column vector. I am considering the following function of a matrix input:
$$ f(X) = \|X \mathbf{v} \|_2.$$
My goal is to compute the ...
- 4,039
0
votes
1
answer
40
views
Show that a harmonic function has no local maxima and minima
Given a differentiable function $f: U \rightarrow \mathbb{R}$, a point $a \in U$ is called a critical point of $f$ (or a singular point) when $d f(a) = 0$, that is,
$$
\frac{\partial f}{\partial x_1}(...
- 2,413
1
vote
0
answers
44
views
How do I make sure my Riemannian hessian is positive-definite?
I am working on L-BFGS optimization on a Grassmann manifold with a projector representation (not a point on the Stiefel manifold).
L-BFGS requires an initial hessian that is positive definite.
I have ...
- 33
0
votes
0
answers
21
views
Providing a formula for the Hessian of a composition $L \circ z$.
Let $z: \mathbb{R}^{m} \rightarrow \mathbb{R}^{n}$ and $L: \mathbb{R}^{n} \rightarrow \mathbb{R}$ be $C^{3}$ functions.
For $\theta \in \mathbb{R}^{m}$, one can use the chain rule to obtain the ...
- 2,167
1
vote
0
answers
64
views
How to tell if cost function (a sum of quadratics) is convex?
I have a mixed integer nonlinear programming optimization problem with the following characteristics:
All the constraints are linear
The cost function (which I want to minimize) is a sum of ...
- 226
0
votes
0
answers
27
views
On finding solution to linear system with norm of independent variable
Let's consider the solution to a well-behaved linear system of equation: $y + Ax + \rho x = 0 \implies x = -(A + \rho I)^{-1}y$.
For now, let's consider the simplest case that $x, y \in \mathbb{R}^n$ ...
1
vote
2
answers
54
views
Second derivative for $-\ln(-f(\vec{x}))$
According to Approach0 or this, this question is new.
I am doing a past exam paper with no solutions, then I am stuck at the following step:
Let $g=-\ln(-f(\vec{x})):\mathbb{R}^n\to\mathbb{R}$ and $f:\...
- 1,151
0
votes
3
answers
72
views
Determining the minimum value of the function $f(x, y) = \frac{x}{y} + \frac{y}{x}$ with $x, y\in \mathbb{R}_{>0}$.
I want to conclude that the minimum value of the function $f(x, y) = \frac{x}{y} + \frac{y}{x}$ with $x > 0, y > 0$ is 2 obtained along the line $x = y$, but I seem to be a bit rusty with my ...
- 1,321
0
votes
0
answers
44
views
Show local extr of function is global one.
Let $f = (x^2+y^2) \exp(-x^2-y^2)$. Find global extremums.
Put $u = \exp(-x^2-y^2)$
$f_x = 2xu - 2xu(x^2+y^2) = 0$
$f_y = 2y*u - 2y*u*(x^2+y^2) = 0$
As $u \neq 0$ we have $2x(1 - x^2 - y^2) = 0 \...
- 11
0
votes
0
answers
18
views
Equivalence of two different forms of the operator norm of the Hessian?
I am currently reading a paper that considers the Hessian of a neural network with two parameter matrices, $\mathbf{A}\in \mathbb{R}^{N\times N}$ and $\mathbf{B} \in \mathbb{R}^{T \times N}.$
They ...
- 579
0
votes
1
answer
28
views
Convexity of function, Hessian
Im trying to understand convexity of a given function $$f(x)=x_1^2+x_2^2+3x_1x_2+10x_1-11x_2+5.$$
My initial thought was to only take the second derivatives and check that $f_{xx} \geq 0$, and $f_{yy} ...
- 593
0
votes
0
answers
51
views
Are second-order conditions in optimization really needed?
I’m participating in optimization course and a lot of time is spent proving second-order conditions for unconstrained and constrained problems. To me these conditions feel rather unnecessary since I ...
- 339
1
vote
1
answer
111
views
Proving legendre transform exists locally if $f$ twice continuously differentiable,hessian determinant non-zero
Backgrounds
Legendre transform is defined as follows:
And the problem I am trying to solve follows:
My thoughts
I searched up to acquired that the matrix concerned is a hessian matrix.
Also, the ...
- 633
0
votes
0
answers
35
views
Positive definite Hessian implies convex function.
Let $A \subseteq \mathbb{R}^n$ be open and $f: A \to R$ be continuously twice differentiable. Suppose $\nabla^2 f(c) > 0$ for some $c \in A$. Prove that there exists a neighborhood $U$ of $c$ such ...
- 968
0
votes
0
answers
33
views
How to calculate $\mathbf{s}_k^{\rm T} \mathbf{B}_k \mathbf{s}_k$ within the L-BFGS approach?
I have recently found a paper dealing with a damped L-BFGS procedure. In that work, the authors used the following quantities in their expressions:
$b_k = \mathbf{s}^{\rm T}_k \mathbf{B}_k \mathbf{s}...
- 528
1
vote
1
answer
51
views
Has anyone seen PDE involving determinant of Hessian?
Travelling through long and convoluted mathematical tracks, I have stumbled upon the following PDE. All I'm really asking is whether someone has seen this PDE before, and if it has a name. So here is ...
- 15
7
votes
2
answers
391
views
Computing the Hessian from the Jacobian of the gradient
I'm trying to compute the Hessian of the function,
$$f(X)=a^tX^2b$$
I used the perturbation approach to expand $f(X+H)$ as
$$f(X+H)=a^t(X+H)^2b=\underbrace{a^tX^2b}_{f} + \underbrace{a^t(XH+HX)b}_{\...
- 8,101
3
votes
0
answers
40
views
Sketchy use of multivariable chain rule under too weak hypoteses
I've found this statement in my real analysis course notes:
Let $f: B_r (x_0) \subseteq \mathbb{R}^m \to \mathbb{R}$ ($B_r (x_0) = \{ x \in \mathbb{R}^m : d(x, x_0) < r \}$) be a function such ...
- 132
0
votes
0
answers
33
views
Definition of covariant Hessian
I read the paper and on page two, it says that a linear function $A(t)$ is the covariant Hessian of a function $f$. Can anyone give me a definition of the covariant Hessian? I couldn't find anything ...
- 423
1
vote
1
answer
40
views
Positive semi-definitiveness condition for multidimensional minimization case
We study the nonlinear problem
\begin{equation} \underset{\mathbb{R}^2}{\text{min}}f(x)
\end{equation}
where $f(x)=x_1^2+x_2e^{x_1}-x_1x_2+x_2^2$
Evaluate whether the problem is convex.
For a ...
- 3,081
0
votes
0
answers
51
views
Least Squares Function Approximation and Convexity of Functions
I have been reading about Least Squares function approximation and am dealing with the following definition:
Let $f$ be continuous on $[a,b]$ and let $W$ be a finite dimensional subspace of $C[a,b]$. ...
- 327
0
votes
1
answer
49
views
This function has no saddle points: correctness of this reasoning
I would like to know if my reasoning is correct. I have the function $f(x, y) = e^{3x}(1+25x^2+25y^2)$ and I have to study the stationary points.
After computing the gradient I found
$$\begin{cases} 3 ...
- 3,521
0
votes
0
answers
29
views
What is the intepretation of a Hessian with only negative and zero eigenvalues?
I'm looking at a high-dimensional optimisation problem. Specifically, one involving a function $F(c_n)$ which depends on N parameters (in my case $N=200$). I have been able to minimise this function ...
0
votes
0
answers
34
views
Double derivative of concave $F$ is negative? $(x-y)\cdot\nabla F(x)=0$ implies $\partial_{x-y}\partial_{x-y}F(x)<0$?
$F:\mathbb R^n\to\mathbb R$ is a strictly concave. $\nabla$ is gradient. $x,y,z\in\mathbb R^n$
Then, is it possible to prove that:
$(x-y)\cdot\nabla F(x)=0$ implies $\partial_{x-y}\partial_{x-y}F(x)&...
- 3,868
0
votes
0
answers
60
views
Block Matrix eigenvalues
I am working on a complicated optimization problem. I define $I\in \mathbb{N}, I > 1, n_I=\binom{I}{2}$. I try to optimize a scalar-valued function $\mathcal{O} : \mathbb{R}^n \longrightarrow \...
- 31
0
votes
1
answer
107
views
Derivatives of multivariate Gaussian
I do not understand how to take the second derivative of the Gaussian:
While I am certain that the result is correct, I do not understand how to get from the first to the second line of the second-...
- 1,175
0
votes
1
answer
57
views
Jacobian and Hessian of a vector
I am dealing with vector derivatives, and I am having a hard time generalising results.
What I mean is that I can compute the results by hand, variable per variable, but I would like to do it in a ...
- 55
0
votes
1
answer
90
views
Does $x$ and $(\nabla f)(x)$ align as we approach the minima for a convex function?
Given a convex function $f(x): \mathbb R^n \to \mathbb R$, let $h(x)$ denote the cosine of the angle between $x$ and $(\nabla f)(x)$.
$$h(x) := \frac{x^\top.(\nabla f)(x)}{\Vert x\Vert \Vert (\nabla f)...
- 103
1
vote
0
answers
81
views
Levenberg-Marquardt algorithm and inverse Jacobian/Hessian
Let’s say I have a function $f(p,q):R^{n+m}→R$, with $p∈R^n$ and $q∈R^m$.
I have a set of $q_{i=1,…,k},y_{i=1,…,k}$ and I want to find $p$ so I use the Levenberg-Marquardt algorithm to resolve the ...
- 131
0
votes
0
answers
36
views
Morse function on a two sphere
For the past few days I've been studying the very basics of Morse theory and its connection to supersymmetric quantum mechanics. I'm following the lectures written by David Tong. To introduce the ...
- 191
-1
votes
1
answer
124
views
Relationship between hessian matrix and curvature [closed]
I am taking vector calculus this semester, and while researching about Hessian matrices for a project, I encountered this formula.
enter image description here
Could anyone explain how it is derived, ...
- 61
2
votes
1
answer
68
views
Why $\nabla f$ do not exactly coincide with $D f$ (it's its transpose)
Is there any reason (historical, or of any other kind) to why $$\nabla f= \begin{bmatrix}\frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \\ \frac{\partial f}{\partial z} \\ \end{bmatrix}...
- 1,301
0
votes
0
answers
24
views
The inverse of a specific case of symmetric matrix (scalar product of d dimensions vectors)
The problem is the following:
For $i \in [N]$, let $v_i$ be a $1 \times d$ vector and $b_i$ a scalar.
Moreover, let $A$ be a $N \times N$ matrix, whose (i, j)-entry is:
$$ a_{ij} = \begin{cases} \...
- 1
0
votes
0
answers
41
views
Matrix derivative of matrix commutator
I'm working on some functional and I have to calculate its second derivative with respect to some matrix-variables. I'm just left with the following derivative to perform:
$$
V''= -6 i \lambda \gamma ...
- 33
2
votes
0
answers
33
views
Find maximum function with 7 variables (REPOST)
My problem was not correctly stated in the old post. (Find maximum function with 7 variables) and because of multiple edits it confuses people.
Here is the function I'm trying to find the maximum:
$$f(...
- 21
0
votes
0
answers
21
views
Inequalities of Hessian of distance functions on complete non-compact Riemannian Manifolds
I am interested in finding some inequalities relating the following expression with the curvature on a non-compact Riemannian Manifold
$\frac{1}{d_1}Hess^{d_1} (\frac{\partial}{\partial x^\alpha},\...
1
vote
0
answers
59
views
Hessian of coordinate function on sphere
Denote by $S^n$ the unit sphere in $\mathbb{R}^{n+1}$, and consider the coordinate function $x_{n+1}$ on it, i.e. the function $(x_1, \ldots, x_{n+1}) \mapsto x_{n+1}$. Denoting by $\mathrm{Hess}(x_{n+...
- 1,917
0
votes
1
answer
184
views
Hessian matrix determinant greater than zero in a saddle point?
I have the function $f:\mathbb{R}^3 \mapsto \mathbb{R}$ defined as $f(x, y, z) = xy+xz+yz+z-x$. I've calculated the Jacobian:
$$
J_f = (y+z-1, x+z, x+y+1)
$$
Which, by setting $J_f = \vec{0}$, reveals ...
- 645
2
votes
1
answer
59
views
${\rm Hess}~r$ is scalar matrix $\implies$ $M$ is isometric to the space form
I'm trying to prove the rigidity part of a theorem in my paper, which requires the use of the classical Hessian comparison theorem's rigidity part:
$${\rm Hess}~r=\frac{{\rm sn}_k'}{{\rm sn}_k}{\rm d}...
- 3,095
1
vote
0
answers
89
views
Relation between Hessian of the squared distance function and the metric tensor
Let $(M, g)$ be a Riemannian manifold and let $d$ be the distance function associated with the metric tensor $g$. Define the function $$ f(x) := \frac12 d^2 (x,y) $$ for some arbitrary fixed $y \in M$....
- 57
0
votes
1
answer
51
views
Classifying critical points when hessian has det0 [closed]
I am trying to find the critical points of
$$f(x,y)=(x^2-y^4)(1-x^2-y^4).$$
There are 9, and I found all their coordinates. I’ve classified 6/9 using the hessian, but the remaining 3 have at least one ...
- 69
0
votes
1
answer
108
views
Possible to have only zero eigenvalues of the Hessian of a harmonic function that is neither of the form $ax+by+cz+d$ nor a constant?
(Following an earlier post here) I intuit that if we restrict to functions $f(x,y,z)$ that are harmonic (i.e. satisfying $\nabla^2f=0)$ but neither of the form $ax+by+cz+d$ ($a,b,c,d\in\mathbb{R}$) ...
- 703
2
votes
1
answer
104
views
Proving nonexistence of local extrema of harmonic functions using Hessian
Edited I want to prove that solution to Laplace's equation do not support local maximum or minimum using Hessian.
Suppose $f(x,y,z)$ is a real-valued function of three real variables that satisfies ...
- 703
0
votes
1
answer
43
views
Is this function convex over a set?
Well, I got a doubt: is this function
$$f(x, y, z) = 2xy + yz$$
Convex over the set
$$V =\{ (x,y,z) \in \mathbb\{R\}^3:\ x+y+2z \leq 1, x\geq 0, y \geq 0, z \geq 0\}$$
?
Being it a quadratic form, I ...
- 3,521
1
vote
2
answers
184
views
Logit Gradient/Hessian derivations
I'm trying to follow the algebra leading from the gradient function to the Hessian in Logistic Regression, but I can't quite understand where I have gone wrong.
I have the gradient function as:
$$
\...
- 113