All Questions
Tagged with hessian-matrix vector-analysis
44 questions
-1
votes
1
answer
124
views
Relationship between hessian matrix and curvature [closed]
I am taking vector calculus this semester, and while researching about Hessian matrices for a project, I encountered this formula.
enter image description here
Could anyone explain how it is derived, ...
- 61
0
votes
0
answers
21
views
Inequalities of Hessian of distance functions on complete non-compact Riemannian Manifolds
I am interested in finding some inequalities relating the following expression with the curvature on a non-compact Riemannian Manifold
$\frac{1}{d_1}Hess^{d_1} (\frac{\partial}{\partial x^\alpha},\...
1
vote
0
answers
59
views
Hessian of coordinate function on sphere
Denote by $S^n$ the unit sphere in $\mathbb{R}^{n+1}$, and consider the coordinate function $x_{n+1}$ on it, i.e. the function $(x_1, \ldots, x_{n+1}) \mapsto x_{n+1}$. Denoting by $\mathrm{Hess}(x_{n+...
- 1,917
0
votes
3
answers
221
views
On the Hessian of the Log-Determinant and the solution provided in Stephen Boyd's textbook
This is a follow up to another question on the second-order approximation to log-determinant in Boyd's textbook, the excerpt can be found here:
Here, $f$ is the log-determinant, $f(Z) = \log(\det(Z))$...
- 5,603
0
votes
0
answers
110
views
Jacobian and Hessian of $f(x) = \langle x, Ax \rangle$
A is a $\Re^{n \times n}$ Matrix.
f is a function from $\Re^n$ to $\Re$ with $f(x) = \langle x, Ax \rangle$.
How can I determine the gradient and hessian of this Matrix at point x?
user1159114
1
vote
0
answers
65
views
Determine the Hessian of this function?
Given the function below, I would like to determine the Hessian with respect to $\mathbf{x}$, which will result in a $2 \times 2$ matrix. Note: $n$ and $\mathbf{z}$ are constants with respect to $\...
- 524
1
vote
1
answer
431
views
Deriving hessian as best linear approximation
I'm having trouble understanding how to derive the Hessian based on the following definition of the derivative of a multivariable function (the derivation is from "An introduction to optimization ...
- 13
1
vote
2
answers
202
views
Hessian matrix of $ \| Ax -b \|_2$
I need to compute the Hessian matrix of $ \| Ax -b \|_2$ with respect to x where $A$ is $ m \times n $ matrix, $x$ is $ n \times 1 $ vector and $b$ is $ m \times 1 $ constant vector.
It is not so ...
- 474
1
vote
3
answers
320
views
How to calculate the gradient and Hessian for composite function?
I have a function that can be written as follows:
$$f(\vec{x},\vec{y}) = f_x(\vec{x}) + f_y(\vec{y}) + k\bigg(f_x(\vec{x})-f_y(\vec{y})\bigg)^2$$
I have to take the first derivative (gradient) and ...
- 267
1
vote
1
answer
709
views
Which is the correct vector calculus notation for the Hessian?
In vector calculus, the nabla symbol $\nabla$ is used to denote three different operations:
the gradient of a scalar function $f$ is vector field: $\mathrm{grad}(f)=\nabla f$
the divergence of a ...
- 145
0
votes
1
answer
895
views
Calculation of hessian and gradient of spherical harmonics
I was searching for relations that can be used to calculate the gradient and the hessian matrix of spherical harmonics in Cartesian coordinates easier. I am using the following definition of the ...
- 194
0
votes
1
answer
74
views
Strongly convexity of a loss function
I want to calculate the strongly convex parameter $\sigma$ for this loss function:
$$
l_Z(Z)=||Z-A||^2_F+\lambda tr[Z^TBZ]
$$
where $Z\in \mathcal{R}^{n\times m}$, the value of $A,B$ and $\lambda$ are ...
- 303
0
votes
0
answers
49
views
How does Lipschitz Continuity of the gradient implies that the Hessian-Matrix minus Lipschitz-Constant is negativ semidefinit? [duplicate]
I was reading following lecture notes https://www.stat.cmu.edu/~ryantibs/convexopt-F13/scribes/lec6.pdf In the proof the author says that if the gradient $\nabla f$ is Lipschitz-continuous than $H_f(x)...
- 1
1
vote
1
answer
716
views
Find the critical points of a multivariable function with constraints
For my vector calculus class, I need to solve this problem:
Let $$f(x,y)=x^3+8xy-8y^3-7x+14y$$
Given that this function has critical points on the lines $3x=7$ and $3x=-3$, find and classify all ...
- 346
0
votes
1
answer
555
views
Jacobian of product of a matrix and a vector functions
Let's say I have a $m\times m$ matrix function $A=(a_{ij})$, where each $a_{ij}:\mathbb R^n\to\mathbb R$ is a scalar function. Let's say I also have a vector valued function $f:\mathbb R^n\to\mathbb R^...
- 3,650
0
votes
1
answer
145
views
Hessian of a squared bilinear form
I have to expand the function $f:\mathbb{R}^n \times \mathbb{R}^m \rightarrow \mathbb{R}$ in Taylor series where
$$ f(x,y) = (x^TAy + B^Tx + C^Ty)^2$$
with $A\in\mathbb{R}^{n\times m}$, $B\in\mathbb{R}...
2
votes
1
answer
2k
views
On "the Hessian is the Jacobian of the gradient"
According to Wikipedia,
The Hessian matrix of a function $f$ is the Jacobian matrix of the gradient of the function $f$; that is: $H(f(x)) = J(\nabla f(x))$.
Suppose $f : \Bbb R^m \to \Bbb R^n,x \...
- 467
0
votes
0
answers
80
views
Possibility of confusion caused by the use of $\nabla$
This is a question about notation. Of course, as long as the notation is clearly defined, it doesn't matter at all which notation we use, but it's still helpful to ask about a few possible confusions ...
- 7,606
1
vote
2
answers
102
views
Gradient and Hessian of a matrix
For $g(y) = f(D^{\frac{1}{2}}y)$ where $D^{\frac{1}{2}}$ is a matrix to the power half and $ x = D^{\frac{1}{2}}y$
Then $\nabla g(y) = \nabla f(D^{\frac{1}{2}}y) = D^{\frac{1}{2}} \nabla f(D^{\frac{1}{...
1
vote
1
answer
210
views
Gradient, Hessian, and minimum of a vector function?
Given the function $f(\vec{x})=(\frac{1}{2})\vec{x}^TP^TP\vec{x}+q^T\vec{x}+r$, where $\vec{x}$ and $q \in \mathbb{R}^n$, and $P \in \mathbb{R}^{n x n}$ is full rank and $r \in \mathbb{R}$, I am ...
- 149
0
votes
1
answer
134
views
Vanishing Hessian on circle
I'm looking for examples of smooth functions $f: \mathbb{R}^2 \longrightarrow [0,\infty[$ such that the determinant of their Hessian matrix does only vanish on the unit circle, i.e.
$$\det \mathsf{H}...
- 360
2
votes
1
answer
1k
views
Gradient and Hessian of vector multiplication
I was asked to find gradient ($\nabla f(x)$) and Hessian ($H(f(x))$) of $f(x)=(a^T x)\cdot (b^T x)$, where $x$, $a$, and $b$ are n-dimensional column vectors.
I was not taught how to find them with ...
- 123
1
vote
1
answer
302
views
How do you show that f has neither a local maximum nor a local minimum?
Be $f : \mathbb R^2 \to \mathbb R$ defined by $f(x_1,x_2) = x_2x^4_1 −x_2$
How do you show that $f$ has for all $x \in \mathbb R^2$ with $Df (x) = 0$ neither a local maximum nor a local minimum?
I ...
1
vote
3
answers
78
views
$\bar f(y) = f(Ty)$, how to compute the Hessian of $\bar f(y) $?
From Convex Optimization by Boyd and Vandenberghe: Let $T \in \Bbb R^{n \times n}$ be nonsingular. Let $f: \Bbb R^n \rightarrow \Bbb R$ convex and twice continuously differentiable. Define $\bar f(y) =...
- 191
0
votes
2
answers
2k
views
Matlab: Gradient and Hessian of a function with vector input of user specified size
I need to write a matlab m file that takes the following function of $x=(x_{1},x_{2},\cdots, x_{2n})$, and for $n=10$, $n=100$, $n=500$, $$f(x) = \frac{1}{2}\sum_{i=1}^{2n}i(x_{i})^{2}-\sum_{i=1}^{2n}...
user100463
0
votes
2
answers
161
views
Assistance with a Vector Calculus Exercise
I am computing $\nabla \nabla$ of a function $\mu_{\gamma}=-\frac{e^{-\gamma}}{4 \pi r}$, and get the following (since the function only depends on $r$).
$$
\begin{split}
\nabla \mu_{\gamma} &= \...
- 3,025
0
votes
1
answer
162
views
Hessian of quadratic form of function using Hadamard and Frobenius notation
Related to this question, I am trying to compute the Hessian of
$$
g(r, \theta) = [r\cos(\theta)]^{\top} A \, [r\cos(\theta)] = f(r, \theta) ^{\top} A \, f(r, \theta) \tag{$*$}
$$
for $r, \theta \in \...
- 2,721
3
votes
0
answers
58
views
$\mathbb{C}\mathbb{R}$-calculus for quadratic form
I've been working through this set of notes on differentials of $\mathbb{R}$-valued functions of complex variables (and this MSE question), and I'm trying to work through a simple example. Do I have ...
- 2,721
0
votes
1
answer
313
views
At a Critical Point the Hessian Equals the Frst Nonconstant Term in the Taylor Series of $f$?
My textbook defines Hessian functions as follows:
Suppose that $f: U \subset \mathbb{R}^n \to \mathbb{R}$ has second-order continuous derivatives $\dfrac{\partial^2{f}}{\partial{x_i}\partial{x_j}}(\...
- 4,430
2
votes
2
answers
2k
views
Clarification of Textbook Explanation of Hessian Matrix, Directional Second Derivative, and Eigenvalues/Eigenvectors
My machine learning textbook has the following section on the Hessian matrix:
When our function has multiple input dimensions, there are many second derivatives. These derivatives can be collected ...
- 4,430
16
votes
2
answers
10k
views
Calculate the Hessian of a Vector Function
I'm working with optimisation. I am trying to obtain the hessian of a vector function:
$$
\mathbf{F(X) = 0} \quad \text{or} \quad
\begin{cases}
f_1(x_1,x_2,\dotsc,x_n) = 0,\\
f_2(x_1,x_2,\...
- 173
3
votes
1
answer
895
views
Hessian matrix vs differential 2-form
Could someone clarify the convention that the second derivative of a scalar function $f: \Bbb R^n \rightarrow \Bbb R$ is sometimes defined as a linear operator $D^2f : \Bbb R^n \rightarrow L(\Bbb R^n, ...
0
votes
1
answer
52
views
How to obtain the following result?
I have a function $f(x)=h([g_1(x),~g_2(x),\ldots, g_k(x)])$ and I want to find $f''(x)$. To this end, first I have $$f'(x)=\nabla h([g_1(x),~g_2(x),\ldots, g_k(x)]) \left[ \begin{array}{c}
g_1'(x) \\...
- 2,738
0
votes
2
answers
1k
views
Gradient and Hessian for use in Newton's Method with regularization term
I am trying to implement newton's method on a strictly convex n-dimensional set. with the following objective function:
$$F(x)\equiv \|g - Hx\|_2^2 + \frac 1 2 \lambda \|x\|_2^2 $$
Where $g= Hx$ ...
0
votes
2
answers
973
views
How to deal with inconclusive Hessian test for maximization of $(x+y)^2/(x^2+y^2)$
I want to maximize the following function
$$f(x,y)=\frac{(x+y)^2}{x^2+y^2}$$
over $\mathbb{R}$.
Equating its gradient to zero gives
$$\nabla f(x,y)=0\Rightarrow x=y$$
Then, I used Wolfram to compute ...
- 1,381
4
votes
1
answer
475
views
Gradient and Hessian of Nonlinear vector function.
I would like to develop a formula for the Gradient and Hessian of $f(x)= \phi(Ax)$
where $f $ is a nonlinear vector function $x$ is a vector $A$ is a matrix and $\phi$ is a nonlinear vector function.
...
- 149
0
votes
1
answer
1k
views
Gradient and Hessian of quadratic form
Let $\mathbf a, \mathbf x$ $\in$ $\mathbb R^n$, consider the function $f(\mathbf x)=\mathbf a^T\mathbf x$ and $g(\mathbf x)=(\mathbf a^T\mathbf x)^2$.
(a) Find $∇f(\mathbf x)$ and the Hessian $H_f(\...
- 635
3
votes
1
answer
3k
views
Connection of gradient, Jacobian and Hessian in Newton's method
Suppose $f: \mathbb{R^n} \to \mathbb{R}$, the gradient of $f(\mathbf{x})$ is $$\mathop{\nabla} f(\mathbf{x}) = \begin{bmatrix} \frac{\partial{f}}{\partial{x_1}} \\ \vdots \\ \frac{\partial{f}}{\...
- 65
0
votes
1
answer
228
views
Hessian and gradient of matrices
Assuming that $f: R^n \rightarrow R$, $a \in R^n$, $g: R\rightarrow R$, and $h: R^n \rightarrow R$
What are the expressions for
$\nabla f(x) $ and $\nabla^2 f(x)$ where $f(x) = g(h(x))$
and
$\...
- 168
1
vote
1
answer
1k
views
About eigenvalues of the Hessian matrix
I've been trying to prove this result:
Let $f:V\rightarrow\mathbb R^2$ be a $C^2$ function, and $b\in V$ a critical point of $f$. If $\phi:U\rightarrow V$ is a $C^2$-diffeomorphism with $\phi(a)=b$, ...
- 1,404
3
votes
1
answer
5k
views
Hessian matrix in spherical coordinates
This seems like a straightfoward question but I cannot find the answer anywhere.
I need to implement the Hessian matrix of a real scalar function f (an Hamiltonian, to be specific) in spherical ...
- 39
0
votes
0
answers
144
views
Unconstrained optimisation and Hessian Matrix solving for convexity/concavity
Is it possible to reduce the Hessian Matrix with Gaussian elimination when we are trying to find if the function is concave or convex, assuming that all the inputs inside the Hessian matrix are ...
- 153
0
votes
1
answer
470
views
Higher order terms in Hessian of $g(x)^T g(x)$, where $g(x)$ is the gradient of underlying $f(x)$: $\mathbb R^n \Rightarrow \mathbb R$
Consider a (continuously differentiable as many times as you need it) function $f(x)$: $\mathbb R^n \Rightarrow \mathbb R$.
Let $g(x)$ = gradient of $f(x)$ w.r.t. $x$.
Let $H(x)$ = Hessian of $f(x)...
- 4,556
2
votes
2
answers
899
views
Function Composition, Derivatives, Gradient, Hessian
Here's the problem:
Let $f : R^n \to R$ be a twice continuously differentiable function. Let $\phi(t) = f(u + td)$ be a composition function from $R$ to $R$, with given vectors $u, d \in R^n$. ...
- 43