Static optimization difficulties

Question

Good morning everyone,

I'm not a mathematician (I'm an economist) but I found myself struggling with a static optimization problem that appears difficultly solvable with my limited knowledge on the topic. I would be very grateful if you could recommand me some readings or materials to solve this type of issues.

I have an objective function, let it be $\mathcal{O} : \mathbb{R}^n \times \mathbb{R}^n \rightarrow \mathbb{R}^n$. The variable of this application is $\alpha \in \mathbb{R}^n \times \mathbb{R}^n$ (or $\alpha \in \mathcal{M}_n(\mathbb{R}))$. I use the following notations: $\alpha = (\alpha_{ij}) = (\alpha_{1}, ..., \alpha_{j}, ... \alpha_{n})$ (in the latter, $\alpha_j$ is the column j of $\alpha$. I denote $\mathcal{O}_i$ the ith-component of $\mathcal{O}(\alpha)$.

For $(s, \gamma, \delta) \in \mathbb{R}^n \times \mathbb{R}^+ \times \mathbb{R}^+$ some parameters, $$ \mathcal{O}_i = \sum^n_j s_j\alpha_{ij} - \sum^n_j \frac{\alpha_{ij}^\gamma}{1-\alpha_{ij}^\gamma} - \left(\sum^n_j \alpha_{ij}\right)^\delta$$

I want to find $\text{argmax}_\alpha\mathcal{O}$. To do so, I computed my first order condition, that is the partial derivative: $$\frac{\partial \mathcal{O}_i}{\partial \alpha_{ij}} = s_j - \frac{\gamma\alpha_{ij}^{\gamma-1}}{(1-\alpha_{ij}^\gamma)^2} - \delta \left(\sum^n_k \alpha_{ik}\right)^{\delta-1} $$

Assume (to stick to the point of this question) that all existence conditions are satified, that is that all parameters allow the existence of the partial derivative, and the problem has at least one solution.

In my lectures in optimization, I was taught to compute the first order conditions and to solve the system to find optimum values. In this case, I find myself unable to do so because of the term $\frac{\gamma\alpha_{ij}^{\gamma-1}}{(1-\alpha_{ij}^\gamma)^2}$. I tried to rewrite the problem in full matrix notation, but I struggle with the same term and I don't really know if it could really help me to do so.

What would you do in my situation? Is there a methodology that I'm not aware of? I thank you in advance for all the tips you could give me!

Answer 1

Define a few variables and their differentials $$\eqalign{ \def\d{\delta} \def\g{\gamma} \def\e{e_i} \def\EE{E_{ii}} \def\A{A_{ij}} \def\E{E_{ij}} \def\M{M_{ij}} \def\D{\operatorname{Diag}} \def\s{\oslash} \def\q{\quad} \def\qiq{\q\implies\q} \def\p{\partial} \def\o{{\tt1}} \def\grad#1{\frac{\p #1}{\p\A}} B &= A^\g &\qiq dB = \g A^{\g-\o}\odot dA &= \g A^{\g-\o}\odot\E \\ g &= A\o &\qiq dg=dA\:\o &= \E\o \:=\: \e\\ C &= B\s(J-B) &\qiq dC=H\odot dB &= M\odot\E \\ H &= (J+C)\s(J-B) \\ M &= \g A^{\g-\o}\odot H \\ }$$ where $(J,\o)$ are the all-ones matrix/vector, $\,(\odot,\s)$ denote elementwise multiplication and division, and the exponentiations are also elementwise.

Then recall that the self-gradient of a matrix $A$ with respect to on of its components $\A$ is the single-entry matrix $\E$ $$\eqalign{ \grad A = \E \:=\: \e\,e_j^T \\ }$$ so replacing the differential $dA$ by the single-entry matrix produces the expressions on the far RHS.

Rewrite the function using the above matrix variables and calculate its gradient wrt $\A$ $$\eqalign{ f &= As - g^\d - C\o \\ df &= dA\,s - \d g^{\d-\o}\odot dg - dC\,\o \\ \grad f &= \E\,s - \d g^{\d-\o}\odot\e - (M\odot\E)\,\o \\ &= \left(s_j - \d g^{\d-\o}_i - \M\right) \e \\ \grad{f_k} &= \left(s_j -\d g^{\d-\o}_i -\M\right) \d_{ik} \\ }$$ This agrees with the gradient that you calculated (except the $\d$ symbol is being overused).

Setting the gradient to zero means that the scalar term in parenthesis must be equal to zero $$\eqalign{ s_j &= \M \;+\; \d g^{\d-\o}_i \q\q &\{ {\sf Index\ notation} \} \\ \o s^T &= M \;+\; \d g^{\d-\o}\o^T \q\q &\{ {\sf Matrix\ equation} \} \\ }$$ Solving this for the optimal $A$ matrix will be extremely difficult.

Are you sure that the objective function isn't scalar-valued?
Something like $$ \tfrac12\big\|f\big\|^2 \q{\sf or}\q w^Tf $$