The general expression of a line element in a space with metric tensor $g_{\mu \nu}$ is $$ds = \sqrt{ g_{\mu \nu} dX^{\mu} dX^{\nu} }$$ If we consider a curve $X^{\mu}(\tau)$ parametrised by $\tau$, the distance between points $X^{\mu}(\tau_1)$ and $X^{\mu}(\tau_2)$ is $S$: $$S = \int^{\tau_2}_{\tau_1} ds$$ $$S = \int^{\tau_2}_{\tau_1} \sqrt{ g_{\mu \nu} \frac{dX^{\mu}}{d\tau} \frac{dX^{\nu}}{d\tau} }d\tau.$$
If we minimise $S$ to find the (geodesic) curve that gives us the shortest distance between the two points in the given, we get the following two terms in the Euler-Lagrange equation. The term $$\frac{\partial L}{\partial X^{k}} = \frac{1}{2} \frac{dX^{\mu}}{d\tau} \frac{dX^{\nu}}{d\tau} \partial_k g_{\mu \nu}$$ would be analogous to the force term $\frac{\partial L}{\partial x} = \frac{1}{2} mv^2$ in classical mechanics. Ofcourse, this is a very different context. But if we let $X^{\mu}$ and $X^{\nu}$ be $(3+1)$-dimensional spacetime coordinates, then we see there is some connection between the metric tensor $g_{\mu \nu}$ of spacetime and the idea of mass in classical mechanics. This connection is explained by Einstein's field equations. But what I am confused about is that in the steps described so far, we made no physical assumptions about relativity. We just considered the shortest distance between points in a general space, and we used the metric tensor as a mathematical object, which just tells us how to find the infinitesimal distance between two points (i.e. how to relate the line element to coordinate differentials). And yet we already see the relativistic connection between mass and spacetime geometry come up.
A similar thing is seen in the "momentum" term $$\frac{\partial L}{\partial \dot{X}^{k}} = g_{k \nu} \dot{X}^{\nu}$$ which is similar to the $\frac{\partial L}{\partial \dot{x}} = mv$ term in classical mechanics.
Is there some deeper connection between the metric tensor and (classical) mass than what is immediately obvious here? It would seem surprising for this connection to appear without making relativistic assumptions, if there wasn't some underlying mechanism behind General Relativity.