2
$\begingroup$

In many machine learning algorithm, it is often assumed that outputs of unknown function and their corresponding inputs are given to estimate the unknown function. However, I wonder whether there exist some algorithms which can estimate unknown function using derivative data of unknown function and their corresponding inputs. For example,

$$ \mbox{Given }D = {(x_1, y_1),\ldots,(x_n, y_n)}$$ $$\mbox{ where }y_{1} = \frac{d{f(x)}}{dx}\Bigg|_{x=x_1},\ldots,y_n=\frac{d{f(x)}}{dx}\Bigg|_{x=x_n}$$ $$\mbox{ Find } f(x).$$

I found one curve fitting method which is called Hermite interpolation in wiki and some papers about Hermite learning. However, in these methods, outputs of unknown function and derivative outputs of it and corresponding inputs are given. Especailly, I want to know there exist some methods that utilize only derivative data. Also is it possible to find unknown function from derivative data, in particular, the error bound between unknown function and our estimator can decrease as the number of training data increases. How can we prove it?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
4
$\begingroup$

If your function is $f:\mathbb{R}\to\mathbb{R}$, you can "learn" $f'$ as a standard regression problem (linear, polynomial, etc.) and then recover $f$ up to an additive constant by integrating $f'$. Obviously, you will only able to recover $f$ up to an additive constant from the derivative.

Update: As for error estimates, here's a simple one. Suppose you've managed to recover $f'$ up to $\epsilon$-accuracy in $L_\infty$. Suppose further that you know that $f(0)=0$ and you only care about $f$ on $[0,T]$. Then your estimate of $f$ (obtained by integrating $f'$) is accurate up to $T\epsilon$ in $L_\infty$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thank you for your kind example. However, as you said, when $f$ is estimated from $f'$ with error $\epsilon$, the error of estimation linearly grows (i.e. $T\epsilon$). In particular, if the interval of integration goes to infinity such as $[-\infty,\infty] = \mathbb{R} $, then the cumulative error $\int\epsilon dt$ also goes to infinity. So the error of estimation through the whole input domain always goes to infinity.In this case, is it possible to discuss the same thing as your answer? Minor question aoubt notation, does $L_{\infty}$ mean the set of function whose infinite norm is bounded? $\endgroup$
    – Kyoungjae Lee
    Commented Dec 5, 2016 at 3:13
  • 1
    $\begingroup$ If you assume something about the noise, you might be able to argue that the errors in the derivative cancel out on average (plausible, since you're integrating) and the deviation is bounded over all of $R$. $\endgroup$
    – Aryeh
    Commented Dec 5, 2016 at 8:00
  • $\begingroup$ Note that you might want to use a different loss function here than in a standard setting as error "accumulates" when you integrate. Particularly, points near the base point of integration matter more than points far from it. And you don't want to do catastrophically badly on a small set of inputs, whereas in a normal setting that's acceptable. $\endgroup$
    – Reinstate Monica
    Commented Sep 3, 2019 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.