Numerical differentiation

https://en.wikipedia.org/wiki/Taylor_series

INTRODUCTION

The Taylor series of a function is an infinite sum of terms that are expressed in terms of the function's derivatives at a single point.

Taylor series

f(x[i+1]) = f(x[i]) + f'(x[i]) h + f''(x[i]) \frac{ h^2 }{ 2! } + ... 
+ f^{(n)}(x[i]) \frac{ h^n }{ n! } + R_n,

where x[i+1] = x[i]+h, R_n = O(h^{n+1}).

Mean value theorem

f(x[i+1]) = f(x[i]) + f'(c) h,

where c is between x[i] and x[i+1].

Approximation of 1st order derivative by forward difference
f'(x[i]) = \frac{ f(x[i+1]) - f(x[i]) }{ h } + O(h).

Approximation of 1st order derivative by backward difference
f'(x[i]) = \frac{ f(x[i]) - f(x[i-1]) }{ h } + O(h).

Approximation of 1st order derivative by central difference
f(x[i+1]) = f(x[i]) + f'(x[i])h + f''(x[i]) h^2/2 + f^{(3)}(c1) h^3/(3!),
f(x[i-1]) = f(x[i]) - f'(x[i])h + f''(x[i]) h^2/2 - f^{(3)}(c2) h^3/(3!),
where c1 is between x[i] and x[i+1].
c2 is between x[i-1] and x[i].

Subtracting
f(x[i+1]) - f(x[i-1]) = f'(x[i]) 2h + f^{(3)}(c3) 2h^3/(3!),
where c3 is between c2 and c1.

Rearanging
f'(x[i]) = \frac{ f(x[i+1]) - f(x[i-1]) }{ 2h } + O(h^2). [better]

Forward difference
f(x[i+2]) = f(x[i]) + f'(x[i])2h + f''(x[i]) 2h^2 + O(h^3).
4f(x[i+1]) = 4f(x[i]) + f'(x[i])4h + f''(x[i]) 2h^2 + O(h^3).

Subtracting
4f(x[i+1]) - f(x[i+2]) = 3f(x[i]) + f'(x[i]) 2h + O(h^3).

Rearanging
f'(x[i]) = \frac{ -f(x[i+2]) + 4f(x[i+1]) - 3f(x[i]) }{ 2h } + O(h^2).

Backward difference
f(x[i-2]) = f(x[i]) - f'(x[i]) 2h + f''(x[i]) 2h^2 + O(h^3).
4f(x[i-1]) = 4f(x[i]) - f'(x[i]) 4h + f''(x[i]) 2h^2 + O(h^3).

Subtracting
4f(x[i-1]) - f(x[i-2]) = 3f(x[i]) - f'(x[i]) 2h + O(h^3).

Rearanging
f'(x[i]) = \frac{ 3f(x[i]) - 4f(x[i-1]) + f(x[i-2]) }{ 2h } + O(h^2).

2nd order forward difference
f(x[i+2]) = f(x[i]) + f'(x[i]) 2h + f''(x[i]) 2h^2 + O(h^3).
2f(x[i+1]) = 2f(x[i]) + f'(x[i]) 2h + f''(x[i]) h^2 + O(h^3).

Subtracting yields
f(x[i+2]) - 2f(x[i+1]) = -f(x[i]) + f''(x[i]) h^2 + O(h^3).

Rearanging
f''(x[i]) = \frac{ f(x[i+2]) - 2f(x[i+1]) + f(x[i]) }{ h^2 } + O(h).

2nd order backward difference
f(x[i-2]) = f(x[i]) - f'(x[i]) 2h + f''(x[i]) 2h^2 + O(h^3).
2f(x[i-1]) = 2f(x[i]) - f'(x[i]) 2h + f''(x[i]) h^2 + O(h^3).

Subtracting yields
f(x[i-2]) - 2f(x[i-1]) = -f(x[i]) + f''(x[i]) h^2 + O(h^3).

Rearanging
f''(x[i]) = \frac{ f(x[i]) - 2f(x[i-1]) + f(x[i-2]) }{ h^2 } + O(h).

2nd order central difference
f(x[i+1]) = f(x[i]) + f'(x[i])h + f''(x[i]) h^2/2 + f^{(3)}(x[i]) h^3/(3!) + O(h^4).
f(x[i-1]) = f(x[i]) - f'(x[i])h + f''(x[i]) h^2/2 - f^{(3)}(x[i]) h^3/(3!) + O(h^4).

Adding
f(x[i+1]) + f(x[i-1]) = 2f(x[i]) + f''(x[i]) h^2 + O(h^4).

Rearanging
f''(x[i]) = \frac{ f(x[i+1]) - 2f(x[i]) + f(x[i-1]) }{ h^2 } + O(h^2). [better]