## Numerical differentiation. What am I actually doing?

For the discussion of the sciences. Physics problems, chemistry equations, biology weirdness, it all goes here.

Moderators: gmalivuk, Moderators General, Prelates

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

### Numerical differentiation. What am I actually doing?

I have some data that I (a colleague) wishes to fit some rate equations.

Raw data
scale.png (5.81 KiB) Viewed 1621 times

We think that there are two regimes, and when fitting with two rate constants, we get better answers. In order to try and find the time at which we should change over equations, we differentiated the data

$y'_i = \frac{y_{i+1}-y_i}{x_{i+1} - x_i}$

and got this:

raw data differentiated
s_dash.png (8.3 KiB) Viewed 1621 times

When playing with the data, we found that if we divided the scale by time, and then plotted against time, we got a nice curve that seemed to be the derivative

scale on time
SonT.png (5.87 KiB) Viewed 1621 times

I then did a regression on the original data to get an analytical equation so I could differentiate that [scale(t)=exp(0.018808*ln(t)^4 - 0.307358*ln(t)^3 + 1.560657*ln(t)^2 - 1.358942*ln(t) - 1.325869)]

analytical representation of scale
calc_s.png (4.44 KiB) Viewed 1621 times

Differentiating that, I got a similar curve to s/t
analytical differentiation of scale
calc_s_dash.png (4.58 KiB) Viewed 1621 times

but different.

If I then do a moving average of the scale, and then numerically differentiate it as above, I get a similar curve to the calculated one

numerical differentiate of moving average scale
s_ave_dash.png (7.78 KiB) Viewed 1621 times

My question is: Does the s/t curve actually mean anything? Why does it look like the derivative? Did we just jag it, or does it have a physical representation somewhere?

Charlie!
Posts: 2035
Joined: Sat Jan 12, 2008 8:20 pm UTC

### Re: Numerical differentiation. What am I actually doing?

Nope, just lucky

Plot on a log/log scale or something like that to really see the difference, I'd guess.
Some people tell me I laugh too much. To them I say, "ha ha ha!"

BlackSails
Posts: 5315
Joined: Thu Dec 20, 2007 5:48 am UTC

### Re: Numerical differentiation. What am I actually doing?

Wont you ALWAYS get better results from fitting with two constants instead of 1?

Sagekilla
Posts: 382
Joined: Fri Aug 21, 2009 1:02 am UTC
Location: Long Island, NY

### Re: Numerical differentiation. What am I actually doing?

Well, when you're doing [imath]\frac{s(t)}{t}[/imath], you are taking a derivative of sorts. A very bad one.

The formula for a numerical derivative is:
$\frac{y(t + dt) - y(t)}{dt}$

If you take t to be 0, and assume y(0) is zero (or at least very tiny) then you get:
$\frac{y(dt)}{dt}$

Which looks just like [imath]\frac{s(t)}{t}[/imath]. That's why it looks so similar to the real numerical derivative.

There's actually some formulas for derivatives that are robust against noise. They essentially smooth out your data,
but their results are pretty good. This guy has a bunch of stuff on it: http://www.holoborodko.com/pavel/?page_id=245

@BlackSails: Most of the time yes. If you're doing modeling the way I think they're doing (f(t) = g(a1, t) + h(a2, t),
where a1 and a2 are your constants) then yes it gives better results since you're using the appropriate function
for the appropriate region (Fit it linearly where it's linear, exponentially where it's exponential, that sort of thing).

If you're just doing it some naive way (say polynomial fit), then more constants isn't always better.

Edit: I swear to God I have the most random toolkit of knowledge when it comes to problem solving.
http://en.wikipedia.org/wiki/DSV_Alvin#Sinking wrote:Researchers found a cheese sandwich which exhibited no visible signs of decomposition, and was in fact eaten.

Zamfir
I built a novelty castle, the irony was lost on some.
Posts: 7602
Joined: Wed Aug 27, 2008 2:43 pm UTC
Location: Nederland

### Re: Numerical differentiation. What am I actually doing?

Your s(t)/t is simply the slope of a line from the origin to every point on your graph. Your graph goes almost through the origin and increases monotonically, so that slope starts out as a decent approximation of the derivative, and is later still somewhat like the derivative.

For comparison: think of a graph that crosses the y-axis at a positive value, and slopes down. For such a grpah, s(t)/t would be nowhere near a derivative.

gorcee
Posts: 1501
Joined: Sun Jul 13, 2008 3:14 am UTC

### Re: Numerical differentiation. What am I actually doing?

Numerical differentiation of quantized/noisy data is not an easy thing to do.

Numerical differentiation involves computing [imath]\frac{y_{i+1}-y_i}{h}[/imath], and when you have noise, it gets greatly amplified in the derivative. There are a few ways that this gets handled in practice.

First, you can put the data through a low-pass filter. This is inelegant but easy.

Second, you can try to fit a curve to the data, and take the derivative of that. This is also inelegant but easy, although finding the right curve is hard, and bad choices can lead to bad results.

Third, you can apply a noisy derivative algorithm. A while back, I implemented one based on Total Variation Regularization. This is a technique derived from Inverse Methods. It works pretty well, but it's a batch algorithm. Actually, it works frakking amazing, but it might be a little more difficult to implement than you have time for. The original paper is by Rick Chartrand: http://math.lanl.gov/Research/Publicati ... erical.pdf

It's not terribly hard to understand or implement. If you really need the noiseless derivative, or a reconstructed noiseless signal, this method is sexy-time fun. Check out this paper to see some results: http://www.ndia-mich.org/downloads/GVSE ... hicles.pdf

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

### Re: Numerical differentiation. What am I actually doing?

ta for that everyone.

@gorcee thanks for the papers, I'm having a look at them now, just for fun...

.

The thing is that using the turning point in the s/t curve as the delimiter between the two models actually gives a nice result...

Oh well, back to the old drawing board