Cross correlation

For the discussion of the sciences. Physics problems, chemistry equations, biology weirdness, it all goes here.

Moderators: gmalivuk, Moderators General, Prelates

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

Cross correlation

Postby masher » Mon Sep 27, 2010 2:28 am UTC

Hi all

I want to do a cross-correlation on two datasets to find the angle offset required for maximum correlation.

The datasets look like this:

Code: Select all

index  angle  intensity
  1    10.00    1000
  2    10.05    1125
  3    10.10    1235
  4    10.15    1325
  5    10.30    1234
  6    10.35    1325
  7    10.40    1256
  ...   
20000  1200.05  1356
 
 
index  angle  intensity
  1    10.00    1032
  2    10.05    1302
  3    10.20    1321
  4    10.25    1234
  5    10.30    1234
  6    10.35    1432
  7    10.40    1564
  ...    
20000  1201.05  1457

The important values are the (angle, intensity) tuple. All cross-correlation algorithms that I have seen work on the index number, and in this case, the indices don't necessarily correspond to the same angle (unless I go through and pre-sort the data by deleting values).

Are there any algorithms that work with an x-value, as opposed to an index number? or am I stuck with pre-treating my data...

User avatar
iop
Posts: 930
Joined: Tue Jul 24, 2007 11:26 am UTC
Location: The ivory tower

Re: Cross correlation

Postby iop » Tue Oct 05, 2010 1:07 am UTC

People usually calculate a cross-correlation in order to compare two sets of data. To make it a fair comparison, you should, however, compare apples with apples. That means, you should compare intensity values measured at the same angles. Otherwise, if one data set had angles spaced 0.1 degrees from 0 to 5, and the other data set had angles spaced 1 degree from 50 to 100, what meaning would a cross-correlation between two intensity curves have?

In short: You need to get equal lists of angles, ideally evenly spaced. If you have to, you can interpolate to 'recover' missing angles, though be warned that interpolation is a form of filtering, which reduces noise, and may affect your cross-correlation result.

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

Re: Cross correlation

Postby masher » Tue Oct 05, 2010 10:58 pm UTC

iop wrote:That means, you should compare intensity values measured at the same angles.

I know. That's why I asked if there were any algorithms that calculate a cross-correlation using an x-value, instead of an index number, so I wouldn't have to pre-treat my data to throw away all the non-equivalent angles in each dataset

iop wrote:In short: You need to get equal lists of angles, ideally evenly spaced. If you have to, you can interpolate to 'recover' missing angles, though be warned that interpolation is a form of filtering, which reduces noise, and may affect your cross-correlation result.

I don't want to interpolate; I've got more than enough data that I can throw some away to ensure I'm comparing intensities from the same angles.

.

I haven't been able to find any routines that can do what I want, so I've had to go through and pre-treat my data to throw away all data that doesn't appear in both datasets.


Return to “Science”

Who is online

Users browsing this forum: No registered users and 14 guests