## Cross correlation

For the discussion of the sciences. Physics problems, chemistry equations, biology weirdness, it all goes here.

Moderators: gmalivuk, Moderators General, Prelates

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

### Cross correlation

Hi all

I want to do a cross-correlation on two datasets to find the angle offset required for maximum correlation.

The datasets look like this:

Code: Select all

`index  angle  intensity  1    10.00    1000  2    10.05    1125  3    10.10    1235  4    10.15    1325  5    10.30    1234  6    10.35    1325  7    10.40    1256  ...   20000  1200.05  1356  index  angle  intensity  1    10.00    1032  2    10.05    1302  3    10.20    1321  4    10.25    1234  5    10.30    1234  6    10.35    1432  7    10.40    1564  ...     20000  1201.05  1457`

The important values are the (angle, intensity) tuple. All cross-correlation algorithms that I have seen work on the index number, and in this case, the indices don't necessarily correspond to the same angle (unless I go through and pre-sort the data by deleting values).

Are there any algorithms that work with an x-value, as opposed to an index number? or am I stuck with pre-treating my data...

iop
Posts: 930
Joined: Tue Jul 24, 2007 11:26 am UTC
Location: The ivory tower

### Re: Cross correlation

People usually calculate a cross-correlation in order to compare two sets of data. To make it a fair comparison, you should, however, compare apples with apples. That means, you should compare intensity values measured at the same angles. Otherwise, if one data set had angles spaced 0.1 degrees from 0 to 5, and the other data set had angles spaced 1 degree from 50 to 100, what meaning would a cross-correlation between two intensity curves have?

In short: You need to get equal lists of angles, ideally evenly spaced. If you have to, you can interpolate to 'recover' missing angles, though be warned that interpolation is a form of filtering, which reduces noise, and may affect your cross-correlation result.

masher
Posts: 821
Joined: Tue Oct 23, 2007 11:07 pm UTC
Location: Melbourne, Australia

### Re: Cross correlation

iop wrote:That means, you should compare intensity values measured at the same angles.

I know. That's why I asked if there were any algorithms that calculate a cross-correlation using an x-value, instead of an index number, so I wouldn't have to pre-treat my data to throw away all the non-equivalent angles in each dataset

iop wrote:In short: You need to get equal lists of angles, ideally evenly spaced. If you have to, you can interpolate to 'recover' missing angles, though be warned that interpolation is a form of filtering, which reduces noise, and may affect your cross-correlation result.

I don't want to interpolate; I've got more than enough data that I can throw some away to ensure I'm comparing intensities from the same angles.

.

I haven't been able to find any routines that can do what I want, so I've had to go through and pre-treat my data to throw away all data that doesn't appear in both datasets.