CODEWITHSUNDEEP
algorithmmatlabfilteringsensorsnoise
John Alberto
John Alberto

Reputation: 437

Filtering (or making an algorithm?) for very noisy/spikey position data in MATLAB

For the longest time now, I've been struggling to filter or develop an algorithm for some data that I've collected from a potentiometer (5K linear pot, sampled at 500 Hz if it matters). Here are two sample sets:

1) http://cl.ly/1N2716032h24?_ga=1.52620968.1407169386.1426657565

2) http://cl.ly/032K0M3f2q2Y?_ga=1.52620968.1407169386.1426657565

As you can see there are many spikes in the data, but it's aggravating because I can SEE what it should look like. I can see that it's simply the "minimum" of where the spikes are occurring (maybe call it the baseline? Im not sure). Basically if those spikes just came down with the rest of the neighboring data it's clear that I would get the correct output, since it should just increase and decrease linearly for the most part. I hope that makes sense. Is there any way I can filter this, or develop some sort of algorithm? I can't seem to get anything working-the latest thing I tried was a median filter but I lost the shape of my data in the process. Is there anyway I can get this filtered into what it obviously SHOULD look like? I would really appreciate any help whatsoever.

Upvotes: 0

Views: 172

Answers (1)

LowPolyCorgi
LowPolyCorgi

Reputation: 5171

This problem is similar to a classical problem of neuroscientists, who often want to detect spikes from a noisy, slowly fluctuating background. There are many solutions to this problem, among the most basic (but quite efficient !) is to use the prctile function on a sliding window.

Here is a try, where I computed the baseline BL and then kept only the points that are close from the baseline:

% --- Parameters
w = 5;                   % Half-width of the sliding window
th = 25;                 % Threshold for closiness to the baseline

% --- Load and plot data
a = load('pos1.mat');
x = 1:numel(a.pos1);
y = a.pos1;
plot(x, y, '.');
hold on

% --- Find the baseline
BL = y*NaN;
for i = 1:numel(x)

    i1 = max(1,x(i)-w);
    i2 = min(x(i)+w, numel(x));

    BL(i) = prctile(y(i1:i2), 25);
end

plot(x, BL, 'r-');

% --- Select the points close to the baseline
I = abs(BL-y)<=th;
scatter(x(I), y(I), '+');

And the result:

Result

Then, depending on your needs you can keep only these data points or interpolate to obtain the missing values.

Best,

Upvotes: 2

Related Questions