# MATLAB and averaging without NaN

1. Jun 10, 2017

### joshmccraney

Hi PF!

Let's say we have a vector $x = [1, \,2,\, 3, \,20,\, 4,\, 5,\, 30,\, 6]$. I am trying to loop through and find the magnitude of the derivative of each $i^{th}$ element with unitary spacing, and if that derivative is sufficiently high, I make that element NaN and continue.

Example: Let's call the derivative of the $i^{th}$ element $d_i\equiv|d_{i-1}-d_{i+1}|/2$. Then $d_2 = |3-1|/2=1$. Similarly, $d_3=|20-2|/2=9$. However, if $d_i$ is greater than, say $5$, I want to make $x_{i+1}=NaN$ and recalculate the derivative using now $x_{i-1}$ and the next non-NaN number. So, as a first loop through $d_3=|20-2|/2=9$ which is above 5, so $x_4=NaN$ and thus $x = [1, \,2,\, 3, \,NaN,\, 4,\, 5,\, 30,\, 6]$. Then recalculating $d_3=|4-2|/2=1$. Then $d_4=|4-3|/2=1/2$. Also, $d_5=|3-5|/2=2$.

If we continue $x = [1, \,2,\, 3, \,NaN,\, 4,\, 5,\, NaN,\, 6]$. So far what I have is
Code (Matlab M):

x = [1 2 3 20 4 5 30 6];
for i = 2:length(x)-1
if isnan(x(i-1))==0
d(i) = abs(x(i+1)-x(i-1))/2;
if d(i) > 5
x(i+1) = NaN;
end
else
???
end
end

I don't know how to write what I've described above, and I could be way off here. Any help would be so awesome!

Last edited by a moderator: Jun 16, 2017
2. Jun 10, 2017

### Staff: Mentor

I would put print statements in the iteration and step through your code and at each step you can verify whether MATLAB is doing what you want if not then you know right where to fix it.

3. Jun 16, 2017

### joshmccraney

My solution for sake of others:
Code (Matlab M):

%% smooths outliers to avoid clusters
critslope = 3;% critical slope filter
for i = 2:length(x)-1
if isnan(x(i-1))==0
slope(i) = abs(x(i+1)-x(i-1))/(2*dz);
if slope(i) >= critslope
x(i+1) = NaN;
end
count = 1;% reset count in case multiple NaN packets
else
count = count+1;% increase count since NaN detected
slope(i) = abs(x(i+1)-x(i-count))/(2*dz);
if slope(i) >= critslope
x(i+1) = NaN;
end
end
end

%% rewrite NaN as previous value
for i = 2:length(x)-1
if isnan(x(i))==1
x(i)=x(i-1);
end
end

Last edited by a moderator: Jun 16, 2017
4. Jun 16, 2017

### Staff: Mentor

How did you figure out what was wrong?

5. Jun 16, 2017

### joshmccraney

It's not so much what was wrong, it was more just trying to get something in the "???" statement of my first post. After lot's of thinking about it the above is what came out. I googled and googled but didn't find anything, else I'd post a link.

I definitely followed your advice and printed the final $x$ results of the simple vector above, knowing exactly what the result should look like, and modified the conditions and the "count" until it worked.

At any rate, this should smooth data assuming the first data point is not an outlier.