How can I use the find function to replace missing values in a 4D array?

Click For Summary

Discussion Overview

The discussion revolves around using the find function in MATLAB to replace missing values in a 4D array representing climate variables. Participants explore alternatives to using if statements for efficiency in handling missing values, which are denoted as 10^15 and need to be converted to NaN for better plotting.

Discussion Character

  • Technical explanation
  • Exploratory
  • Debate/contested

Main Points Raised

  • One participant shares their current method using nested for loops and if statements to replace missing values, expressing concerns about efficiency.
  • Another participant suggests consulting the MATLAB documentation for the find function, indicating it can return linear indices or indices by dimension, but is uncertain if it will improve performance.
  • A participant questions the legality of a specific MATLAB statement for replacing values directly in the array and compares it to Python's numpy syntax, noting potential limitations with 4D arrays.
  • Another participant reiterates the legality of the MATLAB statement, suggesting that instead of using 0/0 to generate NaN, it is more straightforward to assign NaN directly to the array.

Areas of Agreement / Disagreement

Participants express differing views on the efficiency and legality of various methods for replacing missing values. There is no consensus on the best approach, and multiple competing views remain regarding the use of the find function versus direct assignment.

Contextual Notes

Participants note potential limitations regarding the performance of different methods and the applicability of certain statements to 4D arrays. There is also mention of warnings related to division by zero when using 0/0 to generate NaN.

Who May Find This Useful

This discussion may be useful for MATLAB users working with multi-dimensional arrays, particularly in the context of climate data analysis and those interested in optimizing code for efficiency.

Halsey
Messages
3
Reaction score
0
Hi all-

I have a question regarding using the find function as opposed to if statements in finding values in a 4D array. These are climate variables, such as temperature (in Kelvin). Missing values are represented as 10^15, and I want to change them to NaN which plots much nicer. I have the following code so far:


for i = 1:288 %all longitudes
for j = 1:144 %all latitudes
for k = 1:42 %all heights
for l = 1:8 %all times
if temp(i,j,k,l) > 1000
temp(i,j,k,l) =0/0;
end
end
end
end
end

and it is doing what I want, but I fear that it is not very efficient because i have 11 climate variables total to do this for, and the if conditionals waste a lot of time. i have used the find function on 1D and 2D arrays, but I do not know how it works, if at all, on 4D arrays. Any help would be great! Thanks in advance.
 
Physics news on Phys.org
Take a look at the MATLAB documentation for the 'find' function:
http://www.mathworks.com/access/helpdesk/help/techdoc/ref/find.html

You can either get linear indices (i.e. your 4-d array gets squashed down into a single vector), or you can get indices returned to you by dimension (take a look at one of the provided examples; it should generalize to more than two dimensions).

I'm not sure that it'll be any faster (maybe with some parallelization) but you could use 'tic' and 'toc' to see how it compares with the looping and scanning method. If it goes too fast, you may be able to do it several times inside a loop, and then divide the result by the number of times you looped.
 
Is this a legal statement in MATLAB:
temp(temp>1000)=0/0;

I'm more familiar with Python's numpy ( http://www.scipy.org/NumPy_for_Matlab_Users )
and they refer to an example under Linear Algebra Equivalents:
MATLAB: a(a<0.5)=0
NUMPY: a[a<0.5]=0
Notes: a with elements less than 0.5 zeroed out
(If this example only works for 2D arrays, you might have to somehow reshape your temp array or loop over slices to do this operation.)
 
robphy said:
Is this a legal statement in MATLAB:
temp(temp>1000)=0/0;

I'm more familiar with Python's numpy ( http://www.scipy.org/NumPy_for_Matlab_Users )
and they refer to an example under Linear Algebra Equivalents:
MATLAB: a(a<0.5)=0
NUMPY: a[a<0.5]=0
Notes: a with elements less than 0.5 zeroed out
(If this example only works for 2D arrays, you might have to somehow reshape your temp array or loop over slices to do this operation.)

I was going to make a point about this. Instead of doing a 0/0 and getting a warning about your division by 0 (which you can turn off by writing "warning off" at the top of your m-file, or before you execute your statements), you can set them directly to NaN, with an assignment:

>> bob = 0/0;
>>% gives an error
>> bob = NaN;
>>% no error!
 

Similar threads

  • · Replies 10 ·
Replies
10
Views
4K
  • · Replies 3 ·
Replies
3
Views
2K
Replies
5
Views
2K
  • · Replies 5 ·
Replies
5
Views
4K
Replies
3
Views
3K
Replies
2
Views
2K
  • · Replies 12 ·
Replies
12
Views
4K
  • · Replies 2 ·
Replies
2
Views
13K
  • · Replies 3 ·
Replies
3
Views
5K
  • · Replies 2 ·
Replies
2
Views
2K