Replace NaN with next element in Multidimensional Array

Click For Summary

Discussion Overview

The discussion revolves around the problem of replacing NaN values in a multidimensional array (specifically a 49x49x49 array) with the previous non-NaN value. Participants explore various methods and considerations for efficiently handling NaN replacements in MATLAB, including potential pitfalls and edge cases.

Discussion Character

  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant proposes using nested loops to replace NaN values, suggesting that the computational load may not be excessive given the array size.
  • Another participant suggests modifying the data loading process to handle NaN values upfront or creating a list of indexes to avoid repeated searches through the array.
  • A third participant recommends using the repnan function from MATLAB's File Exchange, noting that it requires transforming the N-D array into a vector before reshaping it back.
  • Concerns are raised about edge cases, such as handling NaN values at the beginning of the array or on the edges where no previous value exists to replace them.
  • One participant provides a 2-D example of handling NaN values using imtranslate to shift data and replace NaNs with adjacent values.
  • Another participant emphasizes the importance of considering the column-major order of MATLAB arrays when accessing previous elements for replacement.

Areas of Agreement / Disagreement

Participants express differing views on the best approach to replace NaN values, with no consensus reached on a single method. Concerns about edge cases and the implications of different strategies are also highlighted, indicating ongoing debate.

Contextual Notes

Limitations include the need to address special cases where NaN values are located at the edges of the array, which may complicate the replacement process. The discussion also reflects varying assumptions about the structure and loading of the data.

Ben Wilson
Messages
90
Reaction score
16
I have a a multidimensional array (49x49x49) which has NaN values scattered randomly.

For each NaN, I want to replace the value with the previous non NaN value.

e.g. if i had a 3 x 3 matrix

4 7 9
2 7 NaN
1 2 NaN

should become

4 7 9
2 7 7
1 2 2

I have 3 dimensions [i,j,k], but it doesn't matter which dimension I move back in, so long as it is an adjacent element.

Anybody have a solution to my problem?EDIT:

I guess I could make several loops, find the NaNs and replace e.g i with i-1. But shouldn't there be a more efficient way in Matlab?
 
Last edited:
Physics news on Phys.org
How is the data loaded into the array?
Could you modify the loading to make the adjustments then?
Or at least create a list of indexes so you don't have to search through the array again?
 
cpscdave said:
How is the data loaded into the array?
Could you modify the loading to make the adjustments then?
Or at least create a list of indexes so you don't have to search through the array again?
no sadly not
 
Sadly the only way I know to do it then would be via 3 nested loops. 49*49*49 shouldn't take too long.
 
  • Like
Likes   Reactions: jim mcnamara and Greg Bernhardt
What if the first element is a nan? What if a nan is on the an i=1 edge so that there is no i-1 value? This problem can be quickly solved once those questions are addressed.
 
mfig said:
What if the first element is a nan? What if a nan is on the an i=1 edge so that there is no i-1 value? This problem can be quickly solved once those questions are addressed.
"NaN values scattered randomly."
 
Ben Wilson said:
"NaN values scattered randomly."

That does not answer my question. In your example you replace one nan by the 7 to its left. You are replaced the other nan with the two to its left. What if the nan is on the left-hand column?
 
Here is a 2-D example:

% Create sample data
data = randi(99, 10, 10);
% Introduce 7 nans at random locations
data(randperm(numel(data), 7)) = nan
% Determine location of nans.
nanMap = isnan(data);
% Need to get rid of nan's for imtranslate.
data(nanMap) = -1
% Make an array shifted one column to the right.
replacements = imtranslate(data, [1, 0])
% Initialize a copy of data for our output.
output = data;
% Replace nan's with element to the left
output(nanMap) = replacements(nanMap)
 
  • #10
vNanIdx = find(isnan(mData));

mData(vNanIdx) = mData(vNanIdx - 1);

Remember MATLAB is Column Major, hence if the Previous item is according to the row, permute the array accordingly (Or use something which depends on the dimensions of the array instead 1).

Pay attention it won't work if the first item is NaN.
 
  • Like
Likes   Reactions: FactChecker
  • #11
Drazick said:
Pay attention it won't work if the first item is NaN.
But that is easy to fix as a special case with one line.
 

Similar threads

  • · Replies 2 ·
Replies
2
Views
1K
  • · Replies 4 ·
Replies
4
Views
2K
  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 9 ·
Replies
9
Views
5K
  • · Replies 32 ·
2
Replies
32
Views
4K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 4 ·
Replies
4
Views
5K
  • · Replies 4 ·
Replies
4
Views
7K
Replies
7
Views
2K
  • · Replies 1 ·
Replies
1
Views
4K