How to Handle Outliers in Array Data in Fortran?

[kevin86]

Hello

I have a bunch of data in a text file that got loaded into an array. Now the problem is that there are a lot of outliers. So let's say if I find the outlier, how do I get rid of it, by say move the next one to that spot. If I do that how do I redefine the arrary size. What would be the best way to do it? I don't need to do it my way. I just need to get rid of the outliers anyway possible

 

[mathmate]

Without knowing the order of magnitude of your data size (100 record, or 1000000 records?), and how frequently you find your outliers, that is once and for all, occasionally, 3 times a day, every time you run..., it is not easy to suggest an optimal solution.
For now, I could suggest some possibilities:

1. If possible, eliminate the 'outliers' while reading into the array, this way, you get rid of the problem at the source (once and for all).

2. If option 1 is not possible, you could write a utility program to write the 'good' records to a file and read it back into your original program every time you encounter an 'outlier'.
In the case of F90, you have the option of using allocatable arrays instead of using a disk file. You only have to allocate a new array of the correct size, and copy the good records to the new arrary, and deallocate the old one containing outliers.

3. If the above steps are not acceptable for one reason or another, you will have to take the time to do some copying and shifting of the array elements, which is not really hard to do. Be sure that if the array contents are indexed, the index has to be updated as well.
You do not, however, recuperate the memory space so saved.