Array shuffling in fortran

In summary, the conversation discusses the issue of outliers in a large dataset stored in an array and the possible ways to handle them. The options include eliminating outliers while reading the data, using a utility program to write good records to a file and read them back in, or manually copying and shifting array elements. The most optimal solution depends on the size of the dataset and frequency of encountering outliers.
  • #1
kevin86
20
0
Hello

I have a bunch of data in a text file that got loaded into an array. Now the problem is that there are a lot of outliers. So let's say if I find the outlier, how do I get rid of it, by say move the next one to that spot. If I do that how do I redefine the arrary size. What would be the best way to do it? I don't need to do it my way. I just need to get rid of the outliers anyway possible
 
Technology news on Phys.org
  • #2
Without knowing the order of magnitude of your data size (100 record, or 1000000 records?), and how frequently you find your outliers, that is once and for all, occasionally, 3 times a day, every time you run..., it is not easy to suggest an optimal solution.
For now, I could suggest some possibilities:

1. If possible, eliminate the 'outliers' while reading into the array, this way, you get rid of the problem at the source (once and for all).

2. If option 1 is not possible, you could write a utility program to write the 'good' records to a file and read it back into your original program every time you encounter an 'outlier'.
In the case of F90, you have the option of using allocatable arrays instead of using a disk file. You only have to allocate a new array of the correct size, and copy the good records to the new arrary, and deallocate the old one containing outliers.

3. If the above steps are not acceptable for one reason or another, you will have to take the time to do some copying and shifting of the array elements, which is not really hard to do. Be sure that if the array contents are indexed, the index has to be updated as well.
You do not, however, recuperate the memory space so saved.
 
  • #3
.Hello,

There are a few different approaches you could take to address the issue of outliers in your array. One option could be to use a statistical method, such as calculating the mean and standard deviation of the data and removing any values that fall outside a certain range (e.g. more than 3 standard deviations from the mean). This approach would involve redefining the array size by creating a new array with only the non-outlier values.

Another approach could be to use a sorting algorithm, such as quicksort or mergesort, to rearrange the elements in the array so that outliers are at the beginning or end. You could then truncate the array to remove these outliers. This approach may be more efficient if you have a large amount of data.

It's also important to consider why the outliers are present in the first place. Are they legitimate data points or errors? If they are errors, it may be necessary to go back and correct the source of the data rather than simply removing them from the array.

Ultimately, the best approach will depend on your specific data and goals. I would recommend consulting with a statistician or experienced programmer to determine the most appropriate method for your situation.
 

What is array shuffling in Fortran?

Array shuffling in Fortran is a process of randomly rearranging the elements of an array. It is often used in scientific computing to create new data sets for statistical analysis or to improve the performance of algorithms.

How do you shuffle an array in Fortran?

To shuffle an array in Fortran, you can use the RANDOM_NUMBER function to generate a random number, and then use this number as an index to swap the elements of the array. This process can be repeated multiple times to achieve a more thorough shuffling.

What are the benefits of array shuffling in Fortran?

Array shuffling in Fortran can help to eliminate any patterns or biases in the data, making it more representative and suitable for statistical analysis. It can also improve the efficiency of certain algorithms that rely on random data.

Can array shuffling be applied to multidimensional arrays in Fortran?

Yes, array shuffling can be applied to multidimensional arrays in Fortran. The same process of generating a random number and swapping elements can be used for each dimension of the array.

Are there any built-in functions for array shuffling in Fortran?

No, there are no built-in functions specifically for array shuffling in Fortran. However, the RANDOM_NUMBER function and the ability to manipulate array elements make it relatively easy to implement shuffling in Fortran.

Similar threads

  • Programming and Computer Science
Replies
1
Views
415
  • Programming and Computer Science
Replies
20
Views
3K
  • Programming and Computer Science
Replies
2
Views
1K
  • Engineering and Comp Sci Homework Help
Replies
7
Views
1K
  • Programming and Computer Science
Replies
6
Views
1K
  • Programming and Computer Science
Replies
6
Views
924
  • Programming and Computer Science
Replies
5
Views
1K
  • Programming and Computer Science
Replies
4
Views
2K
  • Programming and Computer Science
Replies
20
Views
2K
  • Programming and Computer Science
Replies
13
Views
1K
Back
Top