Programming style in R - to array-process or not?

  • Thread starter andrewkirk
  • Start date
  • Tags
    Programming
In summary, R enables processing arrays by single statements rather than via loops, by having almost all commands able to apply to arrays as well as to scalars. This allows for more concise and elegant code, as well as faster execution.
  • #1
andrewkirk
Science Advisor
Homework Helper
Insights Author
Gold Member
4,121
1,719
TL;DR Summary
Can array-processing capability sometimes be more of a burden than a boon? Do you feel moral pressure to avoid loops when using an array-processing-capable language? Do you give in to the pressure?
R enables processing arrays by single statements rather than via loops, by having almost all commands able to apply to arrays as well as to scalars. I seem to recall, when I first learned it (a long while ago now) feeling that the intro texts, and user comments online, encouraged the practice of avoiding loops where possible.

I find the ability to process an array without looping very powerful, and it is dead-easy when one wants to do something like add two arrays componentwise, so we just write c <- a + b rather than having to write a nest of loops with one nesting level for each dimension of the arrays. But it becomes difficult when one has to perform operations that are more abstract, and may have several stages. I have become good at this over the years, and enjoy the elegance of a piece of code that does something really complex and multi-dimensional in a handful of lines. But I am often nagged by the thought that I could still do it much faster if I just wrote out the loops and performed the operations componentwise. Some purist (puritanical?) streak in me usually stops me from doing that, even when I feel sure I could write the code for that in two minutes, whereas it takes me half an hour, and much searching to find specialist helper functions, to do it without loops.

I have been learning Python recently and get the feeling that, while Python can do some array processing, there is no philosophy of avoiding loops. I find that rather liberating.

I'd love to hear the opinions of other R users as to whether they have ever felt a similar pressure to avoid loops, the extent to which their code avoids loops as a result, ad whether it takes them a long time to find a non-looping solution where a looping solution would be quicker to write, despite taking up more lines.

There may be other languages for which similar issues arise. I used APL thirty years ago and seem to recall that as having array-processing capabilities and a philosophy of avoiding loops where possible. I'd be interested in comments on this issue for other languages too.

Another reason I try to avoid loops is a belief that code that uses array processing capabilities will generally run faster than code with explicit loops, since R is an interpreted language. I don't know if there's anything in that.
 
Computer science news on Phys.org
  • #2
Sometimes implicit array processing can be parallelized in ways hand written loops cannot And that’s the primary reason for using these constructs.

However array processing can be more memory intensive with intermediate results storage.

one example, I saw was with APL programming Where a very compact program computed the prime numbers from 1 to 100000. It did it by making 1000000x1000000 array with each row representing a number and each column of each row containing a 0 or 1 to indicate that the column number was a factor of the row number. Then is summed over all columns to make a single row of counts.

a filter was applied over the single row to create a list of column numbers where the row sum was 2 Meaning the number had two factors 1 and itself. That list was the primes numbers from 1 to 1000000.

anyway, you can see in this example the heavy use of memory initially.
 
  • Like
Likes andrewkirk
  • #3
In power systems analysis, the matrices are very sparse, with 98% of off-diagonal elements zero. We use special sparse matrix libraries and methods, and of course we really don't allocate or store those 98% zero elements.

I've been retired for a long time, so my info is dated. Can modern compilers and parallel processing features deal efficiently with sparse matrices?
 
  • Like
Likes andrewkirk

Related to Programming style in R - to array-process or not?

1. What is the difference between array-processing and non-array-processing in R?

Array-processing in R involves using vectorized functions to perform operations on entire arrays of data, while non-array-processing involves using loops or other iterative methods to manipulate data one element at a time.

2. Which approach is more efficient in R - array-processing or non-array-processing?

In general, array-processing is more efficient in R because it takes advantage of R's built-in vectorization and avoids the overhead of loops. However, this may vary depending on the specific task and data being processed.

3. What are the advantages of using a programming style that favors array-processing in R?

Using a programming style that favors array-processing in R can lead to more concise and readable code, as well as faster execution times. It can also help avoid common errors that can occur when using loops or other iterative methods.

4. Are there any situations where non-array-processing may be preferred over array-processing in R?

Yes, there may be certain tasks or data sets where non-array-processing may be more efficient or easier to implement. For example, if the data set is very large and cannot fit into memory, using a non-array-processing approach may be necessary.

5. Is there a specific programming style that is considered best practice for array-processing in R?

There is no one "best" programming style for array-processing in R, as it may vary depending on the specific task and individual preferences. However, it is generally recommended to use built-in vectorized functions whenever possible and to avoid using loops unless necessary.

Similar threads

  • Programming and Computer Science
Replies
5
Views
1K
  • Computing and Technology
Replies
9
Views
2K
  • Computing and Technology
Replies
3
Views
2K
  • Programming and Computer Science
Replies
17
Views
2K
  • Engineering and Comp Sci Homework Help
Replies
5
Views
1K
  • Engineering and Comp Sci Homework Help
Replies
7
Views
2K
  • Programming and Computer Science
Replies
13
Views
3K
  • Programming and Computer Science
Replies
2
Views
1K
  • Engineering and Comp Sci Homework Help
Replies
15
Views
3K
  • Computing and Technology
Replies
2
Views
820
Back
Top