MATLAB Array Pre-allocation help

  • Context: MATLAB 
  • Thread starter Thread starter hoffmann
  • Start date Start date
  • Tags Tags
    Array Matlab
Click For Summary

Discussion Overview

The discussion revolves around the topic of array pre-allocation in MATLAB, specifically addressing issues related to memory usage and performance when handling large datasets. Participants explore methods for efficiently managing arrays within a code snippet that processes metabolite pairs.

Discussion Character

  • Technical explanation
  • Debate/contested
  • Mathematical reasoning

Main Points Raised

  • One participant shares a MATLAB code snippet and expresses difficulty with memory management in the second while loop, seeking advice on preallocating arrays to hold approximately 10,000 values and managing excess zeros.
  • Another participant suggests using the zeros() function for preallocation but questions the concern about memory usage, stating that preallocation improves speed but does not reduce memory consumption.
  • A different participant counters that preallocating does indeed lead to faster execution times when working with large datasets, providing an updated code example that includes preallocation for reactant arrays.
  • There is a suggestion to use slice operators to truncate arrays to desired sizes, although the context of this advice is not fully explored.
  • A participant raises a new question about preallocating long strings and how to efficiently create and fill them in a loop.

Areas of Agreement / Disagreement

Participants express differing views on the implications of preallocation regarding memory usage and performance. There is no consensus on the best approach to manage unused space in arrays or on the specifics of preallocating strings.

Contextual Notes

Participants do not fully resolve the issue of how to delete excess zeros from arrays after preallocation, nor do they clarify the implications of memory usage versus execution speed in detail.

hoffmann
Messages
65
Reaction score
0
Hi everyone. I'm working on a piece of code and I'm having a bit of trouble with array pre-allocation. Here it is:

function [react_in1 react_in2 react_out1 react_out2]=oneVtwoXone(masses,prec);

prec = 1e-4;
N=length(masses);
masses=[0;masses]; % adding dummy metabolite with zero mass
masses=masses(:);

%mass2=masses*ones(size(masses))'+ones(size(masses))*masses';
%mass2=mass2(:);

mass2=zeros(N*(N+1)/2,1); % list of mass pairs
met1=mass2; % list of the first metabolite in a pair
met2=mass2; % same for the second metabolite
k=0;
for i=1:N
for j=i:N
k=k+1;
met1(k)=i;
met2(k)=j;
mass2(k)=masses(i)+masses(j);
end
end

disp(['Number of distinct pairs is' length(mass2) ])

[mass2, ord]=sort(mass2); % sorted mass pairs
met1=met1(ord); % metabolites in the same order as sorted mass pairs
met2=met2(ord);

dm=diff(mass2);

while(length(find(dm <= mass2(1:length(dm))*prec)) > 0) % while have metabolite pairs that have about the same mass
ind=find(dm<mass2(length(mass2)-length(dm)+1:end)*prec);




react_in1=[react_in1 met1(ind)];
react_in2=[react_in2 met2(ind)];
react_out1=[react_out1 met1(ind+1)];
react_out2=[react_out2 met2(ind+1)];


dm = dm(1:end-1) + dm(2:end);

end

disp(['Number of conforming reactions is' length(react_in1) ])

The source of my trouble is in the second while loop. The react_in1 etc. takes up too much memory and I need to figure out how to preallocate a matrix to hold values approximately 10000, and how to delete the excess zeros if there is any unused space.

Can anyone help?
 
Physics news on Phys.org
To preallocate a large matrix, just use the zeros() function. Why do you think your matrices are taking up too much memory? Preallocating them will make your code faster, but won't make them take up any less memory.

- Warren
 
well actually I've found that preallocating takes up a significantly less amount of time. i have such a huge data set of 10000 values so my code runs faster. here's what I'm doing now:

while(length(find(dm <= mass2(1:length(dm))*prec)) > 0) % while have metabolite pairs that have about the same mass
ind=find(dm<mass2(length(mass2)-length(dm)+1:end)*prec);


react_in1=zeros(10000, 1); % preallocate a new array for all four reactants
react_in2=zeros(10000, 1);
react_out1=zeros(10000, 1);
react_out2=zeros(10000, 1);
count = 0;

for ind=1:10000

react_in1(ind)=met1(ind);
react_in2(ind)=met2(ind);
react_out1(ind)=met1(ind+1);
react_out2(ind)=met2(ind+1);
end


dm = dm(1:end-1) + dm(2:end);

end

how do i truncate the unused zeros?
 
An array of 10,000 values is actually quite SMALL.

You can use the slice operators (like dm(1:10)) to "truncate" an array to whatever size you want.

- Warren
 
how can I preallocate long strings? in other words, How can I rapidly create a string of a certain length and then fill it in a loop?
 

Similar threads

  • · Replies 8 ·
Replies
8
Views
3K
  • · Replies 9 ·
Replies
9
Views
4K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 1 ·
Replies
1
Views
4K
  • · Replies 15 ·
Replies
15
Views
6K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 45 ·
2
Replies
45
Views
7K
  • · Replies 1 ·
Replies
1
Views
3K
  • · Replies 1 ·
Replies
1
Views
9K