Why Does My Perl Script Crash with Large Data Sets on Raspbian?

  • Thread starter Thread starter Borek
  • Start date Start date
AI Thread Summary
A user encountered issues running a Perl script on a Raspberry Pi with Raspbian, particularly with large data sets. The script performed well with smaller data but failed to complete with larger inputs, initially running but then stopping without error messages. Initial assumptions about memory limitations were dismissed after monitoring CPU and RAM usage, which showed low RAM consumption. Further investigation revealed that the problem was related to differences in line endings (CRLF vs LF) in the data sets. The issue was resolved after identifying this discrepancy.
Borek
Mentor
Messages
29,132
Reaction score
4,556
Disclaimer: the only thing I know about Perl is the language name.

Raspbian on Pi (512 MB RAM).

I have a Perl script that I want to use. And it works - sort of.

Problem is, it works OK for small data sets, but it fails for large ones. When it works, it works for some time and then it spits out the result. In case of known errors (like a wrong file name) it displays an error message (while I don't know Perl I see these are coded in the script). When it doesn't work - it initially works, then just ends, without any messages.

My first odea was that it is limited by the memory. For a large set of data I see it (with top) using 98% of CPU and allocating more and more memory, but when it stops it is at 3% RAM, so just about 15 MB, not that much.

Any ideas what I can do? Is there a way of checking why it crashes? Can it be related to some memory limit per process? Or is there some limit set to amount of memory available for Perl?

I did some blind googling but all I see seems to be suggesting memory should not be a problem here.
 
Technology news on Phys.org
OK, for now ignore my message.

I did some more checks and it looks like the problem can't be memory related. Data sets are just list of words. Script works for a fairly short data set that I entered manually, but it never works for a more complicated ones, prepared by other script (we are talking 3 words vs 84 words). It means there must be some other problem.
 
Sigh. CRLF vs just LF.

Case closed.
 
can you post the script and the dataset?

Edit: Just saw that you solved your problem.
 
Thread 'Is this public key encryption?'
I've tried to intuit public key encryption but never quite managed. But this seems to wrap it up in a bow. This seems to be a very elegant way of transmitting a message publicly that only the sender and receiver can decipher. Is this how PKE works? No, it cant be. In the above case, the requester knows the target's "secret" key - because they have his ID, and therefore knows his birthdate.
Back
Top