C++ help- my code reads from small files but not large

[davidsmith315]

Hello, I'm trying to make part of a code that reads a csv file with data containing the date and time, and the wind speed at that time, then does various output things.
When I copy and paste 1000 entriesinto a csv file and ru it, it works great, but if I use the file that's about 6,000kb it misses every second date and time part out, code anyone help me with this?
I'm a complete novice to programming- this is part of my Electrical Engineering course.

heres my code:

#include <iostream>
#include <sstream>
#include <fstream>
#include <vector>
#include <string>
#include <ctime>
#include <FileHC.h>

using namespace std;

int main ()
{	  ofstream outFile("C:\\WindSpeedAnswer.csv");
  if (!outFile) {
	  cout<< "error opening file"<<endl;
	  return -1;
  }
  
  
    ifstream inFile ("C:\\Copy of Kirkwall_23_hm_proc15.csv");
    string line;
    int linenum = 0;
   if (inFile.is_open())
  {
	  double randomMax;
	  double randomMin;
	  cout<<"would you like to add a random varience, enter 'yes' if you do "<<endl;
	 string Question;
	cin >> Question;
	string answerYes= "yes";
	if (Question==answerYes){
	cout<<"enter the max value of random varience "<<endl;
	cin>>randomMax;
	randomMax=randomMax-1;

	
	cout<<"enter the min value of random varience "<<endl;
	cin>>randomMin;
	randomMin=randomMin-1;}

	while (getline (inFile, line))
    {
        linenum++;
       // cout << "\nLine #" << linenum << ":" << endl;
        istringstream linestream(line);
        string item;
        int itemnum = 0;
		
        while (getline (linestream, item, ','))
        {
            itemnum++;
			double WindSpeed=0;
		string DateAndTime;
	
              //cout << "Item #" << itemnum << ": " << item << endl;
			if (itemnum%2 == 1) {
		
			DateAndTime=item;

				
		string replace= "-99999";	while(DateAndTime==replace){
			DateAndTime= "Data Missing";}

	cout<<"at "<<DateAndTime<<endl;
		outFile<< DateAndTime<<","	;}
			
		
	if (itemnum%2 == 0) {
		
	WindSpeed=atof(item.c_str());
	if(WindSpeed==-99999){
			
	WindSpeed=9;
			}
	if (Question==answerYes){
	for(int index=0; index<itemnum; index++){
	double X = randomMax+ rand() * (randomMax - randomMin) / RAND_MAX;
		if (randomMin<X<randomMax){
		WindSpeed=X;}
			}}
cout<<"the wind speed is "<<WindSpeed<<","<<endl;
		
outFile<<WindSpeed<<","<<endl;
//outFile.write(WindSpeed); 
			}
		}
			}
		
	 inFile.close();
 }

  else cout << "Unable to open file"; 
  

  
	  outFile.close();
  
  char stopchar;
  cin>>stopchar;
  return 0;
}

 

[Hobin]

Here's something to try: See if it still doesn't work when you only take the first 100 lines or so of the 6,000kb file. If it still doesn't work, check the file for errors. It's much easier to check a 100-line file, and it's quite possible that there are a few thingies that make your program go weird.

 

[davidsmith315]

Re Hobin:
Thanks fo reply
It work great if i take the firs 100 lines, or 1000 lines- its only when I run the 6,oookb file it doesntwork

 

[Hobin]

It work great if i take the firs 100 lines, or 1000 lines- its only when I run the 6,oookb file it doesntwork

That's odd. To be perfectly honest, I can't find anything in your code that would make it stop working at some arbitrary amount of data - have you tried running your code on a different machine? Try splitting the file in the middle (if that's possible), if it works, take 3/4 of the file, if it doesn't, take 1/4 - repeat until you find the boundary where it stops working.

 

[M Quack]

Sounds like you have a memory leak somewhere, for example objects are being created inside the loop, but never deleted.

For better readability, I would put the treatment of each complete line (everything inside the while(getline) loop) into a subroutine.

 

[davidsmith315]

Thanks for replies, I tried using half the input file and it failed, but using 1/4 f it worked perfectly.

Re M Quck- "For better readability, I would put the treatment of each complete line (everything inside the while(getline) loop) into a subroutine"
Sorry to ask but how would I go about doing this?
I tried this:
if (itemnum%2 == 1) {

DateAndTime=item;
string replace= "-99999";
while(DateAndTime==replace){
DateAndTime= "Data Missing";}
count<<"at "<<DateAndTime<<endl;
}
if (itemnum%2 == 0) {
WindSpeed=atof(item.c_str());
if(WindSpeed==-99999){
WindSpeed=9;
}
if (Question==answerYes){
for(int index=0; index<itemnum; index++){
double X = randomMax+ rand() * (randomMax - randomMin) / RAND_MAX;
if (randomMin<X<randomMax){
WindSpeed=X;}
}}

count<<"the wind speed is "<<WindSpeed<<","<<endl;

outFile<<DateAndTime<<","<<WindSpeed<<","<<endl;



It nearly worked- except it left out the time and date part- the time and date row was blank, but the WindSpeed row was output perfectly

 

[Hobin]

Thanks for replies, I tried using half the input file and it failed, but using 1/4 f it worked perfectly.

Very good, now try (3/4)/2 part of the file.

I hope you see where I'm going with this. By cutting what's left in half every time, you could theoretically very quickly go to the point where you come to the specific line where your file fails to work properly. Going to that extreme is probably unnecessary, but as you 'zoom in' on the part where it fails to work, take a very good look at possible anomalies in the file.

 

[rcgldr]

Couldn't you use a source level debugger to see where the failure occurs? If you have windows, you can get microsoft visual c++ express for free, and it includes a source level debugger.

 

[Hobin]

^That, too. Alternatively, if you use GCC, you can obviously download and use the GDB.

 

[and9]

Whenever this happens for me, I just use GDB and run the code with the proper arguments and print the stack trace to see where the fault occurred.

Doing what the previous user suggested above in terms of essentially binary searching for the line where the code stops at is helpful but won't solve your problem if the issue is a memory leak. That style of debugging will only be helpful if that's the line of input error.

GDB will tell you if the input is faulty or if you have a memory leak. Also consider running your executable with memcheck to see if you have memory leaks.