BitTorrent file saving problem

  • Thread starter Yoss
  • Start date
  • Tags
    File
In summary, my friend and I are developing a BitTorrent Client for our CS class, but we've run into a dilemma. If anyone can point us in the right direction we'll be thankful.
  • #1
Yoss
27
0
My friend and I are developing a BitTorrent Client for our CS class, but we've run into a dilemma. If anyone can point us in the right direction we'll be thankful.

A little background info if you're unfamiliar with BT. The file that is to be downloaded from multiple peers is broken up into file_length/32K (+ 1) pieces (conventionally.) And typically BT clients do not choose the pieces in order to download (they use the rarest first algorithm.) That ought to be enough!

Anyway, the scheme we developed was to create a 2D byte array of size [num_pieces][piece_length]. So once we've successfully downloaded a piece from a Peer and checked the infohashes and such, we dump the piece into its position in the byte array. We realized the problem of seriously overloading memory because the array in the data part of memory. Obviously this is a terrible solution..imagine downloading a 2Gb file and it all being stored in system memory (or VM) until it is finished.

We developed a partial solution. Once the file is completed, we dump all the pieces to the output file. To note, we create a temporary .dat file when the user prematurely quits before the file is complete, we save meta info (e.g. amount downloaded, uploaded, etc) and the 2D byte array with the downloaded pieces. When the program is loaded again it loads the data through a hashmap, etc. and the byte array is loaded into memory ONLY if the file is not complete. This is what we still have to fix. When the file is complete, this condition is checked when the user quits and the byte array of pieces is dumped to the output file, but not the .dat file. What we came up with, when the program is loaded again and we are uploading to peers, when the peer requests a piece, we just scan the save file and extract the bytes for the piece from there. This works like a charm because we don't have to load all the pieces into memory, at most , 32KB*(num peers uploading to). Much better solution.

When the file is not done, and we are uploading to peers the pieces we have, we need to develop a way to get the pieces off of the disk. But we don't necessarily download the pieces in order, and we might not get all of the pieces with the current peers. If anyone can give us suggestions of how to dump a piece to disk once we get it, and be able to extract it upon request, given that the pieces do not come in order. And how to assemble the pieces in order when we dump it to the final file.

Sorry for the longwindedness! Thanks
 
Technology news on Phys.org
  • #2
What all bittorrent clients I've seen do is pre-allocate the file on disk and then write the data to file when the chunk is completed. You can use fseek to offset the file pointer and write the chunk where you need to.

Also, I don't know if you know but there is a bittorrent library that you could have used:

http://libtorrent.rakshasa.no/ [Broken]
 
Last edited by a moderator:
  • #3
Do you not know how to use the seek() function?

- Warren
 
  • #4
dduardo said:
What all bittorrent clients I've seen do is pre-allocate the file on disk and then write the data to file when the chunk is completed. You can use fseek to offset the file pointer and write the chunk where you need to.

Also, I don't know if you know but there is a bittorrent library that you could have used:

http://libtorrent.rakshasa.no/ [Broken]


Thanks dduardo. I bet that would have been the solution we would come to . We are using the seek() function to extract the pieces from the 100% file, I just didn't think of pre-allocation, duh! I was thinking a solution of keeping track of the order of downloaded pieces, writing the pieces in the order downloaded, and rearranging them at the end. Your solution is definitely more natural (and easier.)

chroot, do I dare say that your reply approaching scorn? Funny feeling, is all.
 
Last edited by a moderator:

1. Why is my BitTorrent file not saving properly?

The most common reason for a BitTorrent file not saving properly is internet connection issues. If your internet connection is slow or unstable, it can interrupt the downloading process and cause the file to not save properly.

2. How can I fix the BitTorrent file saving problem?

One solution is to check your internet connection and make sure it is stable. You can also try restarting the download or using a different torrent client. Another option is to check if there are any firewalls or antivirus software that may be blocking the download.

3. Can I resume a failed download?

Yes, most torrent clients have the option to resume a failed download. Make sure to check the settings or options in your specific torrent client to see if this feature is available.

4. What should I do if my BitTorrent file is stuck at 99%?

This is a common issue and usually means that there are no seeders for the file you are trying to download. Without seeders, the file cannot be fully downloaded. You can try finding another torrent for the same file or wait for more seeders to become available.

5. Are there any other factors that could affect BitTorrent file saving?

Yes, other factors that could affect BitTorrent file saving include low disk space on your computer, incorrect settings in your torrent client, or a corrupted file. It's important to check these factors if you are experiencing issues with saving your BitTorrent files.

Similar threads

  • Programming and Computer Science
Replies
1
Views
174
  • Programming and Computer Science
2
Replies
65
Views
2K
  • Engineering and Comp Sci Homework Help
Replies
5
Views
2K
  • Quantum Interpretations and Foundations
2
Replies
37
Views
1K
  • Set Theory, Logic, Probability, Statistics
Replies
2
Views
2K
  • Computing and Technology
Replies
7
Views
3K
  • Programming and Computer Science
Replies
4
Views
15K
Replies
10
Views
2K
  • Art, Music, History, and Linguistics
Replies
1
Views
1K
Back
Top