C/C++ [C++] Quickly finding directory files using boostfilesystem

  • Thread starter Thread starter PPeter
  • Start date Start date
  • Tags Tags
    files
AI Thread Summary
The discussion centers on optimizing the search for files in a directory containing approximately 14,000 entries using Boost Filesystem. The initial approach involves iterating through all files, but this is deemed impractical due to the volume. Suggestions include using directory iterators for a one-time search or leveraging file masks to narrow down results. For repeated searches, storing filenames in an array allows for binary search, though this method requires knowing the beginning of the filename. The conversation highlights that Boost.Filesystem lacks built-in filtering capabilities, but integrating Boost.Regex can significantly enhance search efficiency by filtering filenames based on patterns.
PPeter
Messages
3
Reaction score
1
I'm working on a program which copies pdf's from one directory to another using boost filesystem. My problem is that the directory I'm grabbing files from contained about 14000 files (not including files in sub-directories). So iterating through each one to find something isn't very practical.

I just started using the library, so I don't know the ins and outs. As far as I can see, the only way to get information from the directory is through the directory iterators (and I believe it automatically sorts entries in alphanumeric order from what I've seen). I've thought about incrementing the iterator by something more than 1, but anything I read about when doing that (for iterators in general) warns against trouble when it gets close to the end.

My initial plan was to keep breaking all the directory entries into smaller segments to find what I'm looking for. IE: jump to the middle entry between 2 limits, compare that filename with what I'm looking for, then split the appropriate segment into half again. Then keep going until it's found. I'm not sure if something like this is even possible, or suggested when using iterators. So if anyone has any suggestions or ideas, I'm open to hearing them.Thanks,

Peter
 
Technology news on Phys.org
So you're searching for a file by name from a list of 14000 files. If this is a once only search then the iterator is the only way unless boost handles file masks so you could ask for a list of files matching the mask and then search the reduced list iteratively.

If it's a repeated operation then you could iterate through the list and place each file name in an array where you can apply the binary search scheme you mentioned earlier. Although this only works if you know the first few letters of the file name not if the search string is embedded inside the filename.

I found this API reference for boost software:

http://www.boost.org/doc/libs/1_34_0/libs/filesystem/doc/index.htm
 
jedishrfu said:
So you're searching for a file by name from a list of 14000 files. If this is a once only search then the iterator is the only way unless boost handles file masks so you could ask for a list of files matching the mask and then search the reduced list iteratively.

If it's a repeated operation then you could iterate through the list and place each file name in an array where you can apply the binary search scheme you mentioned earlier. Although this only works if you know the first few letters of the file name not if the search string is embedded inside the filename.

I found this API reference for boost software:

http://www.boost.org/doc/libs/1_34_0/libs/filesystem/doc/index.htm
Ah, this is very helpful. It doesn't seem that boost::filesystem has the functionality to do that on it's own, but by using boost:regex to filter, it should be able to speed things up dramatically.

Thanks
 
  • Like
Likes Medicol
Thread 'Is this public key encryption?'
I've tried to intuit public key encryption but never quite managed. But this seems to wrap it up in a bow. This seems to be a very elegant way of transmitting a message publicly that only the sender and receiver can decipher. Is this how PKE works? No, it cant be. In the above case, the requester knows the target's "secret" key - because they have his ID, and therefore knows his birthdate.
I tried a web search "the loss of programming ", and found an article saying that all aspects of writing, developing, and testing software programs will one day all be handled through artificial intelligence. One must wonder then, who is responsible. WHO is responsible for any problems, bugs, deficiencies, or whatever malfunctions which the programs make their users endure? Things may work wrong however the "wrong" happens. AI needs to fix the problems for the users. Any way to...
Back
Top