[C++] Quickly finding directory files using boostfilesystem

[PPeter]

I'm working on a program which copies pdf's from one directory to another using boost filesystem. My problem is that the directory I'm grabbing files from contained about 14000 files (not including files in sub-directories). So iterating through each one to find something isn't very practical.

I just started using the library, so I don't know the ins and outs. As far as I can see, the only way to get information from the directory is through the directory iterators (and I believe it automatically sorts entries in alphanumeric order from what I've seen). I've thought about incrementing the iterator by something more than 1, but anything I read about when doing that (for iterators in general) warns against trouble when it gets close to the end.

My initial plan was to keep breaking all the directory entries into smaller segments to find what I'm looking for. IE: jump to the middle entry between 2 limits, compare that filename with what I'm looking for, then split the appropriate segment into half again. Then keep going until it's found. I'm not sure if something like this is even possible, or suggested when using iterators. So if anyone has any suggestions or ideas, I'm open to hearing them.


Thanks,

Peter

 

[jedishrfu]

So you're searching for a file by name from a list of 14000 files. If this is a once only search then the iterator is the only way unless boost handles file masks so you could ask for a list of files matching the mask and then search the reduced list iteratively.

If it's a repeated operation then you could iterate through the list and place each file name in an array where you can apply the binary search scheme you mentioned earlier. Although this only works if you know the first few letters of the file name not if the search string is embedded inside the filename.

I found this API reference for boost software:

http://www.boost.org/doc/libs/1_34_0/libs/filesystem/doc/index.htm

 

[PPeter]

So you're searching for a file by name from a list of 14000 files. If this is a once only search then the iterator is the only way unless boost handles file masks so you could ask for a list of files matching the mask and then search the reduced list iteratively.

If it's a repeated operation then you could iterate through the list and place each file name in an array where you can apply the binary search scheme you mentioned earlier. Although this only works if you know the first few letters of the file name not if the search string is embedded inside the filename.

I found this API reference for boost software:

http://www.boost.org/doc/libs/1_34_0/libs/filesystem/doc/index.htm

Ah, this is very helpful. It doesn't seem that boost::filesystem has the functionality to do that on it's own, but by using boost:regex to filter, it should be able to speed things up dramatically.

Thanks