Searching inside .zip files on internet

  • Thread starter Thread starter Vespero
  • Start date Start date
  • Tags Tags
    files Internet
Click For Summary
SUMMARY

Searching inside .zip files on the internet without downloading them is not feasible with current technology. Users must download the .zip files to access their contents, as no search engines or tools can index files within .zip archives directly. HTTP version 1.1 allows for byte serving, which can help in narrowing down files based on filename, but ultimately the entire .zip file must be retrieved to search its contents. Custom code can be written to open and search .zip files locally after download, but this does not solve the initial problem of searching remotely.

PREREQUISITES
  • Understanding of HTTP version 1.1 and byte serving
  • Familiarity with .zip file structure and compression
  • Basic programming skills for writing custom search code
  • Knowledge of file handling libraries for .zip files in programming languages
NEXT STEPS
  • Research HTTP byte serving techniques and their applications
  • Explore libraries for handling .zip files in programming languages such as Python or Java
  • Investigate existing tools or services that may offer remote searching capabilities for compressed files
  • Learn about web scraping techniques to automate the download and search process for .zip files
USEFUL FOR

Web developers, data scientists, and anyone involved in file management or searching for specific content within compressed files on the internet.

Vespero
Messages
26
Reaction score
0
Is anyone aware of a way to search through .zip files on the internet (such as in an archive site) without having to download and extract the files? For example, if I have a search phrase that may be in a .doc file inside a .zip file which is potentially stored with many other .zip files, I don't want to have to download them all and have to manually search through them, but would like to be able to at least find the correct .zip to download first.

Many thanks.
 
Computer science news on Phys.org
Vespero said:
Is anyone aware of a way to search through .zip files on the internet (such as in an archive site) without having to download and extract the files? For example, if I have a search phrase that may be in a .doc file inside a .zip file which is potentially stored with many other .zip files, I don't want to have to download them all and have to manually search through them, but would like to be able to at least find the correct .zip to download first.

Many thanks.

If your using HTTP version 1.1 then yes because you can use ranges.

http://en.wikipedia.org/wiki/Byte_serving
 
SixNein said:
If your using HTTP version 1.1 then yes because you can use ranges.

http://en.wikipedia.org/wiki/Byte_serving
I think he is hoping to use a search engine - and I don't think any of them do what he wants.

If he's writing the search code himself, your byte range thing would be useful if he could eliminate many of the files based on their filename. But whenever a zip file contained only files like *.txt, *.docx, he would still need to read the whole zip file.
 
I can't think of any way for a search bot to look inside any file out on a web server without first retrieving the file to local disk. It seems irrelevant whether the file is compressed (zipped) or not. If you are writing code, you can certainly get a library to open the zip so it can be searched as plain text or whatever format you expect.

Unless perhaps you were able to inject pernicious code onto the web server itself so that the code runs THERE, but that would be only if you are permitted to add code to a site. It would not apply to most sites. If your code is looking at files on the web, they must first be fetched to your local disk by the HTTP client. Period.
 

Similar threads

  • · Replies 15 ·
Replies
15
Views
2K
  • · Replies 2 ·
Replies
2
Views
2K
  • · Replies 7 ·
Replies
7
Views
3K
  • · Replies 1 ·
Replies
1
Views
2K
  • · Replies 18 ·
Replies
18
Views
4K
  • · Replies 6 ·
Replies
6
Views
2K
Replies
7
Views
15K
  • · Replies 2 ·
Replies
2
Views
2K
Replies
3
Views
3K
  • · Replies 5 ·
Replies
5
Views
901