Finding subpages without links

[patrickbotros]

I'm wondering if there's a way to find all the subpages on a website even if the mainpages don't have any links to them. I bet you could keep guessing the names of the subpages but that would take forever.

 

[Mark44]

I'm wondering if there's a way to find all the subpages on a website even if the mainpages don't have any links to them. I bet you could keep guessing the names of the subpages but that would take forever.

I don't think there is a way to do this without knowing the file structure of the web site.

 

[jtbell]

You can sometimes find files by "backwhacking" a URL, that is, entering partial URLS corresponding to (sub)directories (folders), with a slash at the end. If an Apache server is configured to allow directory listings and a directory does not contain an index.html file (or whatever other files the server recognizes as "index files" e.g. index.php), this will give you a listing of the files in that directory.

However, if the directory contains an index.html or similar file, you will always get that. If the directory listings are disabled, and there is no index.html, you will get an "access denied" message or something similar.

 

[Fooality]

An additional really simple tool for finding pages on a site is to do a search like:
"doctor who" site:physicsforums.com
on Google. Google will index pages not linked to on the main page if they are linked to elsewhere on the net, so if a page here talked about Doctor Who and was not indexed from any page, it could still show up if another page linked to it. (Edit: At least they used to, I haven't tried it lately)

 

[patrickbotros]

You can sometimes find files by "backwhacking" a URL, that is, entering partial URLS corresponding to (sub)directories (folders), with a slash at the end. If an Apache server is configured to allow directory listings and a directory does not contain an index.html file (or whatever other files the server recognizes as "index files" e.g. index.php), this will give you a listing of the files in that directory.

However, if the directory contains an index.html or similar file, you will always get that. If the directory listings are disabled, and there is no index.html, you will get an "access denied" message or something similar.

How?

 

[DaveC426913]

If an Apache server is configured to allow directory listings

Unlikely in this day & age.

 

[jtbell]

How?

How what, exactly?

 

[DaveC426913]

How?

If you hit page www.coolwebsite/about/contactus.html
You'll get a legit page.

If you hack the URL to www.coolwebsite/about/
you might get lucky and see the /about/ directory structure. (But you'd have to be pretty lucky these days.)