Index Website: Learn How to Find Forgotten Files

  • Thread starter Thread starter ssb
  • Start date Start date
Click For Summary

Discussion Overview

The discussion revolves around the concept of indexing websites to access files that may not be directly linked in HTML documents. Participants explore how to view directory structures of websites, the implications of such access, and the limitations imposed by webmasters regarding directory listings.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • One participant describes the concept of website indexing, where a list of files on a server is displayed, including various file types.
  • Another participant states that accessing such lists requires specific configuration by the webmaster, and that it cannot be forced.
  • Concerns are raised about security risks associated with exposing directory structures, suggesting that it could unintentionally grant access to sensitive content.
  • Several participants suggest using Google to search for accessible directories using specific search queries.
  • There is a discussion about the difference between accessing a site via HTTP and FTP, with some participants clarifying the nature of the URLs provided.
  • One participant mentions that most web servers disable directory listing by default due to security concerns.
  • There is a technical clarification regarding the nature of the URL provided, emphasizing that it can be accessed via both HTTP and FTP protocols.

Areas of Agreement / Disagreement

Participants express differing views on the feasibility and implications of accessing website directories. While some agree on the security risks and limitations imposed by webmasters, others provide alternative methods for finding files, indicating that no consensus exists on the best approach.

Contextual Notes

The discussion highlights limitations related to web server configurations, security risks, and the nature of HTTP versus FTP access, which remain unresolved.

ssb
Messages
119
Reaction score
0
Hello, let me explain what I mean by indexing a website:

Sometimes ill click on something or manually type in a URL into the web browser and ill be brought to a "index" of everything on that site. For example at the top of the screen it will say "index of /~whatever"

It is as if there is a folder with a bunch of files in it and I can see all those files, from .html, .exe, .mp3, .txt, .pdf... basically anything that the webmaster has uploaded is displayed in a list that describes file name, size, date created, etc.

How do i manually do this to a website so I can find files that someone may have forgotten to link to in an .html document (or link incorrectly to)?

an example of an index that I found is
http://ftp.mozilla.org/pub/mozilla.org/

I just want to be able to bring up that screen for all websites.

Thanks!
 
Last edited by a moderator:
Computer science news on Phys.org
ssb said:
I just want to be able to bring up that screen for all websites.

In general, you cannot force this to happen. These lists are so-called virtual directories, and must be specifically enabled by the webmaster. If the webmaster chooses to not let you list virtual directories, then you can't, and there's no way to override it.

- Warren
 
It would be a security risk if you were able to browse the structure and possibly the server-side code of a site, and it might grant users access to content the webmasters did not intend them to have, so naturally it's not something any site should have enabled, and most don't.
 
You can use google to see if there are any accessible directories.

Type this into google:

indexof: www.physicsforums.com

And if you're lucky you'd get a listing:

https://www.physicsforums.com/Nexus/misc/
 
Last edited by a moderator:
Browsers, by default, are supposed to pull up the "index.html" file, not the actual directory structure.
 
int_80 said:
You can use google to see if there are any accessible directories.

Type this into google:

indexof: www.physicsforums.com

And if you're lucky you'd get a listing:

https://www.physicsforums.com/Nexus/misc/
Hmm, how 'bout that!

Warren...?
 
Last edited by a moderator:
russ_watters said:
Hmm, how 'bout that!

Warren...?

Oh no! They're going to steal all our... gifs! :smile:

- Warren
 
hmm.. I see you've supplied an ftp link.

Try browsing the site with an ftp client, or just go to ftp://ftp.mozilla.org/pub/mozilla.org/[/URL]
 
Last edited by a moderator:
No, he supplied an http url to a machine called ftp.

- Warren
 
  • #10
pretty sure that's an option most web servers disables by default, unless your web server supports virtual hosting and you can write your own configuration for your custom domain. But like the other posters have stated, this is a major security risk and I rather have it disabled than to have some short-term conveniences ..
 
  • #11
chroot said:
No, he supplied an http url to a machine called ftp.

- Warren
Well, it actually is an ftp site - he's just reached it via http.

You can go directly to the ftp site:
"ftp.mozilla.org/pub/mozilla.org/"[/URL]
and download stuff.

But that;s because they [I]want [/I]you to.
 
Last edited by a moderator:
  • #12
DaveC426913 said:
Well, it actually is an ftp site - he's just reached it via http.

The fact that it's accessed over http means it's a web site. There is no deeper definition. The facts that the same data is also offered via ftp, and that the computer's name is ftp, are irrelevant. He's using a web browser to communicate with a web server. He's getting a virtual directory from this web server, in HTML. This is distinct from using your web browser's built-in ftp client to access an actual ftp site, as your link would do.

- Warren
 
  • #13
chroot said:
The fact that it's accessed over http means it's a web site. There is no deeper definition. The facts that the same data is also offered via ftp, and that the computer's name is ftp, are irrelevant. He's using a web browser to communicate with a web server. He's getting a virtual directory from this web server, in HTML. This is distinct from using your web browser's built-in ftp client to access an actual ftp site, as your link would do.

- Warren


Indeed. The link would look like this if it were an ftp server:

ftp://ftp.mozilla.org/pub/mozilla.org/[/URL]

not like this:

[PLAIN]http://ftp.mozilla.org/pub/mozilla.org/
 
Last edited by a moderator:
  • #14
chroot said:
No, he supplied an http url to a machine called ftp.

- Warren

Well, yes, if you'd like to get technical. I was just keeping things simple, since 999 out of 1000 times, any http link to an ftp.website.com[/URL] URL will have an FTP service on it as well, seeing as how the http server is usually set up for compatibility with older browsers and downloaders that don't have FTP support.. ;)
 
Last edited by a moderator:

Similar threads

Learning about Python application deployment - possible recommendations
  • · Replies 7 ·
Replies
7
Views
3K
Are There Alternative Ways to Display High-Resolution Images on a Website?
  • · Replies 16 ·
Replies
16
Views
2K
Creating a Private File Storage Website with Easy Access
  • · Replies 18 ·
Replies
18
Views
4K
How to save *.wav files from pirsa.org ?
  • · Replies 1 ·
Replies
1
Views
3K
Learn How to View .he5 Files Easily | NASA NOAA Ozone Data Guide
  • · Replies 3 ·
Replies
3
Views
9K
How to Unzip an ACE File in Windows XP
  • · Replies 8 ·
Replies
8
Views
6K
Learn How to Use Siesta Software: Tutorial and Sample Files Available
  • · Replies 4 ·
Replies
4
Views
8K
Looking for formulations used in sea level calculations
  • · Replies 2 ·
Replies
2
Views
2K
Data Organization: Quickly Group Data for Easier Access
  • · Replies 6 ·
Replies
6
Views
3K
How should i do things liek this?
  • · Replies 5 ·
Replies
5
Views
3K