Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Over 3,000 guests and 100 members

  1. Sep 17, 2011 #1
    WHat the heck? There are 3000 people viewing us and not even a tenth of those people are members. We have 100 members active. What are those 3000 people waiting for!??
  2. jcsd
  3. Sep 17, 2011 #2

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    A lot of them are spiders. Ever since Google changed the way search engines rank web sites the web has been crawling with little critters. Some of these spiders emanate from commercial engines, some are written by students just to learn how to create a web crawler, some are commercial entities crawling the web for their own nefarious purposes. Some people who have strong interest in some topic will write their own crawlers rather than relying on search engines.
  4. Sep 17, 2011 #3


    User Avatar

    Staff: Mentor

    These all show up as "guests" which are supposed to be real people. Unless the reporting has changed, the spiders show up as such and will say "yahoo" "google" etc...

    It does seem odd that lately I haven't seen any spiders.
  5. Sep 17, 2011 #4
    PF also has a lot of solved problems in his database. So some people just google a phrase and look at the solved problem at PF. I don't really like this, but I guess that should count for some visitors.
  6. Sep 17, 2011 #5
    Most are real guests. They pop in from random google searches.
  7. Sep 17, 2011 #6
    What are the spiders u guys refering to? I don't want to be in the dark
  8. Sep 17, 2011 #7
  9. Sep 17, 2011 #8
  10. Sep 17, 2011 #9
    Read the wiki.

  11. Sep 17, 2011 #10

    D H

    User Avatar
    Staff Emeritus
    Science Advisor

    To find web pages. Crawling took on much greater importance with PageRank (google that term), where the "Page" in PageRank stands for Larry Page, not web page. Page's PhD thesis was about a new kind of search engine that used a technique similar to how librarians and academicians decide which are the most important journal papers. It's a fairly simple concept: Count the number of times a paper is referenced by some other paper.

    Now think of the web. Suppose you are the author of a sports blog and you write an article about AC Milan (I'm watching AC Milan v Barcelona right now). In this article you happen link to AC Milan's home page. Lots of other people will do the same, in various contexts. In fact, if you want to know about AC Milan the best place to go is to AC Milan's home page. Because so many people link to this page in reference to discussions of AC Milan, PageRank will quickly find that this is the place to go for info on AC Milan.

    This is part of the reason why you rarely need to go to page 13 when you do a google search. Google's goal is to make the page that you think is the best source of information on topic X is the very first page they list in their search (the very first page they list after the paid ads that relate to topic X, that is).
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook