What is the significance of guests, members, and robots on a website?

[DiracPool]

I was just a little curious as how to read the "Members online now" box on the homepage. As I am typing now, it says, Total: 1,442 (members: 35, guests: 1,315, robots: 92). I get the 35 members part, I recognize many of the names. It's the 1,315 guests that's puzzling. Does that mean that there are 1,315 people on the site that are just looking around? Compared to only 35 that are participating? I don't know how many of those 1300 are steady viewers, but it just seems odd that they wouldn't spend 3-4 minutes to get a free account. They don't even have to post anything if they don't want to, what's the difference? Unless I'm misinterpreting what a "guest" is. Also, it seems as though the bulk of the people on the site are in the homework help sections. If they are not a member and can't post a question, what are they doing, hoping some member will fortuitously ask their particular homework question for them? I'm a bit perplexed. Finally, the obvious question, what is a robot (in the context discussed here), and why are there 3 times as many of them as members? Thanks in advance for any clarification :w

 

[Nugatory]

I expect that many of the guests are following links from google and other search engines. They have a question, they google it, they get a link to a PF discussion, they read it and leave.

 

[Danger]

I don't really know anything about computer stuff, but I've been here a while. It has always been my understanding that Members are people who have registered, Guests are people who have not, and Robots are automated web-crawlers such as Google might employ snooping for key words. I might be wrong, but in the old format "Robots" was listed as "Others".

edit: Oops... you snuck in ahead of me, Nugs.

 

[jtbell]

I've sometimes wondered, but never got around to asking: I assume that someone gets included in that list if he has fetched a PF page during some immediately preceding time interval, and then drops off after a while if he doesn't fetch any more pages. What is that time interval?

 

[Greg Bernhardt]

Most of the guests are coming from Google after searching for a specific question. They aren't necessarily looking for a community to join, just an answer to a question. The robots are search engine bots that are scanning pages for indexing. The time interval is 10 minutes. So that count is the number of members, guests and bots that have been active on the site within the past 10 minutes.

 

[phinds]

Most of the guests are coming from Google after searching for a specific question. They aren't necessarily looking for a community to join, just an answer to a question. The robots are search engine bots that are scanning pages for indexing. The time interval is 10 minutes. So that count is the number of members, guests and bots that have been active on the site within the past 10 minutes.

Sorry to resurrect an old thread, but Greg, where do so MANY robots come from? Are these mostly spybots looking for email addresses or what?

 

[Doug Huffman]

https://en.wikipedia.org/wiki/Web_crawler lists twenty-nine web-crawler 'bots, many for sale.
http://www.robotstxt.org/db.html lists 300+
http://www.jafsoft.com/searchengines/webbots.html

 

[Greg Bernhardt]

Just search engine bots indexing our pages

 

[jtbell]

My modest website has had 15 robots crawling it this month, according to Awstats in my cpanel.

(added) A quick search through my raw log files shows that the "bot*" and "*bot" entries probably include multiple robots, including some like Googlebot that are also listed separately. I'd have to do a detailed tally to find the actual number of robots that at least present themselves as distinct.

 

[Northward]

Mind you, the statistics reset, say, every hour. So it is the amount of users active within an hour or the last few hours rather than 'right now.'

 

[Greg Bernhardt]

Mind you, the statistics reset, say, every hour. So it is the amount of users active within an hour or the last few hours rather than 'right now.'

Five minutes

 

[phinds]

Five minutes

Gads. We get 1,500 bots every 5 minutes?

 

[Greg Bernhardt]

Gads. We get 1,500 bots every 5 minutes?

I think you're looking at the guests stat. Also it's not 1500 new guests every 5 minutes, but 1500 active guests within the past 5 minutes

 

[phinds]

I think you're looking at the guests stat. Also it's not 1500 new guests every 5 minutes, but 1500 active guests within the past 5 minutes

OK, guess I did get the bots figure mixed w/ the guests figure. Thanks.

 

[mfb]

Search engines are a large part of this bot traffic, jtbell's stats agree with my own observations. I once added link to a page that generated some random content on a new page, and stored it so the user could share the result. The link was never at a very popular place, but after a while I checked the database and had tens of thousands of stored items in it. I checked the IPs and most entries were from Google bots. Apparently the bots were happy to find new content every time they visited the page, so they checked the page frequently.