A question about search engines.

Click For Summary
SUMMARY

Search engines like Google and Bing may fail to find existing entries on a website due to several factors. Content may not be indexed if it was added after the last visit from search engine spiders. Additionally, site structure can hinder accessibility for these spiders, particularly if content is embedded in JavaScript. Furthermore, search engines may choose not to index certain content based on the site's robots.txt file or if the content is deemed inappropriate or illegal.

PREREQUISITES
  • Understanding of search engine indexing processes
  • Familiarity with robots.txt and the Robots Exclusion Standard
  • Knowledge of website structure and accessibility for crawlers
  • Basic comprehension of JavaScript and its impact on SEO
NEXT STEPS
  • Research how to optimize website structure for better search engine crawling
  • Learn about the Robots Exclusion Standard and its implications for indexing
  • Explore the effects of JavaScript on search engine visibility
  • Investigate tools for monitoring search engine indexing status
USEFUL FOR

Webmasters, SEO specialists, and content creators looking to improve their site's visibility and indexing by search engines.

mech-eng
Messages
826
Reaction score
13
Can you explain me why search engines sometimes can not find existing entries. I can find them in the site, I can see them but Google and Bing can not find them.

Best Regards.
 
Technology news on Phys.org
Maybe the search engines simply haven't found that content yet. The content might have been added after the search engine spiders last visited the site. Or the site might be structured in such a way as to make the content inaccessible (or at least not easily accessible) to the spiders. For example, it used to be the case that Google couldn't crawl links inside Javascript code; I haven't kept up with this so I don't know if it's still true.
 
What you see and what the search engine sees are sometimes two different things.

Even when the search engine sees something it may choose to not index it. Sometimes per the owner site request - http://en.wikipedia.org/wiki/Robots_exclusion_standard, sometimes because it decides site contains stolen/illegal material.
 

Similar threads

Lowering the Golden Ratio: The Impact on Golden Section Search Efficiency
  • · Replies 1 ·
Replies
1
Views
2K
MHB How to check if search problem has no solution in depth limit search?
  • · Replies 1 ·
Replies
1
Views
1K
Java FileFinder using Breadth First Search algorithm fails
  • · Replies 2 ·
Replies
2
Views
2K
MHB Pretty good privacy materials needed
  • · Replies 1 ·
Replies
1
Views
953
Java How to make a search engine by Java ?
  • · Replies 2 ·
Replies
2
Views
5K
Creating Search Queries using indents
  • · Replies 8 ·
Replies
8
Views
1K
Potassium Bichromate/Dichromate
  • · Replies 2 ·
Replies
2
Views
2K
Confused about tiny issues in backend software development
  • · Replies 3 ·
Replies
3
Views
2K
Python Is there any way to web-scrape a website that's down?
  • · Replies 8 ·
Replies
8
Views
2K
Generating electrical energy using an intermediate reference frame
  • · Replies 40 ·
2
Replies
40
Views
4K