A question about search engines.

AI Thread Summary
Search engines may fail to find existing entries on a website for several reasons. One common issue is that the content may have been added after the search engine's last crawl, meaning it hasn't been indexed yet. Additionally, the website's structure could hinder search engine spiders from accessing the content, particularly if it relies on technologies like JavaScript, which may not be fully crawlable. Even if search engines can access the content, they might choose not to index it due to directives from the site owner, such as those outlined in the robots exclusion standard, or if the content is deemed to contain stolen or illegal material.
mech-eng
Messages
825
Reaction score
13
Can you explain me why search engines sometimes can not find existing entries. I can find them in the site, I can see them but Google and Bing can not find them.

Best Regards.
 
Technology news on Phys.org
Maybe the search engines simply haven't found that content yet. The content might have been added after the search engine spiders last visited the site. Or the site might be structured in such a way as to make the content inaccessible (or at least not easily accessible) to the spiders. For example, it used to be the case that Google couldn't crawl links inside Javascript code; I haven't kept up with this so I don't know if it's still true.
 
What you see and what the search engine sees are sometimes two different things.

Even when the search engine sees something it may choose to not index it. Sometimes per the owner site request - http://en.wikipedia.org/wiki/Robots_exclusion_standard, sometimes because it decides site contains stolen/illegal material.
 

Thread 'Learning Assembly and computer architecture for x86'

Dear Peeps I have posted a few questions about programing on this sectio of the PF forum. I want to ask you veterans how you folks learn program in assembly and about computer architecture for the x86 family. In addition to finish learning C, I am also reading the book From bits to Gates to C and Beyond. In the book, it uses the mini LC3 assembly language. I also have books on assembly programming and computer architecture. The few famous ones i have are Computer Organization and...
View full post »

Thread 'Learning data structures and algorithms in different programming languages'

I have a quick questions. I am going through a book on C programming on my own. Afterwards, I plan to go through something call data structures and algorithms on my own also in C. I also need to learn C++, Matlab and for personal interest Haskell. For the two topic of data structures and algorithms, I understand there are standard ones across all programming languages. After learning it through C, what would be the biggest issue when trying to implement the same data...
View full post »

Similar threads

Lowering the Golden Ratio: The Impact on Golden Section Search Efficiency
Replies
1
Views
1K
Java FileFinder using Breadth First Search algorithm fails
Replies
2
Views
2K
MHB How to check if search problem has no solution in depth limit search?
Replies
1
Views
1K
MHB Pretty good privacy materials needed
Replies
1
Views
796
Java How to make a search engine by Java ?
Replies
2
Views
5K
Creating Search Queries using indents
Replies
8
Views
1K
Potassium Bichromate/Dichromate
Replies
2
Views
2K
Confused about tiny issues in backend software development
Replies
3
Views
2K
Python Is there any way to web-scrape a website that's down?
Replies
8
Views
2K
Generating electrical energy using an intermediate reference frame
Replies
40
Views
3K

Hot Threads

Recent Insights

Back
Top