Wayback Machine to the rescue (almost)

  • Thread starter Thread starter jtbell
  • Start date Start date
  • Tags Tags
    Machine
Join the discussion
Ask a follow-up here, or get your own question answered by working scientists, mathematicians and engineers — people, not an autocomplete.
Real named experts · corrections over time · the nuance an AI answer skips
2 replies · 3K views
Messages
16,110
Reaction score
8,368
Last December, my college's Web server crashed because of a hard-disk failure. As I was fiddling with my most recent backup of my own Web pages, I clumsily managed to delete that, too. I had to resort to a much older backup that was missing several pages that I had created since then. I put the missing pages on my "to do" list to re-create, but never got around to doing it.

Just today in a thread elsewhere on PF, someone mentioned the Wayback Machine, which I had forgotten about. I used it to search for my Web site, and voilà, there were the missing pages! :!)

There was just one problem. The pages are part of a large photo gallery, and all the images are in a directory that I've forbidden to Web crawlers via a robots.txt file. (Some people were slurping hundreds of pictures at once, and bogging the server down.) So the Wayback Machine has the Web page text, including the picture captions, but not the images. :cry:

At least I can find the images again in my collection, based on the captions and URLs, but it will take some time to track them down and fix them up in Photoshop again. To make things easier in the future, I've added an entry to robots.txt that allows the Wayback Machine's crawler to fetch my images, while still forbidding other crawlers from doing so.
 
Physics news on Phys.org
"About the Wayback Machine:
Browse through 85 billion web pages archived from 1996 to a few months ago. "


"Thank you very much for considering us in your charitable giving. We appreciate and rely on donations from people like yourselves. The Internet Archive is a 501(c)(3) non-profit organization, therefore your donations are tax deductible as allowed by law."
 
Last edited: