Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Web site update tracking?

  1. Jun 29, 2006 #1
    Anyone know of any app (preferably free, or built into windows) that can track a web site and tell me if the information on that page has been updated.

    I'm working on tracking a large amount of Legislation bills and need to be updated on when they are being ammending/voted/enacted.

  2. jcsd
  3. Jun 29, 2006 #2
    Interesting, not sure how I would go about doing that. How much of the page has to change in order for it to be marked as "new."
  4. Jun 29, 2006 #3


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    The HTTP headers returned by the server include a last-modification timestamp. If you know just about any programming language, it should be an easy task to send requests to the server for a bunch of URLs, record the timestamps, and compare them with the last timestamps.

    It'd probably be 10-15 lines of Python.

    - Warren
  5. Jun 29, 2006 #4
    I was thinking something around the same lines but I checked the headers of these forum pages and didn't see any timestamp.
  6. Jun 29, 2006 #5


    User Avatar
    Staff Emeritus
    Science Advisor
    Gold Member

    Well, these forum pages are generated dynamically by PHP, so they don't have modification times.

    Normal html files should, however.

    I actually just realized an easier way. Send all your requests with an If-Modified-Since: header, and just read the response code.

    - Warren
  7. Jun 29, 2006 #6
    Yep, you are right. I forgot php pages show the modified date as the time you are accessing the page.


    In that case the IF statement would work perfectly.
  8. Aug 13, 2009 #7
Know someone interested in this topic? Share this thread via Reddit, Google+, Twitter, or Facebook

Similar Threads - site update tracking Date
Delete automatic updates .. problem? Dec 10, 2017
How to access blocked sites Jun 10, 2017
Ref: Safe Site Request:Anaconda and Virtual Box, Please Sep 27, 2016
Non-US .edu sites? Apr 24, 2016
Free web site hosting? Jun 27, 2014