Is There an App to Track Website Updates?

  • Thread starter Thread starter 600burger
  • Start date Start date
  • Tags Tags
    Tracking Web
AI Thread Summary
There is a need for an application or tool, preferably free or built into Windows, to track website updates, specifically for monitoring legislative bills. The discussion highlights that the HTTP headers from servers can provide a last-modification timestamp, which can be utilized to determine if a webpage has been updated. For those familiar with programming, a simple script in Python can be created to send requests to multiple URLs, record the timestamps, and compare them. It is noted that dynamically generated pages, such as those created with PHP, may not have modification timestamps, but using an "If-Modified-Since" header in requests can effectively check for updates by analyzing response codes. This approach is suggested as a practical solution for tracking changes on legislative websites.
600burger
Messages
63
Reaction score
0
Anyone know of any app (preferably free, or built into windows) that can track a website and tell me if the information on that page has been updated.

I'm working on tracking a large amount of Legislation bills and need to be updated on when they are being ammending/voted/enacted.

Thanks,
-Andrew
 
Computer science news on Phys.org
Interesting, not sure how I would go about doing that. How much of the page has to change in order for it to be marked as "new."
 
The HTTP headers returned by the server include a last-modification timestamp. If you know just about any programming language, it should be an easy task to send requests to the server for a bunch of URLs, record the timestamps, and compare them with the last timestamps.

It'd probably be 10-15 lines of Python.

- Warren
 
I was thinking something around the same lines but I checked the headers of these forum pages and didn't see any timestamp.
 
Well, these forum pages are generated dynamically by PHP, so they don't have modification times.

Normal html files should, however.

I actually just realized an easier way. Send all your requests with an If-Modified-Since: header, and just read the response code.

- Warren
 
Yep, you are right. I forgot php pages show the modified date as the time you are accessing the page.

heheo:)

In that case the IF statement would work perfectly.
 
Back
Top