Is There an App to Track Website Updates?

  • Thread starter Thread starter 600burger
  • Start date Start date
  • Tags Tags
    Tracking Web
Click For Summary

Discussion Overview

The discussion revolves around finding an application or method to track updates on websites, specifically for monitoring changes to legislation bills. The focus includes technical approaches and programming solutions.

Discussion Character

  • Exploratory
  • Technical explanation
  • Debate/contested

Main Points Raised

  • Andrew inquires about free apps or built-in Windows features to track website updates for legislation bills.
  • One participant questions how much of a webpage needs to change for it to be considered "new."
  • Warren suggests using HTTP headers, specifically the last-modification timestamp, and proposes a programming solution in Python to track changes by comparing timestamps.
  • Another participant notes that certain forum pages do not have modification timestamps due to being dynamically generated by PHP.
  • Warren offers an alternative method using the If-Modified-Since header to check for updates based on response codes.
  • A later reply acknowledges the limitation of PHP pages showing the access time instead of a modification time, agreeing that the If-Modified-Since method would be effective.

Areas of Agreement / Disagreement

Participants express varying levels of understanding and approaches to the problem, with some agreeing on the technical methods proposed while others raise questions about the feasibility and specifics of implementation. No consensus is reached on a single solution.

Contextual Notes

There are limitations regarding the assumptions about website structures, the nature of dynamic content, and the specifics of HTTP headers that may affect the proposed solutions.

600burger
Messages
63
Reaction score
0
Anyone know of any app (preferably free, or built into windows) that can track a website and tell me if the information on that page has been updated.

I'm working on tracking a large amount of Legislation bills and need to be updated on when they are being ammending/voted/enacted.

Thanks,
-Andrew
 
Computer science news on Phys.org
Interesting, not sure how I would go about doing that. How much of the page has to change in order for it to be marked as "new."
 
The HTTP headers returned by the server include a last-modification timestamp. If you know just about any programming language, it should be an easy task to send requests to the server for a bunch of URLs, record the timestamps, and compare them with the last timestamps.

It'd probably be 10-15 lines of Python.

- Warren
 
I was thinking something around the same lines but I checked the headers of these forum pages and didn't see any timestamp.
 
Well, these forum pages are generated dynamically by PHP, so they don't have modification times.

Normal html files should, however.

I actually just realized an easier way. Send all your requests with an If-Modified-Since: header, and just read the response code.

- Warren
 
Yep, you are right. I forgot php pages show the modified date as the time you are accessing the page.

heheo:)

In that case the IF statement would work perfectly.
 

Similar threads

  • · Replies 10 ·
Replies
10
Views
11K
Replies
5
Views
2K
  • · Replies 1 ·
Replies
1
Views
2K
Replies
12
Views
2K
  • · Replies 15 ·
Replies
15
Views
3K
  • · Replies 19 ·
Replies
19
Views
4K
  • · Replies 10 ·
Replies
10
Views
2K
  • · Replies 2 ·
Replies
2
Views
3K
  • · Replies 1 ·
Replies
1
Views
8K
  • · Replies 3 ·
Replies
3
Views
4K