Dismiss Notice
Join Physics Forums Today!
The friendliest, high quality science and math community on the planet! Everyone who loves science is here!

Advise on writing program to collect visual sighting reports online autonomously

  1. Aug 25, 2011 #1


    User Avatar
    Gold Member

    Greetings, i'm an undergraduate electrical engineer with minimal programming experience and i'm here because I would like some advice as to how I should approach writing a program i'm interested in. I'm specifically interested in what programming language I should learn and use.

    The purpose of the program is to autonomously search the internet (with the help of Google's search engine) for user reports of a specific visual phenomenon called ball lightning. This program would be for the purpose of data collection in respect to the date and location (eventually latitude and longitude) of the sightings. This program would compile a matrix of report data of the previously specified 2 data points.

    An example of the data I expect the program to analyze is the following report from http://amasci.com/weird/unusual/bl.html"

    I would expect the program to be able to single out this report from the whole page of them, look for key words including "On Tuesday 29th June 2010 at 12.10am" and "USA". Then input the date and location into a matrix.

    Another, more challenging example:
    The program would need to recognize that there was no specific location listed in the sighting, but the keywords "family, table, kitchen" would indicate the sighting was at the reporter's home and would resort to the location listed next to the date the sighting was listed on the website; opposed to another location listed in the report.
    Data collected from this report:
    Date: 1948
    location: Hopkinton, MA, USA

    What programming language would you recommend I use to write such a program and what is the level of difficulty of such an endeavor?

    Thank you for your time,
    Last edited by a moderator: Apr 26, 2017
  2. jcsd
Share this great discussion with others via Reddit, Google+, Twitter, or Facebook

Can you offer guidance or do you also need help?