Python Loging into a webpage using python.

[dacruick]

Hi,

I am trying to access some data online, but I am having trouble getting past the actual authentication process.

import cookielib
import urllib, urllib2

if __name__ == '__main__':
    urlLogin = '[PLAIN]https://www.hobolink.com'[/PLAIN] 

    uid    = 'userid'
    password = 'xxxxxxx'

    fieldId   = 'username'
    fieldPass = 'password'
    
    ButtonId = 'submit'
    Button = 'Log in'

    cj = cookielib.CookieJar()
    data = urllib.urlencode({fieldId:uid, fieldPass:password, ButtonId:Button})

    opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))

    urllib2.install_opener(opener)
    usock = opener.open(urlLogin)
    usock = opener.open(urlLogin, data)
    #pageSource = usock.read()
    usock.close()
 
    usock = opener.open('FinalWebpage')
    pageSource = usock.read()
    usock.close()
    print(pageSource)

The HTML code which corresponds to the Username, Password, and login button are respectively as follows.

<input id="username" name="username" type="text">

<input id="password" name="password" type="password">

<li id="submit"><input class="button" name="commit" onclick="alertForExplorerBrowserVersion(7, 3);" type="submit" value="Log in"></li>

After I try to access the sought after link, it redirects me back to the authentication page. So two possible things are happening. The first one is, I am not entering anything into the username and password fields. The second possibility is that I am not "clicking" the log in button, but instead am trying to open a page that I am not authenticated to open.

 

[Coin]

One thing I notice is that "hobolink" (do you have their permission to use their site in this way..?) uses POST in their form, whereas urllib will produce GET queries.

 

[dacruick]

One thing I notice is that "hobolink" (do you have their permission to use their site in this way..?) uses POST in their form, whereas urllib will produce GET queries.

hmm, I'm pretty new at this so I'm not sure exactly what the difference is, but that would be consistent with my results thus far