Register to reply

What is the use of http headers?

by adjacent
Tags: headers, http
Share this thread:
adjacent
#1
Aug29-14, 05:56 AM
PF Gold
adjacent's Avatar
P: 1,516
For example, I am downloading an image file from example.com using a python script.
Is there any need to include the headers in the script? I don't see anything happening if I don't include the headers.
So what is the need of it?
Phys.Org News Partner Science news on Phys.org
'Smart material' chin strap harvests energy from chewing
King Richard III died painfully on battlefield
Capturing ancient Maya sites from both a rat's and a 'bat's eye view'
jedishrfu
#2
Aug29-14, 08:10 AM
P: 3,097
Here's a description of the headers:

http://en.wikipedia.org/wiki/List_of_HTTP_header_fields

I know some but not all are required, they give the browser info on how to display the content that follows. In general, if any value is missing the browser defaults or uses cached values.
martiandawn
#3
Aug29-14, 06:20 PM
P: 9
As far as I'm aware of the only mandatory header in HTTP 1.1 is "Host: ". However it might be better if your script can also send appropriate headers to define the encoding, and request the TCP connection to be closed or kept open.

Medicol
#4
Aug29-14, 09:01 PM
P: 51
What is the use of http headers?

[Edit: I thought you were only trying to download an image ]

You need http header options especially in case the website in contact needs e.g your credentials for access (username, password, apikey etc). Plus, it depends on which method you use to send your packet so as to include your appropriate request header i.e partial vs full download stream, request string from your client to the server in transaction etc.

If for example you would want to download an image from the site example.com, you can just need to GET the image location provided that you've been granted your access right to the site.
In case you would want to query something from the site which has supplied you with i.e public api methods, they sure also document them on their website, visit and follow their examples.
adjacent
#5
Aug30-14, 11:59 AM
PF Gold
adjacent's Avatar
P: 1,516
Thanks for the answers. Can the site Identify me as a bot if I don't use the headers?
martiandawn
#6
Aug30-14, 12:59 PM
P: 9
Can the site Identify me as a bot if I don't use the headers?
Some sites do use headers for that, but it's very few of them and in most cases that's just for protecting copyrighted material (e.g. videos). So most likely you won't have trouble.

If you notice that it doesn't work (and you're not doing any nasty things) I'd suggest you use netcat in server mode and send a browser request to it. You'll be able to see on screen all the headers that your browser sends. Alternatively, you can use packet capture software to inspect your browser's requests to the real site.
adjacent
#7
Aug30-14, 01:13 PM
PF Gold
adjacent's Avatar
P: 1,516
Quote Quote by martiandawn View Post
If you notice that it doesn't work (and you're not doing any nasty things) I'd suggest you use netcat in server mode and send a browser request to it. You'll be able to see on screen all the headers that your browser sends. Alternatively, you can use packet capture software to inspect your browser's requests to the real site.
The 'firebug' add-on for firefox can also be used to find the request and response headers.
martiandawn
#8
Aug30-14, 01:15 PM
P: 9
The 'firebug' add-on for firefox can also be used to find the request and response headers.
Thanks for the tip!
Mark44
#9
Aug30-14, 02:29 PM
Mentor
P: 21,397
All of the major browsers (Firefox, Chrome, IE) have debugging capabilities that can be accessed from the F12 key. Using the F12 debugging tools you can see the request and response headers and quite a bit more.
jedishrfu
#10
Aug30-14, 04:03 PM
P: 3,097
Quote Quote by Mark44 View Post
All of the major browsers (Firefox, Chrome, IE) have debugging capabilities that can be accessed from the F12 key. Using the F12 debugging tools you can see the request and response headers and quite a bit more.
I've played with web apps for years and never knew about the F12 key. Thx.


Register to reply

Related Discussions
Cross Plane 180 headers for GM LS V8 Automotive Engineering 1
Excel Headers -> Mathematica Math & Science Software 0
Long tube headers vs. short tube headers. Mechanical Engineering 8
Bb ford 180 degree race headers Mechanical Engineering 19
Headers on titlepage using LaTeX Math & Science Software 5