What are the Differences Between HTTP, FTP, and Telnet Protocols?

[fog37]

TL;DR Summary

difference between different protocols

Hello,

I understand that the internet represents the physical infrastructure of hosts (devices of all sorts) over which digital information is stored and is shared in a distributed manner.

I can come up with three different type of information that use the internet: WWW information (websites and webpages), email information (all emails we send and receive) and information stored files in different format (binary, etc.)

Each different type of information mentioned above uses a different application protocol: http is the protocol to request/send web info, smtp/pop the protocols to exchange emails, and ftp is the protocol for file transferring (files of any type, I guess, even web pages). Regardless of the different type of information, the protocols of the suite TCP/IP are used in all cases during the data transfer and checking phases.

Why aren't webpages transferred using the ftp protocol since it is designed to transfer information? Is ftp still a popular protocol to transfer files? I feel like any type of data we download uses the web these days...

What about telnet? Isn't it a also protocol specific for remote access to the resources of a remote computer? Are VPN a newer alternative to telnet?

Thanks!

 

[DrClaude]

You should explore https://en.wikipedia.org/wiki/Internet_protocol_suite

I can come up with three different type of information that use the internet: WWW information (websites and webpages), email information (all emails we send and receive) and information stored files in different format (binary, etc.)

Streaming should be included in there.

Why aren't webpages transferred using the ftp protocol since it is designed to transfer information? Is ftp still a popular protocol to transfer files? I feel like any type of data we download uses the web these days...

See https://daniel.haxx.se/docs/ftp-vs-http.html

I often use sftp to transfer files between computers over the internet.

What about telnet? Isn't it a also protocol specific for remote access to the resources of a remote computer?

I'm not sure anyone uses telnet anymore. SSH is more common.

Are VPN a newer alternative to telnet?

They are completely different. VPN does not give you remote access to a computer.

 

[anorlunda]

There are many more methods and protocols than the ones you list. There can be much overlap in what they do. Each one has advantages and disadvantages. Designers choose a method that works best for the application.

In the past, we had nets using protocols other than TCP/IP. They are no longer used because of TCP/IP's advantages.

Think of this application. When the button is pushed, it transmits 1 bit (pushed) plus 1 unique ID. What protocol would you use for that? It is not a file, not a page, not an email. What would influence your choice of protocol?

 

[fog37]

Thanks

There are many more methods and protocols than the ones you list. There can be much overlap in what they do. Each one has advantages and disadvantages. Designers choose a method that works best for the application.

In the past, we had nets using protocols other than TCP/IP. They are no longer used because of TCP/IP's advantages.

Think of this application. When the button is pushed, it transmits 1 bit (pushed) plus 1 unique ID. What protocol would you use for that? It is not a file, not a page, not an email. What would influence your choice of protocol?

View attachment 272570

Thanks!

Well, that is binary data being transmitted so a binary protocol. I see that there are many options in that department as well.

The main ideas is that different types of information require different protocols. Maybe there are cases in which the same type of information can be shared/sent using different protocols. I guess ftp is an example: we could view a wepage in browser communicating with the web server via http OR download the webpage from a ftp server using the ftp protocol...

 

[fog37]

You should explore https://en.wikipedia.org/wiki/Internet_protocol_suiteStreaming should be included in there.See https://daniel.haxx.se/docs/ftp-vs-http.html

I often use sftp to transfer files between computers over the internet.I'm not sure anyone uses telnet anymore. SSH is more common.They are completely different. VPN does not give you remote access to a computer.

Thank you DrClaude. I will go through the links.

I guess streaming (video, sound or both), which I was not thinking about at all, is a different type of information that requires its own protocol...

A protocol is a set of rules for intercommunication and data exchange. Assuming the same physical transmission medium (TCP/IP protocol takes care of that), the choice of the protocol is determined by the type of information and the client process (application program) that is used to receive and display the information...

 

[jbriggs444]

Think of this application. When the button is pushed, it transmits 1 bit (pushed) plus 1 unique ID. What protocol would you use for that? It is not a file, not a page, not an email. What would influence your choice of protocol?

Well, that is binary data being transmitted so a binary protocol. I see that there are many options in that department as well.

It is not the data type that is relevant. It is the access pattern, timing and reliability requirements. Here, we have a one-way transmission of a small amount of data. We are not told about any timing, reliability or security requirements, though low latency seems like a likely goal.

Absent any other requirements, this practically begs for UDP: You wait for the button press, put the unique ID into a packet and send it. The receipt of the packet indicates that the button was pushed. The recipient takes the ID from the packet and is done. No TCP connection. No acknowledgments. No retransmissions. Dead simple.

I guess streaming (video, sound or both), which I was not thinking about at all, is a different type of information that requires its own protocol...

Not the data format. The access pattern.

With streaming data, you have a whole different set of requirements. You need to be able to deliver a data stream in near real time.

You might want to use something like https://tools.ietf.org/html/rfc2326

RFCs are often good reading. Although you can certainly get lost in the weeds.

I guess ftp is an example: we could view a wepage in browser communicating with the web server via http OR download the webpage from a ftp server using the ftp protocol...

But how will you know whether that "web page" is formatted in HTML. Or is a .GIF. Or a .JPG. Or some script code? Or is some opaque binary data to be downloaded? The HTTP protocol allows one to attach metadata (headers) to the response. And to attach metadata to the request to tell the server about browser capabilities so that the server has the option to customize the content based on the client's ability to display it.

With FTP all you get is a file name. You do not even get file length unless (by default and by near universal practice) you are willing to trust that end-of-stream means end-of-file.

What about telnet? Isn't it a also protocol specific for remote access to the resources of a remote computer? Are VPN a newer alternative to telnet?

Telnet is barely a protocol. It is a thin layer on top of raw TCP.

A telnet client will take your keyboard input and send it byte for byte as transmit data on a TCP stream to the other end. Meanwhile, data received from the other end is displayed byte for byte on your screen. A telnet server on the other end is most commonly hooked up to a service that creates an interactive login session. Data received on the stream is presented as input to the session. Display output from the session is piped into the TCP stream. [If you look in the RFC, there is provision for a little bit of stream interpretation on top of this (escape characters, and "WILL", "WON'T", "DO" and "DON'T" commands).]

For me, the most common use of telnet is as a poor man's do-anything client.

fly:2:~$ telnet www.google.com 80
Trying 108.177.122.103...
Connected to www.google.com.
Escape character is '^]'.
GET / HTTP/1.1
host: www.google.com

HTTP/1.1 200 OK
Date: Fri, 13 Nov 2020 20:09:41 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2020-11-13-20; expires=Sun, 13-Dec-2020 20:09:41 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=204=geYvM-UggM-GccGryeTQsu8ePGf4JSaV5D0CB46RDw-kyepOwEm51iLxkQVeqEgYDzEqAvbFUxOQQfmSPhfgpVf6nhYMXRIibcZGju1lqzCj9Hbwa3LMPqasz9m2DMBSyEySqaltih2V0K1nBiXyHjF2uSziKla4Hy__vhM0MLM; expires=Sat, 15-May-2021 20:09:41 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Transfer-Encoding: chunked

4e70
<!doctype html><html itemscope=""...

 

[lomidrevo]

Why aren't webpages transferred using the ftp protocol since it is designed to transfer information?

I guess ftp is an example: we could view a wepage in browser communicating with the web server via http OR download the webpage from a ftp server using the ftp protocol...

FTP uses stateful control connection, which means that server needs to maintain current state of the session for each user. It has minimal overhead in the data connection, so it is suitable to transfer big files.
HTTP is stateless protocol, so server doesn't need to maintain any specific information about the status of individual user sessions. That is very useful in case of web servers, where many users access the web page, and with each click they request a new content provided by the page. FTP is totally impractical for such purpose, that is why it is not used to transer web pages (html) to your browser.

 

[fog37]

Thanks everyone. always learning a lot from you.
Since we are on the topic, I am trying to make sense of XML, which is a markup language, to see how it fits in the information transfer process.

During the web client/web server interaction, the client (browser) makes a request to the server which sends a response, the desired HTML file which is rendered by the browser on the screen. The HTML files can be styled with CSS code. On the server-end, languages like PHP take care of the server communication with other servers or databases (web services). XML is also a markup language (tags that the developer defines) like HTML, but XML adds description to the content of in the HTML document.

Does the client, aka the browser, make any specific and direct use of the XML code embedded into the HTML document to enhance the user's experience of the content? I don't think so.
HTML is for formatting, CSS is for styling and XML is for content description. XML, together with JSON, seems to be use to transfer information. Is it about transferring information on the server-end? Which processes are involved when information is exchanged in XML or JSON format?

 

[pbuk]

XML is also a markup language (tags that the developer defines) like HTML, but XML adds description to the content of in the HTML document.

XML and HTML exist entirely independently, XML does not 'add anything' to an HTML document.

Does the client, aka the browser, make any specific and direct use of the XML code embedded into the HTML document to enhance the user's experience of the content? I don't think so.

Whilst it is possible to embed XML in an HTML document, this is not how websites use XML to include dynamic content. The links at the end of this post tell you more.

HTML is for formatting

No

CSS is for styling

Yes

XML is for content description.

No

and XML, together with JSON, seems to be use to transfer information.

Can you give an example of a protocol that is not used to transfer information?

Is it about transferring information on the server-end?

What does this question mean?

Which processes are involved when information is exchanged in XML or JSON format?

https://en.wikipedia.org/wiki/XML
https://en.wikipedia.org/wiki/JSON

 

[lomidrevo]

XML, together with JSON, seems to be use to transfer information. Is it about transferring information on the server-end? Which processes are involved when information is exchanged in XML or JSON format?

From my point of view both, XML and JSON, are just "containers" to store or transfer structured data. No matter whether it is for client to/from server, or server-to-server communication. JSON seems to be more popular these days, as it is easier to work with, generally speaking. But it depends on the application. Here is short comparison:
https://www.w3schools.com/js/js_json_xml.asp

 

[fog37]

I familiarized with both XML and JSON and learned that they are both data formats indeed. If I want to send someone a file with data inside it, should I then consider either XML or JSON as data formats? Why not just send a CSV or else?

In regards to XML, I see how it is a markup format with tags that can be defined by the user. Beside HTML, there is XHTML which seems to incorporate the virtues of XML. Why? What does XML have to do with HTML?

 

[pbuk]

I familiarized with both XML and JSON and learned that they are both data formats indeed. If I want to send someone a file with data inside it, should I then consider either XML or JSON as data formats? Why not just send a CSV or else?

If you want to send someone a file then CSV can be a good choice: it is simple to read and because data in rows and columns is familiar to a human, any errors are relatively easy to identify.

But for communication between computer systems, the simple format of CSV confers no advantage, and its lack of a consistent specification makes it a poor choice.

In regards to XML, I see how it is a markup format with tags that can be defined by the user. Beside HTML, there is XHTML which seems to incorporate the virtues of XML. Why? What does XML have to do with HTML?

XHTML was an attempt to address some perceived inconsistencies in HTML 4, but in many people's opinion the problems it introduced were worse than the ones it attempted to solve. The introduction of HTML 5 solves the problems with HTML 4 and makes XHTML largely irrelevant.

 

[pbuk]

Of course you should also realize that CSV is only suitable for data that fits in an NxM matrix; for communication between humans of more structured data there is no better format IMHO than YAML.

 

[lomidrevo]

Why not just send a CSV or else?

For tabular data, CSV is the simplest option, I agree. But many applications needs to store/send more complex, nested structures. Moreover, JSON and XML (with XSD schemas) can recognize some basic datatypes. In case of CSV, it depends on the end application, how are the text data interpreted.

 

[fog37]

Thank you.

JSON has a fixed number of data structures (object, array, string, Boolean, null, number). A single JSON object is essentially a Python dictionary with various key:value pairs and a JSON file is a .json file containing multiple objects, i.e. multiple dictionaries together.

When learning about JSON, the terms "serialization" and "deserialization" came up. Serialization is the process of converting an object (like a Python dictionary) into a stream of bytes (a byte is an ASCII character) that can be sent across a wire. The process of converting it back to an object is called deserialization. So serializing converts an object to a string and deserializing converts a string to an object. A string is a sequence of alphanumerical characters and each character is represented by a group of bits.

All files, no matter their extension, are made of bit and bytes under the hood. A text file and a binary file are both made of bits but the text file has a special encoding that encodes maps those bit into characters (ASCII, UTF, etc.). Everything that is not a text file is termed a binary file (like images, sound, databases, etc.) and are made of bits representing information in different ways depending on the file extension. Is that correct?
So is there anything unique and specific about JSON and serialization and deserialization? Isn't every file that is transferred and exchanged over the internet sent as a sequence of bytes? Every file needs to be converted to a "string" so there is always serialization... Is that correct?

 

[jbriggs444]

All files, no matter their extension, are made of bit and bytes under the hood.

That is the near-universal convention these days, yes. A file on disk is a sequence of eight-bit bytes.

I learned the trade in an operating environment where things were different (VAX/VMS). A file on disk there was a sequence of records, possibly with some additional organization such as index keys. In that environment, preparing a file for transfer to a non-VMS destination involved a transformation from on-disk format to network transfer stream format.

A text file and a binary file are both made of bits but the text file has a special encoding that encodes maps those bit into characters (ASCII, UTF, etc.). Everything that is not a text file is termed a binary file (like images, sound, databases, etc.) and are made of bits representing information in different ways depending on the file extension. Is that correct?

Yes, mostly. Though the file extension is only a convention. It has no built-in meaning. You can put text into a .EXE or executable code into a .HTML.

So is there anything unique and specific about JSON and serialization and deserialization? Isn't every file that is transferred and exchanged over the internet sent as a sequence of bytes? Every file needs to be converted to a "string" so there is always serialization... Is that correct?

If you have a file on disk (and you are not using VAX/VMS), the data is already serialized, ready for transmission. You do not need to think of it as transferring the data. You can think of it as transferring a file. The meaning of the bits in the file is irrelevant to the transfer process.

Note that you do not always transfer the entire file in one go. Protocols like SMB will let a client retrieve data from specified offsets within a file -- e.g. "give me 50 bytes starting at offset 1024" or "overwrite 3 bytes at offset 12 with this data". That way a client over here can access a file system on a server over there without having to do a bulk copy of every file, every time it is accessed (and incurring the data inconsistency problems associated with doing that).

 

[pbuk]

A single JSON object is essentially a Python dictionary with various key:value pairs

JSON is nothing to do with Python and the comparison is not helpful. The letters stand for 'JavaScript Object Notation' so if you want to compare with the data structure of any language then JavaScript would be the obvious one.

and a JSON file is a .json file containing multiple objects, i.e. multiple dictionaries together.

No, a valid json file contains only one root element.

When learning about JSON, the terms "serialization" and "deserialization" came up. Serialization is the process of converting an object (like a Python dictionary) into a stream of bytes (a byte is an ASCII character)

Characters in JSON are Unicode (in practice almost always UTF-8) not ASCII.

A string is a sequence of alphanumerical characters

No, a string is a sequence of any characters not just letters and numbers.

All files, no matter their extension, are made of bit and bytes under the hood. A text file and a binary file are both made of bits but the text file has a special encoding that encodes maps those bit into characters (ASCII, UTF, etc.). Everything that is not a text file is termed a binary file (like images, sound, databases, etc.) and are made of bits representing information in different ways depending on the file extension. Is that correct?

I think it is better to think of text files as a subset of binary files.

So is there anything unique and specific about JSON and serialization and deserialization?

Yes, JSON is (currently) uniquely specified by RFC 8259.

Every file needs to be converted to a "string" so there is always serialization... Is that correct?

No. When you watch a video on YouTube, look at a picture on Instagram, download a PDF file or in many other circumstances there is no conversion to a string involved.

 

[fog37]

Thanks!

-- I would say that any file that contains bits (all files do) that can be converted into a human-readable format (i.e. text), using whichever encoding scheme we like (ASCII, UTF, etc.), represents a text file. That is the definition I would use to distinguish a text file from a non-text file. Example: a word document file, a notepad .text file and even a JSON file are text files since we can "read" their content in an intelligible way.

-- Images, sound and database data are instead types of data that cannot be read but consume differently: images are not read but viewed, for example. These type of data are therefore not encodable as text. Binary files have extensions like .bin, .jpeg, etc.

-- I didn't know that the extension has no functional meaning but it is only descriptive. Thanks.

-- I am unsure about a database file: we can open a database and "read" its numerical information and make sense of it since the bits/bytes have been encoded as words and decimal numbers which are human-readable characters with a meaning.

-- Also, what about a text file, say .RTF, which contains images? The bits representing the image are not text so it is not completely a pure text file...

 

[jbriggs444]

-- I would say that any file that contains bits (all files do) that can be converted into a human-readable format (i.e. text), using whichever encoding scheme we like (ASCII, UTF, etc.), represents a text file. That is the definition I would use to distinguish a text file from a non-text file.

The useful distinction between text and non-text from the standpoint of a network engineer is based on the FTP protocol as described in RFC 959. If you have a "text file" on your system and want to send that file so that I end up with a "text file" on my system then we will use the ASCII transfer mode.

You start with your standard on-disk format. You convert your text to ASCII. You convert your line ending sequences into standard format. You send the result over the wire. I pick up the wire format version, parse out the line ending sequences, convert the text from ASCII to my native character set and store the result in my standard on-disk format.

If both of our systems share the same on-disk format then we can cut out the middle man, transfer the file in binary mode and save all the fuss over data encoding.

 

[jbriggs444]

-- I didn't know that the extension has no functional meaning but it is only descriptive. Thanks.

Conventions surrounding the file extension vary.

Under VAX/VMS (where I learned) the file extension was treated separately from the file name. Originally the file name was limited to 9 characters and the extension to 3 characters. This was expanded to a 39 character file name with a 39 character extension at the version 4.0 release as I recall. When opening a file, one could specify a default file extension. When compiling a Fortran program, the compiler would default the input to an extension of .for and the output to an extension of .obj. So one could issue a command:

$ fortran myfile​

and the compiler would compile myfile.for to produce myfile.obj

Still, nothing prevented you from doing

$ fortran myfile.xyz​

and the compiler would dutifully compile myfile.xyz to produce myfile.obj.

There was also a versioning scheme built into the file system. But you do not care about that.Under MS-DOS and Windows 3.11 the file name was limited to 8 characters and the file extension was limited to 3. I do not remember when this restriction was relaxed to allow free format file names. Today you can still see traces of this 8.3 legacy if you use the "dir /x" command from the command line.

The association between file extension and file type currently is placed in the registry. For a particular file type, the registry specifies what is to happen (i.e. what program will be used) to "open" a file of that type. If you key in the file name on the DOS command line or right-click or double-click to open the file from an Explorer window, the registry is consulted and the specified application is used to open the file.Under Unix, file names are free format and no meaning is associated with the file extension. Instead, the first few bytes of the file conventionally contain a magic number to identify the file type. The first two bytes of the magic number are ASCII "#!". The next bytes identify the file type. For shell scripts, it will have the path where the shell is to be found. e.g. "#!/usr/bin/sh" or "#!/usr/bin/perl".

When executing a file, the first few bytes will be consulted to determine whether the file is an executable that can be run directly or whether the specified shell needs to be activated to interpret the file.

Still, most folks try to use descriptive file names and use .sh extensions on Korn or Bourne shell scripts, .csh on C shell scripts, .pl for perl, etc. Executables and directory names are normally created without an extension.

-- Also, what about a text file, say .RTF, which contains images? The bits representing the image are not text so it is not completely a pure text file...

Does it matter what we call it? What matters is what we do with it.

If both sender and receiver know how to handle .RTF files then transfer the file as-is (binary) and be done with it.

If only one end can handle .RTF files then some conversion will be required. It may take someone with human intelligence to figure out how to achieve the desired goals.