HTTP using cURL

The Wikipedia says, “HTTP is the foundation of data communication for the World Wide Web.”

And how does it do that? The internet communicates by exchanging data between web servers. HTTP (Hypertext Transfer protocol) standard facilitates this data exchange. HTTP uses the client-server model: An HTTP client opens a connection and sends a request message to an HTTP server; the server then returns a response message, usually containing the resource that was requested. After the response is delivered, the server closes the connection. HTTP uses verbs (or methods) to tell the server what to do with the data identified by the URL.

The most commonly used HTTP verbs are POST, GET, PUT, and DELETE. They correspond to create, read, update, and delete (or CRUD) operations, respectively. There are other verbs too HEAD, OPTIONS, TRACE, CONNECT – more information can be found at the official source IETF.

I’ll use cURL command line tool to demonstrate what these verbs do. cURL is used transfer data with URL syntax, between servers using the supported protocols (HTPP, FTP, LDAP etc). cURL comes installed mostly in all the operating systems. Or a simple yum install curl (or apt-get install curl) does it.

GET only retrieves the information (headers, message-body) and does not do any modifications. GET is the default method if no specific method is mentioned. For instance,

curl -v

This takes the default method GET and returns the headers and the body. The option -v will let the request be more verbose. Given below is what you see as part of the information received for the command given above:

GET / HTTP/1.1
User-Agent: curl/7.32.0
Accept: */*

HTTP/1.1 200 OK
Date: Sun, 07 Dec 2014 10:34:06 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html;

The first part represents the information in the request sent. GET is the method. Slash represents the URL given (we are directly requesting the host and not anything under it ( HTTP/1.1 is the version. The next three lines are the headers giving information about the request.

The next part is the response from the server. The format followed is Version, Response Code, Date and metainformation about the requested resource.This is followed by the document or what can also be called as message-body. The request returns a response code 200 (OK) when there is no error, and often returns 404 (NOT FOUND) or 400 (BAD REQUEST) in error cases.

HEAD returns the headers (metainformation) of the requested resource. It is identical to GET except it does not return a message-body in the response. GET and HEAD are hence considered Safe methods since they do not take any action other than retrieval.

PUT is mostly used for updating or modifying a resource with the request body containing the newly-updated representation of the original resource. It can also create a resource but is considered confusing (since the URI contains the value of a non-existent resource) and POST is recommended for creation (more on that below). On a successful update PUT request returns 200 response code (204 if no content in the body)

curl -v -X PUT /EmployeeInfo/12345/address -d “modified address”

The above contains the information on modified resource while -d is used to pass on the data. Option X is used to specify a method (other than default GET).
For obvious reasons of security you will not see this information being passed in URLs in user-end applications and systems. Instead this is wrapped in a form or some scripting is used. PUT is not a safe operation as it involves modification, but is Idempotent. In a sense, repeating the same operation by PUT results in an unmodified output. There can be scenarios (like incrementing a counter) where PUT is no more idempotent. For non-idempotent cases, POST is recommmended.

POST verb is often used to create resources. It requests the server to accept the entity enclosed as a subordinate of the original resource identified by the URL. The new resource is often assigned by the server and takes care of associating it to the parent (original resource).

curl -v -X POST /EmployeeInfo/

This indicates the creation of a resource under EmployeeInfo by the server. On successful creation, it returns HTTP status code 201. POST is neither safe nor idempotent. Two identical POST requests will result in creation of two resources with same information. Hence it is recommended for non-idempotent requests.

DELETE deletes the resource identified.

curl -v -X DELETE /EmployeeInfo/12345/

On successful deletion of the resource, a response 200 is returned, 202 if the request is accepted and yet to be enacted, 204 if it is enacted and no content in the response body. DELETE is idempotent since deleting the same resource will result in the same output (the entity is deleted. A point to be noted is 404 (NOT FOUND) is the response code once the resource is deleted and cannot be found anymore). In cases where DELETE decrements a counter it is no more idempotent.

OPTIONS represents a request of information about the communication options available with a resource or capabilities of a server. OPTIONS with an asterisk (*) is intended to apply to the server rather than a particular resource. Usually the response is 200 (OK) and an Allow header which specifies the HTTP methods that can be used.

curl -i -X OPTIONS

An example response would be:

HTTP/1.1 200 OK

Detailed information about curl can be found at cURL
More information about HTTP/1.1 at RFC 2616