Best Practices: Understanding REST Headers and Parameters

The REST headers and parameters contain a wealth of information that can help you track down issues when you encounter them.

Headers

In the article World Of API Testing: SOAP vs. REST Challenges, we mentioned that REST is not a standard in itself, but instead makes use of the HTTP standard. Therefore any REST call headers are in fact HTTP headers, a required component of the message. They define the operating parameters of the transaction, the so called metadata. It is considered bad practice to transmit actual data as a header field. Ideally, since each endpoint represents a state transfer, custom headers should only be used for things that do not involve the name of the resource (should be passed in the URL), the state of the resource (should be passed in the body), or parameters directly affecting the resource (should be passed as URL parameters). That leaves only true metadata about the request in custom headers.

In general, the header fields start on the second line of a message, after the request (the verb – discussed in Best Practices: Testing the Other Verbs article - and resource URL) or response (the HTTP status) lines. All fields are name-value pairs (separated by a colon), and terminated by a new line. Long lines can be folded into multiple lines; continuation lines start with either a blank space or tab character. The end of the header fields is indicated by an empty line, which is then followed by the body of the message.

There are several standards that define the required fields. An application can define any number of custom fields. By convention, custom fields started with “X-”, however this convention was dropped after several non-standard fields became standardized. The HTTP standard does not define any size limits on headers, however most servers are configured to some limit for practical reasons; good values are 8kB per field, and 100 fields per message.

Following is a brief selection of some interesting headers:

Header Field NameDescriptionExample
AcceptContent-Types that are acceptable for the responseAccept: text/plain
Accept-CharsetCharacter sets that are acceptableAccept-Charset: utf-8
Accept-EncodingList of acceptable encodingsAccept-Encoding: gzip, deflate
Accept-LanguageList of acceptable human languages for responseAccept-Language: en-CA
Cookiean HTTP cookie previously sent by the server with Set-CookieCookie: $Version=1; Skin=new;
Content-LengthThe length of the request body in octets (8-bit bytes)Content-Length: 348
Content-MD5A Base64-encoded binary MD5 sum of the content of the request bodyContent-MD5: Q2hlY2sgSW50ZWdyaXR5IQ==
Content-TypeThe MIME type of the body of the request (used with POST and PUT requests)Content-Type: application/x-www-form-urlencoded
DateThe date and time that the message was sentDate: Tue, 15 Nov 1994 08:12:31 GMT
AuthorizationAuthentication credentials for HTTP authentication Authorization:
OAuth realm="http://sp.example.test/",

oauth_consumer_key="0685bd9184jfhq22",
  oauth_token="ad180jjd733klru7",
  oauth_signature_method=
"HMAC-SHA1",

oauth_signature="wOJIO9A2W5mFwDgiDvZb
TSMK%2FPY%3D",
  oauth_timestamp="137131200",

oauth_nonce="4572616e48616d6d65724c
61686176",
  oauth_version="1.0"
WWW-AuthenticateIndicates the authentication scheme that should be used to access the requested entityWWW-Authenticate: OAuth realm="http://sp.example.test/"

Parameters

REST parameters specify the variable parts of your resources: the data that you are working with. As was mentioned in a previous article Best Practices: Testing the Other Verbs, in a REST request the resource that you are working with is specified in the URL – Uniform Resource Locator. The URL is a special case of the URI – Uniform Resource Identifier – which consists of four parts::

scheme_name:hierarchical_part?query#fragment

In the article World Of API Testing: SOAP vs. REST Challenges, we discussed that REST is not a standard in itself, but instead makes extensive use the HTTP standard. Therefore for all REST calls, the scheme name will always be “http:”, or “https:” if sent over a secure channel.

Restful Resources

In any RESTful service it is very desirable to have all your resources structured by their hierarchy. These are then specified in the hierarchical part of the URL. The hierarchical parts are all 1) required, and 2) unique. This means that none of them can be omitted, and all of them can appear only once. Certain parts of the URL are going to be fixed (such as the server name, port, and endpoint), and certain parts are going to be parametrized. The parametrized parts are often denoted in code and in documentation by curly braces.

Consider, for example, a web service for ordering books. Your data might be organized into customers, which contain orders, which in turn contain individual books. The resource URL might look like this:

http://server.test:8080/order_api/{customer_id}/{order_id}/{book_id}

Sending a DELETE request to this URL might remove a book from an existing order, while sending a GET request to this URL might retrieve the details of a particular book (such as if it is on back order or out of stock).

Alternatively, an airline booking engine might structure their data by airplane first, which in turn could contain customers. The URL might look like this:

https://airline.server.test/ticketing_api/{flight_id}/{customer_id}

Note that for an airline the flight (airlines refer to specific flights as a “tail”) is the bigger object they need to keep track of, which then contains customers (passengers). While in the previous example, there is really no object in a bookstore that would contain customers.

Query Parameter

The query parameters are sometimes referred to as optional parameters; this is the first distinguishing feature from the hierarchical parameters: they are all optional. The second feature is that they are non-unique, meaning that you can specify any one parameter multiple times. The query parameters are separated from the hierarchical parameters by the question mark. The exact syntax of the actual parameters is not generically defined, but normally are a sequence of key-value pairs (separated by an equal sign), with the sequence separated by either a semicolon or an ampersand.

Building on the above airline example. A customer, when making a booking, may wish to add options, such as vegetarian meal and wheelchair access. The URL for this resource could look like this:

https://airline.server.test/ticketing_api/{flight_id}/{passenger_id}?option=vegetarian&option=wheelchair

Fragment Parameters

The fragment part of the URL, everything after a hash symbol, is information that is normally used only by the client, such as a browser, and not processed by the server. Therefore it is uninteresting when discussing REST parameters. The only interesting item is if you need to send the actual hash character as a value (instead of representing the hash control symbol) to one of the options. In that case you need to encode the URL.

Character Encoding

Special characters are encoded in the URL, by a mechanism called “percent encoding”. In this mechanism any character can be replaced by the percent symbol, followed by a two-digit hexadecimal value of the encoded character. If special characters (such as the hash character) need to be sent as actual data, they must be encoded. All other characters can optionally be encoded.

Size Limits

Although the URI standard does not specify a maximum size of the URL, most clients enforce an arbitrary limit of 2000 characters. Sending data that is difficult to express in a hierarchical manner, and especially data that is larger than this 2000 character limit, should be transmitted in the body of the request. As was discussed in World Of API Testing: SOAP vs. REST Challenges, the data in the body can be structured in any machine readable format, but most often is structured as XML or JSON.