Pattern: Conditional Request

How can unnecessary server-side processing and bandwidth usage be avoided when frequently invoking API operations that return rarely changing data?


The final version of this pattern is featured in our book Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges.

Pattern: Conditional Request

a.k.a. Conditional Retrieval, Conditional Modifications (Google Calendar API terminology)

Context

Some clients keep on requesting the same server-side data repeatedly. This data does not change in between clients requests.

Problem

How can unnecessary server-side processing and bandwidth usage be avoided when frequently invoking API operations that return rarely changing data?

Forces

The following forces apply when solving this problem:

  • Complexity of endpoint, client, and message payload design and programming
  • Accuracy of reporting and billing
  • Size of messages
  • Client workload
  • Provider workload
  • Data currentness versus correctness

Pattern forces are explained in depth in the book.

Solution

Make requests conditional by adding Metadata Elements to their message representations (or protocol headers) and processing these requests only if the condition specified by the metadata is met.

Sketch

A solution sketch for this pattern from pre-book times is:

Example

Many Web application frameworks, such as Spring, support Conditional Requests natively. The Spring-based Customer Care backend application in the Lakeside Mutual scenario includes ETags – implementing the Fingerprint-Based Conditional Request variant – in all responses. For example, consider retrieving a customer:

curl -i http://localhost:8080/customers/1c184cf1-a51a-\
433f-979b-24e8f085a189

A response could start with:

HTTP/1.1 200
ETag: "0c2c09ecd1ed498aa7d07a516a0e56ebc"
Content-Type: application/hal+json;charset=UTF-8
Content-Length: 801
Date: Wed, 20 Jun 2018 05:36:39 GMT

{
  "customerId" : "1c184cf1-a51a-433f-979b-24e8f085a189",
...

Subsequent requests can then include an ETag to make the request conditional:

curl -i -H  'If-None-Match: "0c2c09ecd1ed498aa7d07a\
516a0e56ebc"' http://localhost:8080/customers/1c184c\ 
f1-a51a-433f-979b-24e8f085a189

If the entity has not changed, i.e., If-None-Match occures, the provider answers with a 304 Not Modified response including the ETag:

HTTP/1.1 304
ETag: "0c2c09ecd1ed498aa7d07a516a0e56ebc"
Date: Wed, 20 Jun 2018 05:47:11 GMT

If the customer has changed, the client would get the full response, including a new ETag. RFC 7232

Note that this implementation of Conditional Requests is implemented as a filter applied to the response. This means that the response is still computed, but then discarded by the filter and replaced with the 304 Not Modified status code. This approach has the benefit of being transparent to the endpoint implementation; however, it only saves bandwidth and not computation time. A filter applied to the request would also work in conjunction with a server-side cache.

Are you missing implementation hints? Our papers publications provide them (for selected patterns).

Consequences

The resolution of pattern forces and other consequences are discussed in our book.

Known Uses

Some of the highly visible and frequently accessed Public APIs on the Web implement this pattern:

  • The GitHub API v3 uses ETags in most of its responses. Responses that return a 304 Not Modified do not count against the Rate Limit to encourage clients to use them.
  • In the Google Calendar API, ETags are used, both when modifying and when retrieving calendar data.
  • Clients of Salesforce’s REST API have the choice between ETags and If-Modified-Since or If-Unmodified-Since with a timestamp to make requests conditional.

Web application frameworks typically include some support for Conditional Requests. For example, Spring has a ShallowEtagHeaderFilter that calculates an ETag from the response body. The Play Framework offers a CacheControl library to help developers implement the HTTP caching model.

Some exemplary usages of the patterns in node.js are discussed in this RisingStack blog post.

More Information

Related Patterns

Using a Conditional Request may have a positive influence on a Rate Limit that includes response data volumes in the definition of the limit, as less data is transferred when this pattern is used.

The pattern can be carefully combined with either Wish List or Wish Template. This can be rather useful to indicate the subset of resource data that is requested if the condition evaluates to true and the resource needs to be sent (again).

Chapter 10 in “RESTful Web Services Cookbook” (Allamaraju (2010)) is dedicated to conditional requests. Some of the nine recipes in this chapter even deal with requests that modify data.

References

Allamaraju, Subbu. 2010. RESTful Web Services Cookbook. O’Reilly.