Pattern: Wish List

How can an API client inform the API provider at runtime about the data it is interested in?


The final version of this pattern is featured in our book Patterns for API Design: Simplifying Integration with Loosely Coupled Message Exchanges.

Pattern: Wish List

a.k.a. Data Wish Enumeration, Partial Response Representation Request, Data Selection Profile

Context

API providers need to serve multiple different clients that invoke the same operations. Not all clients have the same information needs: some might just need a subset of the data offered by the endpoint, other clients might need rich data sets.

Problem

How can an API client inform the API provider at runtime about the data it is interested in?

Forces

Multiple clients with different information needs might use an API. Hence, the primary design issues that drive selection and adoption of this pattern are:

  • Performance, scalability, and resource use
  • Information needs of individual clients
  • Loose coupling and interoperability
  • Developer convenience and experience
  • Security and data privacy
  • Test and maintenance effort

Furthermore, API providers also need to balance general quality forces such as response time, throughput and processing time.

Pattern forces are explained in depth in the book.

Solution

As an API client, provide a Wish List in the request that enumerates all desired data elements of the requested resource. As an API provider, deliver only those data elements in the response message that are enumerated in the Wish List (“response shaping”).

Sketch

A solution sketch for this pattern from pre-book times is:

Figure 1: Wish List: A client enumerates the desired data elements of the resource.

Example

In the Lakeside Mutual Customer-Care application1, a request for a customer returns all of its available attributes.

curl http://localhost:8080/customers/gktlipwhjr

For customer ID gktlipwhjr, this would return:

{
  "customerId" : "gktlipwhjr",
  "firstname" : "Max",
  "lastname" : "Mustermann",
  "birthday" : "1989-12-31T23:00:00.000+0000",
  "streetAddress" : "Oberseestrasse 10",
  "postalCode" : "8640",
  "city" : "Rapperswil",
  "email" : "admin@example.com",
  "phoneNumber" : "055 222 4111",
  "moveHistory" : [ ],
  "customerInteractionLog" : {
    "contactHistory" : [ ],
    "classification" : {
      "priority" : "gold"
    }
  }
}

Alternatively, the client can also provide a Wish List of fields in the query string to restrict the result to just those fields. For example, a client might only be interested in the customerId, birthday and postalCode fields:

curl http://localhost:8080/customers/gktlipwhjr?\
fields=customerId,birthday,postalCode

The returned response now contains only the requested fields:

{
  "customerId" : "gktlipwhjr",
  "birthday" : "1989-12-31T23:00:00.000+0000",
  "postalCode" : "8640"
}

This response is much smaller; only the information required by the client was transmitted.

Are you missing implementation hints? Our papers publications provide them (for selected patterns).

Consequences

The resolution of pattern forces and other consequences are discussed in our book.

Known Uses

Wish Lists can be found in many public Web APIs under different names and in many variations:

  • The Google Calendar API has the notion of partial requests, partial responses and patches. These can be seen as a variant of the Wish List pattern: the client sends a field parameter that lists desired values. The API also supports an expression syntax including wild cards.
  • The Facebook Graph API supports a variation of this pattern under “making nested requests”: Field expansion in the Graph API allows client developers to nest multiple graph queries into a single call.
  • Sparse fieldsets in the JSON API specification realize the Wish List pattern as well: A client can request that an endpoint returns only parts of a data element in the response message.
  • Atlassian JIRA supports Wish Lists under the name expansion comprehensively, but in a slightly different way. Initial responses are terse, but contain elements that can be “expanded” in further requests and are marked as such. The client can then provide a Wish List of parameters in the query string to be expanded, which will then be present in the subsequent response. The same concept is supported by the Confluence REST API.
  • The Microsoft Graph API supports optional query parameters. One of them is an expand parameter with expansion semantics.
  • LinkedIn APIs support “Response Decoration”.
  • Flask RESTPlus, a Web development and REST framework for Python, has the notion of Fields masks for this pattern.
  • The TMForum, an association of telecommunication providers and suppliers, offers a number of open APIs. The TMForum REST API allows query operations to select the resource’s attributes that should be returned. Furthermore, it also allows filtering of attributes with various filter expressions.
  • A Swiss software vendor specializing on the insurance industry uses partial representations and expanded GETs in its internal REST API Design Guidelines. These guidelines recommend defining a request variable called fields that lists the requested fields. A wildcard operator exists; if the fields variable is not present, all fields must be returned.
  • Another enterprise information system usage of the pattern is outlined in Brandner et al. (2004).

The Pattern Adoption Story: Dutch Government & Energy Sector contributed by a reader has more known uses and further discussion of this pattern.

More Information

Related Patterns

Wish Template addresses the same problem, but works with a structure rather than a list of wishes; that is, Wish List usually uses the Atomic Parameter List pattern to express the wishes.

Wish List deals with instances of Parameter Tree and Parameter Forest as “origin” and result data structures, as the patterns are usually applied to more complex data structures.

Using a Wish List has a positive influence on a Rate Limit, as less data is transferred when the pattern is used.

The Pagination pattern also reduces response messages sizes, but in a different way (by splitting large repetitive responses into parts).

Other Sources

“Web API Design: The Missing Link”, an eBook by apigee, recommends comma-separated Wish Lists in its chapter “More on Representation Design”.

“Practical API Design at Netflix, Part 1: Using Protobuf FieldMask” in the Netflix Technology Blog (Borysov and Gardiner 2021) mentions GraphQL field selectors and Sparse Fieldsets in JSON:API (JSON API 2022). It then features protobuf FieldMask as a solution for gRPC APIs within the Netflix Studio Engineering. The authors suggest that API providers may ship client libraries with pre-built FieldMask for the most frequently used combinations of fields. This makes sense if multiple consumers are interested in the same subset of fields.

References

Borysov, Alex, and Ricky Gardiner. 2021. “Practical API Design at Netflix, Part 1: Using Protobuf FieldMask.” Netflix Technology Blog. 2021. https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518.
Brandner, Michael, Michael Craes, Frank Oellermann, and Olaf Zimmermann. 2004. “Web Services-Oriented Architecture in Production in the Finance Industry.” Informatik-Spektrum 27 (2): 136–45. https://doi.org/10.1007/s00287-004-0380-2.
Fowler, Martin. 2002. Patterns of Enterprise Application Architecture. Addison-Wesley.
JSON API. 2022. “JSON:API.” 2022. https://jsonapi.org/.


  1. Lakeside Mutual is a fictitious insurance company invented by the authors comprising of several microservices to demonstrate the patterns in action. It can be found at https://github.com/Microservice-API-Patterns/LakesideMutual.↩︎