Pattern Adoption Stories from Readers: Dutch Government & Energy Sector (1/4)

On pages like this, we collect contributions from readers of our patterns.

Book Part 2

This pattern adoption story was contributed by Ton Donker, Ruben Haasjes and their readers group. It covers the 21 patterns presented as API Design Pattern of the Week on LinkedIn and Medium as well as other patterns. This page features part 1 of 4 (part 2, part 3, part 4).

Wish List

Known Uses

The Wish List pattern is widely used in Dutch government APIs. The guidelines for use of this pattern are available online in Section 9 (“Customization”) of “API Designrules Extensions”.

Government APIs are available via a public developer portal. Please find below some examples of Dutch government APIs that make use of the fields parameter for custom representation:

  1. https://developer.overheid.nl/apis/kadaster-bag-current/specificatie/production
  2. https://developer.overheid.nl/apis/logius-cor/specificatie/production

In the Dutch Energy sector I am currently working in, we are about to design APIs that utilize the combination of the fields parameter with the expand parameter. In HAL-formatted responses (responses with links to associated resources) the consumer can retrieve subfields of the associated resource by expanding the returned HAL link. Design rules are available in the Dutch document “API-strategie”. There is no concrete API yet.

Discussion points:

  • To distinguish between “real” query parameters and the more “steering” or “meta” parameters like expand, fields, limit and page, we – the Dutch Energy sector API Working Group – advocate the usage of an underscore prefix, so _fields, _expand, _limit and _page to prevent misinterpretation of the function of the query parameters.
  • The Wish List works in “optima forma” or comes to his right if all the response fields are optional. If there are any required fields in the response, the question arises if these fields should be returned in case they are not requested in the Wish List;
  • The implementation in provider service layers of the Wish List is expensive - sometimes these costs are simply too high and is there no business case to justify the investment.

Some other sources that mention the Wish List pattern:

More comments about the pattern can be found on Linked In.

Pagination

Known Uses

  1. Dutch governmental and Dutch Energy APIs make widely use of the page-based pagination variant. See for instance Section 14 of “API Designrules Extensions”.
  2. Nordic API distinguishes five types of pagination methods, see “Everything You Need to Know About API Pagination”.

We see a current tendency in the industry to favor the use of the cursor-paged pagination variant. Examples include (just to mention a few):

In the Dutch Energy sector we are analyzing this tendency.

The pattern was discussed rather intensively in week 2 of our “API Design Pattern of the Week” series on Linked In.

A very tricky pattern, each variant has its pros and cons and there is no ‘one-size-fits-all solution’. Real-time data and large datasets?: cursor-based pagination… Optimal DX and static data?: offset-based pagination…, just to mention a few. More decision drivers and design concerns in the book.

Error Report

Known Uses

  1. RFC7807 is adopted by Dutch governmental API design rules, see Section 17 of “API Designrules Extensions”;
  2. Same in the Dutch Energy sector, which declared the RFC7807 members type, title and status as required;
  3. Zalando http://opensource.zalando.com/restful-api-guidelines/#176 propose relative URI references in type field;
  4. Swedbank pay https://developer.swedbankpay.com/introduction.html#problems.

Discussion points:

  1. Should error handling include all possible errors or should it simply stop processing after the first error? The second scenario is how most APIs currently work since it less implementation effort. In case of Request Bundle (p292 of the book) first scenario is advisory;
  2. In the book (p290) it is stated: “Timestamps are another common information in Error Reports too”. Good point! However timestamp is not a member of RFC 7807 and seems to be absent in other alternative error response formats;
  3. Error reports should enable the receiver to be self-serviced. A talk given by Asbjørn Ulsberg is on this: https://www.youtube.com/watch?v=ZWIUn5BHDBc.

Also see comments on the pattern on Linked In. Feel free to join the discussion!

API Key

I think the pattern is very well explained in the book.

Known Uses

The Section 3.5 in “API Designrules Extensions” details the authentication and authorization protocol in the Dutch governmental sector (in English).

Discussion Input

In the case of authentication/authorization errors, the details of the error report should be carefully considered and minimized not to unveil any provider-side implementation security and privacy details. So the pattern API Key requires special implementation of Error Report; in that case, an API Key impacts the contents and message representation design of Error Report. This is all very well explained in the book, and it is also well-stated that an API Key is a “lightweight alternative to a full-fledged authentication protocol.” Despite all the concerns and ‘weaknesses’ one can argue that API Keys are a way to identify your consumers and build client communities to announce new releases and improve the client developer experience.

Processing Resource

The notion of Processing Resource as an endpoint role offering an activity-oriented API endpoint automatically brings us to the discussion how to define transactions as (REST) resources in the URI endpoint of the HTTP API. Should the URL be verb-free? This is still a hot debate in the REST API design community (so is the discussion between REST vs RPC). Some will argue that using verbs in data-oriented API endpoints will enrich the semantics of the API, others see it as an anti-pattern. Further inspiration on this subject: https://tyk.io/blog/rest-never-crud/ by James Higginbotham. This valuable link is of course provided in the book (amongst many others)!

Known Uses

n/a

The notion of activity-oriented API roles and processing resources endpoints is in the original literature very often associated with the controller resource archetype. Examples:

  • RESTful Web Service Cookbook (Allamaraju 2010): In order to abstract complex business operations (including transactions), servers may need to employ controller resources to make changes to other resources (e.g. Recipe 2.6);
  • REST API Design Rulebook (Massé 2012): A controller resource models a procedural concept. Controller resources are like executable functions, with parameters and return values; inputs and outputs;
  • https://restfulapi.net/resource-naming/: Use verb to denote controller archetype;

Informative link and resource that talk about RPC and REST (understanding the difference might help in understanding the Processing Resource pattern:

With regard to the API endpoints must be verb-free debate it’s worthwhile to mention:

This is of course a simple fragment of what is being offered by the web community, hopefully it adds some inspiration and ideas!

Information Holder Resource

Perhaps one of the most frequently used design patterns, especially in combination with Retrieval Operations Good to read that joint use of Processing Resource and Information Holder Resource is not recommended for microservices architectures (responsibility separation). And tagging the Information Holder Resource as one of the subtypes certainly adds semantics and expressiveness to the contract, establishing a ‘ubiquitous language’ used in API design… 👍

Known Uses

As already stated in the book, this pattern is widely used in may public Web APIs.

Known uses:

There are many other known uses of this pattern, so above links just mentions a few. It’s a typical pattern used in governmental APIs.

Data Transfer Resource

Known Uses

In the Dutch Energy sector we have EDSN (Energy Data Services Netherlands) as central data platform for market-facilitating. Cooperation and communication between market parties (suppliers, grid operators, metering companies) is facilitated via the central infrastructure of EDSN. There are use cases in which central data needs to be shared from the source with the decentral market party sinks (for instance for reporting purposes). This is achieved via de Data Transfer Resource Pattern, technically implemented by the S3 cloud storage protocol (the so-called S3 buckets). Such cloud storage is a technique that allows for the exchange of large data sets, low implementation costs, availability, durability, scalability, decoupling and it has a broad adoption in the industry (most parties are capable of connecting to cloud storage).

The pattern variant currently in place is Relay Resource: Every market party sink has a unique bucket where only the required semi-finished data products are delivered to. Housekeeping rules (like retention periods) are mutually agreed upon, and based on business processes. Replay and deletion are responsibilities of the data sink. The shared resource address is agreed upon in advance. Future plans are to optimize the process and to use push notifications (event driven) to inform clients proactively in order to enable faster processing of data and discoverability (which links nicely to the Link Lookup Resource Pattern as described in the book 😊!)

Further known uses from public cloud providers are:

Discussion Input

There are of course security and privacy risks when enterprises store and share files via a Web-flavored shared repository (like in the cloud). However, there are ways to avoid these risks, and the advantage of having all your data in one convenient place that can be accessed at any time from any place is worth investigating as long as the implementation is compliant with government regulations, security and privacy rules, and other legal obligations.

References

Higginbotham, James. 2021. Principles of Web API Design: Delivering Value with APIs and Microservices. Addison-Wesley.
Masse, M. 2011. REST API Design Rulebook: Designing Consistent RESTful Web Service Interfaces. O’Reilly Media. https://books.google.ch/books?id=eABpzyTcJNIC.

Back to homepage