Pattern Sections
Pattern: Reference Data
Holder 
a.k.a. Immutable Endpoint/Immutable Data Holder, Static Data Resource, Reference Data Lookup Table
Context
A requirements specification unveils that some data is referenced in most if not all system parts, but changes only very rarely (if ever); these changes are of administrative nature and not caused by API clients operating during everyday business. Such data is called reference data.1 It comes in many forms: units of measurement, zip codes, country codes, currency codes, geo locations, etc.2
The data transfer representations in the request and response messages of API operations may either contain – or point at – reference data to satisfy the information needs of a message receiver.
Problem
How should data that is referenced in many places, lives long, and is immutable for clients be treated in API endpoints? How can such reference data be used in requests to and responses from Processing Resources or Information Holder Resources?
Forces
The following specific forces have to be resolved when dealing with static, immutable data:
- Do not repeat yourself (DRY)
- Performance versus consistency trade-off for read access
Pattern forces are explained in depth in the book.
Solution
Provide a special type of Information Holder Resource endpoint, a Reference Data Holder, as a single point of reference for the static, immutable data.3 Provide read operations, but no create, update, or delete operations in this endpoint.
The Reference Data Holder may allow clients to retrieve the entire reference data set so that they can copy it locally (e.g., for accessing it multiple times), partially filter its content before doing so (e.g., to implement some auto-completion feature in a form), or to lookup individual entries of the reference data (e.g., for validation purposes).
Sketch
A solution sketch for this pattern from pre-book times is:
Example
There is one main usage scenarios for this pattern, simple value data lookup (e.g., for country codes, currency codes). Figure 2 shows an instance of the pattern that allows API clients to lookup zip codes.
Are you missing implementation hints? Our papers publications provide them (for selected patterns).
Consequences
The resolution of pattern forces and other consequences are discussed in our book.
Known Uses
Publicly visible known uses include:
- The country abstractions/endpoints in the public, donation-funded API RESTCountries are instances of this pattern. Similar services exist for currency codes.
- Taxonomies such as the topic keywords used in digital libraries such as the ACM Digital Library Computing Classification System and IEEE Xplore qualify when exposed in APIs such as the IEEE Xplore API Portal.
- Geographic data such as maps, coordinates and zip codes also change
rarely (although they still do change over long periods of time), and
many APIs for them can be seen as
Reference Data Holders.
- An example is the Overpass API for Open Street Map.
- The location database behind the Open Weather Map API uses its
own
city IDs (for more than 200k cites) and also works with country codes, zip codes, and geo coordinateslong(itude)andlat(itudes).
- The unique company identifiers (“UIDs”) assigned by the Swiss government and available via a Web service qualify as reference data.
The core banking integration SOA described in Brandner et al. (2004) works with region codes and product/market categories.
Many digital archives must guarantee that stored documents are never changed and will therefore return them as static data. For instance, Terravis Berli, Lübke, and Möckli (2014) uses such a digital archive and offers only read operations (i.e., Retrieval Operations) on existing documents. Moreover, Terravis offers a process meta-data search feature/service that returns information about process instances that will never change once a process instance has been completed, but might be amended before because additional information about a process instance is collected (e.g., process outcome.) For example, a notary might invite another bank to participate in a land register transaction. This newly added bank will be added to the reference data of the process instance.
Examples of somewhat unexpected changes to static/immutable reference data include the introduction of the Euro currency in many European countries and new zip codes in Germany in the 1990s; APIs that work with such abstractions must make the data catalog version and data types (for instance, units of measures for distances and weights) explicit.
More Information
Related Patterns
The Master Data Holder pattern is an alternative to Reference Data Holder. It also represents long-living data, which still is mutable. Operational Data Holders represent more ephemeral data.
The Reference Management subcategory of our Quality Patterns features two related patterns, Embedded Entity and Linked Information Holder. Simple static data is often embedded (which eliminates the need for a dedicated Reference Data Holder), but can also be linked (with the link pointing at a Reference Data Holder).
Other Sources
A definition of the term reference data can be found at TechTarget: “consisting of sets of values, statuses or classification schema”.
References
See “Data on the Outside vs. Data on the Inside” by P. Helland, for an introduction to reference data (in the broad sense of the word).↩︎
See https://en.wikipedia.org/wiki/Reference_data for links to inventories/directories of reference data.↩︎
To use terms from data warehousing and master data management: “single version of the truth” or “golden copy”.↩︎