Pattern Sections
Pattern: Data Transfer
Resource 
a.k.a. Connector Resource, Integration Resource, Share, Temporary Data Store, Transient Information Holder
Context
Two or more communication participants want to exchange data. The number of exchange participants may change over time, and their existence only partially be known to each other. They might not always be active at the same time. For instance, additional participants may want to access the same data after it has already been shared by its source.
Participants may also only be interested in accessing the latest version of the shared information and do not need to observe every change applied to it. Communication participants may not be able to install any messaging middleware beyond a basic HTTP client library locally.
Problem
How can two or more communication participants exchange data without knowing each other, without being available at the same time, and even if the data has already been sent before its recipients became known?
Forces
The following forces have to be resolved in this setting:
- Coupling (time dimension)
- Coupling (location dimension)
- Communication constraints
- Reliability
- Scalability
- Storage space efficiency
- Latency
- Ownership management
Pattern forces are explained in depth in the book.
Solution
Introduce a Data Transfer Resource as a shared storage endpoint accessible from two or more API clients. Provide this specialized Information Holder Resource with a globally unique network address so that two or more clients can use it as a shared data exchange space. Add at least one State Creation Operation and one Retrieval Operation to it so that data can be placed in the shared space and also fetched from it.
Decide on data ownership and its transfer; prefer client ownership over provider ownership (in this case).
Sketch
A solution sketch for this pattern from pre-book times is:
HTTP support for the pattern: From an implementation perspective, this solution is directly supported in HTTP, whereby client A first performs a PUT request to publish the information on the shared resource, uniquely identified by a URI, and then client B performs a GET request to fetch it from the shared resource. Note that the information published on the shared resource does not disappear as long as no clients perform an explicit DELETE request. Client A publishing the information to the shared resource can do so reliably, as the HTTP PUT request is idempotent. Likewise, if the subsequent GET request fails, Client B may simply retry it to be able to eventually read the shared information. Figure 2 illustrates the HTTP realization of the pattern.
Example
The example in Figure 6 instantiates the pattern for an integration
interface in a fictitious insurance scenario. The
ClaimReceptionSystemOfEngagement is the data source, and a
ClaimTransferResource decouples the two data sinks
(ClaimProcessingSystemOfRecords,
FraudDetectionArchive) from it.
Are you missing implementation hints? Our papers publications provide them (for selected patterns).
Consequences
The resolution of pattern forces and other consequences are discussed in our book.
Known Uses
DropBox and ownCloud created a business model for usage of this pattern (Software-as-a-Service, SaaS) and provide related integration APIs (see here and here). The Doodle scheduling service is not a API, but a Web application that uses this pattern as well.
An article by B. Rücker introduces event command transformation, mentions asynchronous Web-based integration, and provides working examples.
A major German car manufacturer offers a REST level 2 user profile management service for all its clients; this service features a different, queue-based type of Data Transfer Resource to decouple clients from each other and from servers. In the spirit of the Reactive Manifesto, it uses asynchronous message passing (here: Amazon SQS). The queue is populated in the implementation of one API operation. Another API operation offers clients an opportunity to poll (look for) new appearances in the queue via a Web resource; if there are none, the call does not block but returns a “come back later” message after 30 seconds. Clients can select what they are interested in; the server also filters by authentication roles (which can be seen as an implementation of a Retrieval Operation). Consumption policies and message structure vary.
More Information
Related Patterns
The pattern differs from other types of Information Holder Resources with respect to data access. The Data Transfer Resource exclusively owns and controls its own data store; the only way to access its content is via the published API of the Data Transfer Resource. Instances of other Information Holder Resource types may work with data that is accessed and possibly even owned by other parties (e.g. backend systems and their non-API clients). Likewise the Data Transfer Resource acts both as a data source and data sink, unlike other Information Holder Resources which may either store operational data, master data, or reference data. A Link Lookup Resource only exposes metadata; a Data Transfer Resource can hold any data.
65 patterns for asynchronous messaging are described in Hohpe and Woolf (2003). A Data Transfer Resource can be seen as a Web-based realization of a Message Channel, supporting message routing and transformation, as well as several message consumption options (e.g., Competing Consumers and Idempotent Receiver). Queue-based messaging and Web-based software connectors (as described by this Data Transfer Resource pattern) can be seen as two different but related integration styles; these styles are compared in Pautasso and Zimmermann (2018).
Blackboard pattern is a POSA 1 pattern Buschmann et al. (1996), intended to eligible in a different context, but similar in its solution sketch. The Remoting Patterns book Voelter, Kircher, and Zdun (2004) describes the remoting style shared repository; our Data Transfer Resource can be seen as the API for a Web-flavored shared repository.
Other Sources
Interfacer is a role stereotype in Responsibility-Driven Design (RDD) that describes a related but more generic programming-level concept Wirfs-Brock and McKean (2002).