Speaking at the service technology symposium

September 24th, Marc Schmeetz (Vodafone) and myself will be presenting at the service technology symposium in London, UK.

The topic of our presentation will be on how we improved business and technology alignment in a multi-project programme that lasted for several years.

This cross-expertise programme involved many business units, suppliers, developers, business analysts, architects and stakeholders and was particularly challenging.

For more details please see the agenda and presentation profile at the service technology symposium.

Click here for the symposium web site.

Click here for the presentation abstract.

- Roger

SOA, Separation of concerns and agnosticism (5/5)

Separation of concerns and abstraction

Any service that we build should have a certain level of abstraction. Abstraction comes in many kinds but they are all intended to provide separation of concerns.

As an example. if a service accesses a database and the database has an VARCHAR2() field for a numeric value called DBLADR2FLD, then it should not matter to the service consumer what the underlying datatype and column name of the database field is. In essence the service abstracts from the implementation to prevent this.

Additionally, a VARCHAR2() datatype is a specific vendor technology datatype. The service should also abstract from the specific database vendor technology.

Why we do this can be answered pretty simple and straight-forward: the underlying data model and the database vendor technology can change over time or be replaced by a completely different system or technology, and this should not matter to the consumer of the service. This is the most important reason why we have abstraction. By doing contract-first approach, we can already make sure that up-front no implementation-specific or technology-specific dependency can trickle into the contract and as a consequence into the service consumer logic (see [3])

Another form of abstraction is the quality of service abstraction: not all quality of service information does need to be known to the consumer of a service. Important QoS attributes are the availability and the reliability of a service, how much load it can handle and what the average and peak duration of execution is.

Not important however is the fact that the service had 4 failures in the past hour, as this does not say anything about how it will behave in the future and also it might even make the service look unattractive. In a sense we need to be shielding these kind of runtime concerns from potential service consumers. This can be done by creating SLAs that cover the quality of service in a constructive way with meaningful performance indicators.

Many more types of abstraction can be applied - if I can find some more time I will update this article.

- Roger

SOA, Separation of concerns and agnosticism (4/5)

Separation of concerns and interoperability

Because the contract must be interoperable we need to define contracts in a technology that is interoperable and preferably vendor-neutral by nature.

We also need to make sure that both the message protocol as well as the transport protocol are standardized as much as possible. Ie. in case we use a message protocol that cannot be supported on a specific platform or a transport which is not supported by all required platforms, this poses us with a significant interoperability challenge. Sometimes both message and transport are coupled (ie. REST), sometimes decoupled (ie. SOAP where the message protocol remains SOAP but the transport protocol can change from ie. HTTP to JMS).

For this interoperability challenge to be resolved we can choose a technology that allows for cross-platform interoperability, like SOAP, REST or plain XML over HTTP. As you can see the standards used in this paragraph are all industry standards. Industry standards are standards that agreed on and have large industry support. As an example, at the time of writing, SOAP 1.1 has more industry support than SOAP 2.0, so although SOAP 2.0 has some improvements over SOAP 1.1, personally still I would prefer the older standard.

Choosing industry standards for the technical contract, the chances of interoperability increase, as multiple different types of platforms would most likely be compatible, despite the platforms being disparate systems.

As a side note, if technology like SOAP is used, the contract can easily be designed first because WSDL (yes another industry standard) can be used to describe the contract in a standardized way (from technology point of view). This would help creating a (separately governed) contract that can be used by both the service designer as well as the service consumer designer to allow for concurrent design and delivery of both service as well as its consumer. 

SOA, Separation of concerns and agnosticism (3/5)

Separation of concerns and decoupling

To facilitate standardized contracts, they must be decoupled as much as possible. Decoupling can be achieved by designing the contract first, regardless of technology, back end, consumer etcetera.

A number of different coupling types can be identified. Generally speaking we distinguish positive and negative coupling types. Positive coupling types are encouraged whereas negative coupling types should be avoided. Negative coupling is anything on which the contract can form dependencies. In other words, contract-to-* coupling is considered negative coupling and *-to-contract coupling is considered positive coupling as this reduces the amount of dependencies the contract can form.

A way of achieving contracts independent of consumers and implementation is by designing the contract first. If the contract-first approach is not followed the risk exists that a number of different kinds of negative coupling find their way into the contract.

Now that we have identified positive and negative coupling types, the following can be considered positive coupling:

• Consumer-to-contract
• Logic-to-contract

The directly negative coupling types are:

• Contract -to-consumer (contract depends on one or more consumers of the service)
• Contract-to-logic (contract depends on the implemented logic’s interface
• Contract-to-functional (contract only supports specific functional context ie. parent process)

Indirectly another set of negative coupling types can exist, ie:

• Logic-to-implementation(logic depends on how the logic is implemented)
• Logic-to-technology (logic depends on the technology on which it is implemented)

Because the contract can depend on the logic, the contract can indirectly depend on implementation or technology. Examples of negative coupling are situations where contracts are generated from implemented components or classes, or components generated to expose a database implementation.

Regardless of the coupling type, any negative coupling type from contract-to-logic will inadvertedly also expose an indirect level of coupling to the consumer of the service. Ie. the consumer might need to populate parts of a message to our service which represents ie. a tablename or fieldname of the underlying back end or database implementation. This will result in unnecessary changes to cascade though our services into the consumers and should be avoided in general.

Again, a fairly simple way of avoiding these kinds of problems is by following the contract-first approach. Another advantage of contract-first approach is that once the contract is defined (and hopefully also standardized as described in section 2), is that delivery streams of both service provider as well as service consumer can make their design and implementations in parallel, as the interface of the service is predefined with a clear purpose, context and meaning. This would facilitate a quicker delivery model than ie. a straight-forward sequential approach where service implementation had to be finished before the contract can be shared. This would benefit any project manager, scrum manager etc. as there is no sequential dependency before the consumer of a service can start designing and developing. This provides flexibility to the project delivery lifecycle.

SOA, Separation of concerns and agnosticism (2/5)

Separation of concerns and contracts

A significant prerequisite for separation of concerns is the fact that contracts are standardized in multiple ways. Aspects that can be standardized on a contract are the data model (service data representation standardization), and the naming and semantics of the services, operations and data elements (functional expression standardization).

Standardizing the contract forces us to separate the concern of contract design from everything around the contract, this is also called decoupling (see [3]).

A way to standardize and independently govern the contract is simply by approaching a service using the top-down approach. 

This is easier said than done, but a few pointers are:

• Expression of functionality from the viewpoint of business and business processes
• Naming entities from the viewpoint of the business functional area they support
• Applying normalization and logic centralization to services
• Creating the contract-first, only then think about the implementation

Once the contracts are designed first, there is a natural decoupling of contract and logic, as the contract ideally should not expose any implementation details. The separation of concerns is reached specifically by the contract-first approach as this does not allow implementation-specific details to trickle down into the contract.

SOA, Separation of concerns and agnosticism (1/5)

Middleware and Separation of Concerns

Middleware can exist in many ways but most common approach is either the application service or the ESB approach. Regardless of what platform or technology the middleware is hosted on, the main  purpose of the middleware layer is the same: separation of concerns.

As an example the following picture shows that the middleware decouples the consumer from the back end systems by providing services that act on behalf of the back end. This way the consumer does not need to be concerned with the back end, and the back end does not need to be concerned with who the consumer is or how it works.

Decoupling should be explained a bit further. Two systems are coupled if there is a dependency between the two. What we try to achieve with decoupling is that we try to minimize the amount of dependencies between them.

Separation of concerns can be observed in middleware in many ways but starting with offering services on the middleware platform allows us to wrap back end systems. This then allows us to offer a standardized way of accessing underlying back end systems. Standardizing the access to logic can improve interoperability. To provide increased opportunity of interoperability, we need to understand that we have to standardize contracts. If two systems are interoperable it means that they can easily exchange information, or in other words, cooperate. If we put in lots of effort we might be able to provide a system that has something called intrinsic interoperability.

Intrinsic interoperability is achieved if we have standardized a system in such a way (and if we have built logic in such a way that the interoperability can be supported) that if two systems exchange messages, the messages can be easily understood, without the need for (complex) transformations or translations.

For intrinsic interoperability to be achieved, a number of prerequisites must be met, and even then matters are not easy and require significant amount of attention in the governance area.

SOA, Separation of concerns and agnosticism (0/5)

Separation of concerns and agnosticism

Separation of concerns and agnosticism are two closely-related terms. Where separation of concerns is a term commonly used in object-oriented and component-based technologies, it is a way to clearly separate functional concerns by defining containers (services, classes, components) to identify a functional boundary of the contained operations.

Agnosticism (from the Greek word agnostos) is a term traditionally used in religion-related discussions but also in the world of service-orientation to explain that the service is not aware of, or is not concerned with, the parent functional context. Gnostos means “aware of the existence of”, the “a-“ is a negation making the whole term’s meaning “not aware of the existence”.

Both terms convey a similar concept where we keep concerns of a particular piece of code local to that piece of code only, and not have it scattered across the software infrastructure. The term agnostic however intends to apply to a wider scope than only code or logic, for instance on the contract or on the service as a whole, as well as the entire service inventory structure.

This five-part series touches on several aspects of how separation of concerns can be achieved in a service-oriented solution.