Make sure it's not too CLOUDy...

This post addresses some of the concerns that companies might have when dealing with cloud services in general, specifically when dealing with sensitive data in a hosted environment. Where cloud computing in intended to increase the quality of service, the quality of "privacy" is not a generally recognized factor when services are hosted. The environment naturally protects its applications because that's what the hosting environments provide.

With cloud computing, the deal changes with respect to traditional hosted services in ie. a colocated hosting environment for which you know where it is hosted. When services are hosted in a cloud, ie. the service's storage resource is hosted in redundant locations, you can not be sure with a (public) cloud where a service or its resources are hosted. In fact, some of the more innovative cloud service providers have dynamically hosted resources in 'cooler' locations on the planet to save cost by requiring less power for cooling. This means that the services are hosted across multiple continents and in many countries, which naturally leads to the conclusion: what happens with my customer data, what happens with my client data?

Businesses or governments storing data in a cloud may inherently be exposing the data into countries which have different privacy policies or legislation than required for their purpose. For example, if data of a European company is stored outside the European Union, which means that data of customers may not be protected by the privacy laws of Europe.

In the example, a cloud consumer should stick to cloud providers for which he knows the privacy policy, and for which the hosting environments are only located in Europe. Alternatively a cloud consumer should build a private cloud or use a community cloud which has controlled access.

Similar approaches apply for different reasons and for different countries, but the essence stays the same: make sure it's not too cloudy...

Core service logic and auxiliary service logic

Here's a nice scenario I have encountered recently:

Customer management wants every activity performed by a customer and every customer contact logged in the customer data store.

For this purpose, every relevant service and process in the service inventory is fitted with auxiliary logic to create customer logs in the CRM system.

Historically, this has worked for several years, but recently the system administrators started complaining about the sheer amount of data in the system and how it is becoming very hard to keep the system displaying these logs to the CM agent without running expensive (long-running) queries.

To cut a long story short, they want to reduce the amount of data logged for a customer by creating a new policy which only logs significant activities on a customer record.

For this to work, the initial request was to remove the logging logic from certain service capabilities.

In the service inventory, a whole bunch of services has, besides the relevant core service logic, some hard coded auxiliary CRM logging logic which will use a utility service to push the logged data to the CRM system.

As it is impacting many different services which are used in different contexts, removing the auxiliary logic as requested may not be the best way to go about this. Not only does this cost a lot of effort and regression testing, but also does it not solve the issue that the same service should log to the CRM system in one context, but not in another.

An alternative, perhaps more flexible way of changing the system's logging logic, is to change the utility service or apply a policy to this service which acts as a filter, deciding for all services whether or not to execute the actual CRM system update.

This means that the logging logic can still stay in the services as auxiliary logic, and the decision whether or not to perform the actual update will happen in the CRM system can be managed (declaratively), external to the core service logic of the services needing the logging, and external to the actual logging service itself.

This can work only if the context of the service call is also recorded with the logging message.

An alternative to this approach can be that all services log messages to the CRM system per default, but publish the message into a queue or topic and selectively process messages based on business rules in a business rules engine or similar.

SOA Pattern Publishing - why?!

Hi all,

Recently I have been posting a lot of candidate patterns on soapatterns.org, the pattern-related site of soasystems.com.

I was a matter of time before someone might ask "what are you doing and why are you doing it?". Well, it happened. Anyway, the reason why I wrote this post is because people ask why use soa patterns, why document soa patterns and there's only one simple reason for me doing this.

SOA patterns are proven solutions for common problems. Some of the candidate patterns I have documented may seem like open doors or a shot at an empty goal - but that's a good thing actually. What it means is that people recognize a certain solution for a problem, to actually be a good or at least a viable solution. And if people recognize the structure of the solution, does that not mean that they have just recognized a pattern in what they are seeing?

SOA patterns are not rocket science. SOA patterns are not smart or unique solutions. They are merely a common way of fixing a problem that we've encountered more than just once.

Does that mean it should not be documented? I don't think so. By documenting it, I hope to save other people's time and effort; because if someone recognized a documented soa pattern as something that can be applied to their problem, with or without modifications, that particular goal is reached.

Why am I publishing candidate patterns and not patterns? Well, it's in the definition of the soa pattern. For it to be a pattern it must be proven - as in - used in the field by more than one person. A site like soapatterns.org manages exactly that. This is also why I'm asking with every candidate pattern published by myself, to get others to testify that they have used the same pattern or a similar approach for real too. Because only if a group of people would testify they use it, it can be considered a pattern instead of a candidate pattern.

Keep watching this blog for a couple of more patterns to come. Presently finalizing patterns on rules normalization and rule layers and a couple of more service inventory related patterns. And if you've done something similar to what's described in one of my pattern candidates - or any others in the list - be sure to let the people at soapatterns.org know that you did. This way you, the community, can contribute to the field.

Regards,

-Roger

Published Reference Data Distribution candidate pattern

Hi all,

I just published the Reference Data Distribution candidate pattern. I'm looking for people to testify that they have applied this pattern to get it promoted from the candidate to the pattern status. Check it out here at soapatterns.org.

- Roger

Throttling is not trivial

Runtime governance tooling vendors, when they present their tools for ie. throttling, the way this is presented to potential customers is that they explain how installing these is easy and how implementing the actual behavior of the actual service agents is configuration only and this can simply be done by system administrators.

The pitfall is that customers exist who infer from these statements that they can purchase the licenses for any required service agents, give the CD to an administrator, and ultimately let the system administrator configure the throttling agents into the service inventory. All problems solved.

Let’s start by defining what throttling actually is from several points of view and then we can go from there to illustrate that these conclusions can be way off and can be very costly to resolve.

In the eyes of customers, throttling is a way to manage the amount of traffic to a service or a back end system. Expectation is often that they manage the amount of throughput by restricting the access to a service to not exceed a certain metric, ie. a specific number of calls per second or per minute, depending on load.

What does a statement like this mean, to the messages that arrive at the throttling agent and are marked to be beyond the predefined threshold?

For a customer this could mean that the message has to wait until the measured load does not exceed the threshold anymore.
To a throttling agent this usually means that the service call cannot be executed, because executing this would violate the expected threshold even more.

Throttling examples

What should happen to a service call exceeding the threshold? This actually depends on the purpose of a service and the context in which this service call is executed. A few examples are:

1. A read call for customer data intended for display in a client application
2. A read operation in the context of an update service composition
3. An update of an address based upon a client application request

Let’s highlight some characteristics for each of these:

Ad 1)
It seems OK to respond “too busy” to the service consumer. The consumer can retry in less busy times if it’s really important. To the throttling agent it means it’s ok to discard the message and respond “too busy”.

Ad 2)
This scenario is distinctly different from the first one. Discarding the read message to be executed in the context of an update, will most likely trigger a retry mechanism. The retry mechanism will make the same message come back potentially even quicker than a retry attempt coordinated by a service consumer. This will significantly increase the resource load on the message infrastructure as well as on the throttling mechanism. A way to overcome this issue is to have message properties which help identify in which context the message is being executed. And on the same throttled service, but in the different context, you can decide to allow this service call although messages in the ‘regular read’ scenario would be refused. For this you can even use two different throttling statistics.

This does however expose an issue related to throttling in service composition context. For elaborating this see the part of the article titled to “Where to throttle in the SOA”.

Ad 3)
A little less easy to deal with. Discarding the message does most likely not meet the business requirements. If the front end application (service consumer) is a web application which exposes a page to submit address changes, it might even cost customers ifthe data was typed and the system would say “too busy”. But then what? It seems that storing the message and trying later in less busy times should be fine.

One has to ask himself now whether the order in which messages are processed is significant in the context of the core service logic. This determines whether it’s fine to park requests which exceed the threshold for later execution. Even if the requests are parked for later use, this is just moving the problem to another place in the system or to another point in time. If the amount of throttled (parked) messages is big, when processing the parked messages you may face another throttling challenge. If we’re just moving the problem it does not seem like a viable solution.

If the order of execution is to be guaranteed, the solution mentioned cannot be executed as no new messages can be executed until the one exceeding the throttling policy can be successfully processed. This is another solution which does not seem viable to solve the throttling issue.

What other options do we have? If we look at this scenario, the throttling should do no more than manage that messages do not get lost while the availability of the service provider is not guaranteed (the throttling policy exceeded situation is the same as having an availability issue of the service provider). This can be solved by utilizing a queue to mitigate for the times of reduced availability of the service provider. This should be sufficiently supporting that (eventually) the address update is being executed.

A way of throttling in this situation is to have messages read from a queue (the messages posted by a service consumer) at a maximum predefined rate to prevent the throttling policy from being violated. This can only be done if the consumer does not require a synchronous response to its update request message. This is a perfectly fine solution where no messages get lost, the order of execution is maintained only if messages are read from the queue one at a time which then seriously impacts the scalability of the throttled service.

The processing order of messages is relevant if out of order execution causes data integrity issues or service failures. Some examples are: two subsequent address changes on the same customer result in the incorrect address details in the customer database if executed out of order. Or similarly, if the core service logic of a service must create subsequent service calls, and the second one cannot be completed successfully without the first one being executed successfully, this will create similar data inconsistency issues.

A way to make the system more scalable is when messages can be skipped if a more recent one was received; this applies in certain situations only. When taking the two subsequent address changes as an example: only the most recent message for the address change is relevant to the system. This can be achieved by assigning a time based or sequence based message property or header element to a message upon receipt by the service consumer. Perhaps the service consume can already assign this to the request. If an expression can be used to identify whether a processed message can be dropped because a more recent one has been processed before. (A similar system can be used to detect replay attacks). For this to work, a form of data store must be available in the system to keep track of these requests.

Where to throttle in the SOA?

Where should throttling happen anyway? There is no single right answer to this topic. Let’s consider the following layered service inventory:

a - Public services controlling the access to services inside the service inventory, acting as an endpoint for all external access to the service inventory
b - Orchestration services (orchestrated task services) controlling all centralized and long-running processes
c - Business services (ie. task services, utility services) which are the sole access point to any underlying layer
d - Data Services (ie. entity services, utility services) controlling all back end access.

What happens if we throttle on services in each of these layers?

Ad a)
This can control the amount of traffic allowed into the service inventory but what does that achieve? It would only achieve throttling on that level in the infrastructure. It would be good for controlling specific service consumer policies and indirectly keep the load on the underlying system manageable but in the end, many public services can access the same business services or back end services in complex compositions resulting in a significantly greater amount of requests to the back end which can be a multitude of the amount of client requests. Furthermore, not all business service would need to be exposed on public level, meaning that load can exist on the lower layers inside the service inventory that the throttling mechanism would not be measuring.

Ad b)
A system can hardly be throttled on this level, as process starters are often inside the orchestration engine and cannot be exposed to service agents. This means that if a process must be controlled, it’s probably best to throttle externally to the process, ie. in the layers ‘above’ or ‘below’ the orchestration layer. Or by controlling the flow of messages to and from the process in general.

Ad c)
Business services might seem to be the best place to control throttling because they see all the traffic coming into the system has to pass by a business services. They see the traffic from the top layers and the traffic from the layers below. But multiple business services may be composed together in complex compositions, resulting in a throttling nightmare when the need to throttle arises on these kinds of services. For example, if I had an order service which is used in a composition which should also invoice the customer and schedule an electronic payment, on which of these elements (functional business service areas) do I throttle? On one of the composed services? On the service composition controller? On all? Even here a single answer is not possible because it all depends on the purpose of the throttling and potentially this differs per throttled service.

Ad d)
This can control the back end load probably best, but as the data access services usually do not know the context in which they are being called, applying the throttling on this level has its restrictions. Referring back to 2) if the read of underlying data is relevant for an update operation on the same service composition, then what kind of an effect does refusing the read operation have on the service composition, and what is the consequence of that to the bus infrastructure? This is not easy to answer as it is probably different for many services.

In the end I think it all comes down to why the throttling happens from business or technology point of view. Sometimes the throttling happens to protect the legacy and back end systems, which should allow for throttling on data service level. Sometime the throttling happens to protect the middleware from excessive load, which can probably best be manages at business service level. Sometimes it’s one specific consumer which can threaten the entire system’s availability and then a throttling policy for selected relevant service capabilities exposed to that particular consumer may be used to protect other consumers from the load caused in the middleware by the throttled one.

Each method has it’s pros and cons. When looking at the overall picture, it may be perfectly fine to throttle on two or three levels. Although a combined throttling policy may not be the most easy to comprehend and it may not be using the system resources to the best extent, it still remains a popular method as it guards a number of key parameters of the system. This results in a solution which is still manageable without the need for capacity enhancements.

Of course throttling policies can be used in other ways, for example to give priority to certain messages, or messages from certain consumers or customer requests on the system, and many other ways exist, but this post is just an example and I can never convey all the issues and opportunities of throttling in one single post.

Dealing with throttled services in a reference architecture.

I hope that this post conveys that throttling is not trivial. In fact it’s crucial to have an up-front analysis done for your throttling architecture before any policy is applied. This can be formalized by documenting specific approaches for specific situations in a reference architecture document.

A well-respected colleague of mine once said: the throttled message can be discarded and the service consumer can be thrown a technical exception. Although this may be fine for many messages and throttling implementations, please be aware that more options exist and can be addressed by having a throttling (reference) architecture.

A throttled service may throw a technical exception. Usually technical exceptions are treated by consumers as a permanent failure for a service call. If the call was a read operation, probably it may not happen again, but if it’s a write operation, the consumer may have retry mechanisms in place which might immediately result in another call with the same message, This however is the easiest and most straight-forward implementation and can be introduced without really big implementation issues in the system. Most initial implementations may be using this method. Some caution with this statement: if the consumer treats this as an invalid message call to another system, some elaborate log analysis sessions may come from this, since people cannot tell the difference between a back end availability issue and a throttled message response. To make this difference, you may not be able to avoid customization of the services.

A throttled service may throw a technical or functional status (“not now”) but in the end this means that the service consumers must be able to understand this message. What it means is that at present the message cannot be completed. Probably it does not make sense to retry the message at this point in time but a retry at a later time may work perfectly well. This means that a delayed retry may succeed after all, whereas an immediate retry would not.

Once a reference architecture exists, it should be easier for system administrators to think about and implement new policies, and fine-tune throttled entities. But be aware that, depending on how elaborate the throttling architecture is, the complexity of throttling may dramatically increase. Even if current throttling parameters are understood perfectly well, a dependency analysis must be conducted to be able to fully assess and understand the implementation of a new throttling policy.

A similar risk may apply for making changes to an existing throttling policy. As soon as you tune the throttling policy to switch close to a point near “typical load”, or the typical load increases to a value near the configured throttling policy, dramatic changes in system performance and behavior can be expected.

My advice would be to have and changes or new policies investigated by a team which consists of administrators (system current knowledge), architects (system dependencies and consequences) and capacity planners (system future load).

Throttling is not trivial; it's as simple as that!

Published the Reference Data Observer candidate pattern

Hi all,

I just published the Reference Data Observer candidate pattern. I'm looking for people to testify that they have applied this pattern to get it promoted from the candidate to the pattern status. Check it out here at soapatterns.org.

- Roger

Published the Service Data Forward Cache candidate pattern

Hi everyone,

I just published the candidate pattern Service Data Forward Cache. Looking for people to testify that they have applied this pattern to get it promoted from the candidate to the pattern status. Check it out here at www.soapatterns.org.

-Roger

Published the Configurable Contract SOA Design Pattern Candidate

Hi all,

I just published the Configurable Contract SOA Design Pattern Candidate on www.soapatterns.org.
I'm looking for people who can verify that they have successfully applied this pattern to be able to get it promoted to a recognized SOA pattern.

-Roger

Certified SOA Architect

Finally managed to make some time and get the SOA Architect Certification done. For more info see www.soaschool.com.

-Roger

Agnostic / Non agnostic

Recently I had a conversation with a customer who did not understand the difference between agnostic and non agnostic.

An example from PCI compliance pov could elaborate which I would like to share.

In service orientation one of the guiding principles is to separate agnostic and non-agnostic service logic. This can be done by separating agnostic service logic into one service and any non-agnostic parts into (an) other service(s).

The definition of agnostic/non agnostic did not help him onto the right track so we used PCI compliance to allow for an appropriate example.

To check the definition of agnostic/non-agnostic, check it at SOAGlossary.com

Summarized, agnostic logic is supposed to be more reusable than non-agnostic logic. Separating agnostic logic from non-agnostic logic is done to increase the reuse potential of a service.

In PCI compliance, one of the scenarios is where customer management representatives view customer details. The business rules dictate that unless explicitly necessary, a card number should not be readable. In this example, the method of making a card number not readable is by masking parts of the number when displayed.

Several places for masking can be identified:

1. In the service which retrieves the customer details including payment data (as the customer had requested)
2. In the front end(s) which use this service to display the payment details are changed where necessary
3. Somewhere else?

Regarding 1) 

Introduces the issue that when the service mask the payment details all the time, the service can never be used in other scenario’s ie. when requirements for viewing unmasked details or when the service is used on other service compositions and unmasked data is necessary. Since the service is agnostic, it does not know the reason as to why the details are retrieved.

Regarding 2) 

Introduces the issue that during the initial implementation one might catch all the scenarios, but as new screens and applications (front ends) are built, no guarantees exist that these comply with the PCI requirements. Furthermore, if the amount of front ends or screens is large, the effort to incorporate the masking logic may be considerable, resulting in an expensive implementation.

Regarding 3) 

This is where the agnostic/non-agnostic can be explained more easily and this is what the remainder of this post will elaborate on.

Agnostic logic should not be aware of the reason why it’s used. This means it would lack functional or process context. If the knowledge like reason or process context must not be in the service which retrieves the customer’s details we can move that knowledge and corresponding logic into a separate service which is non-agnostic. In fact, the task service type as defined by Thomas Erl et al is intended to do exactly this: encapsulate non-agnostic logic.

So we’ve ended up with a customer details service which is agnostic and will retrieve unmasked credit card number data. This can be considered an entity services as per Thomas Erl et al definition.

And we’ve ended up with one or more task service(s) which should be able to distinguish from the agnostic service in the sense that it knows whether or not to mask credit card details.

As indicated the customer details service is agnostic and will not perform the masking logic. If we use the contract centralization pattern and the logic centralization pattern, we have created an official end point pattern. This means we can force users in the service inventory to not use the agnostic service directly but force them to use the non-agnostic task services.

A task service can then deal with the non-agnostic context. A task service is built in a certain context, ie. it retrieves the customer details for sales process purposes. A sales process should typically not be interested in the credit card number. The non-agnostic service, intended for the sales process would always mask the card data.

In the context of a billing process, another task service could expose the unmasked number so that particular service support the actual act of making payment with the card number.

The end result is that:

- the reusable agnostic entity service is not publicly exposed

- one or more non-agnostic task services are publicly exposed, managing the masking logic depending on the context it's called in.

Note that the two task services could be replaced by one task service that could perform masking of card holder data, based upon additional parameters, ie. a reason code, which represents the context in which the service is called. The task service would then decide, based upon the reason code, whether or not to mask any data.

Hope this one helps a bit - I know it's a bit over-simplified but this should paint the picture.