Arriving in Chennai

Just arrived in Chennai and already had my first conference call - I guess work at home continues while we travel :)
Boy is this area hot - it was 35 in Bengaluru but the air was quite dry so the heat was bearable. Here in Chennai, it's 'only' 32 but as it's located at the Indian Ocean the air is very humid. When I got out of the airport I was wet all over my body in an instant!
I like the cabs! These I would usually see in movies only. Lacking airco, seat-belts or any kind of comfort they really have a certain charm.

Leaving for Chennai

Our engagement in the Bengaluru area is coming to an end. Thanks to Babu, Raj, Arun and Madhu for hosting us. These were great sessions and we had good progress. Tomorrow I'm travelling on to Chennai where I will be continuing the visit to the Indian offshore offices.

Follow the process and thinking outside the current constraints

Today I am the second day in India. The hotel is really great!

While here, I have encountered a few great examples of how people can become stuck in processes and how continuous process improvement as well as better exception/compensation handling could have helped.

A lady arrives early from the airport and the staff had just begun clearing up the breakfast buffet. Everything was still there so there should not have been a problem. Unfortunately, because the staff to clean this up had already entered the breakfast room they apparently could not stop the process and the lady could not get anything to eat ; so much for hospitality in the hospitality business...
- later I have asked the staff about the how and why and the explanation was fairly simple: "this is how we have agreed to work sir. We first warn everyone in the room that we are about to close the buffet, and when everyone has indicated they are all fine with this we can start the cleanup procedure". Apparently the procedure does not cope with people who arrive to the room when they have finished asking around....

Anyway to cut a long story short, I found out that the improvement process is typically: when a customer complains and the number of complaints in a certain period becomes too high, the process can be changed to compensate for the increased number of incidents. Despite all good intentions, I believe there is just not enough room for common sense or for improvement suggestions triggered by the people who are in the process themselves.

Extending this, an analogy can be found in (IT) processes. Very often I encounter that "we all" work according to procedures and we can hardly accommodate for anything that is not covered in these procedures. Also in IT, common sense often simply not used as we have proven time after time that we cannot accommodate for this in our procedures. Apparently, thinking outside the current constraints is still hard to do. We do everything by the book and still the customer is not happy :(

Bengaluru

Today it's Sunday and I'm working in the hotel room but I'd like to share a few photos I took yesterday on my first day in India.

Ganesha, remover of obstacles, aka Godess of Luck

Lord Shiva, aka the destroyer of evil

and follower of Ganesha

Unfortunately only a few hours are available for sightseeing to not many pictures.

Jetlag!

This morning at 9am local time I arrived safely in Bangalore, India. Funny to see yourself being filmed with an IR camera. My temperature was OK :). I cannot sleep in planes but I saw a couple of nice movies "Twilight" and "New Moon Saga" both were ok but had very sudden endings; the plot was not really properly ended and they were bad cliffhangers. By the time I arrived at the hotel around 10.30am, I had gotten over my tiredness and could start with a normal day with breakfast. At the time of writing this it's about 8pm in the evening and I guess it's more than time for another shower and get something to eat; I'm starving... Keep you posted...

End of my Egypt Visit

So unfortunately my visit to Egypt has come to an end.

Roger @ Sphinx

I met nice and friendly people and we had a very productive week. It was way too short for the agenda and also for meeting these fine collegues. Now I will get a good night of sleep. Fortunately, some time was available to visit the pyramids and other fun stuff late after work. Oh and Egypt has nice food: Fool, Kosheri and Molockheya are great to eat. Tomorrow you should find me in the Egyptian Museum in Cairo and in the evening I will leave for Dubai and Bangalore. Shokran likum for taking care of me this week and hi to Ahmed, Ahmed, Hani, Heba, Islam, Wael and of course Hoda, Nadine and Mona!

- Tesba7 3ala 7'eer!

View from Al Azhar park on old Islamic Cairo - city of 1000 mosques

Separation of concerns: Exception handling

This week (26-02-2010 ... 05-03-2010) I'm in Egypt for business.

While using a local ATM machine, I got an error message on screen similar to "Error xxx occurred while performing activity yyy". What was clearly visible from the descriptive text was that the activity I was performing, that activity yyy was actually a service on a very specific back-end. Even the back-end name was mentioned in the descriptive text. Also, from the error xxx, I could derive that this was a specific session management issue with the back-end itself.

Why is this wrong?

Well for a number of reasons but here are the main ones:

Security


- Security wise, this message exposed sensitive implementation details to the user, on-screen. Mentioning system names, nature of exceptions to a user etc. is a risk. Any mischievous user could easily have used this information to their own benefit. It is these kinds of mistakes which can easily be avoided. The user should not be bothered with this kind of information. The ATM user is simply not concerned. A simple "Service not available" or "Service temporarily unavailable" message should have sufficed. The user is not interested in this information. A system administrator for example, should be concerned.

- Similarly activity names expose functional context (expose implementation details on functional level) to the ATM user. Same issue: why would you give out such information to a user. Shield the "activity concerns" from  the user by not telling them too much "Service temporarily not available" would suffice.

- The back-end exception was almost 1:1 delivered to the front-end. The builder had implemented specific back-end exception handling to be "pass-through" - meaning that any system which needs to react on these exceptions, would have to be concerned with the actual implementation of the back-end exceptions. Note that this last one is a 'guestimate' as obviously I cannot look into the system. But from interpreting the error message I have strong reason to believe so.

Separation of back-end exceptions from integration-level exceptions

What strikes me as odd that there are still many applications which would have to integrate with back-end systems one way or another, expose back-end details to consumers or even to user interfaces. Good practice is to shield domain specific exceptions (back-end) from the consumer of a service. This is commonly referred to as "Separation of concerns". The consumer of a SOA service does not need to know about the domain specific service because that would cause what Thomas Erl refers to as "contract-to-implementation" coupling, one of the four negative coupling types.

While we are at it, think about this one, It is not directly related to the issue at hand but it is a good practice anyhow:

Separation of technical exceptions from functional exceptions

What I encounter many times in my profession is that in the way exceptions are implemented, people often do not make a difference between technical and functional exceptions. This is related to the fact that technical issues (real exceptions) and functional concerns (not really exceptions) are easily intertwined - sometimes the conceptual difference between the two are ignored or not recognized leading to complex implementations.

Technical exceptions are defined for situations in the system which break the normal operation (ie. Service Not Available, Connection timeout, Timeout, Service down for maintenance etc). These happen typically when something is wrong with a system, the network, the database etc. As you can see these are significant failures in the system as they prevent the software system from working properly, that is, from executing the core service logic. Typically these problems require rigorous kind of resolving and are considered disruptive to the execution of the core service capability. If you encounter one of these, a retry mechanism may make sense; but not always.

Functional exceptions are not really exceptions. They typically happen while executing the core logic of a service, and are conditions which make sense to the core logic of the service. Note that in these situations, nothing is wrong with the system. Examples are Customer not found, Customer status is inactive etc. Business rules are typically based on these kinds of exceptions. If you encounter one of these then typically the consumer can anticipate that another successive call to this service will return the same result.

A strategy I tend to follow is that both technical status and functional status are mapped into typically two separate fields. TECH_STATUS and FUNC_STATUS.

Whenever a service capability is to be executed, the response to any consumer would always contain a technical status. As the tech status illustrates whether something technically went wrong it is important to ie. both request-response patterns, as well is fire-forget.
The functional status however, may not always be available especially ie. when a fire-forget pattern is executed.

Some examples are listed below.

request response:

• tech status = service down for maintenance; func status = don't care/not available
• tech status = ok; func status = ok
• tech status = ok; func status = no data found

fire forget:

• tech status = connection not established; func status = not available
• tech status = ok; func status = not available

Obviously, a fire-forget pattern will not return a functional status as it will be executed offline from the consumer logic execution. Potentially a functional status may be returned asynchronously but this would not always be necessary. Sometimes it is sufficient to know that -eventually- the service core logic gets executed.

Hope you had fun reading; until next time...

SOA Certified professional

Oops, I forgot to share this...
For some time now I'm officially a SOA Certified Professional :). For details on SOA certification, look at www.soaschool.com and www.soacp.com.
If you want decent, vendor agnostic training ;) I suggest to try one of these programmes. I am now working on the SOA Certified Architect programme. I was on the Rotterdam November 2009 class. Currently I'm preparing for the module 3 examination; wish me luck!!!

Published a SOA pattern

I just published a SOA pattern for review at SOAPatterns.org.

Please review it and give comments and feedback on the Process Orchestration Recomposition pattern. This would include suggestions for pattern name by the way.

Thank you very much;  I really appreciate your input!

Some thoughts on service naming / service capability naming conventions (1/2)

A topic which has had a lot of discussions in the past and will still have a lot of discussions and disagreements for a long time to come, is service naming / service capability naming.

Recently I got another trigger on this topic: imho service naming which does not reflect the purpose of a service properly. A lot can be said about service naming and there are almost as many opinions to the subject as there are people interested in the subject.

This post tries to describe my personal humble opinion on service naming. You can agree or disagree and I'm willing to hear your comments, just leave me a comment and I'll be glad to take your input into this article.

Service naming is a very important subject for any SOA. Any self-respecting SOA architect has/uses a SOA reference architecture with service naming guidelines in it as it directly contributes to one of the most misunderstood but very important SOA design concepts: service (capability) discoverability. In the past, discoverability had lost a lot of its charms for architects and designers as the intended purpose (runtime) discoverability as a goal was never really reached in the way it was intended when initially conceived. Instead, nowadays, design time discoverability is becoming more and more important and service naming is one of the key concepts for defining discoverable services. If the service name does not properly match the scope and purpose (the core logic implemented by the service), discoverability becomes inherently more complex. If your service name is chosen wisely, the service sells itself better (real life example: if a hotel is not properly/oddly/ambiguously named, you will probably skip it when you are browsing the hotel directory for a place to sleep, especially if the hotel is called "amour toujours").

In this post we will use the following structure and definitions when referring to services:

service::capability where:
service - represents the unit of service logic to which the SOA concepts 'have been applied to a meaningful extent' as per Erl's definition :).
capability - represents the operation(s) which can be performed by the service.


Some examples of bad service names are service names which are
- scope and context issue: Inventory (what is the scope of "Inventory"? Just managing inventory levels? Or inventory locations, both, something else?)

- core logic: ::update (what does it update? even if we know the definition of the service itself, what is the service context in which updates happen here?)

- ambiguous naming: OrderManagement::execute (what is it that the order management service is actually executing? Ie. execute order fulfillment or ordering?)
etc.

How can we deal with this problem? How do we prevent these problems from being created in the first place?

One answer to this question (besides "governance"), is to define naming rules and conventions  (which must then be appropriately governed). Before starting  the actual discussion, we need to understand that different kinds of services exist and they are meant to perform different types of logic. I will use the "Erl" definition here which distinguishes between Entity Services, Task Services and Utility Services. More service models can be created but I have intentionally stuck tot he basics. Roughly, you can take these assumptions for this post:
- Entity service: performs their capabilities on business entities like Customer, Invoice, Order etcetera.
- Utility services are services for 'shared system logic' or cross-cutting technology services like logging, notification etc. They help the system to be operable while separating concerns like notification out of the other service types.
- Task services perform operations triggered by and executed in a certain (business) process - hence they contain business (process) logic.

Below you can find some suggestions to setup such a guide for designers to follow. I'm happy to receive any feedback on these.

As said, whether it be services, or service capabilities, apply naming standards. Don't do it for one and not for the other as it is useful for both.

Service Names should place the service capabilities in the appropriate context and define the service scope wisely. Knowing this is not a smart statement, I can say that this is up to you. Service names should be assigned differently, depending on service type (Erl definition of service types):

• Entity services - use the [entity name] service naming like Order, Invoice etc.
  
• Task services - use the [process name] service naming like OrderManagement, Collections Management, Credit Management etcetera. It's perfectly fine to sub-categorize if necessary, like in this example: OrderFulfillment instead of Order Management.
  
• Utility services - use the [verb-or-noun] service naming like Notification, Logging etc.
  

Regarding service capability naming, apply the following pattern:

[verb][noun][subject]

The verb is used to indicate what happens to the noun.
The noun is used to indicate, to what the verb is applied.
The (optional) subject allows for distinguishing between different logical subjects which may be meaningful to use for certain nouns).
Sometimes, this paradigm seems insufficient, ie. in case of notifications - see proposals below. Try to find a pattern which suits your needs.

Examples are:
- readOrder
- readOrderHeader
- readOrderLines
- updateOrderCustomer
- updateOrderPricing
- updateOrder
- notifyOrderUpdate or sendOrderUpdateNotification
- receiveOrderUpdateNotification

Note that when we expect readOrder to return an entire order structure, updateOrder expects to update an entire order datastructure. I would be concerned if these would be the only two operations I could perform on an order, referring to the service granularity concern: these would cause issues in that area: an updateOrder service would probably be much to big to be useful from a true SOA design principles point of view, but feel free to discover this yourself. Let me make the following statement: if you only need to update the shipping address on an order, and for you to do that, you had to read the entire order, change the applicable order parts and then save the entire order, you are most likely heading down the wrong path regarding service granularity. Continuous evaluating and re-evaluating the granularity of services and service capabilities is a constant factor anyway as the only constant factor we value is "change" ;)

Whether the service capability name may repeat (part of) the service name remains a debate on itself. An example of this is OrderManagement::updateOrderShippingAddress. Typically, the service name implies the context for the capability, making it useless to repeat the service name in the capability name. Applying this to the example would result in the OrderManagement::updateShippingAddress name.

For entity services, I can give one additional advice: try to use standardized (predictable) names, purposely chosen to make it easier for designers to define them for any other entity service. A suggestion for entity services would be: create, read, update, delete (aka. CRUD).
Following this kind of an approach reduces the number of discussions you will have on service naming. In fact, apply this predictable naming concept to all service types whenever possible, by establishing a standard vocabulary. Especially the verb naming requires some extra attention as there are many ways to explain similar behaviour using different verb names. If the intended behaviour is the same, then use the same name. Do not use aliases for the standard verb names unless it removes ambiguity if the standard verb name is used.

Examples of predictable verb names in a vocabulary are:
Get - retrieves data
Retrieve - same as get but discouraged: name too long which makes it hard to read service capability names.
Notify - creates a notification
Create - creates something (and in case of entity service, may store records in a database)
Update - updates something synchronously
Post - updates something a-synchronously as in fire-forget mode

A service capability name must describe what the core service logic does, not what the consumer does. Recently I have been involved in a discussion on service naming. Suppose we have a service exposing events about changes in a customer database object the Entity Service's service and capability names would be similar to CustomerService::postChangeNotification. This service name indicates that it is related to Customers; the service capability name indicates that it is sending notifications about changes on the service's underlying distinct functional scope: the customer objects. The replication service capability consuming these events would probably be named something like OfflineCustomerPortalCacheReplicationService::processCustomerChangeEvent. (Please don't flame me for insanely long names). The service name is referring to an underlying object "offline customer cache" and also the service is responsible for replication ( this would most likely be a utility service). The service capability name indicates that it processes the events fired by changes on the customer object (as generated by the CustomerService), in the context of the replication service for a certain offline customer cache for the portal (this example could be the implementation of a forward cache pattern).

Last advice: please remember not to try defining services and service names in splendid isolation! Get other people's opinions and make sure that you all (well, most of the lot anyway) agree to avoid different naming by different project members even if they are in the same team! This obviously also applies to other architecure and design activities you and your team members do.  Perhaps it's a good idea to setup the naming guidelines together. The discussion around service capability naming will probably never go away, but at least you will apply the same set of concepts and rules to service naming and service capability naming. In the end, life will be easier I hope.