Integration monitoring - How to maximize business performance .

Being able to have control and structure in your integration landscape is one of the basic capabilities that needs to be in place. This is easier said than done and with this blog post we start a blog series where we discuss problems, risks, possibilities and some solutions to shed some light on the matter. 
 

Integration monitoring 

Integrations are in the middle of applications orchestrating, transforming and enriching transactions between different systems. This can be enabled with large ESB products, API´s or a cloud-based iPaaS service. Or all together in hybrid solutions. Irrespectively all have one challenge in common, no matter what product or code language they are implemented with. How do you keep track of everything that passes the platform? 
 

How do we keep track of everything? 

Maintaining a high level of service is an important factor when talking about integration. A statement that almost everyone agrees with. But what is a high level of service for your business? What are your demands on keeping track of transactions between systems? All businesses have different needs and ambitions when it comes to insights on the flow of information between endpoints. I believe that defining ambition and deciding the business goal for logging is where it all should start. 

Do you want to track a transaction from where the information was created to where it is used to create business value? Is it possible to follow a value stream, not separate from the different systems or domains but looking at it from a process view? Or is it just enough to be able to verify that the transaction did not get stuck in “my application” or domain? The reality for most implementations is usually somewhere in between. It is not uncommon that logging is developed ad-hoc with a bare minimum of resources put into it, because it isn’t generating direct business value. But when transactions stop reaching their destination or the invoices can’t be sent, then it becomes a problem for the business.  

Why not treat logging as any other demand?  

Who is the user? Is it only for IT operations that are working with message management and keeping the transaction flowing between the systems? Or does the business want to be able to have insights in the orders or invoices that are sent in real time?  

What does your organization look like? Is the goal to have a dedicated team to manage the operational aspect or is the goal to have a self-service approach where each department are responsible for their own transactions between the systems. Who will act when transactions get stuck for a reason or another? And who has the competence to sort it out? Different scenarios also have different security requirements. 

Security requirements. On what granularity do you need to control access to different information objects or technical resources. Who should be able to access those salary transactions or the financial results? Who should be able to stop the order flow? This kind of question are just the tip of the iceberg when addressing logging and monitoring in the integration domain. But work actively with it, to set a goal and a road map. Iterate over the solutions to meet that defined goal, instead of always try to catch up, reacting on missing capabilities and identifying them when you can’t find those lost transactions. Is logging and monitoring and alerts of failed transactions between systems the only goal connected to creating a well governance integration domain? Keeping track of your information in well-defined documentation. BI and Analytics could also be a consumer of transactional data that flows between systems that can provide that real-time information edge you need.  

Next step  

What level of logging meets your needs? Analyzing your business requirements and set a goal for integration logging and after that, define what product/products and services that will tick those boxes and deliver that edge you can gain with the right choices. 

Should the architecture be built adapted to the logging and monitoring products or should the architecture first support the logging and monitoring process that is needed and then implement the suitable products. What tools are needed to meet your demands on each level and what tools are available on the market? In a set of blog post different technologies will be focused and what capabilities they can meet.  

Please stay tuned for the next post in this series where Henrik Wallenberg and Björn Hackberg will talk about a possible logging implementation in Azure. 

Magnus Petersson 
Integration Architect 
magnus.petersson@enfogroup.com 
LinkedIn