Native logging and monitoring for Azure .

This post is the second in our series on integration logging and monitoring. If you’re interested in a more tech agnostic view on this subject and how you should go about choosing the solution that’s right for your organization, then have a look at the first post of the series by Magnus Petersson.

Introduction

In the massive toolbox called Azure, it can feel a bit of a jungle when it comes to choosing the right services to support you with the specific business-needs that your organization is facing. Whether this business-need is related to servers, networks, or security. Or as in the case of this blog, the need to keep track of your integration transactions. In the following post we will shed some light on how you can go about doing just that, by using the services we have at hand natively in Azure. 

Common logging object

Central to being able to expose logging in a structured way is to organize the data you log. Remember to slice the context both in a business-oriented way and in a technical way. The logging should be able to support both business and integration operations. Find a structure that contains the central information in an easy way to access and provides space for tagging the information with more specific info. An important part of your common logging object is the CorrelationId, that should tie different building blocks in your integrations together so that you can follow all logging for an entire run. A strong suggestion is also to include a status code for different kinds of warnings and errors. 

The Azure Integration Platform

Azure Integration Services (AIS) is the collective name for the services (products) that Microsoft points to for enterprise application integration and are normally the ones that an integration platform in Azure revolves around. AIS are separate services and they exist in an integration landscape that also consists of other supporting services. Most services have some sort of out-of-the-box logging and up to a certain point it can be sufficient to investigate the logs of each service to gather information. This will become more and more difficult as the volume of integrations and transactions within the platform increases. So how can you keep track of your transactions in Azure when they are spread across multiple services? The answer to that question is to use Azure Monitor.
 

The Azure Monitor toolbox – where do you start? 
 

Picture: Overview of Azure Monitor - Azure Monitor | Microsoft Azure
 

Azure Monitor is the collective name for the best services that we have at hand to view and manage metrics and logs in Azure.

For AIS integrations, Log analytics is your friend and the cornerstone. It consists of both the log where you store your log data (as common logging objects) and tools for accessing the data. You enable extensive automatic logging from, for example your logic apps by:
⦁    Creating a Log Analytics workspace
⦁    Deploying Diagnostic Settings for Logic Apps to Log Analytics workspace

Structure your log data by implementing your common logging object by logging it as tracked properties in your logic apps. Log from your Azure Functions and support Apps by using the HTTP Data Collector API.

Design dashboards, use Log Analytics Workbooks, Log Analytics - Logic Apps Management, alerts and “Custo Queries” to monitor your integrations within Azure Portal and build Power BI reports to let the business monitor their own data. The alerts could in many cases create issues in your external issue management system or simply in Azure DevOps. 
 

Extend your dashboards to include visual representation of your Metrics data to be able to proactively mitigate problems and understand performance challenges. Don’t overdo this just because it looks nice but think about the value each graph should create. 
 

Next in turn is Patric Kronberg that will tell how we do it with Baseline integration Portal powered by Nodinite. 
 

Henrik Wallenberg
Integration Architect
henrik.wallenberg@enfo.se
LinkedIn

Björn Hackberg 
Integration Architect
bjorn.hackberg@enfogroup.com ​​​​​
LinkedIn