When the Purchaser role is impatient and have high expectations, BI/DW- project providers need to look for solutions – how can we become better? How can we achieve more speed and improve quality in our deliveries? Are tools for automatic code generation code a way forward? Is it simple and carefree, or does it require some extra care?
Curious? Please read on!
There is an abundance of different automation tools on the market, aiming for BI in general and DW in particular. These tools promise gold and diamonds and has the Great Promise of reducing development time in projects, ranging from 50% to 90%. It sounds absolutely wonderful, doesn’t it?
Is it that simple? Really? Well, “kind of” is the honest answer. Another answer is "it depends". The tools that are available as full platform products frequently compete with in-house automation tools for narrower development that every consulting firm of dignity has built, or for that matter any line organization where the responsible developer is lazy and clever enough to automate mundane tasks.
What is it actually that third party products offer and how is this different from tools developed in-house? A part from to the price issue of course, ranging between SEK 15000:- per developer license and year, up to SEK 350000:- per company per year. About now, the realization dawns that it is hardly impossible to compare the tools; they have different purposes, serves the needs of different target groups, have different sets of features and have different license models.
A simple sorting provides two main groups as follows:
Category 1: a simple and effective way to a full stomach - Mac-n-cheese
This category contain the tools that are developed in-house, which automates the "monkey coding". Or put in sales terms "we reduce the impact of the human factor on the end result"; how well put! The tools focus entirely on reducing development time, while providing better end-results in the simplest, fastest possible way. Just connect the tool to many data sources and retrieve or submit data in tidied or untidied state.
Category 2: the nine course special tasting menu with Extra Everything and chili sauce on it
The major full platform products includes auto-generated data streams (ETL/ELT), data modeling, visualization & analysis and finally monitoring and automatic documentation.
The meticulous person quickly notes that "but Governance, it is not a technical solution. And Change Management isn’t either. Or… well, we do have Jira for it... ".
That’s just it. The advantage of a full platform product is that it addresses all possible needs in an extremely consistent way. The disadvantage with a full platform product is that you get a whole box of tools that it may already be paying for and may not have use for. In addition, extremely few full platform products can solve all needs in EVERY possible situation. So, you end up building exceptions. Is it even possible, does the product allow it? Many of the extra features support processes that are not specific to BI and DW, they are common processes and affects many systems and business areas – why on earth should all be managed from the BI-box then?!?
To the extent that you would like a Happy Meal for breakfast, lunch and dinner for the rest of your life, it is obviously a very attractive solution.
The grey area between category 1 and 2 can contain real bargains for the conscious shopper. This would be tools that has started adding extra functionality but still has not reached all the way up to category 2. In some cases - wise in my opinion – they do not even aspire to reach there.
"The conscious shopper". How easy it sounds. This elusive mental state can be relatively easy to achieve for the bargain hunter trying to find the best price on a kid’s bike using PriceRunner.
To become a good shopper for products in the automation of BI and DW, there are fewer aids. Until this blog. Congratulations!
As always, the devil lies in the details. This is especially true when we are talking about technology, and BI and DW where personal opinions and requirements completely control what, when, and how something is developed. Do we want a tool without the HDMI port? Or should the color be pink to match the Group's colour palette?