Big Data, BI, Data Analytics 2 — Consolidation
When the market changed
It didn’t take long. From the moment when the first BI tools entered the market, until the cutting-edge companies had started using them — and until they got bought by the big enterprises.
Some of the first were Siebel, which originally made a CRM tool but added BI too - and was bought by Oracle. Cognos appeared and was soon bought by IBM, and BusinessObjects, one of the first to use modern BI technology (OLAP), was bought by SAP. Spotfire was bought by TIBCO, and Tableau remained independent for a long while, but was eventually bought by SalesForce.
A few more remained independent, such as MicroStrategy and Qlik, and Microsoft began adding BI functionality to their SQL Server database but had no significant market share for a long time to come. That changed along the way when they launched Power BI.
You can search for BI tools and find many more, but most of them are quite insignificant today - and those that were bought by the big enterprises vanished quickly - among those, only Siebel’s BI tool that was renamed to Oracle BI, and Tableau, are still major players today.
All this happened from the beginning of the 2000s and a few years ahead, when I was moving on from being a consultant for a large consulting company, via my own BI focused company that eventually closed, and into finding my feet again by first taking on a students jobs with BI in a government institution, then upgrading it to a real job, then again back in the consulting business with a focus on BI, and from there considering a handful of offered BI jobs - deciding on a job that had almost nothing to do with BI.
While establishing my own consulting company, the market saw other similar companies being established. BI was hot, and people with better knowledge than me on how to start and run a company also entered the business, growing their businesses faster than I did. I had no ambitions on making it a big company, though, just a handful of people working for a few key customers, creating “reporting systems” for them - the term BI had not yet been introduced when I started in 2000.
The first years of the 2000s saw a parallel focus on BI and CRM, but the latter was actually more popular. “Data-driven” as a concept had not yet been invented, and the transaction systems in use didn’t collect all the consistent data needed for making great analyses and reports. CRM was much easier to go to, as it was about keying in the information, along the way, that sales and support people were using anyway.
So, the front-runners in the BI business did have a good business, but only with a small group of first moving customers among the bigger enterprises. It all started then, after several of these companies had begun on their own with advanced Excel sheets and Microsoft Access reports in some sort of static setup - often difficult to use for others than the people who had made it, such as the CFO or someone else in the Finance department.
There had been some even earlier front-runners, using statistics software from SAS or SPSS, but the majority of even large companies didn’t have the expertise needed to work with these rather complex tools.
What was new from effectively around the year 2000 was OLAP - the so-called Online Analytical Processing, which was implemented as a different database structure but based on input from the OLTP - Online Transactional Processing - databases that served the transactional systems such as general ledger and production systems. Data warehouse was the buzz term for the OLAP databases, and with a data warehouse, it was possible to use a BI front-end tool for making dashboards.
Exactly dashboards were the other new thing. KPIs (Key Performance Indicators) had become popular around the year 2000, along with the Balanced Scorecards - a few calculated numbers that were supposed to tell if the company was going well and according to plan. These scorecards were then put into dashboards for management to see them directly, updated every night.
Hence, the introduction of BI came as a continuation of the KPIs and the scorecards, only requiring the needed data to be present and in a suitable form.
With this move, databases became sources of knowledge and, as some companies in the business like to call it, wisdom. Previously, databases just allowed for computer programs to store data somewhere, until they were needed by another part of the program, but now, suddenly, data became valuable even outside the program that had generated them.
We saw this way, during the first few years of the 2000s, a consolidation of different uses of computer systems - as an integration of systems through data and a new kind of collaboration between different departments of the enterprises, some producing data, others consuming them.
And we saw a quick rise and fall of many players in the new and promising market for BI software where all the most popular tools were bought and faded out.
Already in 2004, this race was close to end. Not completely, but all the ideas introduced during just 2–3 years before, were already being replaced with new ideas. When I talked to a new and growing BI consulting company in 2004, they were already speaking about dashboards as something “so last year”.
So much had happened since the late 1990s, where I had invented an OLTP-based data warehouse to support the growing reporting needs for a customer, and we were still only in the beginning of the 2000s.