Business Intelligence

Hi everyone, The latest Excel template for IBI (Individual BI) is published.  Again, as usual this is a generic template which if you like to use you should customize to suit your needs.  The generic template is here - https://drive.google.com/open?id=1z2_iEDLkkBqvBHhf-X0qwA0m_niZezAY it has dates up to year 2030. It can be easily extended beyond 2030. To download, once you click on the above link, please click on the top right corner download button. To give you an idea of how to customize and use please see below a customized template that I created for my 11-year old nephew. https://drive.google.com/open?id=1cmTGNU3xlhvKQjna3KR5Vg784OG4ydp7   . And here you can find the customized template that I created for my 6-year old son https://drive.google.com/open?id=1EpxjXFy-k4uD6PxLwvUopQgNc_nPDkbz  . All the best for those who are starting their IBI journey from 1st Jan 2020. IBI - Discover yourself factually. Update:  How to easily capture t...

Are you one among those who feels that everyone is talking about machine learning but no one seems to provide a simple example to easily understand it, if yes, this short video should help. And yes, you don't have to be a data scientist to use machine learning.

Latest 2 minutes video on IBI provided below

For those who have been following my IBI (Individual Business Intelligence or Individual BI) journey I am very happy to announce that I have completed more than 1000 days of data capture. On 15th November, 2019 I completed 1000 days. I had started capturing data from 19th Feb 2017. Based on the analysis of own data I have found some interesting patterns, learnt a lot about myself, stopped myself from doing the same mistakes again, noticed changes in patterns, etc. The benefits that I get from Individual BI are way too many. And therefore I will definitely continue IBI as long as I am able to do it. I will be happy to conduct a session based on demand. Let me know. Some of the charts are provided below to give you an idea of the possibilities with own data captured in a structured way consciously. Gaps in the above chart are because there was no weighing scale on those days or weight was not measured on those days. 

Presenting at EU DataViz 2019, Luxembourg on 12th November was such an honour. Thanks to the Publication Office of the European Union for the opportunity. It was a great experience both in listening to other inspiring speakers and also to present the topic of "Before and beyond data visualization". In my talk, I used two examples; Individual Business Intelligence (Individual BI or IBI) and PublicBI EUProc solutions to talk about before and beyond data visualization sections respectively. It was interesting to note that there were many people who were interested in the IBI example than I had expected, and some also had questions about data security. If any organizations (companies, colleges, etc,) would like me to present the topic of Individual BI at their premises I will be more than happy to do that.  My slides are attached below, if there are problems with the pages below, use this link - https://drive.google.com/file/d/1vll-ugWPh6MoCwxqDj3Eo1d6BkcR...

On 15th of November, 2019 I completed 1000 days of Individual Business Intelligence (Individual BI or IBI).  Everybody says data is an asset. I say, the data I have about myself is priceless. But, I was just wondering  if I were to estimate the price of the data I have collected over last 1000+ days, how much would it be worth? I have 104 data points (ideas, mistakes, dreams, health, work, happy days, etc.) captured for over 1000 days. Let's consider 1 EUR per data point.  So 104000 EUR Time spent in filling IBI data capture sheet everyday until now (1014 days) = 110.8 hours So around 11000 EUR (assuming 100 EUR per hour) Time spent in analyzing data until now (1014) = 76.5 hours So around 7600 EUR  So in total it is minimum worth 104K+ 11K+ 7.6K = 122600 EUR Would I sell my data? I don't think so. Would I share my data for a good cause? May be some of it.  I think if data is an asset, then we sh...

We often find ourselves in situations where the BI teams, because of the tools we use and skills we poses in dealing with data, are misused by other departments to carry out non-BI work. I would like to know how companies deal with these situations? Examples of non-BI work 1) Migrate data from old OLTP application to a new OLTP system 2) Create data extracts to be provided to other systems, example billing system or to be provided to B2B customers Because of the tight deadlines on the non-BI work these tasks get pushed on to the BI teams and the BI work is stalled. The non-BI task is also important but for the BI team it is a misfit.

A number of interesting topics have been shortlisted to be presented at the EU DataViz 2019, Luxembourg. I have the honour to be one of the speakers. I will be speaking on the topic "Before and Beyond Data Visualization" Those interested please go ahead and register at https://publications.europa.eu/en/web/eudataviz/home

Happy to announce that I have been selected as one of the speakers in EU DataViz 2019.  Check the official page here - https://publications.europa.eu/en/web/eudataviz/home EU DataViz 2019 is an international conference organised by the Publications Office of the European Union. It addresses for the first time the specific needs of the community engaged in data visualisation for the public sector in Europe, bringing together experts, practitioners and solution seekers.

Are you tired of working as an external and researching for reasons to switch to an in-house BI team? Or are you in a situation where you have to choose between an IT consulting/services company position and an in-house BI team position for the next step in your career? Or just curious to know how it is to work in an in-house BI team? Irrespective of the reasons, I hope that the content in this part 1 of the in-house BI teams multi-part article provides you a well-rounded perspective about the topic and probably even triggers your interest to be a part of it. As there are various definitions for BI and confusions around it, it’s important that we clearly state what we mean by BI to ensure we are all on the same page before we start comparing how it is to work in an in-house BI team vs as an external. Business Intelligence In simple words BI (Business Intelligence) is all about deriving information and insight from data efficiently at scale to enable fact-based decision making in...

It's been over 2 years that I have started IBI . A small subset of the results is shared below (charts) with an intention to create awareness about IBI. One of my personal objectives for this year is to create awareness about IBI among high school and university students.  So if any of you have connections with academic world and would like me to present the IBI concept, tools, my experience and learning from it, I would be glad to present. Almost everyone today, especially data professionals, preach that organizations should be data-driven, however, how many of the data professionals are themselves  data-driven individuals ? How many of us put in any effort in capturing and accumulating our own data in a way it can be used for self-discovery? Shouldn't we practice what we preach? We all agree that if we don't measure something, it can't be improved. Why don't we measure ourselves? Don't we want to improve? On 19th Feb 2019 I completed 2 years of capturi...

What exactly does data, information and insight mean in the context of BI? Data : The raw form from which information and insight can be derived. Usually any recorded values, numbers, text, audio, video, stored in any form, any size and any location that gets generated in any event or transaction or just based on the current state or status.  It could be internal data like employee related data (name, address, phone number, gender, etc.) or products, services related data or customer related data or server logs, web clicks, call center data , product reviews, ratings, etc. Anything and everything that can be used to derive information is data. The much hyped big data is also data. It could be stored as files or in a database or just as logs. Burger Chain Example   For example, the event of a customer buying a burger from a fast food restaurant generates lot of data. Time of purchase, terminal used for payment, employee who served the customer, amount and curre...

I wanted to profile data,  CSV files with 60+ columns and 1 million plus rows. I started searching for a easy to use tool that I could use for data profiling of these files.  That's when I noticed that data profiling is not clearly explained anywhere.  So here is my attempt to cover this important topic, and I will also introduce the tool, that I found out as part of my search, which helped me a lot to get to know the data. What is data profiling?  In simple words data profiling is a process in which we try to understand the characteristics of the data without associating it with a business process. So basically anyone can carry out data profiling for any data. You don't have to know who generates the data, where and how the data is generated, what is the context of that data. What are the answers we are looking for?  Some are listed below to give you an idea How many columns are actually there in the file? Does it match specification/documentati...

Regular manual download of data from a portal (frontend) and loading it into a backend system is like exiting an airport and then reentering the same airport to catch a connecting flight from the same terminal when you have no business outside the airport.  I have seen people doing this, downloading data from portals and then loading into other systems manually. If there is no one looking at the data in the UI and no decision is to be made and only regular data feeds are required then don't do it via frontend (GUI), just get a data extract job created that will automatically load the backend system.  This is partly related to one of the previous posts -  BI can take you to places

A marketing manager, or a HR manager, a sales manager, or an account manager should he (or she) be spending time in creating reports or using reports to make decisions? On one side, total dependence on BI team for all information needs can slow down business users. On the other hand if business users have to create their own reports or work their way through the dashboards or self-service BI to get to the numbers they are looking for, it could kill their time and thereby decreasing the time for their real work, part of which is to take decisions based on information and insights. And that's why there needs to be a balance to ensure basic first level information can be self-served and for complex requirements BI teams spend time in delivering the information. 

In the context of data analysis/BI  when is something called as a prediction and when is something called as a forecast? Quite a lot of people use these terms interchangeably. Dictionary also can't help here, see below. Source :  https://en.oxforddictionaries.com/definition/prediction So in the context of data analysis/BI I would say forecast is based on past trends.  Time series  is involved. Based on previous behavior future behavior is forecasted for a specific time period.  On the other hand prediction may or may not be based on past trends. So all predictions are not forecasts, but all forecasts are predictions. In this way forecast is like a subset of prediction. Example of a prediction which is a forecast - No of books that will be sold each month in the next 6 months. Example of a prediction which is not a forecast - Country X will win the world cup because they are a good team and in the best form compared to other teams. What do ...

A few people who have attended my presentations this year and became aware of the concept of IBI  (Individual Business Intelligence) and a few people with whom I have been in contact this year have shared their interest to start with IBI starting from 2019 or have already started. It feels nice to get feedback like below.  " I attended your presentation at the publication office. I just read your article on IBI and I love your idea."   -   Message in LinkedIn by a senior professional who attended my presentation of PublicBI BI solution for EU Public Procurement at the Publication Office of the European Union, Luxembourg. This is really great that people have started it or plan to start.  I wish you all the best. For those who are still not sure what IBI is and how to use it, below links should be useful. Basically in this post I am placing all the important IBI links in one post in a sequence so it is easy for people to find and understand.  ...

Using BI team only as a data extract team is similar to using a car headlamp to light a room.  You are in a dark room in the ground floor, somebody starts a car outside the room and the light from headlamp of the car enters your room through the glass windows. You can now see some of the things in your room. You now order (you have authority unfortunately) the driver to keep the car on with headlamp on. Driver tries his best to convince you to please get a bulb soon for your room so that he can take the car to go places, but you don't understand, because you have never seen a car and don't know that it can move.  This is how some of the uninformed business users and uninformed non-BI technical people view BI teams. They think BI team has the expertise in moving data so let us use them for moving data. No, BI teams move data to consolidate, to combine, to integrate data, to harmonize, etc., so that users can get full picture of the business based on the information a...