Business Intelligence

Do you know of a company in this generation, from any part of the world, that has annual revenue of 100+ million US dollars and growing and doesn't use business intelligence / business analytics / data science at all? If yes, please do let me know. I have a few questions that I would like to ask as part of a research. To be very clear, I am looking for an active company that fulfills below criteria : Yearly revenue of 100+ million US dollars Doesn't have BI / Analytics / Data science teams Doesn't have data warehouses, data lakes, data marts, etc Doesn't use any SaaS or other cloud variants of BI / Analytics solutions Doesn't use any app analytics, web analytics, etc, basically doesn't use data for analytical purposes at all. 

Thanks to the Publication Office of the European Union for sharing my Corona Virus (COVID-19) Analytics Dashboard  on their website ( https://op.europa.eu/en/web/eudatathon/covid-19 ). In these difficult times all that some of us can do is to create as much awareness as possible and provide good quality information so that people understand the seriousness of the issue we have.  In this way we try our best to reduce the number of people getting infected and thereby saving a few lives at the least. 

In continuation to my previous post on Analytics using Corona Virus data, I managed to come up with a chart that shows prediction for next 30 days and will be auto updated based on new trends. The predicted numbers look very bad, I wasn't sure if I should share it or not, decided to share it with a thought that it is better to share it, create awareness and prove the prediction wrong than not share it and then later find out that the prediction was right. Now in total there are 5 pages (a pdf version is attached below) in the dashboard. For the interactive dashboard continue using the same link as in the previous post.

This is an interactive dashboard that can be used to find trends, patterns and eventually insights related to COVID-19 (Corona Virus) from a reliable data source. Additional notes provided below the dashboard. Direct link to the Google Data Studio Dashboard is here - https://datastudio.google.com/u/0/reporting/2e4f9914-8905-4cc9-b36a-75dc3f220438/page/7JbY Best viewed on desktop/laptop Notes : This is an interactive dashboard that can be used to find trends, patterns and eventually insights related to COVID-19 (Corona Virus) You can use the filters on the top or click on the charts to filter. Feel free to embed the dashboard in your websites (of course for non commercial use). Currently dashboard doesn’t have real-time information. There is a delay of about 24 hours as the data is mainly sourced from https://github.com/CSSEGISandData/COVID-19/tree/master/csse_covid_19_data ( 2019 Novel Coronavirus COVID-19 (2019-nCoV) Data Repository by Johns Hopkins CSSE). For some o...

I would like to know the answers for these below listed questions. I would be happy to get answers from as many organizations as possible to understand how it works in various organizations. It would be good to share it here in comments so that everyone can benefit. Who ensures that the data scientist has chosen all the required data fields (features) that are required? For example, for a customer churn prediction, let's assume that the data scientist didn't consider one of the data sources (call center data), who is going to ensure that data scientist considers call center data too? If the data (for example in this case call center data) is not available in the data lake/data warehouse or any of the other databases other than OLTP DBs will the data scientist fetch the data from OLTP systems or will they just raise request to the data engineering team to bring the data to data lake/DWH? Who actually tests the code written by a data scientist? Who verifies that...

Question : Which one adds more value and has more pay?  Answer : Firstly, big data is a buzzword. There is data, and there are processes to derive information and insights from data. Business Intelligence is the concept/process of deriving information and insight from data to enable fact-based decision making to improve a business. So if you see it from that context, BI is an umbrella term and therefore all activities that are to do with deriving information and insights from data falls under BI. However, currently because of the hype almost every organization wants to showcase that they are also making use of big data, truth is, some of these companies have not even made use of basics (“easily available data”). I know at least two companies in Europe (Germany) where people were hired as big data developer but there was nothing for them to do there as there was no big data tech stack yet. So the answer to your question depends on which organization and what stage of data j...

Question : As an aspiring BI analyst which tools should I learn? Answer :  Please note that there are differences between a BI Analyst , BI BA , Data Analyst and Data Scientist. I will keep that for another post if you are interested. In general for a BI Analyst it is not mandatory to have programming skills or SQL skills. So based on this, here is the list as of today. SQL (must have) Python (good to have) Experience with at least one of the below listed tools is a "must have". Good to have experience in multiple tools. Power BI Tableau Microstrategy Desktop and Enterprise MS Excel (advanced) Google Sheets Google Data Studio Qlik Sense IBM Cognos Analytics Apart from these it is also important to have good communication and presentation skills. 

Question from a BI aspirant " Can I switch from Finance domain to BI? I have been working in Finance for 3 years, do I have to do masters in Business Analytics or Business Intelligence to get into BI? " Answer : Doing a masters in Business Analytics or Business Intelligence is one of the ways to get into BI and therefore is not mandatory. Please note that BI has various roles, for example ; BI Developer (Frontend/Reporting and Analytics platform, Backend/ETL developer), BI Analyst, BI BA, etc. So the path you take depends on the role you are interested in.

This week (on 19th Feb 2020) I completed 3 years of IBI. To celebrate it I have created a Dashboard in Google Data Studio. So from now on instead of using the charts in the Google Sheets I will use the Dashboard. Google Data Studio is more user friendly and mobile friendly too, also it comes with easy to use filters and drill down/up capabilities. PDF version of the first 2 pages of the shareable dashboard is attached below. I am happy to share the first 2 pages of the IBI Dashboard, reach out to me if you would like to get the Google Data Studio IBI dashboard. Once you get the dashboard (without my data of course) you can make a copy of it and point it to your IBI data (assuming you have started), add more charts, pages, etc., and use it for your benefit. Like with every year, this IBI year too I have learnt a lot about myself, made new mistakes, changed several things about me. The benefits of IBI is much more than we can think of. Use it to know it. And what are some of the...

I am curious to know, approximately how many copies of a payment transaction record are stored across all systems across all parties involved in the payments sector?  Based on a simple and rough calculation I think it is a minimum of 60 copies of each payment transaction. Of course every party doesn't have access to every field (column or data point) and has access only to that part of the record which it is authorized to have access.  So, it looks like we (all parties combined) use at least 60 times more storage than it is required to store a payment transaction record. I'm open to learn from others if I have missed some parties/systems/data layers. And is there a possibility to centralize storage across all parties to store it once and to provide access to parts of the records to parties/systems instead of duplication? May be a point to think. 

Very happy to know that at least 3 people, whom I know (and have informed me), have started with IBI  (Individual Business Intelligence) from 1st Jan 2020. One is a close friend and ex-Wirecard who has started with 20+ data points, another is a colleague who would like to capture data and do predictive analytics using his data and check if his predictions comes true, and the 3rd one is my 6 year old son with his mama's help who has started capturing various data points including how many times he cried in a day, how many times he fought, how much time he spent learning something, etc. All the best wishes to 3 of them. I am sure more people will realize the benefits of IBI.  To create awareness about IBI I plan to introduce IBI in different languages that I speak. I have started the series with one introductory video in Kannada language. Please find it below, and please share it with friends and family who speak Kannada. Goal is to create as much awareness, including those,...

Hi everyone, The latest Excel template for IBI (Individual BI) is published.  Again, as usual this is a generic template which if you like to use you should customize to suit your needs.  The generic template is here - https://drive.google.com/open?id=1z2_iEDLkkBqvBHhf-X0qwA0m_niZezAY it has dates up to year 2030. It can be easily extended beyond 2030. To download, once you click on the above link, please click on the top right corner download button. To give you an idea of how to customize and use please see below a customized template that I created for my 11-year old nephew. https://drive.google.com/open?id=1cmTGNU3xlhvKQjna3KR5Vg784OG4ydp7   . And here you can find the customized template that I created for my 6-year old son https://drive.google.com/open?id=1EpxjXFy-k4uD6PxLwvUopQgNc_nPDkbz  . All the best for those who are starting their IBI journey from 1st Jan 2020. IBI - Discover yourself factually. Update:  How to easily capture t...

Are you one among those who feels that everyone is talking about machine learning but no one seems to provide a simple example to easily understand it, if yes, this short video should help. And yes, you don't have to be a data scientist to use machine learning.

Latest 2 minutes video on IBI provided below

For those who have been following my IBI (Individual Business Intelligence or Individual BI) journey I am very happy to announce that I have completed more than 1000 days of data capture. On 15th November, 2019 I completed 1000 days. I had started capturing data from 19th Feb 2017. Based on the analysis of own data I have found some interesting patterns, learnt a lot about myself, stopped myself from doing the same mistakes again, noticed changes in patterns, etc. The benefits that I get from Individual BI are way too many. And therefore I will definitely continue IBI as long as I am able to do it. I will be happy to conduct a session based on demand. Let me know. Some of the charts are provided below to give you an idea of the possibilities with own data captured in a structured way consciously. Gaps in the above chart are because there was no weighing scale on those days or weight was not measured on those days. 

Presenting at EU DataViz 2019, Luxembourg on 12th November was such an honour. Thanks to the Publication Office of the European Union for the opportunity. It was a great experience both in listening to other inspiring speakers and also to present the topic of "Before and beyond data visualization". In my talk, I used two examples; Individual Business Intelligence (Individual BI or IBI) and PublicBI EUProc solutions to talk about before and beyond data visualization sections respectively. It was interesting to note that there were many people who were interested in the IBI example than I had expected, and some also had questions about data security. If any organizations (companies, colleges, etc,) would like me to present the topic of Individual BI at their premises I will be more than happy to do that.  My slides are attached below, if there are problems with the pages below, use this link - https://drive.google.com/file/d/1vll-ugWPh6MoCwxqDj3Eo1d6BkcR...

On 15th of November, 2019 I completed 1000 days of Individual Business Intelligence (Individual BI or IBI).  Everybody says data is an asset. I say, the data I have about myself is priceless. But, I was just wondering  if I were to estimate the price of the data I have collected over last 1000+ days, how much would it be worth? I have 104 data points (ideas, mistakes, dreams, health, work, happy days, etc.) captured for over 1000 days. Let's consider 1 EUR per data point.  So 104000 EUR Time spent in filling IBI data capture sheet everyday until now (1014 days) = 110.8 hours So around 11000 EUR (assuming 100 EUR per hour) Time spent in analyzing data until now (1014) = 76.5 hours So around 7600 EUR  So in total it is minimum worth 104K+ 11K+ 7.6K = 122600 EUR Would I sell my data? I don't think so. Would I share my data for a good cause? May be some of it.  I think if data is an asset, then we sh...

We often find ourselves in situations where the BI teams, because of the tools we use and skills we poses in dealing with data, are misused by other departments to carry out non-BI work. I would like to know how companies deal with these situations? Examples of non-BI work 1) Migrate data from old OLTP application to a new OLTP system 2) Create data extracts to be provided to other systems, example billing system or to be provided to B2B customers Because of the tight deadlines on the non-BI work these tasks get pushed on to the BI teams and the BI work is stalled. The non-BI task is also important but for the BI team it is a misfit.

A number of interesting topics have been shortlisted to be presented at the EU DataViz 2019, Luxembourg. I have the honour to be one of the speakers. I will be speaking on the topic "Before and Beyond Data Visualization" Those interested please go ahead and register at https://publications.europa.eu/en/web/eudataviz/home

Happy to announce that I have been selected as one of the speakers in EU DataViz 2019.  Check the official page here - https://publications.europa.eu/en/web/eudataviz/home EU DataViz 2019 is an international conference organised by the Publications Office of the European Union. It addresses for the first time the specific needs of the community engaged in data visualisation for the public sector in Europe, bringing together experts, practitioners and solution seekers.

Are you tired of working as an external and researching for reasons to switch to an in-house BI team? Or are you in a situation where you have to choose between an IT consulting/services company position and an in-house BI team position for the next step in your career? Or just curious to know how it is to work in an in-house BI team? Irrespective of the reasons, I hope that the content in this part 1 of the in-house BI teams multi-part article provides you a well-rounded perspective about the topic and probably even triggers your interest to be a part of it. As there are various definitions for BI and confusions around it, it’s important that we clearly state what we mean by BI to ensure we are all on the same page before we start comparing how it is to work in an in-house BI team vs as an external. Business Intelligence In simple words BI (Business Intelligence) is all about deriving information and insight from data efficiently at scale to enable fact-based decision making in...

It's been over 2 years that I have started IBI . A small subset of the results is shared below (charts) with an intention to create awareness about IBI. One of my personal objectives for this year is to create awareness about IBI among high school and university students.  So if any of you have connections with academic world and would like me to present the IBI concept, tools, my experience and learning from it, I would be glad to present. Almost everyone today, especially data professionals, preach that organizations should be data-driven, however, how many of the data professionals are themselves  data-driven individuals ? How many of us put in any effort in capturing and accumulating our own data in a way it can be used for self-discovery? Shouldn't we practice what we preach? We all agree that if we don't measure something, it can't be improved. Why don't we measure ourselves? Don't we want to improve? On 19th Feb 2019 I completed 2 years of capturi...

What exactly does data, information and insight mean in the context of BI? Data : The raw form from which information and insight can be derived. Usually any recorded values, numbers, text, audio, video, stored in any form, any size and any location that gets generated in any event or transaction or just based on the current state or status.  It could be internal data like employee related data (name, address, phone number, gender, etc.) or products, services related data or customer related data or server logs, web clicks, call center data , product reviews, ratings, etc. Anything and everything that can be used to derive information is data. The much hyped big data is also data. It could be stored as files or in a database or just as logs. Burger Chain Example   For example, the event of a customer buying a burger from a fast food restaurant generates lot of data. Time of purchase, terminal used for payment, employee who served the customer, amount and curre...