Business Intelligence

 Completed 6 years of IBI and started 7th year of IBI, hopefully continuing it forever. 

Before data analysis is carried out, it is important to understand the context of the available data. For example, if we analyze hotel reviews data and draw conclusions without understanding the context of the hotel reviews, then we could be drawing totally wrong conclusions.  Let's assume that we have hotel reviews data. Firstly, we should ask several questions about the hotel reviews data before we try to find answers about business from hotel reviews data. Questions such as: Who is the provider of the reviews?  How are the reviews provided? When are the reviews provided?  What are the limitations involved in the review collection process? In how many languages can guests provide reviews in the reviews system? What % of the hotel guests provide reviews? What % of the hotel guests were able to read at least one of the languages in which reviews collection system works? Which channels are there for guests to provide reviews? When did review collection start? What is the s...

It's satisfying to know that the book Business Intelligence Demystified is gradually gaining recognition and more people are purchasing and more importantly reading it and also sharing their feedback.  Happy to let you all know that I have completed reading of the paperback edition of the book. It took me around 12 to 14 hours to read it. It took me more time than I had expected because I was also making a note of all mistakes that I should have avoided in the first place and also a note of all the improvements that can be made in the next edition (if any).  I would like to list all the mistakes and keep it transparent, so in the link below you can check the list of mistakes. If you happen to notice any other mistakes that are not listed, please do let me know. Also, if you would like to share how this book has helped you, please do.  https://docs.google.com/document/d/1Sdc68dgzZxcedEPaaUoeAW5Zi8a215ScXBhunzXgrPs/edit?usp=sharing

OTA Insight, the global leader in cloud-based hospitality business intelligence, has raised $80 million from Spectrum Equity.  Source :  https://www.otainsight.com/resources/news/ota-insight-raises-80m-series-b-funding

  Hatch invests in leading augmented business intelligence firm, Brainnwave  

Amadeus Selected as Business Intelligence Provider for The Leading Hotels of the World. Source: https://www.amadeus-hospitality.com/insight/the-leading-hotels-of-the-world-selects-amadeus-as-its-exclusive-provider-of-business-intelligence/ 

Bullhorn, global leader in software for the staffing industry acquires cube19, business intelligence platform of choice for the global staffing industry.  Source: https://www.cube19.com/bullhorn-acquires-cube19/ https://www.bullhorn.com/news-and-press/press-releases/bullhorn-acquires-cube19-to-deliver-real-time-actionable-business-intelligence-for-staffing-agencies/

Here's an incomplete list of free data visualization tools in no particular order. More tools to be added as and when known. Some of the tools are more than a data visualization tool.  Microsoft Excel Google Sheets MicroStrategy Workstation PowerBI Desktop Tableau Public Google Data Studio

Thanks to everyone who has trusted and ordered the book. Happy (and a bit scared actually) to notice that the book Business Intelligence Demystified is currently in the bestseller No 1 in its category. I hope everyone who has bought it get its full worth. After you have read please share your honest feedback as it will help me to improve, all feedback welcome.

Finally after almost two years of starting the work of writing the book, Business Intelligence Demystified , it's published and available in Amazon and BPB websites. I very much underestimated the work that it involved. I have invested over 800 hours in writing this book, and don't forget the hours and effort spent by all of the reviewers and the publishing team. While I have based this book mostly on own experience of last 15 years in BI projects, where required, I have done quite a bit of research too. Therefore I am sure quite a lot of people will find it useful for starting/restarting their BI journeys irrespective of the current role/profession and also for clarifying all of their doubts about BI.  For whom is this? Simply put, if you want to know BI, you could read this book. If you think you know BI, you should read this book. If you don’t know whether you know BI or not, you must read this book. Irrespective of where you are at your career, or which stage your business...

When we schedule Python jobs through Windows Task Scheduler, there is one tab in which we have to specify the script name which we want to trigger with the location of the script, location of the Python program and arguments. If we don't supply the right values in the right fields, the script will not be triggered. This simple post is limited to provide information about what to provide in each of those fields in that tab (Start a Program).   Once you have opened Windows Task Scheduler and progress through the options in Create Basic Task, the fields that you can see in the "Start a Program" are Program/Script , Add arguments and Start in as shown in the image below.  Program/Script : python Add arguments : provide the path and the python script such as C:\NSE\code\abc.py Start in : Provide the location of the Python folder such as  C:\Users\Win10\AppData\Local\Programs\Python\Python38 as shown in the image below. That's it. Now the python script will be trigg...

Many BI aspirants have these questions "Which BI tool/s should I learn?" , "Learning which BI tools will improve my chances of getting a job sooner?", etc. When such questions are asked in online Q&A forums, most of the answers are biased for one or the other reasons. Therefore, I would like to provide  a way using which you can figure out the answer on your own based on facts.   A random set of of BI tools - not comprehensive Search for current BI job openings either in sites such as LinkedIn, XING, job portals or directly on the website of various companies.  Go through the job specification in detail and exclude those job specs that do not have specific tool requirements. Also exclude those job ads by agencies. Focus on those BI job specs that are directly from the companies that wants to hire for their own needs rather than for their clients.  If you have constraints such as location, industry (domain), or role constraint, then search BI job openings app...

For all those who are looking forward to my book on BI, here's an update. After 14+ months, 700+ hours, lots of sacrifices, a lot of support from my wife, and multiple revisions based on reviews by the publishing house, the book on BI (10 chapters, 270 pages) is now complete and ready for technical review. 

In my view, Big data is a big distraction, especially when full value has not yet been extracted from the set of data that is not big data. Big data requires big investments, big teams, big timelines, but whether big value will be derived from so-called big data is still a question mark. I don't disagree that dealing with so-called big data is a big challenge, it may motivate, make it look fancy and also challenge some technical people, however, question remains, does it bring big return on investments for the business? Happy to be proven wrong. Open to learn. I would like to get some real examples of big data investments - please don't point to abstract articles in some random blogs which doesn't have the details, the details I am looking for are: 1) Why did the project term it as a big data investment, what makes it big data? What exactly makes it different from a BI or Analytics investment?  2) How much was spent? Total cost that includes people or talent cost and all th...

Currently what we see is that tools and technology limitations are used as a basis for classifying data, and even worse is that the classification is in itself incorrect. The so-called big data is so wrongly named. I have already explained why the naming is incorrect in another article - Why big data is actually small?   Wouldn't it be much simpler, better, more meaningful and a standardized approach to classify data into primary and secondary data instead of using misnamed, meaningless and non-standard terms such as small data and big data?  Primary data is the essential or core data without which the business cannot function. For example, a purchase transaction in a retail store has to be captured and stored for billing, payment, compliance,  etc. These are mandatory requirements. Business Intelligence (BI) on top of that data is not mandatory but very useful, but that is not the main purpose of storing that core data in the first place. Secondary data is all of that da...

Rephrased question : ETL developer vs Data engineer Answer : Unfortunately there are no strict industry standards on these job titles. That is just one part of it. Before ETL tools such as DataStage, Informatica, Ab Initio, etc., became popular, developers were hand coding every ETL flow. These ETL tools shortened the ETL flow development time to a great extent and allowed ETL developers to focus on business rule/logic/requirement (what to implement) than how to code it or optimize the code. There are many other benefits of using a tool but I won’t go into that in this answer. So an ETL developer with experience in these tools without any programming (coding) experience was/is able to design and develop end to end data flows. Whenever new types of source/target data format comes up, these tools catch up but it takes time, i.e., the ETL tool provider (e.g. Microsoft, IBM, etc.,) adds new components/connectors within the ETL tool to be able to work with new data format. For example, let’...

Rephrased question : Does data warehouse store future data (meaning forecasts or predictions) ? Answer : Data warehouses stores whatever the company/business has decided to store. There is nothing that stops anyone from storing forecasts. So if forecasts are created and if these needs to be stored then yes it is stored. For example, for a Gas group in the UK we did a project in which we had to store the forecasts for the next 25 years and this was an yearly exercise, which means every year the forecasts for the next 25 years was stored and versioned, so we would be able to fetch the differences between the forecasts too. 

Rephrased question : What is the level of importance of data privacy compared to the level of importance of data governance? Answer : Data privacy is one of the aspects within data governance. In data governance, simply put, on one side there is a need to ensure data is secure, protected and that it doesn’t fall into wrong hands, on the other side there is a need to ensure that value is derived out of data and data is monetized. Data governance should come up with policies, framework, principles, etc., that satisfies/balances both sides.

Rephrased question : Which job would lead to a data engineer job? Is it data analyst or business analyst? Answer : Sorry, there are two things wrong in the question from my point of view There is a wrong assumption in the question that to become a data engineer there is a need to first get into some other job. A business analyst or a data analyst is a job of its own and is not an entry-level job to lead to data engineer job. From a BA the natural progression is that you become a senior BA, and similarly from a data analyst you become a senior data analyst and so on.  Now that there is some clarity, from any of these roles if the person is interested, is ready to learn/unlearn and there is an opportunity in the organization the person can switch between these roles. As data analyst is in general more technical than a business analyst (in the sense of requirements analyst) it would be easier for a data analyst to switch to data engineer role compared to a business analyst. However, t...

Best affordable business intelligence software in 2020? This was one of the questions asked.  Answer: You are actually referring to BI software in the narrow sense, i.e., limited to frontend tools such as Microstrategy, Tableau, Power BI, etc., instead of referring to BI in the correct broader sense as an umbrella term for the end to end process of deriving information and insights from data. In any case the answer depends on the specific requirements and current situation (current set of tools, software, strategy, etc.) of the business. In my view it is not right to state that one tool is best for all purposes and situations. We should select the tools based on the needs and the situations. For example, if a company has invested a lot in Oracle products and if Oracle provides additional software for BI free then why wouldn't you want to consider that as an option? Same with Microsoft, let's say some department already has purchased license for MS SQL, SSIS, SSAS, SSRS, MS Exce...