GDPR Compliant Business Intelligence Solution - Part 1

Introduction

With GDPR coming into action on 25th May 2018 some adjustments are required for most of the systems that run in an organization to ensure technical compliance along with several other organizational measures - see Measures for GDPR compliance for a company, and obviously it's very much applicable for data intensive systems like BI. In this post I will limit the scope to BI solution. If you are new to BI please take few minutes to glance through this page Business Intelligence to get a rough idea about BI. 
Now on one side there is no point in building a GDPR compliant BI solution in such a way that it can't be used for business improvement purposes, or for decision-making at all levels in a company, because BI users will eventually stop using it and the system will become obsolete.  On the other hand there is huge risk (both financial and loss of reputation) in building or maintaining a BI solution that's fully in-use but not compliant with GDPR. So the goal now is to ensure that the useful BI solution is also GDPR compliant.

What is GDPR see What is GDPR? post. For all posts related to GDPR see GDPR.

Business Intelligence
In the past BI solutions were mostly used by large companies, but now most of the companies use BI solutions in one form or other. BI is no longer a "good to have" but a "must have". BI teams have become almost like any other key functional teams such as Sales or HR or Marketing without which a company can no longer scale up. So it's a critical requirement that we ensure BI solution is GDPR compliant. First let's look at a typical BI solution architecture, see the below figure. Note that there are many other possible variations.

Typical BI Solution Architecture
Click to enlarge : Typical BI Solution Architecture

Simply put BI solution enables business users to derive information and insights efficiently from all sorts of data. This is usually made possible by storing data in a specialized way in a data warehouse (DWH). In simple words DWH is where the data from various data sources are integrated, historized, and stored in a clean and consistent manner such that it's better suited for reporting and analysis purposes. The data stored in the DWH could be about various categories of data subjects. To see different categories of data subjects in the context of a company check this post - data subjects. To keep it simple for explanation purpose let's consider that the personal data stored in the DWH is of the B2C customers from the above mentioned post. For ease of understanding a much more simplified version of BI solution architecture is given below

Simplified BI Solution Architecture
Click to enlarge : Simplified BI Solution Architecture

As it can be seen from the figure, technically BI broadly serves two purposes - 1) Information needs of all types of business users and 2) High quality integrated data requirements of downstream systems.

GDPR Compliant BI Solution

So, how do we ensure that the BI solution is GDPR compliant?

While the transaction systems store the data necessary to perform the day to day operations of the business, data is stored in the the DWH mainly with an intention to know more about the performance of the business, to measure the business, to know the current status and to know how the business is performing over a period of time, which product or service is doing good, which needs to be improved,  to predict how the business will perform and to prescribe what changes needs to be done to meet goals of the business. This is where BI systems are different from all other operational/transaction systems.

To explain this for those who are new to BI, let's take an example of an imaginary EduTech company called Languages4All (L4A) that is into language education. L4A provides language education through various channels like multiple mobile apps,  multiple websites and offline both at own training centers and other locations  across the world. In this case the B2C customers of L4A are the language students. So the company processes personal data of the students. And obviously the company also processes personal data of employees (both trainers and non trainers), partners, vendors, etc. The transaction systems in this case for example store personal data to enroll a student,  to track progress of the course, to provide necessary support during the training and to provide certificate at the end of the course. Whereas DWH data is primarily meant for analyzing performance and to answer questions such as how many students enrolled, which location, split by age, gender, income, education, mode of delivery, type of device used, number of calls to call center, etc., basically to get good insights about the business and to get a 360 degree view of the customer. 

Now that you have an understanding of a BI solution we will explore different options to ensure GDPR compliance of BI solution in my next post -  GDPR Compliant Business Intelligence Solution - Part 2 (to be posted in the following weeks).

Disclaimer : I am not a legal expert nor a certified GDPR consultant (not sure if there is one certification yet). I am a data enthusiast (and now GDPR enthusiast) and I like to envisage, conceptualize and design solutions for real problems. All posts related to GDPR are only to present my understanding and to start a good discussion with the audience. As every business is different please consult legal experts to understand obligations specific to your company. For official documentation check the official website - https://ec.europa.eu/info/law/law-topic/data-protection/data-protection-eu_en.

Comments

  1. GDPR course Thank you because you have been willing to share information with us. we will always appreciate all you have done here because I know you are very concerned with our.

    ReplyDelete
  2. GDPR awareness course Hello I am so delighted I located your blog, I really located you by mistake, while I was watching on google for something else, Anyways I am here now and could just like to say thank for a tremendous post and a all round entertaining website. Please do keep up the great work.

    ReplyDelete
  3. Great blog. All posts have something to learn. Your work is very good and I appreciate you and hopping for some more informative posts. Analítica empresa

    ReplyDelete

Post a Comment

Thanks for your comment. It will be posted after checks.

Popular posts from this blog

ETL developer vs Data engineer

3 years of IBI