Business Intelligence

The Data Analysis of Data Scientists Survey Results Report has been published on Reportpedia.com.  The report is available on this link -  https://www.reportpedia.com/2018/07/data-analysis-of-data-scientists-survey.html . 

Feel very happy and motivated that the PublicBI team has been shortlisted as one of the finalist in the EU Datathon 2018 competition organized by the Publications Office of the European Union. For more details about EU Datathon see the official website  https://publications.europa.eu/en/web/eudatathon/speakers .   Congratulations to all the finalists and all the best! This now also means that I have to put some of my planned articles (example GDPR) on hold so that I can work on the Datathon topic.

Today I received my certificate for attending the AWS Innovate 2018 Online Conference on 19th July 2018. I had managed to attend all of the sessions from the Big Data and Analytics track and couple of sessions in the AI and Machine Learning track (see agenda below). All of the sessions that I attended were quite interesting and useful. If you have missed don't worry there is still an option to watch the sessions using the on-demand option for free.  See  https://aws.amazon.com/events/aws-innovate/ . If you have attended other tracks and can recommend a must watch session that would be great. I would like to sincerely thank all those who were involved in making this event successful. I can imagine the hard work that would have gone in making this happen based on my experience in organizing the first ever BI Online Conference ( PublicBI BIKON May 2018 ). Looking forward to more such events from AWS team.  AWS Innovate 2018 Agenda Screenshot - Click to enlarge

For all those people who are following my blog, especially on the IBI topic, I am happy to announce that I have now completed 500 days of data capture. For all posts related to IBI see  IBI posts . Just by being disciplined enough to capture the data for 500 days and the fact the I managed to do it and still continue doing it, there is some sense of achievement even though nothing is achieved.  So today I spent some time in data analysis, reflected upon the days that went by using data. Went through the details of the mistakes that I have made, the days I was not happy, etc., to understand if there is a recurring pattern and to tell myself that I shouldn't repeat those. I also made some changes to the data capture template and included new charts too.  I have now added a "Year" column in the data capture template because now I have data for more than a year.  With this addition there is possibility to compare same months across years and also to compare year...

Introduction W ith GDPR coming into action on 25th May 2018 some adjustments are required for most of the systems that run in an organization to ensure technical compliance along with several other organizational measures - see  Measures for GDPR compliance for a company , and obviously it's very much applicable for data intensive systems like BI. In this post I will limit the scope to BI solution. If you are new to BI please take few minutes to glance through this page  Business Intelligence  to get a rough idea about BI.  N ow on one side there is no point in building a GDPR compliant BI solution in such a way that it can't be used for business improvement purposes, or for decision-making at all levels in a company, because BI users will eventually stop using it and the system will become obsolete.  On the other hand there is huge risk (both financial and loss of reputation) in building or maintaining a BI solution that's fully in-use but not compliant wi...

Continued from part 1 , now we go more deeper about handling the requests from data subjects. This is explained using the simple flow chart provided below. Click to enlarge B ased on the above flow chart we can now easily list various processes that needs to be carried out by PDRHS. Note that the processes mentioned here are specific to PDRHS with underlying assumption that all other systems are already GDPR compliant. Facilitate reception of various types of requests from data subjects. Store the request. Classify the request as fake or genuine. Verify the identity of the data subject. Collect additional personal details if necessary to verify identity.  Categorize based on type of request. Check the frequency between the requests. Estimate the charge/fee to be applied for too frequent requests. Set the level based on which too frequent requests may be rejected.  Find and consolidate data about data subject. Collect information about automated decision making...

In one of my previous posts ( Measures for a Company for GDPR compliance ) on GDPR topic I listed the measures that a company has to take to ensure GDPR compliance. One of the measures is to implement PDRHS (Personal Data Request Handling System).  In this post I will go into more details about PDRHS. Again, I will limit the scope to companies (exclude public bodies and others).  PDRHS is an abstract of a system that facilitates data subjects to exercise their rights related to personal data. PDRHS is expected to manage the life cycle of data subjects' requests related to personal data. PDRHS in terms of complexity could be anywhere between very simple to very complex, and in terms of automation could be anywhere between fully manual to fully automated solution depending upon the type and size of the company and number of data subject requests the company receives. Companies like Facebook and LinkedIn already provide means to exercise some of our personal data ...

Below diagram provides typical data subjects in the context of a company. Click to enlarge Users are represented separately from customers to differentiate the group that uses a service, software, etc., but does not pay for it. For example Facebook users, LinkedIn standard account users Duolingo users, etc. We as users don't pay to these companies so we are ideally not customers. For all posts related to GDPR see GDPR For rights related to data subjects see How do I as a data subject benefit from GDPR? Disclaimer : I am not a legal expert nor a certified GDPR consultant (not sure if there is one certification yet). I am a data enthusiast (and now GDPR enthusiast) and I like to envisage, conceptualize and design solutions for real problems. All posts related to GDPR are only to present my understanding and to start a good discussion with the audience. As every business is different please consult legal experts to understand obligations specific to your company. For offi...

If you thought that GDPR is all about providing more rights to data subjects and that it's about making it difficult for companies to run businesses then you have not understood GDPR fully. Whether you consider GDPR as an opportunity or as a threat depends on the level of preparedness and your positioning. From my point of view GDPR creates a level playing field for all and thereby helps smaller companies or new entrants. How can companies benefit from GDPR? Some random and disconnected examples of benefits for companies are provided below Companies that plan to build next Facebook or LinkedIn have better chances of getting existing user data from those platforms into their platforms. How? By ensuring that their platform can consume data from Facebook or LinkedIn easily, and informing people about their right to data portability and giving them incentives to exercise that right.  Companies can now compete in the market not just based on pricing but also based on ...

How do I as a data subject benefit from GDPR? GDPR provides several rights to data subjects Right to personal data protection Right to know or obtain confirmation of personal data processed Right to access personal data processed Right to receive personal data Right to data rectification Right to erasure (Right to be forgotten) Right to object to fully automated decision-making / Right to obtain human intervention Right to object further processing Right to data portability / Right to transmit data from one controller to another Right to lodge a complaint with a supervisory authority Right to effective judicial remedy We now have the rights to know Who / Which companies are processing our personal data? What purposes are they processing it for? To whom / which company is our personal data shared? Are there any automated decision-making involved? Where possible, to know the details of the logic involved in the automated decision-making that concerns us.  How...

What is GDPR? GDPR stands for General Data Protection Regulation ( Regulation (EU) 2016/679 )  that came into force on 24 May 2016 and is applicable from 25 May 2018 onwards. For official documentation please check the official website of EU - https://ec.europa.eu/info/law/law-topic/data-protection/data-protection-eu_en. From my point of view EU (European Union) has taken a visionary, bold and an exemplary step in creating a regulation that is so comprehensive for personal data protection of natural persons and free movement of data within the EU. GDPR provides more rights to all of us natural persons (data subjects). How do I as a data subject benefit from GDPR? See  How do I as a data subject benefit from GDPR? How can companies benefit from GDPR? See How can companies benefit from GDPR For all posts related to GDPR see  GDPR Disclaimer  : I am not a legal expert nor a certified GDPR consultant (not sure if there is one certification yet). I am a data enth...

What measures should a company take to be GDPR compliant? GDPR is not only applicable for companies but also for organizations like public administration. In this diagram given below I present my understanding of what measures a company should take to be GDPR compliant. Click to enlarge I guess all of the measures, except for the Personal Data Request Handling System (PDRHS), mentioned in the above diagram should be easily understood. I will explain PDRHS in my future posts in this blog. I also plan to cover how companies can leverage existing DWBI tools and expertise of DWBI professionals within the company to implement some of the technical measures. GDPR Compliance Bottom-up approach Click to enlarge Related posts : What is GDPR? For all posts related to GDPR see - GDPR Disclaimer: I am not a legal expert nor a certified GDPR consultant (not sure if there is one certification yet). I am a data enthusiast (and now GDPR enthusiast) and I like to envisage, co...

Question from one of my connections in LinkedIn who has over 8 years of experience in BI in various roles. Question I am having a tough time these days in deciding about my future. Want to move ahead into leadership role for a data science team. Want to know if I should do a masters degree course in Data science or an MBA. Which will pave a better path for future 10-15 years of professional career. Can u please recommend something on this.??? My answer My current view based on current market situation is as given below I would definitely go with Master's degree in Data Science among the two options. If there is a possibility to get to leadership roles in data science team without doing both I would also recommend that because you already have good work experience unlike freshers. i.e. instead of going for a full-time master's course, choose to learn data science and implement own projects or contribute to data science teams at work and then eventually ...

To clear confusion, to help everyone, especially aspiring data scientists, PublicBI has come up with a survey called Data Analysis of Data Scientists. Request all data scientists to take around 5 minutes of their time and participate in the survey and provide valuable inputs. The summarized results will be published in reportpedia.com . Results will be free to use for all. If you are a Data Scientist please take part in it. If you are not a Data Scientist please share the survey link with Data Scientists you know. Survey link is here -  Data Analysis of Data Scientists

This was one of the questions by a person via LinkedIn.  He wants to know if it's easy to learn another ETL software once you know how to use one ETL tool? Also if you know one of the Reporting and Analytics tool like SAP BO is it easy to learn another one like MicroStrategy? My answer : In general, yes, it's easier to learn the second tool in the same category. From own experience, it was quite easy to learn Ab initio after having worked with DataStage, and it was again quite easy to learn BODS after Ab initio and DataStage.  After working with these 3 tools it was quite easy to understand what Pentaho Data Integration (PDI) offers when I led a team which worked with PDI. Similarly, it was quite easy to pick up MicroStrategy after working with SAP BO.  And similarly other tools like Tableau, Excel, Google sheets, GDS, etc.

Question : Is it possible to work in banking as a business intelligence analyst and then after some years change to work in the retail industry or pharmaceutical industry? This was one of the questions by a LinkedIn connection.  He wants to know if it's possible to switch domains (for example from banking to retail or pharma) while performing business intelligence analyst role. My answer : In general, yes, it's possible. The skills you need are the same. Your experience will definitely support you. You will have to pick up the domain knowledge quickly. Domain knowledge is the most important aspect in this role from my point of view.     

Happy to announce that, among others, one of the consulting firms in El Salvador got interested in KABI methodology and have started publishing and promoting KABI in Spanish. See  http://blog.zanzivar.com/2018/05/29/buscando-una-metodologia-agil-para-proyectos-de-bi-kabi-una-excelente-opcion/ For more about KABI see -  http://akvkbi.blogspot.com/2016/09/kabi-new-agile-methodology-for-bi.html If you need 2 hours KABI training (online or onsite) or KABI coaching (a few days onsite) please email info@publicbi.com or use the PublicBI contact us page. 

As part of the welcome note during the recently organized PublicBI BIKON event I have clarified what exactly is Business Intelligence and also cleared the confusion around the definition of BI.  Watch this short video to get an understanding.

As part of PublicBI I will be delivering a crash course on BI called PublicBI EBIT (Essentials of Business Intelligence - Training). See agenda slides below. For more details see PublicBI EBIT . 

All of the videos (except for Q&A part) of PublicBI BIKON event are now available on YouTube on PublicBI's YouTube channel. Welcome note and introduction to BI. A revolutionary IoT reputation project becomes economically viable through pattern based Real-Time Big Data Technology Significance of ETL in this era of Data Science, Analytics and Cloud Technologies Data to Insights - A Personal Journey by Rajesh Patel, Active Intelligence Effective Testing Strategies and Tools for DWH/BI Projects by Wayne Yaddow Business Intelligence Roles, Skills and Tools in Demand, and Tips for BI Jobs in Germany Humanizing data science by Jeff Baird Introducing Samni Systems

7 speakers from 7 cities in 5 time zones and 3 continents. 8 talks * 2 rounds. 4 "Ask any BI questions" breaktime sessions. 25+ international cities from where people have registered as participants. Registrations from Melbourne to Los Angeles.  30+ job titles. Including Developers, Consultants, and CXOs.  20 hours for the start of first ever PublicBI BIKON (International Business Intelligence Conference) event.  PublicBI BIKON is BI conference that covers all time zones in a single day.  Think of PublicBI BIKON as the Car Show for Business Intelligence! We have got some test drive offers too! And the most important part is, It's FREE! No travel, and No payment required!  Sounds interesting? Register at https://www.publicbi.com/bikonmay2018 . Participant link will be sent to the registered participants shortly.

Germany, India, Tunisia, UK, Austria, and Singapore are some of the countries from where people have registered for the Individual Business Intelligence free public webinar scheduled for today at 3 PM CEST (Munich Time). Thank you for the registrations. I have sent invites to each one of you. Couple of emails have bounced back because of typos in the email id provided. You may join the webinar on my conference wall at https://www.freeconferencecall.com/wall/anoopkumarvk . I guess this will be my last voluntary public webinar on IBI, so for those who missed the chance to register are also welcome and may join via the above provided link. Looking forward to e-meeting you all.

BI/Data professionals, here is your chance to get some exciting offers from the sponsors of PublicBI BIKON.  Register for the event at  https://www.publicbi.com/bikonmay2018  .

PublicBI UG is organizing PublicBI BIKON, the 1st ever online International Business Intelligence Conference. This is your chance to listen to the BI experts on interesting BI and related topics. Get to know some of the interesting BI players in the market. You also get to ask any BI related questions during the break time to entire audience apart from the session related questions that you may ask the speakers during the sessions. Also your chance to get some of the promotional offers. For example take some of the tools for a test drive for longer duration or get 1000$ worth training from AIS. Open now for registrations. PublicBI BIKON May 3rd event -  https://www.publicbi.com/bikonmay2018 PublicBI BIKON page -  https://www.publicbi.com/bikon

Most of us, especially those in BI and related professions always knew that companies are using our data. And now it's all over the news about how companies, even political parties are misusing our data. What does this mean? It's very evident that your personal data has lot of value and that's why companies and others are trying their best to get their hands on it, sometimes even without your active consent. On one side, they are able to see value in your data. They are using your personal data for their benefit. And on the other side, most of us as individuals are not even able to see the value in our own data, our individual data. We think data is an asset only for companies and not for individuals, and so we don't even attempt to capture our own data, mostly because of lack of awareness about the possibilities and lack of know-how of data capturing and data analysis.  To create awareness about the possibilities, I will be conducting a free public webinar about I...