Don't use front-end where it is not required

Regular manual download of data from a portal (frontend) and loading it into a backend system is like exiting an airport and then reentering the same airport to catch a connecting flight from the same terminal when you have no business outside the airport. 

I have seen people doing this, downloading data from portals and then loading into other systems manually. If there is no one looking at the data in the UI and no decision is to be made and only regular data feeds are required then don't do it via frontend (GUI), just get a data extract job created that will automatically load the backend system. 

This is partly related to one of the previous posts - BI can take you to places

Comments

Post a Comment

Thanks for your comment. It will be posted after checks.

Popular posts from this blog

ETL developer vs Data engineer

Image
Rephrased question : ETL developer vs Data engineer Answer : Unfortunately there are no strict industry standards on these job titles. That is just one part of it. Before ETL tools such as DataStage, Informatica, Ab Initio, etc., became popular, developers were hand coding every ETL flow. These ETL tools shortened the ETL flow development time to a great extent and allowed ETL developers to focus on business rule/logic/requirement (what to implement) than how to code it or optimize the code. There are many other benefits of using a tool but I won’t go into that in this answer. So an ETL developer with experience in these tools without any programming (coding) experience was/is able to design and develop end to end data flows. Whenever new types of source/target data format comes up, these tools catch up but it takes time, i.e., the ETL tool provider (e.g. Microsoft, IBM, etc.,) adds new components/connectors within the ETL tool to be able to work with new data format. For example, let’...
Read more

KABI - The new Agile Methodology for BI Projects - Implement BI projects quicker happily

Image
KABI is a new agile software development methodology useful for achieving quicker implementation of Business Intelligence (BI) solutions. The methods defined in KABI can also be used in full or in part for any other non BI projects that share similar characteristics as BI development projects.  The word KABI is created by combining "KA" from KA nban and "BI" from B usiness I ntelligence. KABI is a lightweight, iterative, continuous feedback based agile software development methodology that enables every team member to work to their full potential even when there are several unknowns, resource constraints, dependencies and unpredictability of work load. Thereby KABI ensures optimal team productivity throughout the project duration. At the heart of KABI is " Peer Inspiration ".  Every team member plays an important role and every team member works in an exemplary way. They do their part of the job so well that it inspires the whole team. " Mutual...
Read more

Business Intelligence Demystified published and is now available

Image
Finally after almost two years of starting the work of writing the book, Business Intelligence Demystified , it's published and available in Amazon and BPB websites. I very much underestimated the work that it involved. I have invested over 800 hours in writing this book, and don't forget the hours and effort spent by all of the reviewers and the publishing team. While I have based this book mostly on own experience of last 15 years in BI projects, where required, I have done quite a bit of research too. Therefore I am sure quite a lot of people will find it useful for starting/restarting their BI journeys irrespective of the current role/profession and also for clarifying all of their doubts about BI.  For whom is this? Simply put, if you want to know BI, you could read this book. If you think you know BI, you should read this book. If you don’t know whether you know BI or not, you must read this book. Irrespective of where you are at your career, or which stage your business...
Read more