Introduction to Business Intelligence and Data Warehousing


In this post we will briefly discuss about Introduction to Business Intelligence and Data Warehousing concepts and their uses and market trends in IT industry.

Business Intelligence

What is Business Intelligence?

Business Intelligence is the set of processes, technologies, and tools that help an organization to transform raw data into meaningful and useful information for business analysis (Identifying business needs and determining solutions to business problems).

What is the need for Business Intelligence?

Below are some of the major benefits of Business Intelligence in any Organization.

  • Get deeper insights in business operations, Identifying new opportunities and implementing an effective strategy based on insights can provide businesses with a competitive market advantage and long-term stability
  • Business Intelligence Provides historical, current and predictive views of business operations.
  • Sales and marketing – Understanding the profitability of customer segments and answers to valuable questions like,
    • Which customers should an organization target?
    • Which are my most profitable campaigns per region?
    • What is the most profitable source of sales leads and how has that changed over time?
  • Improve Productivity and efficiency
  • Informed decision making
  • Improve Customer Service and satisfaction
  • Streamline budgeting and planning
  • Financial decisions based on results for important questions like
    • What is the full cost of new products?
    • How are forecasts trending against the annual plan?
    • What are the current trends in cash flow, accounts payable and accounts receivable and how do they compare with plan?
  • Overall business performance tracking based on
    • What are the most important risk factors impacting the company’s ability to meet annual profit goals?
    • Should we expand internationally and, if so, which geographic areas should we first target?

Common Functions of Business Intelligence are

  • Reporting
  • OLAP (Online Analytical Processing)
  • Data Mining
  • Process mining
  • Complex event processing
  • Business performance management
  • Text mining, predictive analytics and prescriptive analytics.

Stages of BI

Below are the five stages of Big data Business Intelligence in any organization.

  • Data Sourcing – Defining the data to be loaded into the system. Usually BI applications gathers data from a data warehouse (Data marts, OLTP or OLAP).
  • ETL (Extract Transform Load) – Extracting the source data and transforming per business rules and loading into the Data Warehouses.
  • Data Warehousing – Storing transformed data into various Data warehouses types and making it available for business analysis.
  • Data Analysis – Applying various techniques like data mining, text mining, Process mining to identify trends and patterns in business operations.
  • Decision Making – Based on the reports, dashboards and alerts from previous stage, making valuable business decisions and bench marking future growth.

Data Warehousing

What is Data Warehousing?

The process of extracting and transforming internal and external data into useful business information and loading it into a central database so that it can be explored by business users across the company is known as Data warehousing.

What is a Data Warehouse?

A data warehouse is a relational database that is designed for query and analysis. Enterprise Data warehouses store current and historical data and are used for creating trending reports for business management like annual and quarterly comparison reports.

Business Intelligence and Data warehousing architecture

Below is the typical Business Intelligence and Data warehousing platform architecture.

Data ware house environment

Data Warehouse(DW or DWH) Types

  • Data Marts As shown in the above architecture, a data mart is a simple form of a data warehouse that is focused on a single functional area, like sales, finance or marketing.
  • Online Analytical Processing (OLAP) – OLAP databases store aggregated, historical data in multi-dimensional schemas. OLAP systems typically have data latency of a few hours, as opposed to data marts, where latency is expected to be closer to one day. Mainly used for Reporting and allows complex analytical and ad-hoc queries
  • Online Transaction Processing (OLTP) – OLTP systems support online transactions like INSERT, SELECT, UPDATE, DELETE within fraction of seconds. OLTP is mainly aimed at fast response, simplicity and efficiency but not for reporting purpose.

Below is the high level comparison chart between data warehouse types.

Function OLTP OLAP Data Marts
 Operation  INSERT, UPDATE, SELECT  Complex Queries  Report Generation
 Latency  Within Seconds  Within Hours  Within Days
 Analytical
Requirements
 Low  High  Medium
 Age of Data  Current  Historical, current
and projected
 Historical and
Current
 Business Events  React  Predict  Anticipate

But to handle Big data, the above regular data marts are not capable and Hadoop (HDFS, Hive, HBase) plays the role of OLAP data ware house type in typical Big data Business Analysis using Hadoop. Below is the Hadoop Perspective of the Data warehousing architecture.

Hadoop and Datawarehouse


Profile photo of Siva

About Siva

Senior Hadoop developer with 4 years of experience in designing and architecture solutions for the Big Data domain and has been involved with several complex engagements. Technical strengths include Hadoop, YARN, Mapreduce, Hive, Sqoop, Flume, Pig, HBase, Phoenix, Oozie, Falcon, Kafka, Storm, Spark, MySQL and Java.

Leave a comment

Your email address will not be published. Required fields are marked *


Review Comments
default image

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA Hadoop in Dec/2016 December 22, 2016

.