Objective

In this course, you will learn how big data is driving organisational change and the key challenges organizations face when trying to analyse massive data sets. This course focuses on learning fundamental techniques, such as data mining and stream processing. You will also learn how to design and implement PageRank algorithms using MapReduce, a programming paradigm that allows for massive scalability across hundreds or thousands of servers in a Hadoop cluster. You will learn how big data has improved web search and how online advertising systems work.

By the end of this course, you will have a better understanding of the various applications of big data methods in industry and research.

Eligibility

Candidates interested must have with prior knowledge in any programming language, Data Structures and Algorithms and SQL. This course is more suitable for freshers who seek for a fundamental understanding of Big Data.

Package Requisites

Software- Apache Hadoop, Java Version 1.8

Modules

Module 1: Basics and Characteristics of Big Data and Dimensions of Scalability

1. Understand the four V’s of Big Data (Volume, Velocity, and Variety)
2. Build models for data
3. Understand the occurrence of rare events in random data.

Module 2: Web and social networks

1. Understand characteristics of the web and social networks
2. Model social networks
3. Apply algorithms for community detection in networks.

Module 3: Clustering big data

1. Clustering social networks
2. Apply hierarchical clustering
3. Apply k-means clustering.

Module 4: Google web search

1. Understand the concept of PageRank
2. Implement the basic
3. PageRank algorithm for strongly connected graphs
4. Implement PageRank with taxation for graphs that are not strongly connected.

Module 5: Parallel and distributed computing using MapReduce

1. Understand the architecture for massive distributed and parallel computing
2. Apply MapReduce using Hadoop
3. Compute PageRank using MapReduce.

Module 6: Computing similar documents in big data

1. Measure importance of words in a collection of documents
2. Measure similarity of sets and documents
3. Apply local sensitivity hashing to compute similar documents.
4. Module 7: Products frequently bought together in stores (2 Hours)
5. Understand the importance of frequent item sets
6. Design association rules; Implement the A- Priori algorithm.

Module 8: Movie and music recommendations

1. Understand the differences of recommendation systems
2. Design content-based recommendation systems
3. Design collaborative filtering recommendation systems.

Module 9: Google’s AdWordsTM System

1. Understand the AdWords System
2. Analyse online algorithms in terms of competitive ratio
3. Use online matching to solve the AdWords problem.

Module 10: Mining rapidly arriving data streams

1. Understand types of queries for data streams
2. Analyse sampling methods for data streams
3. Count distinct elements in data streams
4. Filter data streams.

Outcome

• Basic knowledge of Big Data
• Candidates will be able to navigate through Hadoop
• Applying tools like MapReduce on Hadoop