Mô tả

Big Data Developers are in high demand and it's only going to increase as data grows. However, mastering the skills needed to become a Big Data Engineer can be overwhelming. That's why we created "The Big Data Developer Course" with the help of industry experts. Our course provides an end-to-end implementation of the most in-demand Big Data skills, including Hadoop, Spark, Kafka, Cassandra, and more. With 33 hours of hands-on training, you'll start with the basics and work your way up to production-level deployment, troubleshooting, and performance improvement. We cover everything from local development to integrating with complex data sources, such as NOSQL databases, and even streaming data. Our team is available to address any questions you have, and our video tutorials are all explained with examples. By the end of this course, you'll be a Big Data expert, ready to take on any job in the industry. Don't miss this opportunity to join the world of Big Data!

Here is a short description of what you will be learning in this course:
Understand the world of Big Data. What is Big data and why it is important
Understand and learn the concepts behind Hadoop. Understand its architecture
Install the software and start writing code
Learn important Hadoop Commands
Learn the file formats and understand when to use each of the file formats
Dive deep into Sqoop- a tool used for transferring data between RDBMS and HDFS
Dive deep into Hive- a tool used for querying the data on HDFS
Learn Scala -  a top programming language
Dive deep into Spark which is very hot in the market
Learn NOSQL Databases - Cassandra and HBase and integrate them with Spark
Work with Complex data and process them effectively
Make your code production ready and deploy them onto the cluster
Learn Apache NIFI- a powerful and scalable open source tool for data routing
Work with Streaming data
Learn Kafka and integrate it with Spark
Learn troubleshooting techniques and performance improvement tips

This is complete end-to-end implementation course and we are very proud to bring this course to you. 

Enroll now and join the world of Big Data !


Update:
We have added interview Preparation videos for Hadoop, Sqoop , Hive, Scala

Bạn sẽ học được gì

Understand the architecture of Hadoop

Understand file formats and the ability to choose the right format for a given use case

Develop applications on local system and then deploy them into production

Parameterize the code and make it production ready

Import data from mysql database into sqoop. Export data from hdfs to mysql. Get a deep understanding of sqoop

Query and analyze the data effectively using Hive. Get a strong understanding of hive

Learn Scala - one of the top programming languages

Learn basic, intermediate and Advance concepts of Spark which is very hot in the market

Work with complex data and learn how to process them effectively

Learn Cassandra and integrate it with Spark

Learn HBase and integrate it with Spark

Learn Apache NIFI

Work with Spark Streaming - Learn about Kafka and how it integrates with Spark

Get a good understanding of end to end big data pipeline

Yêu cầu

  • You should have good internet connectivity. Should have 6 GB of free RAM. This course will work with 4GB of free RAM but the applications may run slow. So recommend to have atleast 6GB of Free RAM. SSD Hard disk will increase the speed. If possible(not mandatory) have SSD hard disk instead of HDD
  • A basic familiarity with the Linux commands will be helpful

Nội dung khoá học

17 sections

Introduction to the course

3 lectures
What is this course about
04:54
How to make best use of this course
05:55
PPT used in this course
00:06

Introduction to the Big Data World

7 lectures
Introduction to Hadoop
12:54
How MapReduce works
08:48
What is Big Data
06:22
[Notes] What is Big Data
00:19
Hadoop 1.0 Architecture
22:15
Hadoop 2.0 Architecture
15:24
Hadoop 3.0 Architecture
09:01

Setting up Cluster and doing hands on with Hadoop

8 lectures
Cloudera Software Installation
31:30
[Notes] Cloudera Software Installation
00:35
Hadoop Commands
25:31
[Notes] Hadoop Commands
01:43
Row Storage vs Column Storage
11:06
Serialized File Formats
16:06
[Notes] Serialized File Formats
02:30
Hadoop and Big Data Interview questions and Answers
00:15

Sqoop

26 lectures
Sqoop Introduction
07:43
Sqoop Import
09:51
[Notes] sqoop import
00:14
Sqoop Multiple Mappers
09:06
[Notes] Sqoop Multiple Mappers
00:38
import portion of data
15:51
[Notes] import portion of data
00:37
Sqoop eval and change the file delimiter
05:19
[Notes] Sqoop eval and change the file delimiter
00:36
incremental import
18:11
[Notes] incremental import
01:24
Password Protection
10:21
[Notes] Password Protection
00:53
Using Last Modified
12:32
[Notes] Using Last Modified
00:53
Import multiple File Formats
12:55
[Notes] Import multiple File Formats
01:10
Import multiple Tables
07:29
[Notes] Import multiple Tables
00:26
Handling Null during Import
05:24
[Notes] Handling Null during Import
00:31
Sqoop export
06:14
[Notes] Sqoop export
00:30
Sqoop Performance Tuning
06:31
[Notes] Sqoop Performance Tuning
00:38
Sqoop Interview Preparation
00:14

Hive

30 lectures
Hive-Data Preparation
20:58
[Notes] Hive-Data Preparation
00:38
What is Hive
07:15
[Notes] What is Hive
00:50
Create and load a table in Hive
33:44
[Notes] Create and load a table in Hive
02:14
Hive Table Types
04:50
[Notes] Hive Table Types
00:24
Hive Partitions
49:08
[Notes] Hive Partitions
03:48
Hive Use Case
05:15
[Notes] Hive Use Case
00:25
Hive Buckets
15:43
[Notes] Hive Buckets
01:00
Schema Evolution in Hive
27:38
[Notes] Schema Evolution in Hive
02:30
Execute hive queries using a script
05:23
[Notes] Execute hive queries using a script
00:31
Working with Dates in Hive
05:15
[Notes] Working with Dates in Hive
00:16
Joins in Hive
23:43
[Notes] Joins in Hive
01:55
MSCK Repair
06:03
[Notes] MSCK Repair
00:52
Performance Tuning in Hive
04:36
[Notes] Performance Tuning in Hive
00:29
Hive vs SQL
01:35
[Notes] Hive vs SQL
00:11
Hive Additional Resources
04:58
Hive Interview Preparation
00:14

Installation for Spark and Scala

2 lectures
Installing and setting up Spark and Scala
12:44
[Notes] Installing and setting up Spark and Scala - Download links
00:27

Let's learn Scala

32 lectures
Introduction to Scala
02:44
Executing our First Scala Program
13:24
Scala Basics
29:14
Conditional Statements
23:14
Loops in Scala
22:48
Functions in Scala
19:05
Scala Class
12:24
Constructors in Scala
02:04
Scala Inheritance Introduction
02:35
Single Inheritance
08:28
Multilevel Inheritance
05:18
Hierarchical Inheritance
05:15
Scala Traits - for Mutliple Inheritance
05:57
Hybrid Inheritance
02:59
Method overriding and Method Overloading
12:13
Singleton and Companion Object
04:53
Case Class
04:16
Abstraction and Final
09:10
Higher Order Functions and Lambda Expressions
11:08
What is Partially Applied Function
06:57
What is Currying
03:11
What is Option Type
10:56
Pattern Matching in Scala
12:46
Exception Handling in Scala
15:36
Scala Collections
44:26
[Notes] Scala Collections
01:30
Collection Methods
36:47
[Notes] Collection Methods
00:59
Group By vs Grouped
06:30
Variable Arguments - What is it and how is it useful ?
05:36
Working with Files
17:13
Scala Interview Questions and Answers
00:04

Introduction to Spark

2 lectures
What is Spark
06:58
Why is Spark Faster than MapReduce
11:35

Spark RDDs

10 lectures
RDD Basics - Reading and Writing a File
28:59
[Notes] RDD Basics - Reading and Writing a File
00:06
Deploying code to Cluster
14:21
Use Case - Analyze the Log Data
16:03
[Notes] Use Case - Analyze the Log Data
00:04
Common RDD Transformations and Actions
26:11
What is Pair RDD
20:22
Use Case - The word count example
05:36
Using Schema RDD
13:14
Using Row RDD
04:28

Spark DataFrames

12 lectures
What is Spark DataFrame
02:29
Creating DataFrames from RDD
33:25
Spark Seamless Dataframe- Reading and Writing
30:50
[Notes] Spark Seamless Dataframe- Reading and Writing
00:08
Reading and Writing AVRO Data
16:11
Reading and Writing XML Data
14:31
[Notes] Reading and Writing XML Data
00:18
Reading Multi Lines Json
10:18
[Notes] Reading Multi Lines Json
00:09
Write Modes in Spark
08:00
Passing schema to a file
14:35
Applying Transformations using tempView and DSL
17:40

Spark Advance

16 lectures
Let's explore more transformations
46:32
[Notes] Let's explore more transformations
00:26
How to remove duplicates
12:35
Sorting the Data
16:34
Handling Nulls in Spark
17:46
Working with String Functions
20:15
Working with Dates
18:18
Applying aggregation
09:33
Spark Windowing Functions
25:03
Pivoting in Spark
06:27
Passing List of columns to Dataframe
06:45
Joins in Spark
29:04
Use case - Bank Transaction Data
29:14
Reading Current day's file
09:07
Working with Fixed Width File
30:26
[Notes] Working with Fixed Width File
00:37

Productionalizing your Code

6 lectures
How does your code run in Production
04:53
Deploy in prod - parameterizing the filenames
35:06
Parameterize using Config File
51:29
[Notes] Parameterize using Config File
00:59
Spark Hive Integration
06:39
Memory Tuning in Spark
07:20

Complex Data Processing

10 lectures
Working with Json
16:15
Working with MultiLine Json
05:19
Working with Nested Json
10:17
Working with Nested Json - Struct and Array
20:29
Reading Json from a web URL and flattening it
10:18
[Notes] Reading Json from a web URL and flattening it
00:39
Flattening data by creating a Function
27:21
[Notes] Flattening data by creating a Function
00:43
Complex Data Generation
09:15
Flatten XML File
13:43

NOSQL Databases

11 lectures
What is a NOSQL Database
09:12
Working with HBase
34:01
[Notes] Working with HBase
00:07
Spark HBase Integration
37:52
[Notes] Spark HBase Integration
01:07
Cassandra Introduction
05:03
Cassandra setup and working with Cassandra
20:59
[Notes] Cassandra setup and working with Cassandra
00:17
Cassandra Spark Integration
13:46
[Notes] Cassandra Spark Integration
00:47
Cassandra Query Limitations
03:21

Apache NIFI

7 lectures
Apache NIFI Introduction
06:18
Apache NIFI Installation
07:20
[Notes] Apache NIFI Installation
00:21
Let's work with Apache NIFI Tool
14:19
Streaming from a web URL
10:27
Back Pressure and Connection Queue
06:02
Promoting to Another Environment
06:00

Working with Streaming Data

14 lectures
Spark DStream
17:01
Kafka Introduction
07:44
kafka installation and topic creation
18:37
[Notes] kafka installation and topic creation
00:15
Kafka NIFI Integration
06:17
How Kafka Works
11:49
Spark Kafka Integration
17:48
[Notes] Spark Kafka Integration
00:34
Using Kafka Offset Explorer tool
14:22
Kafka Delivery Guarantee Feature
05:02
Spark Structured Streaming
13:29
[Notes] Spark Structured Streaming
00:20
Spark Kafka Integration
11:37
[Notes] Spark Kafka Integration
00:23

Extra

1 lectures
Performance Tuning Tips
05:34

Đánh giá của học viên

Chưa có đánh giá
Course Rating
5
0%
4
0%
3
0%
2
0%
1
0%

Bình luận khách hàng

Viết Bình Luận

Bạn đánh giá khoá học này thế nào?

image

Đăng ký get khoá học Udemy - Unica - Gitiho giá chỉ 50k!

Get khoá học giá rẻ ngay trước khi bị fix.