Apache Spark Big Data Boot Camp

Course Code: 1950

$2695

Overview
Schedule classes
What you'll learn
Outline
Prerequisites
Who should attend

Overview

Apache Spark is a popular toolset for powering Big Data solutions with distributed cluster computing, owing to its speed, expanded versatility and access to powerful APIs and libraries. Spark gives applications the ability to support data science capabilities with R-type dataframes and Big Data streaming helping overcome the time constraints. This fast-paced three day course provides a thorough, hands-on overview of the Apache Spark platform as well as the technologies and paradigms that form a part of Spark. The course will help participants master all the skills necessary to be able to use Apache Spark for their own applications.

Schedule Classes

Looking for more sessions of this class?

Course Delivery

This course is available in the following formats:

Live Classroom
Duration: 5 days

Live Virtual Classroom
Duration: 5 days

What You'll learn

The origin of Apache Spark
Apache Spark vs. Apache Hadoop
Apache Spark use cases
Streaming architecture of Spark
SQL architecture in Spark
Apache Spark and Machine Learning
Machine Learning libraries
Apache Spark GraphX

Outline

Module 1: Introduction to Big Data and Apache Spark

Introduction to data analysis
Introduction to Big Data
Definition of Big Data
Introduce the techniques and challenges in Big Data
Introduce the techniques and challenges in Distributed Computing
Show how the functional programming approach is particularly useful in tackling these challenges
Short overview of previous solutions – Google’s MapReduce and Apache Hadoop
Introduction to Apache Spark
Exercise: Exposure to Admin and setup

Module 2: Deploying and understanding Apache Spark architecture

Spark architecture in a cluster
Spark ecosystem and cluster management
Deploying Spark on a cluster
Deploying Spark on a standalone cluster
Deploying Spark on Mesos cluster
Deploying Spark on YARN cluster
Cloud-based deployment
Exercise: Learn to deploy and begin using Spark

Module 3: Spark Core, RDDs and Spark Shell

Dig deeper into Apache Spark
Introduce Resilient Distributed Datasets (RDD)
Apache Spark installation
Introduce the Spark Shell
Actions and Transformations (Laziness)
Caching
Loading and saving data files from the file system
Exercise: Get hands-on with Spark Code and RDDs

Module 4: Deep dive into RDD

Tailored RDD
Pair RDD
NewHadoop RDD
Aggregations
Partitioning
Broadcast variables
Accumulators
Exercise: You’ll learn expanded RDD capabilities

Module 5: Spark SQL and DataFrames

SparkSQL and DataFrames
DataFrame and SQL API
DataFrame Schema
Datasets and Encoders
Loading and saving data
Aggregations
Joins
Exercise: Learn to use one of Spark’s most powerful features – DataFrames using R-style modelling supported by supercomputer clusters

Module 6: Spark Streaming

A brief introduction to streaming
Spark streaming
Discretized streams
Structured streaming
Stateful/stateless transformations
Checkpointing
Inter-operability with streaming platforms (Apache Kafka)
Exercise: Another of Spark 2.1’s most exciting features is the ability to provide Big Data streaming to allow beating the timeframe constraints of previous Big Data solutions

Module 7: Spark MLib and ML

Introduction to Machine Learning
Spark Machine Learning APIs
Feature extractor and transformation
Classification using logistic regression
Best practices in machine learning for the practitioners
Exercise: Use Spark to perform production-friendly calls for powerful machine learning service and predictive analysis

Module 8: GraphX

Brief introduction to Graph theory
GraphX
Vertex and Edge RDDs
Graph operators
Pregel API
PageRank/ Travelling salesman problem
Exercise: get hands-on practice using GraphX

Module 9: Testing and debugging Spark

Testing in a distributed environment
Testing Spark application
Debugging Spark application
Exercise: Lab practice supporting Spark solutions with best practices for testing, debugging and normal-day production issues for Spark solutions

Prerequisites

There are no mandatory prerequisites for this course, however, completing having a basic understanding of Scala/Python would be beneficial. It is also recommended to complete the Fundamentals of DevOps course prior to taking the Apache Spark Big Data boot camp.