Big Data Hadoop Architect Master Program

·       Introducing Scala

·       Deployment of Scala for Big Data applications and Apache Spark analytics

·       Scala REPL, lazy values, and control structures in Scala

·       Directed Acyclic Graph (DAG)

·       First Spark application using SBT/Eclipse

·       Spark Web UI

·       Spark in the Hadoop ecosystem.

·       The importance of Scala

·       The concept of REPL (Read Evaluate Print Loop)

·       Deep dive into Scala pattern matching

·       Type interface, higher-order function, currying, traits, application space and Scala for data analysis

·       Learning about the Scala Interpreter

·       Static object timer in Scala and testing string equality in Scala

·       Implicit classes in Scala

·       The concept of currying in Scala

·       Various classes in Scala

·       Learning about the Classes concept

·       Understanding the constructor overloading

·       Various abstract classes

·       The hierarchy types in Scala

·       The concept of object equality

·       The val and var methods in Scala

·       Understanding sealed traits, wild, constructor, tuple, variable pattern, and constant pattern

·       Understanding traits in Scala

·       The advantages of traits

·       Linearization of traits

·       The Java equivalent

·       Avoiding of boilerplate code

·       Implementation of traits in Scala and Java

·       Handling of multiple traits extending

·       Introduction to Scala collections

·       Classification of collections

·       The difference between iterator and iterable in Scala

·       Example of list sequence in Scala

·       The two types of collections in Scala

·       Mutable and immutable collections

·       Understanding lists and arrays in Scala

·       The list buffer and array buffer

·       Queue in Scala

·       Double-ended queue Deque, Stacks, Sets, Maps, and Tuples in Scala

·       Introduction to Scala packages and imports

·       The selective imports

·       The Scala test classes

·       Introduction to JUnit test class

·       JUnit interface via JUnit  suite for Scala test

·       Packaging of Scala applications in the directory structure

·       Examples of Spark Split and Spark Scala

·       Introduction to Spark

·       Spark overcomes the drawbacks of working on MapReduce

·       Understanding in-memory MapReduce

·       Interactive operations on MapReduce

·       Spark stack, fine vs coarse-grained update, Spark stack, Spark Hadoop YARN, HDFS Revision, and YARN Revision

·       The overview of Spark and how it is better than Hadoop

·       Deploying Spark without Hadoop

·       Spark history server and Cloudera distribution

·       Spark installation guide

·       Spark configuration

·       Memory management

·       Executor memory vs driver memory

·       Working with Spark Shell

·       The concept of resilient distributed datasets (RDD)

·       Learning to do functional programming in Spark

·       The architecture of Spark

·       Spark RDD

·       Creating RDDs

·       RDD partitioning

·       Operations and transformation in RDD

·       Deep dive into Spark RDDs

·       The RDD general operations

·       Read-only partitioned collection of records

·       Using the concept of RDD for faster and efficient data processing

·       RDD action for the collect, count, collects map, save-as-text-files, and pair RDD functions

·       Understanding the concept of key-value pair in RDDs

·       Learning how Spark makes MapReduce operations faster

·       Various operations of RDD

·       MapReduce interactive operations

·       Fine and coarse-grained update

·       Spark stack

·       Comparing the Spark applications with Spark Shell

·       Creating a Spark application using Scala or Java

·       Deploying a Spark application

·       Scala built application

·       Creation of the mutable list, set and set operations, list, tuple, and concatenating list

·       Creating an application using SBT

·       Deploying an application using Maven

·       The web user interface of Spark application

·       A real-world example of Spark

·       Configuring of Spark

·       Learning about Spark parallel processing

·       Deploying on a cluster

·       Introduction to Spark partitions

·       File-based partitioning of RDDs

·       Understanding of HDFS and data locality

·       Mastering the technique of parallel operations

·       Comparing repartition and coalesce

·       RDD actions

·       The execution flow in Spark

·       Understanding the RDD persistence overview

·       Spark execution flow, and Spark terminology

·       Distribution shared memory vs RDD

·       RDD limitations

·       Spark shell arguments

·       Distributed persistence

·       RDD lineage

·       Key-value pair for sorting implicit conversions like CountByKey, ReduceByKey, SortByKey, and AggregateByKey

·       Introduction to Machine Learning

·       Types of Machine Learning

·       Introduction to MLlib

·       Various ML algorithms supported by MLlib

·       Linear regression, logistic regression, decision tree, random forest, and K-means clustering techniques

·       Why Kafka and what is Kafka?

·       Kafka architecture

·       Kafka workflow

·       Configuring Kafka cluster

·       Operations

·       Kafka monitoring tools

·       Integrating Apache Flume and Apache Kafka

·       Introduction to Spark Streaming

·       Features of Spark Streaming

·       Spark Streaming workflow

·       Initializing StreamingContext, discretized Streams (DStreams), input DStreams and Receivers

·       Transformations on DStreams, output operations on DStreams, windowed operators and why it is useful

·       Important windowed operators and stateful operators

·       Introduction to various variables in Spark like shared variables and broadcast variables

·       Learning about accumulators

·       The common performance issues

·       Troubleshooting the performance problems

·       Learning about Spark SQL

·       The context of SQL in Spark for providing structured data processing

·       JSON support in Spark SQL

·       Working with XML data

·       Parquet files

·       Creating Hive context

·       Writing data frame to Hive

·       Reading JDBC files

·       Understanding the data frames in Spark

·       Creating Data Frames

·       Manual inferring of schema

·       Working with CSV files

·       Reading JDBC tables

·       Data frame to JDBC

·       User-defined functions in Spark SQL

·       Shared variables and accumulators

·       Learning to query and transform data in data frames

·       Data frame provides the benefit of both Spark RDD and Spark SQL

·       Deploying Hive on Spark as the execution engine

·       Learning about the scheduling and partitioning in Spark

·       Hash partition

·       Range partition

·       Scheduling within and around applications

·       Static partitioning, dynamic sharing, and fair scheduling

·       Map partition with index, the Zip, and GroupByKey

·        Spark master high availability, standby masters with ZooKeeper, single-node recovery with the local file system and high order functions

·       Introduction to Splunk and Splunk developer roles and responsibilities

·       Writing Splunk query for search

·       Auto-complete to build a search

·       Time range

·       Refine search

·       Working with events

·       Identifying the contents of search

·       What is a Field

·       How to use Fields in search

·       Deploying Fields Sidebar and Field Extractor for REGEX field extraction

·       Delimiting Field Extraction using FX

·       Writing Splunk query for search, sharing, saving, scheduling and exporting search results

·       How to create alerts

·       Understanding alerts

·       Viewing fired alerts

·       Describe and configure scheduled reports

·       Introduction to Tags in Splunk

·       Deploying Tags for Splunk search

·       Understanding event types and utility

·       Generating and implementing event types in search

·       What is a Macro

·       What are variables and arguments in Macros

·       Creating get, post and search workflow actions

·       Studying the search command

·       The general search practices

·       What is a search pipeline

·       How to specify indexes in search

·       Highlighting the syntax

·       Deploying the various search commands like fields, tables, sort, rename, rex and erex

·       Using top, rare and stats commands

·       Using following commands and their functions: addcoltotals, addtotals, top, rare and stats

·       iplocation, geostats, geom and addtotals commands

·       Explore the available visualizations

·       Create charts and time charts

·       Omit null values and format results

·       Calculating and analyzing results

·       Value conversion

·       Roundoff and format values

·       Using the eval command

·       Conditional statements

·       Filtering calculated search results

·       How to search the transactions

·       Creating report on transactions

·       Grouping events using time and fields

·       Comparing transactions with stats

·       Learning data lookups

·       Examples and lookup tables

·       Defining and configuring automatic lookups

·       Deploying lookups in reports and searches

·       Creating search charts, reports and dashboards

·       Editing reports and dashboards

·       Adding reports to dashboards

·       Working with raw data for data extraction, transformation, parsing and preview

·       Describe pivot

·       Relationship between data model and pivot

·       Select a data model object

·       Create a pivot report

·       Create instant pivot from a search

·       Add a pivot report to dashboard

·       What is a Splunk CIM

·       Using the CIM Add-On to normalize data

·       Introduction to the architecture of Splunk

·       Various server settings

·       How to set up alerts

·       Various types of licenses

·       Important features of Splunk tool

·       The requirements of hardware and conditions needed for installation of Splunk

·       How to install and configure Splunk

·       The creation of index

·       Standalone server’s input configuration

·       The preferences for search

·       Linux environment Splunk installation

·       The administering and architecting of Splunk

·       How to install Splunk in the Linux environment

·       The conditions needed for Splunk

·       Configuring Splunk in the Linux environment

·       Introducing Splunk distributed management console

·       Indexing of clusters

·       How to deploy distributed search in Splunk environment

·       Forwarder management

·       User authentication and access control

·       Introduction to the Splunk app

·       How to develop Splunk apps

·       Splunk app management

·       Splunk app add-ons

·       Using Splunk-base for installation and deletion of apps

·       Different app permissions and implementation

·       How to use the Splunk app

·       Apps on forwarder

·       Details of the index time configuration file

·       The search time configuration file

·       Understanding of Index time and search time configuration filesin Splunk

·       Forwarder installation

·       Input and output configuration

·       Universal Forwarder management

·       Splunk Universal Forwarder highlights

·       Implementing the Splunk tool

·       Deploying it on the server

·       Splunk environment setup

·       Splunk client group deployment

·       Understanding the Splunk Indexes

·       The default Splunk Indexes

·       Segregating the Splunk Indexes

·       Learning Splunk Buckets and Bucket Classification

·       Estimating Index storage

·       Creating new Index

·       Understanding the concept of role inheritance

·       Splunk authentications

·       Native authentications

·       LDAP authentications

·       Splunk installation, configuration

·       Data inputs

·       App management

·       Splunk important concepts

·       Parsing machine-generated data

·       Search indexer and forwarder

·       Introduction to Splunk Configuration Files

·       Universal Forwarder

·       Forwarder Management

·       Data management, troubleshooting and monitoring

·       Converting machine-generated data into operational intelligence

·       Setting up the dashboard, reports and charts

·       Integrating Search Head Clustering and Indexer Clustering

·       Understanding the input methods

·       Deploying scripted, Windows and network

·       Agentless input types and fine-tuning them all

·       Splunk user authentication and job role assignment

·       Learning to manage, monitor and optimize Splunk Indexes

·       Understanding parsing of machine-generated data

·       Manipulation of raw data

·       Previewing and parsing

·       Data field extraction

·       Comparing single-line and multi-line events

·       Distributed search concepts

·       Improving search performance

·       Large-scale deployment and overcoming execution hurdles

·       Working with Splunk Distributed Management Console for monitoring the entire operation

·       Cluster indexing

·       Configuring individual nodes

·       Configuring the cluster behavior, index and search behavior

·       Setting node type to handle different aspects of cluster like master node, peer node and search head