Nos clients

Code formation

hadoopdev

Durée

28 heures (généralement 4 jours pauses comprises)

Pré requis

comfortable with Java programming language (most programming exercises are in java)
comfortable in Linux environment (be able to navigate Linux command line, edit files using vi / nano)

Lab environment

Zero Install : There is no need to install hadoop software on students’ machines! A working hadoop cluster will be provided for students.

Students will need the following

an SSH client (Linux and Mac already have ssh clients, for Windows Putty is recommended)
a browser to access the cluster. We recommend Firefox browser

Aperçu

Apache Hadoop est le framework le plus répandu pour le traitement de Big Data sur des clusters de serveurs. Ce cours présentera aux développeurs divers composants (HDFS, MapReduce, Pig, Hive et HBase) de l’écosystème Hadoop .

Machine Translated

Plan du cours

Section 1: Introduction to Hadoop

hadoop history, concepts
eco system
distributions
high level architecture
hadoop myths
hadoop challenges
hardware / software
lab : first look at Hadoop

Section 2: HDFS

Design and architecture
concepts (horizontal scaling, replication, data locality, rack awareness)
Daemons : Namenode, Secondary namenode, Data node
communications / heart-beats
data integrity
read / write path
Namenode High Availability (HA), Federation
labs : Interacting with HDFS

Section 3 : Map Reduce

concepts and architecture
daemons (MRV1) : jobtracker / tasktracker
phases : driver, mapper, shuffle/sort, reducer
Map Reduce Version 1 and Version 2 (YARN)
Internals of Map Reduce
Introduction to Java Map Reduce program
labs : Running a sample MapReduce program

Section 4 : Pig

pig vs java map reduce
pig job flow
pig latin language
ETL with Pig
Transformations & Joins
User defined functions (UDF)
labs : writing Pig scripts to analyze data

Section 5: Hive

architecture and design
data types
SQL support in Hive
Creating Hive tables and querying
partitions
joins
text processing
labs : various labs on processing data with Hive

Section 6: HBase

concepts and architecture
hbase vs RDBMS vs cassandra
HBase Java API
Time series data on HBase
schema design
labs : Interacting with HBase using shell; programming in HBase Java API ; Schema design exercise

Nos Clients témoignent

★★★★★

Les exercices pratiques en relation avec le contexte

Guillaume Martens

Formation:Web Development with Symfony3

Assistance personnelle lorsque j'avais des problèmes techniques :)

Elena Rouvier

Formation:Web Development with Symfony3

l'humour, disponabilité pour répondre à d'autre types de questions

koen depoorter

Formation:Web Development with Symfony3

Exercices

Fabrice Limbourg

Formation:Introduction to SQL Server 2012 Integration Services (SSIS)

La compréhension du formateur pour le produit OpenLDAP

Patrice Massey - Ville de Montréal

Formation:Ateliers OpenLDAP

Les conseils fournis par l'instructeur concernant les processus d'implantation et de migration. Le partage de ses connaissances, acquises lors de son travail. Les choses à ne pas faire ou à faire attention.

Ville de Montréal

Formation:Ateliers OpenLDAP

La grande place laissée aux exercices

ONEM