Hadoop Ecosystem

Jetzt loslegen. Gratis!
oder registrieren mit Ihrer E-Mail-Adresse
Rocket clouds
Hadoop Ecosystem von Mind Map: Hadoop Ecosystem

1. Coordinator jobs are recurrent Oozie Workflow jobs that are triggered by time and data availability.

2. Reduces aggregated results into answers

3. Links jobs

3.1. Workflow processing

4. Bundle provides a way to package multiple coordinator and workflow jobs and to manage the lifecycle of those jobs

4.1. Connects non-Hadoop stores (RDBMS)

4.2. Moves data to & from RDBMS to Hadoop

5. Workflow jobs are Directed Acyclical Graphs (DAGs), specifying a sequence of actions to execute. The Workflow job has to wait

6. Hive

6.1. SQL-like querying

6.2. Combiner can be used to optimize reducer performance

6.3. Structured data warehousing

6.4. Partition columns instead of indexes

7. Pig

7.1. Scripting for Hadoop

8. HBase

8.1. Non-relational

8.2. Column store

8.3. Transactional lookups

9. Flume

9.1. Log collector

9.2. Integrates into Hadoop

10. Oozie

11. Avro

11.1. Data parsing

11.2. Binary data serialization

11.3. RPC

11.4. language-neutral

11.5. optional codegen

11.6. schema evolution

11.7. untagged data

11.8. dynamic typing

12. Mahout

12.1. Machine learning

12.2. Applied to MR

13. Sqoop

13.1. Autogens Java InputFormat code for data access

14. MapReduce

14.1. Distributed compute

14.2. Maps query onto nodes

15. Ambari

15.1. Cluster deployment and admin

15.2. Driven by Hortonworks

16. ZooKeeper

16.1. Coordinator of shared state between apps

16.2. Naming, configuration, and synchronization services

17. YARN

17.1. cluster management

17.2. Hadoop 2

17.3. resource manager

17.4. job scheduler

18. BigTop

18.1. Package Hadoop ecosys

18.2. Test Hadoop ecosys package

19. Related Apache Ecosystems

20. HDFS

20.1. Distributed storage

21. Spark

22. Impala

22.1. SQL query egnine

22.2. Query data stored in HDFS and HBase

22.3. Real time

23. Cascading

23.1. Higher abstraction from MR

23.2. Creates Flow that assembles Map/Reduce jobs