Amaltas

Anay Tamhankar

HDFS Java API

HDFS Java API

Starting Point There are two important Java classes which are the starting point for using the Java API. Upto Hadoop Version 1.x org.apache.hadoop.fs.FileSystem Hadoop Version 2.x and later org.apache.hadoop.fs.FileContext FileContext class has better handling of multiple fileystems (A single FileContext

HDFS Architecture

HDFS Architecture

HDFS Architecture Diagram Data Replication How does HDFS manage to provide data reliability and high fault tolerance when it uses commodity (and therefore cheap and prone to failure) hardware ? HDFS does it using a very simple idea. 1. Break a large file into series of chunks or blocks (each block

Hadoop Distributed File System (HDFS)

Hadoop Distributed File System (HDFS)

What is HDFS ? HDFS is a distributed file system designed to run on commodity hardware. HDFS is highly fault tolerant and provides high throughput access to "application data". It is suitable for applications which have large datasets. Distributed File Systems are file systems that manage storage across a network of

Introduction to Hadoop

Introduction to Hadoop

What is Hadoop ? Hadoop is a collection of open source software utilities that facilitate solving big data problems using a network of computers. Hadoop comprises of a software framework for distributed storage (HDFS) and a software framework for query and analysis (Map Reduce). Design Principal 💡Hardware failure are common occurrences

Big Data - Challenges and Opportunities

Big Data - Challenges and Opportunities

From a technical standpoint, Big Data can be defined as data that is so large, complex and fast changing that none of the traditional data management tools are able to store and process it efficiently. Therefore the term "big data" or "big data technologies" refers to the technologies that can