flume spooling directory source example


Flume Data Collection into HDFS with Avro Serialization 4

In this post, we will provide proof of concept for Flume Data collection into HDFS with Avro Serialization by using HDFS sink, Avro Serializer on Sequence Files with Snappy Compression. Also we will use the formatting escape sequences to store the events on HDFS Path. In this post, we will create a flume agent with Spooling directory source with JDBC Channel and HDFS Sink. Now lets create our agent Agent7 in flume.conf […]


Flume Data Collection into HBase 5

We will discuss about collection of data into HBase directly through flume agent. In our previous posts under flume category, we have covered setup of flume agents for file roll, logger and HDFS sink types. In this, we are going to explore the details of HBase sink and its setup with live example. As we have already covered File channel , Memory channel and JDBC Channel, so we will try to make […]


Review Comments
default gravatar

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA

.

Next Batch Spark, Scala Training

  • Next Batch on Spark Developer Online Training starts around 26th of February. If any one interested to attend this batch please register by sending email to me on siv535@gmail.com or calling on +91-9704231873
  • Training Course Includes below topics:
  • Scala
  • Spark
  • Real Time projects
  • Contact For Job Support Also

Next Training Batch on Hadoop Starts on 21st Feb

  • Next Batch on Spark Developer Online Training starts around 21st of February. If any one interested to attend this batch please register by sending email to me on siv535@gmail.com or calling on +91-9704231873
  • Training Course Includes below topics:
  • HDFS
  • Mapreduce Essentials
  • Pig
  • Hive
  • Impala
  • Hbase
  • Sqoop
  • Flume
  • Oozie
  • 3 Real Time Projects