flume hdfs.useLocalTimeStamp


Expected timestamp in the Flume event headers, but it was null 1

Error Scenario: Expected timestamp in the Flume event headers, but it was null – NullPointerException This error message is received in ~/logs/flume.log file when starting a flume agent with HDFS sink with format escape sequences.(%Y, %M, %D, %H, %M, %S).

If a sink expects a header but does not find it, events will become stuck in the channel and Flume will log NullPointer and EventDelivery exceptions. Root Cause: We receive […]


Flume Data Collection into HDFS with Avro Serialization 4

In this post, we will provide proof of concept for Flume Data collection into HDFS with Avro Serialization by using HDFS sink, Avro Serializer on Sequence Files with Snappy Compression. Also we will use the formatting escape sequences to store the events on HDFS Path. In this post, we will create a flume agent with Spooling directory source with JDBC Channel and HDFS Sink. Now lets create our agent Agent7 in flume.conf […]


Review Comments
default image

I am a plsql developer. Intrested to move into bigdata.

Neetika Singh ITA Hadoop in Dec/2016 December 22, 2016

.