writing source and custom sink using flume-ng -

-

i new flume-ng. want source send unique xml files channel 1 one. channel validate xml files , send validity(either true or false) , xml file th custom sink. sink write valid files , invalid files different directories in hdfs. not sure source use. please help.

none of current ones fit use case. spoolingdirectorysource line-oriented xml files confuse , not arrive in 1 piece.

i suggest write custom source application.


Comments

Post a Comment

Popular posts from this blog

c# - How to get the current UAC mode -

-
can check current uac type set in device before running application through c#. i got know approach check whether current user administrator or not. but, need know uac type set (i.e. default/never notify) internal logic. is possible? please help. this article self-elevation , checking level program runs in. due security reasons i'm afraid won't able retrieve uac level set. http://code.msdn.microsoft.com/windowsdesktop/csuacselfelevation-644673d3
Read more

python - Redis Error 8 connecting localhost:6379. nodename nor servname provided, or not known -

-
my environment mac os 10.9.2, python3.3, redis-2.6.9 (64-bit). i have many threads (nealy 2000) use same redis instance write data, threads throw following exceptions: during handling of above exception, exception occurred: traceback (most recent call last): file "/usr/local/lib/python3.3/site-packages/redis/connection.py", line 250, in connect sock = self._connect() file "/usr/local/lib/python3.3/site-packages/redis/connection.py", line 268, in _connect self.socket_timeout) file "/usr/local/cellar/python3/3.3.4/frameworks/python.framework/versions/3.3/lib/python3.3/socket.py", line 417, in create_connecti on res in getaddrinfo(host, port, 0, sock_stream): socket.gaierror: [errno 8] nodename nor servname provided, or not known traceback (most recent call last): file "/usr/local/cellar/python3/3.3.4/frameworks/python.framework/versions/3.3/lib/python3.3/threading.py", line 901, in _bootstrap_i nner self.run() fi...
Read more