Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: RSyslog: users

ElasticSearch Auto-Discovery

 

 

RSyslog users RSS feed   Index | Next | Previous | View Threaded


vladg at cmu

Jul 18, 2012, 8:11 AM

Post #1 of 2 (294 views)
Permalink
ElasticSearch Auto-Discovery

Hello,

I've written the following example code for ElasticSearch auto discovery
<https://gist.github.com/3133526>.

~ % ./autodiscovery
{"response":{"cluster_name":"elasticsearch","version":{"number":"0.19.7","s
napshot_build":false},"transport_address":"inet[/192.168.1.2:9300]","http_a
ddress":"inet[/192.168.1.2:9200]","attributes":{}}}


This would really help the resiliency of the omelasticsearch plugin -
currently the logs are sent to a hardcoded IP address. This would enable
omelasticsearch to get a list of cluster members at start, and if they
ever go down, it could get a new IP to send to. For more details about
auto discovery, see:
<http://www.elasticsearch.org/guide/reference/modules/discovery/zen.html>.

I created this example as part of my work integrating ElasticSearch with
our network IDS, and I'm afraid that I don't have the necessary time to
create a patch for omelasticsearch, but I figured I'd share it out in case
anyone was interested.

--
Vlad Grigorescu | Senior Security Engineer
Information Security Office | 412.268.1447
Carnegie Mellon University | 0x632E5272




_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards


rgerhards at hq

Jul 30, 2012, 6:22 AM

Post #2 of 2 (239 views)
Permalink
Re: ElasticSearch Auto-Discovery [In reply to]

Hi Vlad,

I am catching up with my backlog of patches and good suggestions. I really like that autodiscovery feature and wanted to integrate it. However, for me the test program aborts with

Error binding - : Address already in use

Have you seen this? Is it due to multicast not being enabled? Do you know what I need to do in my test lab to make the sample code work? I am currently using Ubuntu 12.04 with the default ES packages.

Thanks,
Rainer

> -----Original Message-----
> From: rsyslog-bounces [at] lists [mailto:rsyslog-
> bounces [at] lists] On Behalf Of Vlad Grigorescu
> Sent: Wednesday, July 18, 2012 5:11 PM
> To: rsyslog [at] lists
> Subject: [rsyslog] ElasticSearch Auto-Discovery
>
> Hello,
>
> I've written the following example code for ElasticSearch auto
> discovery
> <https://gist.github.com/3133526>.
>
> ~ % ./autodiscovery
> {"response":{"cluster_name":"elasticsearch","version":{"number":"0.19.7
> ","s
> napshot_build":false},"transport_address":"inet[/192.168.1.2:9300]","ht
> tp_a
> ddress":"inet[/192.168.1.2:9200]","attributes":{}}}
>
>
> This would really help the resiliency of the omelasticsearch plugin -
> currently the logs are sent to a hardcoded IP address. This would
> enable
> omelasticsearch to get a list of cluster members at start, and if they
> ever go down, it could get a new IP to send to. For more details about
> auto discovery, see:
> <http://www.elasticsearch.org/guide/reference/modules/discovery/zen.htm
> l>.
>
> I created this example as part of my work integrating ElasticSearch
> with
> our network IDS, and I'm afraid that I don't have the necessary time to
> create a patch for omelasticsearch, but I figured I'd share it out in
> case
> anyone was interested.
>
> --
> Vlad Grigorescu | Senior Security Engineer
> Information Security Office | 412.268.1447
> Carnegie Mellon University | 0x632E5272
>
>
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com/professional-services/
> What's up with rsyslog? Follow https://twitter.com/rgerhards
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com/professional-services/
What's up with rsyslog? Follow https://twitter.com/rgerhards

RSyslog users RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.