Login | Register For Free | Help
Search for: (Advanced)

Mailing List Archive: Varnish: Misc

Backend DNS lookup refresh

 

 

Varnish misc RSS feed   Index | Next | Previous | View Threaded


thierry.magnien at sfr

Nov 3, 2011, 9:14 AM

Post #1 of 2 (466 views)
Permalink
Backend DNS lookup refresh

Hi all,

We are dealing with a great number of backends in our VCL file and we have problems with some backends changing IP address (for example when hosted on services such as Amazon S3, that switch IP address without telling anyone).

We would like to be able to refresh the DNS resolution of backends when needed and after a quick look at the source code, it seems there are several ways to handle this.

I would like to get your feeling about what would be the best solution before trying to write a patch, so here are my thoughts, just as it comes out of my mind ;-)

DNS resolution of backends is done when the VCL is loaded. IP address is stored in the compiled VCL as a sockaddr struct.

The first way to solve the problem would be to implement a "vcl reload" CLI function. In that case, this needs some tricks to be able to recompile a VCL which has the same name of the one used.

A 2nd way would be to make backend IP addresses resolved when needed (at first backend request and then cached for a "TTL" time), and not at VCL loading. However, this breaks most of the the optimization brought by the sockaddr structure.

A point discussed quickly with Per Buer on IRC was to have probes checking the IP addresses of the backends and update when an IP change is detected.

All this makes me think that it's not so easy as it could seem, and that's why I would accept any advice/idea/comment about this.

Regards,
Thierry


_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc


marc.fournier at camptocamp

Nov 6, 2011, 2:56 AM

Post #2 of 2 (437 views)
Permalink
Re: Backend DNS lookup refresh [In reply to]

Hello Thierry,

Excerpts from MAGNIEN, Thierry's message of Thu Nov 03 17:14:50 +0100 2011:
>
> We are dealing with a great number of backends in our VCL file and we
> have problems with some backends changing IP address (for example when
> hosted on services such as Amazon S3, that switch IP address without
> telling anyone).

A couple of years ago, we worked around the specific case of S3 by putting
a simple nginx proxy between S3 and varnish.

Amazon S3 emits HTTP redirects when DNS changes occur[¹]. Unfortunately
varnish doesn't handle them either. But nginx does, as well as backend DNS
changes.

It doesn't really answer your question, but this somewhat ugly workaround
has been working very reliably for us since then. It would be nicer if
varnish could deal with these 2 cases, though.

Cheers,
Marc

[¹] http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?Redirects.html

_______________________________________________
varnish-misc mailing list
varnish-misc [at] varnish-cache
https://www.varnish-cache.org/lists/mailman/listinfo/varnish-misc

Varnish misc RSS feed   Index | Next | Previous | View Threaded
 
 


Interested in having your list archived? Contact Gossamer Threads
 
  Web Applications & Managed Hosting Powered by Gossamer Threads Inc.