ted.dunning at gmail
Apr 10, 2009, 5:30 PM
Post #2 of 2
(735 views)
Permalink
|
At Veoh, we pulled machines from the pool during updates, but we did the
updates by copying entirely new indexes to the machines. We might have been
able to avoid pulling the machines from the pool if we had used better I/O
scheduling, but we were very risk averse by that time.
My current system uses elastic computing instead based on Katta. When doing
a full update, we just launch an entire new search cluster and then flip
traffic to it. Nodes download shards autonomously from an HDFS file
system. Updates in the form of additions (our dominant form) are propagated
by simply adding a new shard and letting Katta distribute it. This seems to
work *much* better than the Solr based system we had a Veoh.
On Fri, Apr 10, 2009 at 3:18 AM, sunnyfr <johanna.34 [at] gmail> wrote:
> How people do when they have frequent update to do ? to they turn off their
> servers during the warmup ???
> what is the solution, should I make it lower ??
>
--
Ted Dunning, CTO
DeepDyve
|
