openstack at newgoliath
Apr 15, 2012, 6:31 PM
Post #1 of 1
(101 views)
Permalink
|
|
SWIFT - slogging - hours still odd
|
|
Hi Folks,
I was having MAD time skews because my machines weren't running UTC. Now
they are. BUT STILL, I have a 1 hour skew:
account stats and container stats are gathered at the top of every hour,
say 14:00, for data from 13:00 to 13:59
this data is gatered into /var/log/swift/hourly/2012041514, indicating the
hour it was processed, not the hour of the contents.
the access log processor reads this file, and provides a CSV with the data
in the file for the 13:00 hour
access logs are gathered and uploaded nearly at the same time, 14:00, with
the data accounting for the same hour - 13:00 - 13:59
this data is gatered into /var/log/swift/hourly/2012041514, indicating the
hour it was processed, not the hour of the contents.
HOWEVER the stats processor only knows the timestamp from the filename to
use as the key data, which is not 13, but is 14, in error.
This leads to complete key match misses for all the data, because stats
data is considered 1400 hour data, and access log data is 1300 data. And
the resultant log_processing_csv files have two lines for every account.
Aren't other folks seeing this?
-judd
|
