After reviewing some data I collected with the spider, I'm trying to figure out where the content of the Last_Update and Last_Visited fields is coming from.
I thought Last_Update would be the the date pulled from the last modified header, and the Last_Visited would be the date the page was crawled, but that is not what the fields contain.
What I get is numbers that look like this:
Last_Update 1020026956
Last_Visited 10200226956
That came from a page that was crawled yesterday that returned a LM header of April 12th.
So my question is what do these numbers represent?
I thought Last_Update would be the the date pulled from the last modified header, and the Last_Visited would be the date the page was crawled, but that is not what the fields contain.
What I get is numbers that look like this:
Last_Update 1020026956
Last_Visited 10200226956
That came from a page that was crawled yesterday that returned a LM header of April 12th.
So my question is what do these numbers represent?