[users] Metadata problems

[Filipe Brandenburger]

On Dec 17, 2007 4:42 AM, Dag Wieers <dag at wieers.com> wrote:

> > Ok, but then, why did it fail exactly when downloading the
> primary.xml.gzfrom
> > apt.sw.be? If the repomd.xml supposedly came from the same source? From
> the
> > output on David's e-mail, apt.sw.be was the first mirror tried:
>
> Because the list of checksums was not from that server, I would think.
> But maybe my analysis based on output from users is wrong. Please prove me
> wrong and tell me how to fix it.


I was looking at yum's code, and it sticks to one server as long as there
are no errors. If there is an error (which may be a 404, a timeout, or a
checksum error) it goes to the next mirror. It's done on a layer below yum
(on Python's urlgrabber) so to change yum to do the right thing is not so
easy, it would be an intrusive patch. Anyway, I'm looking at it and thinking
about a way to do it, I'll probably get to it.

When I said the problem was in apt.sw.be, I didn't consider that yum
sometimes caches repomd.xml, probably the user had the repomd.xml from a bad
mirror cached, and then when it compares with apt.sw.be's primary.xml.gz the
checksum doesn't match. If you have this situation (checksum doesn't match)
just after an "yum clean cache", it means that the first server on the list
has a repomd.xml that doesn't match the checksum of primary.xml.gz.


> > Isn't it possible that the problem is on the script used for mirroring
> > instead of on yum only?
>
> If you come up with the cause, I can fix the problem. What I gave you was
> my theory.


Ok, I'm trying to prove this. I'm writing a script that gets repomd.xml,
primary.xml.gz and the other two files from the five mirrors, and then
checks which ones are broken.

Right now (8:15 am EST), this is the situation:

* apt.sw.be, ftp-stud.fht-esslingen.de and mirror.cpsc.ucalgary.ca are
consistent. Their repomd.xml matches the primary.xml.gz with sha1
477aa22403eee4d874d747492e51fed4f03fd881.
* rpmfind.net is inconsistent with itself. Its repomd.xml says
477aa22403eee4d874d747492e51fed4f03fd881, but its primary.xml.gz has sha1
5e065dadf5d688177cffdc0df06aa118fa80bc7f. Strangely, timestamps seem to
match on the files, all of them updated 17-Nov-2007 1:41. The November date
doesn't make much sense, but the strange is that all the files have the same
date, maybe the server has the wrong time...
* www.mirrorservice.org is consistent with itself, but has different
(probably older) files. Both repomd.xml and primary.xml.gz have sha1
f25d5fafe8f3f7cb375036a7145b2a884e5cc107. I couldn't find the file date from
browsing the http directory, because it's using a CGI to show the package
list. From HTTP headers, it seems to be Tue, 13 Nov 2007 15:23:10 GMT.
Apparently the mirror is not updating, that's all.

I'm watching x86_64 arch.

The one I'm worried about is rpmfind.net, its behaviour is really strange...

Anyway, I'll continue looking at it, and I'll report any news. Hope it'll
help to solve the problem.

Thanks,
Filipe
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.rpmforge.net/pipermail/users/attachments/20071217/33981d31/attachment.html