[tools] mrepo ... questions

[Justin Hochstetler]

Hi Dag and the tools at lists.rpmforge.net list,

After the release of Fedora 7, I've been fooling with mrepo serving as
a repository for it (f7-i386 for the moment).  I've noticed some
places where mrepo (latest svn 5990 (according to the mrepo file
itself)) has troubles with it:  [Note: I'm using a FC3 and a FC5
*server* for mrepo .]

Fedora 7 appears to use sqlite-based metadata, which the latest
createrepo I found / compiled (0.4.9) can create using the
"--database" option.  However, this option ("createrepo-options") does
not appear to be settable for anything but the 'main' section [not per
dist or per-repo].  And, if you use '--force', even the 'main' opts
are overridden to just be ' --pretty', which prevents the option from
making it to createrepo.

Fedora 7 also uses a new/non-standard comps.xml file name:
("comps-f7.xml") , which doesn't appear to be picked up by the
auto-finder/copier/linker portion of mrepo, since it isn't
"comps.xml".

So, I've sort of patched mrepo to work around those, by making
'createrepo-options' and 'groupfilename' be 'dist'-level attributes,
but I was kinda treating my patch attempt as a learn-python project,
so I'm not sure it's quite up to production level just yet.  When
people send in patches, do you want them as 'diff -Naud' from a recent
svn repo version?

There were a couple of other things I noticed that seemed weird or
less than optimal:

a. mrepo downloads the rpms, sends an email, then processes the
metadata.  I've seen the metadata process fail sometimes, but since
the email has already gone out, it looks like everything succeeded,
until users can't yum update the next morning.

b. mrepo allows you to specify isos to be used in each dist.  but, if
those isos aren't there, it doesn't stop, and if the 'os' repo is
defined it just starts downloading 2.4GB of stuff (or so) per dist.  I
modified my development version to error out if it couldn't find a
requested iso, so it doesn't try to download the internet. (I just
added a check after the findisos where if isos is false/none/whatever,
but .iso was specified, it dies).

c. one of my servers was old-school yam before I upgraded it.  It
still functions the same, but now the url starts with
"http://server/mrepo" instead of "http://server/yam".  I just added an
one-line alias to the httpd mrepo configuration file that points both
/yam and /mrepo at the same place.  It seems to work, and my
already-configured-client machines are happy again.

"Alias /yam /var/www/mrepo"



But, those things aside, mrepo is a pretty cool app, and not all the
stuff I'm thinking about is bugs/etc.

Some of the stuff that I am thinking of writing for mrepo:

(with interspersed questions about the overall design philosophy in a
couple cases)

#1

The mrepo system uses sections for 'dist's and individual line entries
for 'repo'/url-of-repo entries.  Some of the things I'm thinking of
trying to add to mrepo would seem to require that repos be definable
as sections.  For example:

I have a 'local' repo (actually, I made it a full dist since it's not
'release' dependent in my case) to hold things like nxserver,
vmware-server, etc, that are mostly available as binary RPMs for a
collection of distros (fc3..7, for example) in one rpm.  I wanted to
gpg sign those rpms using a my-lan local gpg-key, (which can be done
easily, manually), but right now, there's not a way to put something
like 'rpm-signwith=//gpgkeypath'  (for example, as a concept) on a
single repo, since the repo isn't a section-object like a dist is.

GPG-Key wise, i'm thinking this would be a good something to be able
to assign to a repo, so that mrepo could create a web folder of all
gpg-keys you'd need to import to use any software in the dist/repos.
Maybe mrepo could also double-check the signatures for each rpm just
to be double-safe, since users will trust an mrepo repo like it's the
real thing.

I'm thinking it might also be possible to re-sign (or add a second gpg
signature to, rather) other 'official' rpms (like from 'extras' or
'updates', except that doing that would screw up the sha1 / md5 /
checksum of the rpms in question, confusing the rsync/etc processes.
I was thinking of de-sign-ing and re-sign-ing before/after the rsync
so that they would stay up to date, and I could set my client systems
to trust a single gpg-key and not have to worry about yum updates
failing when they either don't have, or want you to approve of, the
appropriate gpg key for that individual repo.

#2

I know mirrorlist support is planned/mentioned/to-do-ed for mrepo.  I
know yum-on-client repo-rpm-generation is as well.  I found an example
that looks like it might work well to build repo-client-configuration
rpms for each repo so that they could be yum installed and would then
enable all the other repos in the repository, etc.  One thing about
the URLs that mrepo uses is that they're consistent for mrepo, but
they really tend to look nothing like the various (and sundry) urls
that the distros tend to use: (for example)
".../freshrpms/pub/freshrpms/pub/fedora/fc$release-$arch...etc",

But, if mrepo allowed you to mangle your local web directory structure
to match the repo it was mirroring from, you'd be able to change from
internet to intranet repo usage just by editing the client yum line
from "downloads.redhat.com" to "mrepo.local.lan" , leaving the path
alone.  Further, you could do something like a squid proxy and
lan-DNS-server name-spoof to make 'downloads.redhat.com' resolve to
point instead to 'mrepo.local.lan', and if the path was 'as expected
by a virgin client, the client would use your repo automatically, no
fuss, no muss. :-) [well, assuming mirrorlist support was added, i
suppose]

Okay, this is bit longer than I set out to write, but I think mrepo is
great, and I think there's definitely some neat things that can be
added.

Thanks everybody,
Justin H