rpm2html/mirror.html - view

File: [Public] / rpm2html / mirror.html
Revision 1.4: download - view: text, annotated - select for diffs
Fri May 15 16:17:57 1998 UTC (26 years ago) by veillard
Branches: MAIN
CVS tags: HEAD

Added a Faq + question about the [localbase], Daniel.

<HTML> <HEAD> <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1"> <META NAME="GENERATOR" CONTENT="Mozilla/4.04 [en] (X11; I; Linux 2.1.101 i686) [Netscape]"> <TITLE>Setting up a mirror of Rufus.W3.Org RPM database</TITLE> </HEAD> <BODY TEXT="#000000" BGCOLOR="#FFFFFF" LINK="#0000EF" VLINK="#51188E" ALINK="#FF0000"> <CENTER> <H1> Setting up a mirror of Rufus.W3.Org RPM database</H1></CENTER> This page explain how to set-up a Web database for RPM packages similar to <A HREF="../RPM/">the one running on rufus.w3.org</A> . You should first get acquainted on the mirroring principle described shortly on <A HREF="mirroring.html">the mirroring proposal</A>. However the setup should be fairy simple: <H3> Prerequisites</H3> <OL> <LI> You must of course have a Web server running, <A HREF="../rpm/apache.html">I suggest Apache </A>the obvious choice for a Linux machine, it's probably installed by default anyway.</LI> <LI> You should run a mirror of the RDF database available on <A HREF="ftp://ftp.rpm.org/pub/RDF">ftp://ftp.rpm.org/pub/RDF </A>. To help boostraping the mirroring process it may prove more efficient to fetch first a <A HREF="ftp://ftp.rpm.org/pub/RDF.tar.gz">compressed archive of the whole RDF tree </A>and expand it. Note that you don't need to mirror the full tree, you can select to prune some of the subtrees (but do not break the overall structure !). I suggest using <A HREF="../RPM/mirror.html">mirror-2.8 perl script</A> to do the mirroring. [<B>NOTE</B>: until the FTP server is available on ftp.rpm.org, use the <A HREF="ftp://rufus.w3.org/linux/RDF/">RDF database on rufus</A> and the <A HREF="ftp://rufus.w3.org/linux/RDF.tar.gz">RDF.tar.gz </A>here]</LI> <LI> You should get a recent copy of rpm2html, you can <A HREF="ftp://rufus.w3.org/pub/rpm2html/">grab an rpm</A> for example :-) (the version must be >= 0.90, and it's generally a good idea to follow closely the releases), install it.</LI> <LI> Of course, you need disk space, currently the RDF tree requires 60 MBytes while the full HTML tree built consumes nearly 300 MBytes (which is still small compared to the initial 8 GBytes needed for the RPM initial mirroring !).</LI> <LI> Subscribe to the rpm2html mailing-list, send a mail to <A HREF="mailto:majordomo@rufus.w3.org">majordomo@rufus.w3.org</A> with the line</LI> <BR><TT>subscribe rpm2html</TT> <BR>in the body of the message. The list <A HREF="messages/">archive are on-line.</A></OL> <H3> Setting up the mirror</H3> You need to replicate the RDF database available on <A HREF="ftp://ftp.rpm.org/pub/RDF">ftp://ftp.rpm.org/pub/RDF </A>. Basically you need to install mirror (this is a set of perl script dedicated to the job of mirroring FTP sites), and to add to the default configuration (usually named mirror.defaults) an entry for the RDF repository. Just add the following lines at the end of your mirror.defaults: <PRE>package=rdf         site=ftp.rpm.org         remote_dir=/linux/RDF         local_dir=/home/ftp/pub/linux/RDF         remote_user=anonymous         remote_password=hunter@esprit.net RDF mirroring</PRE> Try it by launching "mirror -d -p rdf" and check for possible problems. <BR>[<B>NOTE</B>: until the FTP server is available on ftp.rpm.org, use the <A HREF="ftp://rufus.w3.org/linux/RDF/">RDF database on rufus</A> and the <A HREF="ftp://rufus.w3.org/linux/RDF.tar.gz">RDF.tar.gz </A>here] <H3> Setting up the rpm2html config file</H3> I suggest <A HREF="ftp://rufus.w3.org/pub/rpm2html/rpm2html.config.mirrors">grabbing my existing config file</A> and modify it, this is a bit painful, but hopefully has to be done only once: <H4> Modify the Global section</H4> <OL> <LI> Change the <B>maint</B> and <B>mail</B> values to reflect your name and prefered E-mail address for feedback</LI> <LI> Change the <B>dir</B> path to the actual directory where the HTML file have to be produced (something like /home/httpd/html/RPM if you use the standard apache setup). This has to be in your server exported space and <B>the tree may grow to 200 MBytes</B> so check first that you have enought space !</LI> <LI> Change <B>url</B> to the prefix to access teh pages on your HTTP server. For example if you are serving them from <B>/home/httpd/html/RPM</B>, the full URL to access them is <B>http://my.server.org/RPM</B> and the correct value would be : <B>url=/RPM</B> .</LI> <LI> Remove any <B>rdf=true</B> or <B>rdf_dir=/linux/RDF</B> if present, those are used on rufus to create the .rdf files from the .rpm ones. You don't need them on a mirror.</LI> </OL> <H4> Modify each Directory section</H4> After the global section, the config file is a list of directory specific informations, usually related to one specific distribution. The goal here is to adapt it to your local filesystem and point to the local FTP mirrors (for example, you wouldn't point directly to RedHat site but to one of the mirrors in your area). You may drop some for the directories of you are too tight on space or if there is no near mirror for this specific distribution. Let's examine one entry: <OL> <LI> <TT>[/linux/RDF/redhat/5.0/i386]</TT> :  <B>change /linux</B><TT> </TT>to the actual location on your disk for the mirror, e.g.:</LI> <BR><TT>[/home/ftp/pub/mirror/redhat/5.0/i386]</TT> <LI> <TT>name=RedHat-5.0 for i386</TT> : You probably don't have to change the name of the distribution, unless you want to translate it.</LI> <LI> <TT>subdir=redhat/5.0/i386</TT> : local path, don't change it !</LI> <LI> <TT>ftp=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/alpha/RedHat/RPMS</TT> : The origin server for the packages, don't change it !</LI> <LI> <TT>ftpsrc=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/SRPMS</TT> : The origin server for the sources, you may want to point to a near server providing the sources RPMs.</LI> <LI> <TT>color=#ffe0ff </TT>: Color code for this distribution, you can change that but avoid giving nearly the same color for two different distribution.</LI> <LI> <TT>mirror=ftp://rufus.w3.org/linux/redhat/redhat-5.0/alpha/RedHat/RPMS</TT> : The <B>first nearest mirror</B>, customize to reduce the bandwidth traffic (don't reference rufus server if you are located in Australia !).</LI> <LI> <TT>mirror=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/alpha/RedHat/RPMS</TT> : <B>additionnal mirrors</B> may be added, rpm2html currently don't use this feature, but will in a near future ...</LI> </OL> <B>Note</B> that if you changed the configuration file for an existing setup, you need to pass the -force option to rpm2html to ensure that all the pages are updated. <H3> Run rpm2html</H3> Try it: <P><TT>rpm2html config.rpm2html.mirrors</TT> <P>Check for error messages, indicating path or directory rights problems, then point your favorite browser to the Web pages and ensure that the links generated internally are correct, as well as the outside links to the actual RPM mirrors. <BR>  <H3> Automate the process</H3> Add the mirror command to update the RDF directory and the call to rpm2html to your crontab. <B>Note </B>that rpm2html never clean up old pages generated but no more accurate, you need to add this to your cron job <B>before</B> running rpm2html: <UL> <LI> 0 4 * * * /usr/local/lib/mirror/mirror</LI> <LI> 30 6 * * * find /serveur/WWW/public/linux/RPM -not -type d -mtime +15 -exec rm {} \; ; /usr/bin/rpm2html -q /usr/share/rpm2html.config.mirrors</LI> </UL> <H3> Announce it and register</H3> Once you have a working setup, it would be cool to announce it to the <A HREF="mailto:rpm2html@rufus.w3.org">rpm2html mailing-list</A>, and to your local linux users group Don't forget to give location (country, state) information as well as the dataset indexed if you don't run the full archive. this has to be shared ! <a href="mailto:veillard@w3.org">Contact me</a> if you want to localize the output of rpm2html, it's not that hard ! <BR>  <ADDRESS> <A HREF="mailto:Daniel.Veillard@w3.org">Daniel Veillard</A></ADDRESS> <BR>$Id: mirror.html,v 1.4 1998/05/15 16:17:57 veillard Exp $ <BR>  </BODY> </HTML>