Annotation of rpm2html/mirror.html, revision 1.3

1.3     ! veillard    1: <HTML>
        !             2: <HEAD>
        !             3:    <META HTTP-EQUIV="Content-Type" CONTENT="text/html; charset=iso-8859-1">
        !             4:    <META NAME="GENERATOR" CONTENT="Mozilla/4.04 [en] (X11; I; Linux 2.1.101 i686) [Netscape]">
        !             5:    <TITLE>Setting up a mirror of Rufus.W3.Org RPM database</TITLE>
        !             6: </HEAD>
        !             7: <BODY TEXT="#000000" BGCOLOR="#FFFFFF" LINK="#0000EF" VLINK="#51188E" ALINK="#FF0000">
        !             8: 
        !             9: <CENTER>
        !            10: <H1>
        !            11: Setting up a mirror of Rufus.W3.Org RPM database</H1></CENTER>
        !            12: This page explain how to set-up a Web database for RPM packages similar
        !            13: to <A HREF="../RPM/">the one running on rufus.w3.org</A> . You should first
        !            14: get acquainted on the mirroring principle described shortly on <A HREF="mirroring.html">the
        !            15: mirroring proposal</A>. However the setup should be fairy simple:
        !            16: <H3>
        !            17: Prerequisites</H3>
        !            18: 
        !            19: <OL>
        !            20: <LI>
        !            21: You must of course have a Web server running, <A HREF="../rpm/apache.html">I
        !            22: suggest Apache </A>the obvious choice for a Linux machine, it's probably
        !            23: installed by default anyway.</LI>
        !            24: 
        !            25: <LI>
        !            26: You should run a mirror of the RDF database available on <A HREF="ftp://ftp.rpm.org/pub/RDF">ftp://ftp.rpm.org/pub/RDF
        !            27: </A>. To help boostraping the mirroring process it may prove more efficient
        !            28: to fetch first a <A HREF="ftp://ftp.rpm.org/pub/RDF.tar.gz">compressed
        !            29: archive of the whole RDF tree </A>and expand it. Note that you don't need
        !            30: to mirror the full tree, you can select to prune some of the subtrees (but
        !            31: do not break the overall structure !). I suggest using <A HREF="../RPM/mirror.html">mirror-2.8
        !            32: perl script</A> to do the mirroring. [<B>NOTE</B>: until the FTP server
        !            33: is available on ftp.rpm.org, use the <A HREF="ftp://rufus.w3.org/linux/RDF/">RDF
        !            34: database on rufus</A> and the <A HREF="ftp://rufus.w3.org/linux/RDF.tar.gz">RDF.tar.gz
        !            35: </A>here]</LI>
        !            36: 
        !            37: <LI>
        !            38: You should get a recent copy of rpm2html, you can <A HREF="ftp://rufus.w3.org/pub/rpm2html/">grab
        !            39: an rpm</A> for example :-) (the version must be >= 0.90, and it's generally
        !            40: a good idea to follow closely the releases), install it.</LI>
        !            41: 
        !            42: <LI>
        !            43: Of course, you need disk space, currently the RDF tree requires 60 MBytes
        !            44: while the full HTML tree built consumes nearly 300 MBytes (which is still
        !            45: small compared to the initial 8 GBytes needed for the RPM initial mirroring
        !            46: !).</LI>
        !            47: 
        !            48: <LI>
        !            49: Subscribe to the rpm2html mailing-list, send a mail to <A HREF="mailto:majordomo@rufus.w3.org">majordomo@rufus.w3.org</A>
        !            50: with the line</LI>
        !            51: 
        !            52: <BR><TT>subscribe rpm2html</TT>
        !            53: <BR>in the body of the message. The list <A HREF="messages/">archive are
        !            54: on-line.</A></OL>
        !            55: 
        !            56: <H3>
        !            57: Setting up the mirror</H3>
        !            58: You need to replicate the RDF database available on <A HREF="ftp://ftp.rpm.org/pub/RDF">ftp://ftp.rpm.org/pub/RDF
        !            59: </A>. Basically you need to install mirror (this is a set of perl script
        !            60: dedicated to the job of mirroring FTP sites), and to add to the default
        !            61: configuration (usually named mirror.defaults) an entry for the RDF repository.
        !            62: Just add the following lines at the end of your mirror.defaults:
        !            63: <PRE>package=rdf
        !            64: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; site=ftp.rpm.org
        !            65: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; remote_dir=/linux/RDF
        !            66: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; local_dir=/home/ftp/pub/linux/RDF
        !            67: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; remote_user=anonymous
        !            68: &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; remote_password=hunter@esprit.net RDF mirroring</PRE>
        !            69: Try it by launching "mirror -d -p rdf" and check for possible problems.
        !            70: <BR>[<B>NOTE</B>: until the FTP server is available on ftp.rpm.org, use
        !            71: the <A HREF="ftp://rufus.w3.org/linux/RDF/">RDF database on rufus</A> and
        !            72: the <A HREF="ftp://rufus.w3.org/linux/RDF.tar.gz">RDF.tar.gz </A>here]
        !            73: <H3>
        !            74: Setting up the rpm2html config file</H3>
        !            75: I suggest <A HREF="ftp://rufus.w3.org/pub/rpm2html/rpm2html.config.mirrors">grabbing
        !            76: my existing config file</A> and modify it, this is a bit painful, but hopefully
        !            77: has to be done only once:
        !            78: <H4>
        !            79: Modify the Global section</H4>
        !            80: 
        !            81: <OL>
        !            82: <LI>
        !            83: Change the <B>maint</B> and <B>mail</B> values to reflect your name and
        !            84: prefered E-mail address for feedback</LI>
        !            85: 
        !            86: <LI>
        !            87: Change the <B>dir</B> path to the actual directory where the HTML file
        !            88: have to be produced (something like /home/httpd/html/RPM if you use the
        !            89: standard apache setup). This has to be in your server exported space and
        !            90: <B>the tree may grow to 200 MBytes</B> so check first that you have enought
        !            91: space !</LI>
        !            92: 
        !            93: <LI>
        !            94: Change <B>url</B> to the prefix to access teh pages on your HTTP server.
        !            95: For example if you are serving them from <B>/home/httpd/html/RPM</B>, the
        !            96: full URL to access them is <B>http://my.server.org/RPM</B> and the correct
        !            97: value would be : <B>url=/RPM</B> .</LI>
        !            98: 
        !            99: <LI>
        !           100: Remove any <B>rdf=true</B> or <B>rdf_dir=/linux/RDF</B> if present, those
        !           101: are used on rufus to create the .rdf files from the .rpm ones. You don't
        !           102: need them on a mirror.</LI>
        !           103: </OL>
        !           104: 
        !           105: <H4>
        !           106: Modify each Directory section</H4>
        !           107: After the global section, the config file is a list of directory specific
        !           108: informations, usually related to one specific distribution. The goal here
        !           109: is to adapt it to your local filesystem and point to the local FTP mirrors
        !           110: (for example, you wouldn't point directly to RedHat site but to one of
        !           111: the mirrors in your area). You may drop some for the directories of you
        !           112: are too tight on space or if there is no near mirror for this specific
        !           113: distribution. Let's examine one entry:
        !           114: <OL>
        !           115: <LI>
        !           116: <TT>[/linux/RDF/redhat/5.0/i386]</TT> :&nbsp; <B>change /linux</B><TT>
        !           117: </TT>to the actual location on your disk for the mirror, e.g.:</LI>
        !           118: 
        !           119: <BR><TT>[/home/ftp/pub/mirror/redhat/5.0/i386]</TT>
        !           120: <LI>
        !           121: <TT>name=RedHat-5.0 for i386</TT> : You probably don't have to change the
        !           122: name of the distribution, unless you want to translate it.</LI>
        !           123: 
        !           124: <LI>
        !           125: <TT>subdir=redhat/5.0/i386</TT> : local path, don't change it !</LI>
        !           126: 
        !           127: <LI>
        !           128: <TT>ftp=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/alpha/RedHat/RPMS</TT>
        !           129: : The origin server for the packages, don't change it !</LI>
        !           130: 
        !           131: <LI>
        !           132: <TT>ftpsrc=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/SRPMS</TT> : The
        !           133: origin server for the sources, you may want to point to a near server providing
        !           134: the sources RPMs.</LI>
        !           135: 
        !           136: <LI>
        !           137: <TT>color=#ffe0ff </TT>: Color code for this distribution, you can change
        !           138: that but avoid giving nearly the same color for two different distribution.</LI>
        !           139: 
        !           140: <LI>
        !           141: <TT>mirror=ftp://rufus.w3.org/linux/redhat/redhat-5.0/alpha/RedHat/RPMS</TT>
        !           142: : The <B>first nearest mirror</B>, customize to reduce the bandwidth traffic
        !           143: (don't reference rufus server if you are located in Australia !).</LI>
        !           144: 
        !           145: <LI>
        !           146: <TT>mirror=ftp://ftp.redhat.com/pub/redhat/redhat-5.0/alpha/RedHat/RPMS</TT>
        !           147: : <B>additionnal mirrors</B> may be added, rpm2html currently don't use
        !           148: this feature, but will in a near future ...</LI>
        !           149: </OL>
        !           150: <B>Note</B> that if you changed the configuration file for an existing
        !           151: setup, you need to pass the -force option to rpm2html to ensure that all
        !           152: the pages are updated.
        !           153: <H3>
        !           154: Run rpm2html</H3>
        !           155: Try it:
        !           156: 
        !           157: <P><TT>rpm2html config.rpm2html.mirrors</TT>
        !           158: 
        !           159: <P>Check for error messages, indicating path or directory rights problems,
        !           160: then point your favorite browser to the Web pages and ensure that the links
        !           161: generated internally are correct, as well as the outside links to the actual
        !           162: RPM mirrors.
        !           163: <BR>&nbsp;
        !           164: <H3>
        !           165: Automate the process</H3>
        !           166: Add the mirror command to update the RDF directory and the call to rpm2html
        !           167: to your crontab. <B>Note </B>that rpm2html never clean up old pages generated
        !           168: but no more accurate, you need to add this to your cron job <B>before</B>
        !           169: running rpm2html:
        !           170: <UL>
        !           171: <LI>
        !           172: 0 4 * * * /usr/local/lib/mirror/mirror</LI>
        !           173: 
        !           174: <LI>
        !           175: 30 6 * * * find /serveur/WWW/public/linux/RPM -not -type d -mtime +15 -exec
        !           176: rm {} \; ; /usr/bin/rpm2html -q /usr/share/rpm2html.config.mirrors</LI>
        !           177: </UL>
        !           178: 
        !           179: <H3>
        !           180: Announce it and register</H3>
        !           181: Once you have a working setup, it would be cool to announce it to the <A HREF="mailto:rpm2html@rufus.w3.org">rpm2html
        !           182: mailing-list</A>, and to your local linux users group, this has to be shared
        !           183: !
        !           184: <BR>&nbsp;
        !           185: <ADDRESS>
        !           186: <A HREF="mailto:Daniel.Veillard@w3.org">Daniel Veillard</A></ADDRESS>
        !           187: 
        !           188: <BR>$Id: mirror.html,v 1.2 1998/05/12 00:54:29 veillard Exp $
        !           189: <BR>&nbsp;
        !           190: </BODY>
        !           191: </HTML>

Webmaster