 |
Debian-AMD64 Mirror Howto |
 |
 |
 |
Simply said: A normal mirror, with the only difference that we tell you when something has changed in the archive. No need to go and just try a sync, you know when it changed. A good page describing it, and some parts of the setup, is available on the debian.org servers. Please read it if you are not familiar with it.
The setup is simple. You need a sane mirror script to get the archive, a sample one is available here. Copy the configuration part into $HOME/.sync-debian-amd64.conf, where $HOME is the the home of whatever user your mirror script runs under. Modify them to fit your environment, but most probably the RSYNC_HOST and RSYNC_PATH are already correct, except you get told otherwise as answer to your mirror offer.
You are of course free to use any script you want, but please be sure to have a sane one. We do NOT accept mirrors that use debmirror or similar tools to mirror. These tools are fine for a private mirror, but not usable for full mirrors, as they exclude many of the useful files! Which basically means: Keep a full working mirror the whole time. The following options to rsync (besides what you normally have) do the trick:
| Run | Options | Affects |
|---|---|---|
| 1 | --exclude Packages* --exclude Sources* --exclude Release* | You download new binary files, sources and Packages, but not the metadata for them that apt-get
uses. Also note that no delete option is given. |
| 2 | --delete --delete-after | The second run now fetches the files excluded in the first run and also deleted all files not any longer in the archive. |
This two-way mirror approach makes sure that users with their apt-get can always fetch the files that are announced to be there (via the Packages files), even when your mirror is updating itself at the moment they try to download stuff. You are free to use any other option to rsync you may like, but we strongly suggest to use at least -lH.
Now you need to add an ssh key to the mirror-user's .ssh/authorized_keys file.
What file you need to add is determined by the region of the world you are in, see the table below, non-listed
regions - please contact joerg@debian.org.
| Region | Mirror Master | Contact | SSH Pubkey |
|---|---|---|---|
| Europe | syncproxy-amd64.debian.net | Joerg Jaspert <joerg@debian.org> | id_amd64push.pub |
| America | debian.csail.mit.edu | Noah Meyerhans <noahm@debian.org> | id_amd64america.pub |
If you look into the keys you see that they are limited to only one command. This means that we are NOT able to log on to your system with this keys. There is no way for us to do anything other to your system then to just trigger the mirror - so the only harm we can do would be to trigger your mirror every few seconds, but for that the script we provide as an example uses a lockfile to avoid multiple runs at the same time. You need to change that to whatever script you use, but do NOT delete the & at the end.
As soon as you finished setting it up you are ready to be added to the list of "pushed" servers. For that please email the hostname and the username where the signal should be sent to the contact as listed in the table above. If there is no master for your region yet then mail the contact from Europe.
Simple, after the above is finished and running you are normally added to our mirror list. While you send the hostname/username to the address above please include the following information for the listing, the data will be forwarded to get included in our main mirrorlist:
Site: Your Primary Hostname Aliases: If you are known under other names also, maybe multiple times Type: Push-[TYPE] (Type can be Primary or Secondary, whatever you want to be) Archive-ftp: /debian-amd64/debian/ Archive-http: /debian-amd64/debian/ Archive-rsync: debian-amd64/debian/ Mirrors-from: syncproxy-amd64.debian.net Maintainer: Your Name <your@ema.il> Country: ISO Countryname Location: City
Change the paths for Archive-* to fit your layout and delete lines with services you don't have. An example for an entry follows.
Site: ftp.de.debian.org Aliases: ftp1.de.debian.org Aliases: debian.inf.tu-dresden.de Type: Push-Primary Archive-ftp: /debian-amd64/debian/ Archive-http: /debian-amd64/debian/ Archive-rsync: debian-amd64/debian/ Mirrors-from: amd64.debian.net Maintainer: Adam Lackorzynski <adam@os.inf.tu-dresden.de>, <debian-ftp@ftp.inf.tu-dresden.de> Country: DE Germany Location: Dresden Sponsor: Technical University of Dresden, Dept. of Computer Science http://www.inf.tu-dresden.de/ Comment: DFN; Push-Primary
Yes, that was it. THANK YOU for offering a mirror service!
Maybe you want to know what traffic you will get - or how much disk space you need. I can't say much about the bandwith usage, as that's dependent on the users, how they use it and what mirror they choose. I know that we have a constant traffic of 80 - 200GB a day with the master server, depending on the day / users / moon / whatever. Some mirrors do have more, some less traffic.
At the moment our archive takes about 43GB of space. This contains ~ 13GB amd64 specific files, the rest are binary indepent packages and the source files, which we MUST distribute to follow the laws. Which of course means wasting a bit of space on systems already mirroring Debian.
As the "waste" of space may not be what you want we offer a way to save it. This way is usable for "Secondary" mirrors, i.e. mirrors that do not want other people mirroring from it, just be accessed by users. If you want to provide other mirrors the possibility to sync from your server (and probably giving out pushes yourself), then the second option isn't for you, please mirror all of us.
| Type | Description / What to mirror |
|---|---|
| Primary Mirrors | You are considered a primary mirror, who has the full amd64 archive. Other mirrors near to you (same
country for example) are ok to mirror from you. Please sync the whole debian-amd64 share from the mirror master. For that set RSYNC_PATH for your mirror-script to :debian-amd64.
|
| Secondary Mirrors | You want to only hold the changes between the Debian and the amd64 archive. Other mirrors should not
sync from you, the service is primarly destined for users. You want to sync the debian-amd64/ directory from the debian-amd64 share from your mirror master, nothing else. That would mean to set RSYNC_PATH for your mirror-script to :debian-amd64/debian-amd64.Please also read below what you need todo to make it work for the users. |
To use the special debian-amd64 setup you need to have an existing Debian mirror on your machine.
Assume you have your mirror of Debian in /mirror/debian, then mirror the debian-amd64 directory from us into
/mirror/debian-amd64, and you are done. Basically you need a debian directory
or symlink right besides the debian-amd64 paths, pointing to a real Debian mirror, as all links are relative
to that.
Real-Life Scenario: You mirror :debian-amd64/debian-amd64 to /home/ftp/debian-amd64, so the files end up
living in /home/ftp/debian-amd64/debian-amd64. Assume you have your regular Debian mirror in /home/ftp/debian,
then you need to create a symlink called debian in /home/ftp/debian-amd64 which points to
/home/ftp/debian to actually get the link farm to work.
Timestamp: 21.01.2006 / 13:40:00 (UTC)
(C)2005 Jörg Jaspert. - License: GPL