MirroringUMD
Jump to navigation
Jump to search
Technology | Software Component Delivery | Software Provisioning | UMD Middleware | Cloud Middleware Distribution | Containers Distribution | Technology Glossary |
Software Provisioning menu: | Software Provisioning Process | UMD Release Process | Quality Assurance | UMD | Staged Rollout |
Mirroring of UMD Repositories
We provide some basic information needed in order to mirror UMD repositories. Available repos:
- CAS
- SAM
- UMD production (including ReleaseCandidate
- UMD staged-rollout
- UMD testing- includes everything that is currently in the Stage Rollout or UMD-Store State
- UMD untested - includes everything that is currently in the Unverified or In Verification State
Communication
- Mailing-lists
- for announcements (changes, new repos, etc): mirror-egi at egi.eu - TO BE SET-UP
- for discussion: FORUM - mirror-egi - TO BE SET-UP
Repos size estimates
- please see the content of the following file for insfo on available directories and their estimated sizes:
Export License
Mirroring
Using lftp
Depending on what repositories you want to mirror and the space you have available (see #Repos size estimates you can use something similar:
- to create the mirror for the first time:
# lftp -c mirror --verbose --exclude 'candidate/' --exclude '1/' --exclude '2/' http://repository.egi.eu/sw/production/umd/ ${LOCAL_DIR}
- to removes any local packages that no longer exist on the source, and only downloads newer package versions from the host mirror:
lftp -c mirror –-only-newer --ignore-time -–delete --exclude 'candidate/' --exclude '1/' --exclude '2/' http://repository.egi.eu/sw/production/umd/ ${LOCAL_DIR}
Using wget
- at the moment "wget -r" will not work, please use the lftp solution described above.
Using rsync
- A rsync server is availabe at rsync.athena.hellasgrid.gr
- it can be use with:
# rsync -avzH --delete --exclude cas/ --exclude sam/ --exclude umd/1/ --exclude umd/2/ --exclude umd/candidate/ rsync://rsync.athena.hellasgrid.gr/production/ <path_to_your_directory/
- or you can create a script to be run as a cron job:
# cat mirror_UMD.sh #!/bin/sh DATE=`/bin/date +%Y-%m-%d` OUTDIR='/tmp' MIRROR=/rep/repo/UMD/production/umd/ [ -d $OUTDIR ] || mkdir -p $OUTDIR OUTFILE=$OUTDIR/mirror-UMD-$DATE.txt /bin/nice /usr/bin/rsync --verbose --progress \ --stats --archive --partial -H --delete \ --exclude cas/ \ --exclude sam/ \ --exclude umd/1 \ --exclude umd/2 \ --exclude umd/candidate \ rsync.athena.hellasgrid.gr::production/ $MIRROR/ \ >> $OUTFILE