MirroringUMD
Jump to navigation
Jump to search
The printable version is no longer supported and may have rendering errors. Please update your browser bookmarks and please use the default browser print function instead.
Technology | Software Component Delivery | Software Provisioning | UMD Middleware | Cloud Middleware Distribution | Containers Distribution | Technology Glossary |
Software Provisioning menu: | Software Provisioning Process | UMD Release Process | Quality Assurance | UMD | Staged Rollout |
Mirroring of UMD Repositories
We provide some basic information needed in order to mirror UMD repositories. Available repos:
- CAS
- SAM
- UMD production (including ReleaseCandidate
- UMD staged-rollout
- UMD testing- includes everything that is currently in the Stage Rollout or UMD-Store State
- UMD untested - includes everything that is currently in the Unverified or In Verification State
Communication
- Mailing-lists
- for announcements (changes, new repos, etc): mirror-egi at egi.eu - TO BE SET-UP
- for discussion: FORUM - mirror-egi - TO BE SET-UP
Repos size estimates
- please see the content of the following file for insfo on available directories and their estimated sizes:
Export License
Mirroring
Using lftp
Depending on what repositories you want to mirror and the space you have available (see #Repos size estimates you can use something similar:
- to create the mirror for the first time:
# lftp -c mirror --verbose --exclude 'candidate/' --exclude '1/' --exclude '2/' http://repository.egi.eu/sw/production/umd/ ${LOCAL_DIR}
- to removes any local packages that no longer exist on the source, and only downloads newer package versions from the host mirror:
lftp -c mirror –-only-newer --ignore-time -–delete --exclude 'candidate/' --exclude '1/' --exclude '2/' http://repository.egi.eu/sw/production/umd/ ${LOCAL_DIR}
Using wget
- at the moment "wget -r" will not work, please use the lftp solution described above.
Using rsync
- A rsync server is availabe at rsync.athena.hellasgrid.gr
- it can be use with:
# rsync -avzH --delete --exclude cas/ --exclude sam/ --exclude umd/1/ --exclude umd/2/ --exclude umd/candidate/ rsync://rsync.athena.hellasgrid.gr/production/ <path_to_your_directory/
- or you can create a script to be run as a cron job:
# cat mirror_UMD.sh #!/bin/sh DATE=`/bin/date +%Y-%m-%d` OUTDIR='/tmp' MIRROR=/rep/repo/UMD/production/umd/ [ -d $OUTDIR ] || mkdir -p $OUTDIR OUTFILE=$OUTDIR/mirror-UMD-$DATE.txt /bin/nice /usr/bin/rsync --verbose --progress \ --stats --archive --partial -H --delete \ --exclude cas/ \ --exclude sam/ \ --exclude umd/1 \ --exclude umd/2 \ --exclude umd/candidate \ rsync.athena.hellasgrid.gr::production/ $MIRROR/ \ >> $OUTFILE