Ceph read-only mirror on gitlab

The gitlab-mirrors scripts are installed to setup a a read-only Ceph mirror, updated hourly. It is used for permalinks such as src/osd/ClassHandler.cc#L170.

The gitlab-mirrors config.sh is as follows:

#Environment file

#
# gitlab-mirrors settings
#

#The user git-mirrors will run as.
system_user="gitmirror"
#The home directory path of the $system_user
user_home="/home/${system_user}"
#The repository directory where gitlab-mirrors will contain copies of mirrored
#repositories before pushing them to gitlab.
repo_dir="${user_home}/repositories"
#colorize output of add_mirror.sh, update_mirror.sh, and git-mirrors.sh
#commands.
enable_colors=true
#These are additional options which should be passed to git-svn.  On the command
#line type "git help svn"
git_svn_additional_options="-s"
#Force gitlab-mirrors to not create the gitlab remote so a remote URL must be
#provided. (superceded by no_remote_set)
no_create_set=false
#Force gitlab-mirrors to only allow local remotes only.
no_remote_set=false
#Enable force fetching and pushing.  Will overwrite references if upstream
#forced pushed.  Applies to git projects only.
force_update=false
#This option is for pruning mirrors.  If a branch is deleted upstream then that
#change will propagate into your GitLab mirror.  Aplies to git projects only.
prune_mirrors=false

#
# Gitlab settings
#

#This is the Gitlab group where all project mirrors will be grouped.
gitlab_namespace="Ceph"
#This is the base web url of your Gitlab server.
gitlab_url="http://workbench.dachary.org"
#Special user you created in Gitlab whose only purpose is to update mirror sites
#and admin the $gitlab_namespace group.
gitlab_user="gitmirror"
#Generate a token for your $gitlab_user and set it here.
gitlab_user_token_secret="$(head -n1 "${user_home}/private_token" 2> /dev/null || echo "")"
#Verify signed SSL certificates?
ssl_verify=false
#Push to GitLab over http?  Otherwise will push projects via SSH.
http_remote=false

#
# Gitlab new project default settings.  If a project needs to be created by
# gitlab-mirrors then it will assign the following values as defaults.
#

#values must be true or false
issues_enabled=false
wall_enabled=false
wiki_enabled=false
snippets_enabled=false
merge_requests_enabled=true
public=true

It is configured for mirrors to be accessible to unauthenticated users by default public=true and disables SSL verification because the gitlab does not have https anyway : ssl_verify=false.
Although the three easy steps contain many sub steps, they can be followed to completion without problems.
For fine control over what is mirrored (release branches, tags and pull requests from github), the following can be used instead:

#!/bin/bash
git fetch --force origin +refs/heads/*:refs/heads/* +refs/tags/*:refs/tags/* +refs/pull/*:refs/pull/*
git for-each-ref 'refs/pull/*/head' | cut -f2 | xargs --no-run-if-empty --max-args 1 git update-ref -d
git push --prune --force gitlab $(for branch in dumpling emperor firefly giant next master ; do echo +refs/heads/$branch:refs/heads/$branch ; done) +refs/pull/*:refs/heads/pull/*
git prune

The merge ref exists if the head of the pull request refs/pull/XXX/head can successfully be merged. They are removed to keep only one ref per pull request. All pull request refs are mapped under the refs/head so that they are noticed by GitLab CI. If they were kept under refs/pull, they would not be.
It is run hourly with:

$ cat ~/crontab
@hourly ( date ; /home/gitmirror/mirror.sh ) > /home/gitmirror/cron.log 2>&1
This entry was posted in ceph, gitlab. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>