Merge branch 'development' version v0.2.6

CHANGELOG

-gitlab-mirrors v0.2.5
-* Fixed a critical bug with pagination when user can view more projects than 20.
-  * Had to patch upstream python-gitlab library for this functionality.
-* Updated documentation to reference my bugfix-edition of python-gitlab in the
+## gitlab-mirrors v0.2.6
+
+* *New Feature* command `ls-mirrors.sh`!
+* Converted `CHANGELOG` to markdown.
+* Fixed bug in preflight check where `merge_requests_enabled` was not being checked.
+* Fixed bug with bad formatted error output for booleans in preflight check.
+* Fixed bug with `lib/manage_gitlab_project.py` where group namespace resolution was not properly using API pagination.
+  * Required an upstream merge request to the `python-gitlab` library.
+* Fixed bug added `merge_requests_enabled` to `config.sh.SAMPLE`.
+
+--
+## gitlab-mirrors v0.2.5
+
+* Fixed a critical bug with pagination where API user can't view more projects than `20`.
+  * Had to merge request upstream `python-gitlab` library for this functionality.
+* Updated documentation to reference my `bugfix-edition` of `python-gitlab` in the
   prerequisites.
 
-gitlab-mirrors v0.2.4
-* Fixing critical git svn mirror bug.
+---
+## gitlab-mirrors v0.2.4
+
+* Fixing critical `git-svn` mirror bug.  SVN mirroring did not update properly prior to this version.
+
+---
+## gitlab-mirrors v0.2.3
+
+* Adding a final catch error message to `add_mirror.sh`.
+* `RELEASE` file for more consistent releases.
 
-gitlab-mirrors v0.2.3
-* Adding a final catch error message to add_mirror.sh
-* RELEASE file for more consistent releases.
+---
+## gitlab-mirrors v0.2.2
 
-gitlab-mirrors v0.2.2
-* Safer environment variable option checking for config.sh in add_mirror.sh command.
-* Added Features do README
+* Safer environment variable option checking for `config.sh` in `add_mirror.sh` command.
+* Added Features to `README`
+
+---
+## gitlab-mirrors v0.2.1
 
-gitlab-mirrors v0.2.1
 * CHANGELOG update
 
-gitlab-mirrors v0.2
-* Renamed project from git-mirrors to gitlab-mirrors.
+---
+## gitlab-mirrors v0.2
+
+* Renamed project from `git-mirrors` to `gitlab-mirrors`.
 * SVN repository mirroring now supported!
-* Project creation defaults can now be set in config.sh.
-* New delete_mirror.sh command.
+* Project creation defaults can now be set in `config.sh`.
+* New `delete_mirror.sh` command.
 * Colorized output enabled for all commands.
 * Better argument handling on all commands.
-* New options for add_mirror.sh, see ./add_mirror.sh -h
+* New options for `add_mirror.sh`, see `./add_mirror.sh -h`.
 * Knit and grit changes
-  * add_mirror.sh has more robust error checking.
-  * add_mirror.sh options can be out of order.  Now using getopt for
-    better argument handling.
-  * lib/create_gitlab_project.py has been renamed to 
-    lib/manage_gitlab_project.py
-  * manage_gitlab_project.py has a little better error handling.
-    Added optparse for better argument handling.
-
-git-mirrors v0.1.1
+  * `add_mirror.sh` has more robust error checking.
+  * `add_mirror.sh` options can be out of order.  Now using `getopt` for better argument handling.
+  * `lib/create_gitlab_project.py` has been renamed to `lib/manage_gitlab_project.py`.
+  * `manage_gitlab_project.py` has a little better error handling.
+    * Added `optparse` for better argument handling.
+
+---
+## git-mirrors v0.1.1
+
 * Minor update to documentation adding project URL to docs.
 
-git-mirrors v0.1
-* Initial project release.  Project gives admins the ability to have
-  mirrors of remote git repositories.
-* Comes with simple add_mirror.sh, update_mirror.sh, and git-mirrors.sh
+---
+## git-mirrors v0.1
+
+* Initial project release.  Project gives admins the ability to have mirrors of remote git repositories.
+* Comes with simple `add_mirror.sh`, `update_mirror.sh`, and `git-mirrors.sh`.
   * Note very little error checking on all commands.
-  * add_mirror.sh arguments must be in a specific order.
-* add_mirror.sh utilizes lib/create_gitlab_project.py to check for
-  a gitlab project.  If it doesn't exist then create it.  When the
-  project exists simply return the project remote git url over
-  ssh.
-  * create_gitlab_project.py has very little error checking.  Arguments
-    must be in a specific order.
+  * `add_mirror.sh` arguments must be in a specific order.
+* `add_mirror.sh` utilizes `lib/create_gitlab_project.py` to check for a gitlab project.  If it doesn't exist then create it.  When the project exists simply return the project remote "git url over ssh".
+  * `create_gitlab_project.py` has very little error checking.  Arguments must be in a specific order.
 * Project is fully documented with documentation.

README.md

@@ -6,10 +6,11 @@ The [gitlab-mirrors](https://github.com/sag47/gitlab-mirrors) project is designe
 ## Features
 
 * Mirror git and SVN repositories.
-* When adding a mirror if the project doesn't exist in GitLab then `add_mirrror.sh` will automatically create it.
-  * Specify initial project defaults when a project is created (e.g. issues enabled, wiki enabled, etc.)
-* Update a single mirror with the `update_mirror.sh` command.
-* Update all known mirrors with the `git-mirrors.sh` command.  Useful for adding to a `cron` job for updating all repositories on a regular schedule.
+* When adding a mirror if the project doesn't exist in GitLab it will be auto-created.
+  * Set project creation defaults (e.g. issues enabled, wiki enabled, etc.)
+* Update a single mirror.
+* Update all known mirrors.
+* List all known mirrors.
 
 
 ---
@@ -23,15 +24,16 @@ The [gitlab-mirrors](https://github.com/sag47/gitlab-mirrors) project is designe
 ---
 # License
 
-Created by Sam Gleske under [MIT License](LICENSE).  This project is meant to temporarily fill in a gap left by GitLab for managing remote git mirrors.  See the following user voice topics which made me create this project in the mean time.
-
-* [Mirror git/svn into repo.][3]
-* [Feature request -- Multi-Master mirroring][4]
+Created by Sam Gleske under [MIT License](LICENSE).  
 
 
 ---
 ## References
 
+This project is meant to temporarily fill in a gap left by GitLab for managing remote repository mirrors (namely git and svn).  See the following user voice topics which enabled me to create this project in the mean time.
+
+* [Mirror git/svn into repo.][3]
+* [Feature request -- Multi-Master mirroring][4]
 * [Git mirror][5]
 * [Git push all branches][6]
 * [Git update mirror][7]
@@ -39,6 +41,9 @@ Created by Sam Gleske under [MIT License](LICENSE).  This project is meant to te
 * [Git-svn Tutorial][9]
 * [Why git can't clone into a bare repository][10]
 * [How to convert a git repository from normal to bare][11]
+* [Converting CVS repositories to git][12]
+
+This project will still be useful after repository mirror support is native in GitLab because it allows admins to mirror massive amounts of repositories in an automated fashion.  When native mirror support is included this project will likely be refactored into a script to automate adding mirrors via the API.
 
 [1]: https://github.com/gitlabhq/gitlabhq/blob/master/doc/api/README.md
 [2]: https://github.com/sag47/python-gitlab
@@ -51,3 +56,4 @@ Created by Sam Gleske under [MIT License](LICENSE).  This project is meant to te
 [9]: http://trac.parrot.org/parrot/wiki/git-svn-tutorial
 [10]: http://stackoverflow.com/questions/12544318/why-git-svn-cannot-clone-a-bare-repo
 [11]: http://stackoverflow.com/questions/2199897/how-to-convert-a-git-repository-from-normal-to-bare
+[12]: http://stackoverflow.com/questions/7344941/converting-cvs-repositories-to-git

add_mirror.sh

@@ -164,9 +164,9 @@ function preflight() {
   if [ ! "${enable_colors}" = "true" ] && [ ! "${enable_colors}" = "false" ];then
     red_echo -n "enable_colors="
     yellow_echo -n "${enable_colors}"
-    red_echo -n "is not a valid option for enable_colors!  Must be "
+    red_echo -n " is not a valid option for enable_colors!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1
@@ -175,9 +175,9 @@ function preflight() {
   if [ ! "${issues_enabled}" = "true" ] && [ ! "${issues_enabled}" = "false" ];then
     red_echo -n "issues_enabled="
     yellow_echo -n "${issues_enabled}"
-    red_echo -n "is not a valid option for issues_enabled!  Must be "
+    red_echo -n " is not a valid option for issues_enabled!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1
@@ -186,9 +186,9 @@ function preflight() {
   if [ ! "${wall_enabled}" = "true" ] && [ ! "${wall_enabled}" = "false" ];then
     red_echo -n "wall_enabled="
     yellow_echo -n "${wall_enabled}"
-    red_echo -n "is not a valid option for wall_enabled!  Must be "
+    red_echo -n " is not a valid option for wall_enabled!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1
@@ -197,9 +197,9 @@ function preflight() {
   if [ ! "${wiki_enabled}" = "true" ] && [ ! "${wiki_enabled}" = "false" ];then
     red_echo -n "wiki_enabled="
     yellow_echo -n "${wiki_enabled}"
-    red_echo -n "is not a valid option for wiki_enabled!  Must be "
+    red_echo -n " is not a valid option for wiki_enabled!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1
@@ -208,9 +208,9 @@ function preflight() {
   if [ ! "${snippets_enabled}" = "true" ] && [ ! "${snippets_enabled}" = "false" ];then
     red_echo -n "snippets_enabled="
     yellow_echo -n "${snippets_enabled}"
-    red_echo -n "is not a valid option for snippets_enabled!  Must be "
+    red_echo -n " is not a valid option for snippets_enabled!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1
@@ -219,9 +219,20 @@ function preflight() {
   if [ ! "${public}" = "true" ] && [ ! "${public}" = "false" ];then
     red_echo -n "public="
     yellow_echo -n "${public}"
-    red_echo -n "is not a valid option for public!  Must be "
+    red_echo -n " is not a valid option for public!  Must be "
     yellow_echo -n "true"
-    red_echo -n "or "
+    red_echo -n " or "
+    yellow_echo -n "false"
+    red_echo "." 1>&2
+    STATUS=1
+  fi
+  #test merge_requests_enabled environment variable (must be bool)
+  if [ ! "${merge_requests_enabled}" = "true" ] && [ ! "${merge_requests_enabled}" = "false" ];then
+    red_echo -n "merge_requests_enabled="
+    yellow_echo -n "${merge_requests_enabled}"
+    red_echo -n " is not a valid option for merge_requests_enabled!  Must be "
+    yellow_echo -n "true"
+    red_echo -n " or "
     yellow_echo -n "false"
     red_echo "." 1>&2
     STATUS=1

config.sh.SAMPLE

@@ -8,7 +8,7 @@
 system_user="gitmirror"
 #The home directory path of the $system_user
 user_home="/home/${system_user}"
-#The repository directory where gitlab-mirrors will copies of mirrored repositories before pushing them to gitlab.
+#The repository directory where gitlab-mirrors will contain copies of mirrored repositories before pushing them to gitlab.
 repo_dir="${user_home}/repositories"
 #colorize output of add_mirror.sh, update_mirror.sh, and git-mirrors.sh commands.
 enable_colors=true
@@ -21,7 +21,7 @@ git_svn_additional_options="-s"
 
 #This is the Gitlab group where all project mirrors will be grouped.
 gitlab_namespace="Mirrors"
-#This is the web url of your Gitlab server. no trailing slash, just the protocol and server name.
+#This is the base web url of your Gitlab server.
 gitlab_url="https://gitlab.example.com"
 #Special user you created in Gitlab whose only purpose is to update mirror sites and admin the $gitlab_namespace group.
 gitlab_user="gitmirror"
@@ -38,4 +38,5 @@ issues_enabled=false
 wall_enabled=false
 wiki_enabled=false
 snippets_enabled=false
+merge_requests_enabled=false
 public=false

delete_mirror.sh

@@ -110,6 +110,8 @@ if ! ${quiet};then
   echo -n "Will DELETE "
   red_echo "${repo_dir}/${gitlab_namespace}/${project_name}"
   echo
+  red_echo "This action CANNOT be undone!"
+  echo
   echo -n "Are you sure you wish to delete project "
   yellow_echo -n "${gitlab_namespace}/${project_name}"
   echo -n "? (y/N): "

docs/management.md

@@ -18,6 +18,12 @@ Create an SVN repository mirror.
 
 The `--authors-file` option is an optional argument.
 
+### List all known mirrors
+
+    su - gitmirror
+    cd gitlab-mirrors
+    ./ls-mirrors.sh
+
 ### Delete a mirror
 
     su - gitmirror

docs/prerequisites.md

@@ -3,7 +3,7 @@
 ### Required software
 
 * [GitLab 6.x][1]
-* [python-gitlab][2] (note due to certain bugs you should obtain my fork [python-gitlab @ 6d6b270][8])
+* [python-gitlab @ 9c5e375][2]
 * [GNU coreutils][3]
 * [git 1.8.4][4]
 
@@ -15,9 +15,9 @@ If you plan on mirroring SVN repositories as well then you'll need the following
 python-gitlab
 
     yum install python-setuptools
-    git clone https://github.com/sag47/python-gitlab.git
+    git clone https://github.com/Itxaka/python-gitlab.git
     cd python-gitlab
-    git checkout 6d6b270e9829012b99d700eb1ca3802ef05f40b4
+    git checkout 9c5e375599a6d89ab1f4520224f47b43b40bcf9b
     python setup.py
 
 You can find the source to git at the [git-core project][5].  For instructions on other platforms see the [Getting Started - Installing Git section of the git book][6].
@@ -45,4 +45,3 @@ Next up is [Installation and Setup](installation.md).
 [5]: http://code.google.com/p/git-core/
 [6]: http://git-scm.com/book/en/Getting-Started-Installing-Git
 [7]: https://www.kernel.org/pub/software/scm/git/docs/git-svn.html
-[8]: https://github.com/sag47/python-gitlab/tree/bugfix-edition

lib/VERSION

-VERSION="v0.2.5"
+VERSION="v0.2.6"

lib/manage_gitlab_project.py

@@ -45,14 +45,16 @@ git=gitlab.Gitlab(gitlab_url,token_secret)
 
 def findgroup(gname):
   #Locate the group
-  found_group=False
-  for group in git.getGroups():
-    if group['name'] == gname:
-      return group
+  page=1
+  while len(git.getGroups(page=page)) > 0:
+    for group in git.getGroups(page=page):
+      if group['name'] == gname:
+        return group
+    page += 1
   else:
-    if not found_group:
-      print >> stderr, "Project namespace (user or group) not found or user does not have permission of existing group."
-      exit(1)
+    print >> stderr, "Project namespace (user or group) not found or user does not have permission of existing group."
+    print >> stderr, "gitlab-mirrors will not automatically create the project namespace."
+    exit(1)
 
 def findproject(gname,pname):
   page=1

ls-mirrors.sh

+#!/bin/bash
+#Created by Sam Gleske
+#MIT License
+#Created Sat Sep 14 15:50:13 EDT 2013
+#USAGE
+#  ./list-mirrors.sh
+
+#bash option stop on first error
+set -e
+
+#Include all user options and dependencies
+git_mirrors_dir="$(dirname "${0}")"
+. "${git_mirrors_dir}/config.sh"
+. "${git_mirrors_dir}/lib/VERSION"
+. "${git_mirrors_dir}/lib/functions.sh"
+
+PROGNAME="${0##*/}"
+PROGVERSION="${VERSION}"
+
+pushd "${repo_dir}/${gitlab_namespace}" &> /dev/null
+echo -n "Namespace: " 1>&2
+#red and bold combined
+red_echo "$(bold_echo -n "${gitlab_namespace}")" 1>&2
+ls -1 "${repo_dir}/${gitlab_namespace}" | while read mirror;do
+  pushd "${mirror}" &> /dev/null
+  if git config --get svn-remote.svn.url &> /dev/null;then
+    repo="$(git config --get svn-remote.svn.url)"
+  else
+    repo="$(git config --get remote.origin.url)"
+  fi
+  green_echo -n "${mirror}"
+  echo -n " -> "
+  yellow_echo "${repo}"
+  popd &> /dev/null
+done
+popd &> /dev/null
+