Portal Home > Knowledgebase > Web Hosting > Tech Tips > Using Gitolite to securely deploy a website with version control based on the GitHub Flow


Using Gitolite to securely deploy a website with version control based on the GitHub Flow




Setup a secure development environment using Gitolite for security and version control so you can deploy to a staging server and/or a production server by simply running git-push.   I've written a post-receive hook for Gitolite that is very useful for managing web applications and it is based off the GitHub Flow for deploying websites.

 

It can be expanded to work on very large website deployments... just create more repos for different sections or servers of your site.

 

*** This tutorial requires some understanding of SSH accounts and Git, there are some recommended reading documents linked at the bottom.

 

The basic idea of this workflow and how it works with the deployment script

1) You create a feature branch off of master
2) You do some work then git push
3) when you push the deployment script will deploy to $develop_path/$branch_name
4) you check the results then do some more work and repeat.
5) when you think your feature is ready for production you merge it into master and deploy to staging
6) the deployment script will then deploy into $staging_path
7) you make sure everything is a go on the staging web path
8) if a problem was found let your team know not to push to production until you have fixed it.
9) if everything checked out on the staging server then you deploy to live/production.

 

It is important to note if there was a problem on the staging server, it is tempting to roll back to the last commit but because others base their work off of master this can cause many problems, instead it is advised to let your team know not to deploy production while you push a fix.


Dont get discouraged by the amount of commands there are in this workflow... I simply wrote out step by step in order to give you an understanding of what commands are needed to make this workflow successful when working with a team of other contributors. Keep reading once done I will show some shortcuts with git aliases and repeat the same workflow using those aliases.

 

ExsysHost.com Web Hosting Accounts meet these requirements:

  • SSH access to the user account in which the web files will be owned
  • Git installed on server (on centos install EPEL repo -> yum install git)
  • For YOUR local computer install git (for windows we recommend installing msysgit )

 

Generate SSH keys

Start by generating your ssh public and private keys if you dont already have one (be sure to change the "key comment" to the name you will use for signing git commits eg. John): http://katsande.com/using-puttygen-to-generate-ssh-private-public-keys

 

Then login to your SSH account user (not root) which will own the web files

 

Setup Gitolite ready to serve git repos

Upload the ssh public key to your shell then inside the shell issue the following command:

git clone git://github.com/sitaramc/gitolite
gitolite/src/gl-system-install
gl-setup YourName.pub

 

it will launch an editor for the config. Edit your .gitolite.rc and update the config options to match the ones below

$GL_GITCONFIG_KEYS = "hooks\\..*";
$REPO_UMASK = 0022;

 

when done it will output the location of your ~/share/gitolite/hooks/common/ directory, create a "post-receive" file in that directory and chmod it +x and then paste the below code in, be sure to read the notes in the script and make any necessary changes so it will work in your environment, then save and exit. If you require any other post receive hooks such as the email post receive hook, just call it from the bottom of the file before the exit 0 with the arguments received from git example:  ./my-second-post-receive-hook "$@"

#!/bin/sh

# ***
# Author: Stephen Major (www.exsyshost.com)
#
# This post update script is intended for use as a deployment script for websites
# It supports deploying to live/production, staging, develop/testing
#
# it was designed for deploying all three on a single server but the functions
# could easily be broken up into separate scripts to have develop, staging and live 
# on different servers


#****
# IMPORTANT: edit the regex below to match the site/domain name being used, the regex
#            is broad and will match any domain.
#
#            this script will rewrite urls to match the develop and staging subdomains
#            when pushed to the remote... the urls are only rewritten on the staging
#            and develop file tree so the repo and the live site will remain in tact
#            this allows you to code urls for the main site and test under deploy
#            and staging.
#
#            replace just domain: [a-zA-Z0-9\-\.]+ with "domain" leave off . and tld
#            replace subdomains of domain: [a-zA-Z0-9\-\.]+domain
#            replace subdomains and domain: [a-zA-Z0-9\-\.]+?domain
#            then edit the tld list to those you wish to replace as well (com|org|net)
#
#            a tool to help debug regex: http://gskinner.com/RegExr/
#
#            leave the "cat" commands so they are not indented, not doing so will break them
#            you may have to update the path inside the cat command to match where your htpasswd file
#            is stored, if you dont need to password protect the directories then remove the entire cat command


# ***
# Do not edit below this line unless you know what you are doing
#

# Safety check
if [ -z "$GIT_DIR" ]; then
    echo "Don't run this script from the command line." >&2
    exit 1
fi

while read oldrev newrev refname
do
    case "$refname" in

        refs/tags/*)

            # We grab the path from the repo config, if the config item doesnt exist then we skip
            live_path=`git config --get hooks.live-path`
            [ -z "$live_path" ] && continue

            # ***
            # We update the production server when a new tag has been pushed
            # and the tag file has been updated
            #
            live_release=$(cat $staging_path/tag)
            live_release_tag=${refname##refs/tags/}

            if [ "$live_release_tag" == "$live_release" ]; then

                echo "Resetting production tree $live_path" >&2
                git gc
                git --work-tree=$live_path reset --hard $live_release

                echo "*** Release $live_release has been pushed to production" >&2
            fi

            ;;

        refs/heads/master)

            # We grab the path from the repo config, if the config path doesnt exist then we skip
            staging_path=`git config --get hooks.staging-path`
            [ -z "$staging_path" ] && continue

            # ***
            # we update the staging server with the latest push to master
            #
            echo "*** Resetting staging tree $staging_path" >&2
            rm -rf staging.git $staging_path
            git --work-tree=$staging_path clone -b master ./ staging.git

            staging_dir=$(dirname $staging_path)

cat > $staging_path/.htaccess << EOF
php_flag display_errors on

AuthGroupFile /dev/null
AuthType Basic
AuthUserFile $staging_dir/.htpasswd/public_html/.htpasswd
AuthName "Authorized Users Only"
require valid-user
EOF

            # we replace any urls for the live site with our staging urls
            staging_domain=$(basename $staging_dir)

            # matches any domains which end in .com .net .org add more if needed can be improved to be more specific to your site.
            find $staging_path/$branch -type f -exec perl -pi -e "s#[a-zA-Z0-9\-\.]+\.(com|net|org|gov|biz|COM|NET|ORG|GOV|BIZ)#${staging_domain}#g" {} +

            # There is some debate over the use of exec vs xargs... the method used is actually more efficient than xargs
            # above is the command we run on the files (many at a time). The "{}" gets replaced by file names. The + at the end 
            # of the command tells find to build 1 command for many filenames. 

            # From the find man page: "The command line is built in much the same way that xargs builds its command lines."
            # Thus it's possible to achieve your goal without using xargs -0, or -print0 . .

            ;;

        refs/heads/*)

            # We grab the path from the repo config, if the config path doesnt exist then we skip
            develop_path=`git config --get hooks.develop-path`
            [ -z "$develop_path" ] && continue

            zero="0000000000000000000000000000000000000000"
            branch=${refname##refs/heads/}

            if [ "$newrev" == "$zero" ]; then
                echo "*** Removing development tree $develop_path/$branch" >&2
                rm -rf $develop_path/$branch
            else

                # ***
                # we update the development server with the latest push to feature/topic branch
                # $staging_path/feature_name
                #
                echo "*** Resetting development tree $develop_path/$branch" >&2
                rm -rf develop.git $develop_path/$branch
                git --work-tree=$develop_path/$branch clone -b $branch ./ develop.git

                # we replace any urls for the live site with our dev url
                develop_dir=$(dirname $develop_path)
                develop_domain=$(basename $develop_dir)

cat > $develop_path/$branch/.htaccess << EOF
php_flag display_errors on
EOF

                # matches any absolute path that begins with / and not // and ends with a space after the appropriate closure
                # this helps prevent incorrect matches where a url might have a double or single quote within it.

                find $develop_path/$branch -type f -exec perl -pi -e "s#(href=\"|href=\'|src=\"|src=\'|url\(\"|url\(\'?)/{1}([^/].*?)(\"\s|\'\s|\'?\)\s|\"\)\s)#\1/${branch}/\2\3#g" {} +

                # matches any domains which end in .com .net .org add more if needed can be improved to be more specific to your site.
		find $develop_path/$branch -type f -exec perl -pi -e "s#[a-zA-Z0-9\-\.]+\.(com|net|org|gov|biz|COM|NET|ORG|GOV|BIZ)#${develop_domain}/${branch}#g" {} +

            fi
            ;;

        refs/remotes/*)
            # tracking branch
            ;;

        *)
            # Anything else (is there anything else?)
            echo "*** Post-receive hook: unknown type of update to ref $refname" >&2
            exit 1
            ;;
    esac
done

# --- Finished
exit 0

Create the admin repo on local machine

Now it is time to setup your Git/Gitolite admin repo so you have control over adding and removing users, repos/ projects etc:

git clone user@server:gitolite-admin

example:

git clone mysshuser@sr1.exsyshost.com:gitolite-admin

 

Configuring your access controls and setting up projects/repos

The gitolite-admin repository contains a directory named keydir/ and a file named conf/gitolite.conf. The keydir/ contains the SSH public keys for your users in files named in the convention of [username].pub. Each user of your git repositories will have their own file in keydir/ the username is for internal gitolite use, and needn't correspond with any shell username. The gitolite.conf file contains a set of access control rules that can be used to provide people access to a particular repository.

Here is a working example config:

# gitolite conf # *** # repo groups # @oss_repos = mywebsite android_project some_other_project # *** # user groups # @admins = YourName @ateam = @qa = @managers = @engineers = @staff = @admins @ateam @qa @managers @engineers # *** # Group our protected branches # @protected = master$ refs/tags # *** # @managers @engineers and @qa can read/write but not rewind or delete master or tags # @managers @engineers and @qa have full control over all other branches in @oss_repos # # @admins and @ateam have full control over all @oss_repos # repo @oss_repos RW @protected = @managers @engineers @qa - @protected = @managers @engineers @qa RW+ = @managers @engineers @qa @ateam @admins # *** # We want this repo to deploy to a web server so we set some deployment paths # we can optionally leave out some paths if we dont want to deploy to all three # develop, staging and production # # If for whatever reason you want to stop deploying on any given path don't just # delete the line instead set it to nothing eg. config deploy.develop_path = # # Notice we set these paths for a specific repo, do not use repo @group # instead duplicate this block for each repo you want to be deployed # with its own set of paths # repo mywebsite config hooks.develop-path = "/usr/myaccount/domains/develop.domain.com/html" config hooks.staging-path = "/usr/myaccount/domains/staging.domain.com/html" config hooks.live-path = "/usr/myaccount/domains/domain.com/html" # *** # We give @admins group full rights on admin repo # repo gitolite-admin RW+ = @admins

 

 

To create a new repository, just add it to the oss_repos group. All repositories will have "clone" or "remote" URLs in the following form:

sshuser@example.com:reponame

 

Setting up your web development repo

1) First edit your gitolite.conf and add your web development repo as outlined above

git clone user@domain.com:mywebsite
cd mywebsite

# Setting up the initial repo
echo v1.0.0 > tag
git add tag
git commit -am "added release version file"
git tag -a v1.0.0 -m "Creating our first official version"
# -u flag sets up tracking on the master branch...only needed on the first push
git push --tags -u origin master

 

Starting a new feature / topic

# first make sure your local master reflects all changes from the remote master
git checkout master
git fetch origin
git merge --ff origin/master

# then create a feature/topic branch with good description
git checkout -b my-features-name master

# push the topic to the server and setup tracking with -u flag
git push -u origin my-features-name

 

To work on a feature / topic:

git checkout my-features-name
git fetch origin
git rebase -p origin/master

# .... do stuff ....

git add -A
git commit -am "added and delted some stuff"

# clean up our commits every once in a while
git rebase -i origin/master

# once ready to push to develop tree for testing
# git up -- we fetch any changes from origin master and rebase our changes on top
git fetch origin
git rebase -p origin/master

# This deploys to the develop tree on the web server as well as back up our changes to the remote repo
git push

# now we check the dev branch on the web server and repeat for any more work needing to be done

 

Deploying to staging

git checkout master
git fetch origin
git merge --ff origin/master

git merge --no-ff topic

# Git up -- we fetch any changes from remote and rebase our changes on top
# preserving the merge with -p

git fetch origin
git rebase -p origin/master
git push

 

Deploying to production after staging has been cleared

git checkout master
echo v1.0.3 > tag
git commit -am "updated tag file to 1.0.3"
git tag -a v1.0.3 -m "Release v1.0.3 - Release name: Funky Monkey"

git fetch origin
git rebase -p origin/master
git push --tags origin master

 

To do a hotfix where the history doesnt need to show a new feature being merged in

git checkout -b hotfix master
git fetch origin
git rebase -p origin/master

# .... do stuff ....

git add -A
git commit -am "Fixed some bug"

git fetch origin
git rebase -i origin/master
git push origin master

# remove the local hotfix branch
git checkout master
git branch -D hotfix

# This will deploy your hotfix to staging, if the fix works on the staging server 
# then follow the procedure for deploying to production

 

Wow thats a lot of commands... Aliases to the resecue!

# Here are the aliases I use everyday that support this git workflow. Place these in your ~/.gitconfig.

[alias]
  c = commit -am
  a = add -A
  co = checkout
  new = !sh -c 'git checkout master && git fetch origin && git merge --ff origin/master && git checkout -b $1 master && git push -u origin $1' -
  up = !git fetch origin && git rebase -p origin/master
  ir = !git rebase -i origin/master
  next = !git add -A && git rebase --continue
  done = !git fetch origin && git rebase -p origin/master && git push
  staging = !sh -c 'git checkout master && git fetch origin && git merge --ff origin/master && git merge --no-ff $1 && git fetch origin && git rebase -p origin/master && git push' -
  deploy = !sh -c 'git checkout master && echo $1 > tag && git add tag && git commit -am \"updated tag file to $1\" && git tag -a $1 -m \"Release: $1  - Release name: $(cat release)\" && git fetch origin && git rebase -p origin/master && git push --tags origin master' -
  fix = !sh -c 'git checkout -b $1 master && git fetch origin && git rebase -p origin/master' -
  fixed = !git fetch origin && git rebase -i origin/master && git push origin master
  rmb = !sh -c 'git branch -D $1 && git push origin :$1' -
  l = log --graph --pretty=format:'%Cblue%h%d%Creset %ar %Cgreen%an%Creset %s'


# most of these aliases are self explanatory with a couple exceptions:

git new  new-branch-name  # will update master then create a new branch then push it to origin
git fix new-branch-name  # will update master then create a new branch
git staging branch-name
git deploy branch-name  # eg: git deploy v1.0.3
git rmb branch-name # will removed a branch both locally and remotely when it is no longer needed

 

A workflow with using the above aliases

git clone user@domain.com:myrepo.git
cd myrepo

# Setting up the initial repo only needs to be done when first creating the repo... not when cloning an existing one.
echo "Funky Monkey" > release
git a
git c "added releasename file"
git push -u origin master


# Starting a new feature / topic
git new my-features-name

# to work on a feautre / topic:
git co my-features-name
git up

# .... do stuff ....

git a
git c "Added some stuff"

# if you want to squash some stuff or 
# clean up commit messages you can use "git ir" at any time
git ir

# deploy to the develop/testing when everything is ready for testing
git done


# deploy to staging
git staging my-features-name


# Deploying to production after staging has been tested
git deploy v1.0.0

# When a feature / topic branch is not needed anymore
git rmb my-features-name


#
# To do a hotfix where the history doesnt need to show a new feature being merged in
#
git fix hotfix-123

# .... do stuff ....

git a
git c "Fixed some bug"

git fixed


# When the fix has been tested on staging follow the procedure for pushing to live / production
# once you are sure everything works you can safely delete the hotfix branch
git deploy v1.0.1
git branch -D hotfix


## Oh and one last thing... if you are moving up to say version 2.0 with a new release name
echo "My New Fancy Name" > release
git deploy v2.0.0

 

Recommeded reading:

Github Flowhttp://scottchacon.com/2011/08/31/github-flow.html

Streamline your git workflow with aliaseshttp://robots.thoughtbot.com/post/4747482956/streamline-your-git-workflow-with-aliases

git_tag_it_like_its_hotthttp://chrissloan.info/blog/git_tag_it_like_its_hott/

Rebasing merge commits in githttp://notes.envato.com/developers/rebasing-merge-commits-in-git/

git trickshttp://www.codingdomain.com/git/tricks/

http://www.syntevo.com/smartgit/index.html

http://code.google.com/p/msysgit/

http://code.google.com/p/tortoisegit/

 

 

 



Was this answer helpful?

Add to Favourites Add to Favourites    Print this Article Print this Article

Also Read