Git, gitweb, gitosis and git-daemon in harmony on Debian

Posted by mish

After a bit of mucking around, we've got a shared git repository going on escapegoat.org with read only access over http and git, and commit access using git (without having to make shell accounts. Oh and there is a nice browsable version of the repository aswell.

Installing Packages

Debian provides most of the packages for you, so start off with:

sudo apt-get install git-core gitweb git-daemon-run gitosis

Debian Etch

Gitosis requires a version of git >= 1.5, however the version in etch is 1.4.x. So we need to enable the backports repository. So add this line to your /etc/apt/sources.list (if it is not there already):

deb http://www.backports.org/debian etch-backports main contrib non-free

Then we need to update, remove a package that doesn't like to be upgraded, and install the required packages from backports:

sudo apt-get update
sudo apt-get remove python-setuptools
sudo apt-get install -t etch-backports python-setuptools git-core gitweb \
      gitosis git-daemon-run

gitosis - allowing commits without shell accounts

Note a lot of this is cribbed from this blog post about gitosis - see that for more detailed discussion.

So let's start by adding a user account to hold the repositories.

sudo adduser \
    --system \
    --shell /bin/sh \
    --gecos 'git version control' \
    --group \
    --disabled-password \
    --home /home/git \
    git

You may change the home path to suit your taste (the git-daemon-run package assumes a home of /var/cache/git, but I ended up changing the git-daemon-run params to fit with gitosis). A successful user creation will look similar to:

Adding system user 'git'...
Adding new group 'git' (211).
Adding new user 'git' (211) with group 'git'.
Creating home directory '/home/git'.

Then you need to copy your ssh public key to your server to add yourself as the first user. (See the above blog post for some help if you don't know how to). So if you copy your ssh key to the /tmp/ directory, you can run the following command to set up gitosis:

sudo -H -u git gitosis-init < /tmp/id_rsa.pub

Success looks like:

Initialized empty Git repository in ./
Initialized empty Git repository in ./

(Yes, two times)

For good measure, let's make sure the post-update hook is set executable. It doesn't always get set (problem with older setuptools):

sudo chmod 755 /home/git/repositories/gitosis-admin.git/hooks/post-update

Here some cool magic happens. Run this on your local machine:

git clone git@YOUR_SERVER_HOSTNAME:gitosis-admin.git
cd gitosis-admin

You will now have a gitosis.conf file and keydir/ directory:

~/dev/gitosis-admin (master) $ ls -l
total 8
-rw-r--r--   1 garry  garry  104 Nov 13 05:43 gitosis.conf
drwxr-xr-x   3 garry  garry  102 Nov 13 05:43 keydir/

This repository that you just cloned contains all the files (right now, only 2) needed to create repositories for your projects, add new users, and defined access rights. Edit the settings as you wish, commit, and push. Once pushed, gitosis will immediately make your changes take effect on the server. So we're using Git to host the configuration file and keys that in turn define how our Git hosting behaves. That's just plain cool.

At this point you may aswell carry on reading the excellent blog post I've copied from a bit already. If you read "Creating new repositories" and "Adding users" there and then come back.

Final gitosis set up

Gitosis can help you work well with gitweb and git-daemon if you add the right bits to the gitosis.conf file. git-daemon can be helped by gitosis automaticaly adding the 'git-daemon-export-ok' file in the repositories. gitweb can also be helped, and you can add an owner and description (which gitweb will show) in the gitosis.conf file. So here is a sample file:

[gitosis]
gitweb = yes
daemon = yes

[group gitosis-admin]
writable = gitosis-admin
members = user1@computer1 user2@computer2

[repo gitosis-admin]
gitweb = no
daemon = no

[group hyper-team]
writable = hyperactive
members = user1@computer1 user2@computer2

[repo hyperactive]
owner = escapegoat
description = A community news and reporting system

So this configuration allows gitweb and git daemon to access all repositories by default, though we have turned it off in the gitosis-admin repository. And we have a put a description and owner into the hyperactive section.

Allowing read only access using git-daemon

git-daemon is installed from the repositories and will run automatically, however it expects the repositories to be in /var/cache/git. I changed this by editing the file /etc/service/git-daemon/run to be

#!/bin/sh
exec 2>&1
echo 'git-daemon starting.'
#exec git-daemon --verbose --base-path=/var/cache /var/cache/git
exec git-daemon --base-path=/home/git/repositories/ --export-all

After this change, and a 

sudo /etc/init.d/git-daemon restart

I was able to download the repository by doing:

git clone git://git.escapegoat.org/hyperactive.git

Allowing read only access over http

To allow access over the web, ie

git clone http://git.escapegoat.org/git/hyperactive.git

we did

sudo mkdir /var/www/git.escapegoat.org/git/
sudo ln -s /home/git/repositories/hyperactive.git /var/www/git.escapegoat.org/git/

Now apache needs to be able to read the repository, and so we added the www-data user to the git group. To do this, you need to find out what groups www-data is in already and then update the list of groups. So the commands we used were:

groups www-data
sudo usermod -G www-data,svnowner,git www-data

Make sure you add any other groups www-data is already in to the list, and remove svnowner if you don't have that group. You may also need to do

cd /home/git/repositories/hyperactive.git
sudo git-update-server-info

after each checkin, but this should be done by /home/git/repositories/hyperactive.git/hooks/post-update. If not then make sure it is executable by

sudo chmod 755 /home/git/repositories/hyperactive.git/hooks/post-update

A user should now be able to do

git clone http://git.escapegoat.org/git/hyperactive.git

Browsing the repository using gitweb

gitweb is installed, and may work out of the box with a simple webserver set up, but we have several virtual hosts. So to put the gitweb.cgi file in the path we did

sudo ln -s /usr/lib/cgi-bin /var/www/git.escapegoat.org/gitweb
sudo cp /usr/share/gitweb/* /var/www/git.escapegoat.org/

(The second line copies the css and images required to make gitweb look nice). We also need to link in the repository. (And do the second and third line below if gitosis has not done it for you already).

sudo ln -s /home/git/repositories/hyperactive.git /var/cache/git/hyperactive
sudo touch /home/git/repositories/hyperactive.git/git-daemon-export-ok
sudo chown git:git /home/git/repositories/hyperactive.git/git-daemon-export-ok

And then it all seemed to work like magic. Go see for yourself at http://git.escapegoat.org/gitweb

Extra apache setup

For now I've done a little bit of extra stuff in the apache set up - specifically in /etc/apache2/sites-available/git.escapegoat.org

# redirect the / to /gitweb/ 
    RedirectMatch ^/$ /gitweb/

ScriptAlias /gitweb /usr/lib/cgi-bin/gitweb.cgi
<Location /gitweb >
            Options +ExecCGI
</Location>

So if you go to http://git.escapegoat.org/ you will be redirected to http://git.escapegoat.org/gitweb/ and be able to browse the repository in a nice way.

Sources

The two main articles I used were

So I have taken those two guides and made them work together - standing on the shoulders of other bloggers ...

Other useful articles include this more DIY guide by the good folks at riseup.net.

Installing Merb

Posted by yossarian

I took a crack at installing Merb today, just to see how it works out. It’s a very minimal Ruby framework which in my (admittedly not very scientific) testing appears to be a lot faster than Rails, especially under conditions of high concurrency.

To get it running, I followed the instructions at the Merb book which is currently a work in progress at http://4ninjas.org. A quick tip: due to a dependency on extlib 0.9.3, the sake edgy technique didn’t work for me at first. I got it all running happily by doing the following.

First, install the Git source-code management tool and the Debian build tools if you don’t already have them:

sudo apt-get install build-essential git-core
sudo gem install rack mongrel json erubis mime-types rspec hpricot mocha rubigen haml markaby mailfactory  english addressable templater

Update: in the last few weeks gem dependencies have changed and you need to ensure you’ve got some specific versions available:

sudo gem install ruby2ruby --version=1.1.8
sudo gem install ParseTree --version=2.1.1

You’ll also need to ensure that you’ve got the MySql headers available in order to build the datamapper MySql libraries.

sudo apt-get install libmysqlclient15off libmysqlclient15-dev
git clone git://github.com/sam/extlib.git  
git clone git://github.com/sam/do.git

cd extlib
rake install ; cd ..
cd do
cd data_objects
rake install ; cd ..
cd do_mysql  # || do_postgres || do_sqlite3
rake install

For whatever reason, DataMapper 0.9.4 has a dependency on Merb-core 0.9.3, so you’ll need to install it and then proceed to build the newest DataMapper:

sudo gem install merb-core

git clone git://github.com/sam/dm-core.git
git clone git://github.com/sam/dm-more.git

cd dm-core ; rake install ; cd ..
cd dm-more
rake install; cd ..

After that stuff, the

sudo gem install sake
sake -i 'http://edgy.4ninjas.org/edgy.sake'
sake edgy:install packages="merb-stack"

commands worked just fine for me.

Bonus: install CouchDb, the wicked distributed database system which is currently an Apache incubator project.

sudo apt-get install build-essential erlang libicu38 libicu-dev libmozjs-dev
wget http://www.apache.org/dist/incubator/couchdb/0.8.0-incubating/apache-couchdb-0.8.0-incubating.tar.gz
tar -xvzf apache-couchdb-0.8.0-incubating.tar.gz 
cd apache-couchdb-0.8.0-incubating/
./configure
make && sudo make install

You can set up the CouchDb datastore as a service, with its own user, like this (thanks to these instructions, slightly modified to avoid the creation of a “couchdb” home directory):

sudo useradd couchdb
sudo mkdir -p /usr/local/var/lib/couchdb
sudo chown -R couchdb /usr/local/var/lib/couchdb
sudo mkdir -p /usr/local/var/log/couchdb
sudo chown -R couchdb /usr/local/var/log/couchdb
sudo mkdir -p /usr/local/var/run
sudo chown -R couchdb /usr/local/var/run

sudo cp /usr/local/etc/init.d/couchdb /etc/init.d/
sudo update-rc.d couchdb defaults
sudo /etc/init.d/couchdb start

After that, go to http://localhost:5984/_utils/index.html and you can administer your new distributed datastore. Of course, you could just ignore CouchDb and use Merb with a more “normal” database server like Mysql or Postgres.

If any of the version dependencies change again please leave a comment and I’ll update these instructions until Merb 0.9.4 is a little easier to get.