I recently noticed that I couldn’t view my Bazaar repositories anymore using the Loggerhead/Apache combination. It turns out that my previous post isn’t completely correct anymore as there seems to be a bug in Loggerhead for Ubuntu 12.04 and 12.10. I’ve updated the old post with a work-around and now everything works as expected again.
Tag: programming (Page 3 of 3)
Recently I added the following lines to my ~/.emacs file:
;; Don't insert tabs when indenting regions (setq-default indent-tabs-mode nil) |
The idea behind disabling the indent-tabs-mode was that (especially) while programming I want any tabs to be converted to spaces. Since different people have different settings for a tab width this seemed like a good choice.
However, once I opened a Makefile I ran into trouble. In a Makefile tabs are a requirement, not an option. Since all my tabs were converted to spaces the moment I saved a Makefile compiling became a nightmare. To solve this problem I added the following to my ~/.emacs file, after the aforementioned statement:
(add-hook 'makefile-mode-hook (lambda () indent-tabs-mode t)) |
This enables tabs again for modes that involve Makefiles.
Related Images:
On November 7th I released version 0.2.2 of ProbABEL, a set of programs that allow scientists (usually geneticists and epidemiologists) to run Genome-wide association studies (GWAS) in a fast and efficient way, even on machines with low amounts of RAM.
ProbABEL is part of the GenABEL suite, wich is a set of open source package for statistical genomics. Its main developer is Yurii Aulchenko, my former supervisor at the Erasmus Medical Centre.
This update contains a few small bug fixes and an update of the probabel.pl wrapper script that enables the use of chunked imputation output files as input. For more detailed changes, check the announcement.
For those of you running Ubuntu Linux (or one of its derivatives and probably also Debian) I have set up the GenABEL PPA (personal package archive) where you can download and install the ProbABEL .deb package and stay up to date with future updates.
ProbABEL is also available for MS Windows, although we don’t have much experience running it on that platform.
Development of ProbABEL (and other members of the GenABEL suite) takes place on this R-forge page. If you are in search of an open source project to contribute to, feel free to contact us!
User support for the GenABEL suite can be found at our forum.
Related Images:
For the ProbABEL project I’m working on I wanted to test for the presence of the Eigen header files using . Eigen is a C++ template library for linear algebra. It basically consists of a bunch of header files. On my PC the Eigen files are installed in /usr/include/eigen3/ since I used the Debian/Ubuntu libeigen3-dev package.
I first tried to test for the headers by simply including
AC_CHECK_HEADERS([eigen3/Eigen/Dense])
in configure.ac, but that didn’t work:
... checking eigen3/Eigen/Dense usability... no checking eigen3/Eigen/Dense presence... no checking for eigen3/Eigen/Dense... no ...
It turns out that you have to add the following lines to configure.ac:
AC_LANG_PUSH([C++]) AC_CHECK_HEADERS([eigen3/Eigen/Dense]) AC_LANG_POP([C++])
Now I get the following output when running ./configure:
... checking how to run the C++ preprocessor... g++ -E checking eigen3/Eigen/Dense usability... yes checking eigen3/Eigen/Dense presence... yes checking for eigen3/Eigen/Dense... yes ...
To go into a bit more detail, these are the errors in config.log without the AC_LANG options:
... configure:4877: checking eigen3/Eigen/Dense usability configure:4877: gcc -c -g -O2 -Wall conftest.c >&5 In file included from /usr/include/eigen3/Eigen/Core:35:0, from /usr/include/eigen3/Eigen/Dense:1, from conftest.c:67: /usr/include/eigen3/Eigen/src/Core/util/Macros.h:188:5: error: unknown type name 'namespace' /usr/include/eigen3/Eigen/src/Core/util/Macros.h:188:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token In file included from /usr/include/eigen3/Eigen/Dense:1:0, from conftest.c:67: /usr/include/eigen3/Eigen/Core:98:12: error: expected identifier or '(' before string constant In file included from /usr/include/eigen3/Eigen/Dense:1:0, from conftest.c:67: /usr/include/eigen3/Eigen/Core:144:18: fatal error: cerrno: No such file or directory ...
It seems to be the case that AC_LANG is set to C instead of C++ and consequently compilation of autoconf’s test programme fails. The AC_LANG_PUSH option forces autoconf to use C++.
Thanks to this post on Stack Overflow I could solve this problem.
Related Images:
Yesterday I was working on ProbABEL, an open source package for running GWAS (genome-wide association studies). We use R-Forge and they provide us with a Subversion (SVN) server for revision control.
Some time ago we created a branch in which one of the co-developers is doing some major refactoring of the code. In the mean time I have been fixing bugs and adding new features to trunk. Now that the work in the refactoring branch comes to an end I thought it was high time to integrate the changes in trunk with the changes in the branch so that we can later promote the branch to trunk.
Since I had never done this before I decided to try the merge in the doc directory first, because I knew that in that directory nothing had changes since the branch was created, so all changes from trunk should be imported. At first I followed the SVN book instructions so I went into the doc dir in the branch and ran
$ svn merge ^/pkg/ProbABEL/doc
Unfortunately that didn’t work out. For some reasons conflicts appeared as wel as files that weren’t supposed to be there at all.
Thanks to Google and this blog post I found a solution. It boils down to explicitly telling SVN which revisions to use for the merge.
First I used
$ svn log | grep -C3 branch
to find out at which revision I created the branch:
------------------------------------------------------------------------ r864 | lckarssen | 2012-03-27 17:38:05 +0200 (Tue, 27 Mar 2012) | 1 line Creating ProbABEL branch for code refactoring
Next I went to trunk and ran
$ svn update At revision 987.
to find out at which revision trunk currently was. Back in the doc directory in the brach I ran
svn merge -r 864:987 ^/pkg/ProbABEL/doc
to merge al the changes since the branch was split off and it worked like a charm! All changes in trunk applied cleanly.
I then id the same for the other directories which also had changes in the branch. It turns out that when SVN find a conflict it is easier to postpone resolving the conflict because Emacs has a great SVN merge minor mode called SMerge! It highlights your changes vs. the incoming ones and allows you to select a resolution and move to the next conflict with a few easy keystrokes. After all conflicts have been resolved Emacs automtically removes the intermediate files SVN created and you are ready to commit.
Related Images:
Introduction
Loggerhead is a webfrontend for Bazaar (usually abbreviated as bzr) repositories. Bazaar is a so-called distributed version control system. So, if you have one or more bzr repositories you can use Loggerhead to look at the files, read the change logs and see the differences between revisions from within your web browser.
The main purpose of this post is to document the steps needed to configure Loggerhead and Apache to work together to publish your bzr repos on the web. The need for this post arose when I tried to get this setup to work and found that there isn’t a lot of documentation on how to get this done and most of it is out of date. The folowing steps were performed on a Linux server with Ubuntu 11.04 installed.
Basic Loggerhead configuration
First, let’s install Loggerhead:
$ aptitude install loggerhead |
Although the package is called loggerhead, the actual binary that is run is called serve-branches. The package provides start and stop scripts for the service (/etc/init.d/loggerhead
), but to start successfully the file /etc/serve-branches.conf
needs to exist. Older documentation I found on the web refers to the file /etc/loggerhead.conf
, but that file has become obsolete.
The serve-branches.conf
file contains three lines:
served_branches=/home/bzr prefix= port=8080 |
Here, the line served_branches
points to the directory under which you store your bzr repositories. Each repo needs to be stored in its own directory. So in this example all the repos are in subdirectories of /home/bzr/
.
You have to make sure that loggerhead can read the files in that directory. Loggerhead runs as the loggerhead
user but I made the directories readable and accessible by all users:
$ chmod -R a+rx /home/bzr/ |
If you now start Loggerhead:
$ service start loggerhead |
you should be able to visit http://localhost:8080 in your browser and see your repositories.
NOTE for Ubuntu 12.04 and 12.10: There seems to be a bug in Loggerhead for these Ubuntu releases (see the link to the Launchpad bug report at the end of this post). In order to start the Loggerhead daemon correctly in these Ubuntu releases the file /etc/init.d/loggerhead must be edited. The line
start-stop-daemon -p $PIDFILE -S --startas /usr/bin/serve-branches --chuid loggerhead --make-pidfile --background --chdir $served_branches -- --prefix=$prefix --port=$port --host=$host --log-folder /var/log/loggerhead 2>/dev/null
must be changed to
start-stop-daemon -p $PIDFILE -S --startas /usr/bin/serve-branches --chuid loggerhead --make-pidfile --background -- file://$served_branches --prefix=$prefix --port=$port --log-folder /var/log/loggerhead 2>/dev/null
Once this is done run restart the Loggerhead service as stated above and it should work again (if you run Loggerhead behind an Apache webserver as detailed below, don’t forget to restart Apache also).
How to publish your branch to this shared repository?
Now that our repository browser is set up, how do we publish our branches to it so that there actually is something to browse through? Here is how you publish your branch to the server, assuming that you are in a directory that contains a branch and want to publish it as myTests
:
$ bzr push --create-prefix sftp://username@server.yourdomain.com/home/bzr/myTests |
As you probably suspected, the --create-prefix
option is only necessary the first time you push your branch. Note that we are using sftp here. Loggerhead itself doesn’t allow writes to the published repos. So, every user that want to push his/her changes to this repository needs to have sftp access to the /home/bzr
directory. I solved that problem by adding all people that need to be able to push changes to a Linux group called vcs (for Version Control Systems) and then set the primary group of /home/bzr/
to vcs as well as giving group write permissions to this directory:
$ ls -ld /home/bzr/ drwxrwxr-x 4 root vcs 4096 2011-08-16 23:10 /home/bzr/ |
Adding Apache to the mix
In my case I already have a web server (Apache) running on port 80. Since I’d rather not open yet another port (8080 in this case) on my router, I wanted to use Apache to hand over the requests for bzr page to Loggerhead. For that I needed to install the following packages:
$ aptitude install python-pastedeploy |
Next, I needed to change the contents of the /etc/serve-branches.conf
file to this:
served_branches=/home/bzr prefix=/bzr port=8080 |
The prefix indicates the location in the URL where Apache will serve the repos. In this case that will be http://server.yourdomain.com/bzr/.
And finally I needed to configure Apache. First, make sure that the proxy
and proxy-http
modules are loaded:
$ a2enmod proxy proxy_http |
Next, create a file /etc/apache/conf.d/sites-available/loggerhead
with the following contents:
# Configuration for browsing of Bazaar repos. Make sure loggerhead is running. <Location "/bzr/"> ProxyPass http://127.0.0.1:8080/ ProxyPassReverse http://127.0.0.1:8080/ </Location> |
Note that Loggerhead and Apache run on the same host, that’s why I set the IP to 127.0.0.1.
Finally it’s time to enable the site and restart Apache:
$ a2ensite loggerhead $ service apache2 restart |
Now it should be possible to browse your repos at http://server.yourdomain.com/bzr/. Note the final /
, it’s important.
Securing access with an LDAP connection
I have stored all my Unix user and group information in an LDAP server. To make sure that only people in the Unix group vcs
are allowed access to the loggerhead pages, change the Apache configuration file loggerhead
to the following:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | # Configuration for browsing of Bazaar repos. Make sure loggerhead is running. <Location "/bzr/"> ProxyPass http://127.0.0.1:8080/ ProxyPassReverse http://127.0.0.1:8080/ # LDAP authentication AuthType Basic AuthName "Karssen.org VCS users" AuthBasicProvider ldap AuthLDAPURL "ldap://ldap.yourdomain.com/ou=Users,dc=yourdomain,dc=com?uid" AuthLDAPGroupAttribute memberUid AuthLDAPGroupAttributeIsDN off Order Allow,Deny Allow From All Require ldap-group cn=vcs,ou=Groups,dc=yourdomain,dc=com </Location> |
Lines 11 and 12 are needed because the vcs
group is not an LDAP group. I store my Unix (POSIX) groups in a separate OU in the LDAP tree (see line 15).
Don’t forget to restart Apache after making these changes.
References
- Bazaar documentation on Apache integration
- Launchpad bug 563118 describing the unexpected change from
loggerhead.conf
toserve-branches.conf
- Apache, LDAP and groups
- Launchpad bug report for Ubuntu 12.04 and 12.10