Notes about open source software, computers, other stuff.
This morning version 0.9-6 of the DatABEL R package was published on CRAN. This is only a minor update that consists of a few small changes and one bug fix. See the official announcement for more information.
DatABEL is an R package that allows users to access files with large matrices (of several gigabytes or more in size) in a fast and efficient manner. The package is mainly used for genome-wide association analyses using e.g. ProbABEL or OmicABEL.
It was quite a long time in the making and then a bunch of other stuff came in between, but I finally managed to release v0.4.4 of ProbABEL!
ProbABEL is a toolset for doing fast, memory (RAM) efficient genome-wide regression tests.
This is a bugfix release, but a major one for those who use the Cox proportional hazards regression module. Thanks to some of our users on the GenABEL forum, a serious bug leading to way to many NaN’s in the output was discovered, fixed and tested. This is one of the best examples of community collaboration I have seen in the GenABEL project.
Another bug fixed in this release is one that caused a failed install on MacOS X and FreeBSD. Again a bug reported on the forum by one of our users. Great work!
Uploads to Debian and the Ubuntu PPA are coming ASAP.
Now, let’s get ready for a new feature release, which will include p-value calculation (a long-standing feature request) and major speed-ups (implemented by former colleague Maarten Kooyman). Time to get to work ;-)!
During the Christmas holidays I released a new version of ProbABEL (v0.4.2). The official release announcement can be found here. ProbABEL is a toolset that allows running GWAS (Genome-Wide Association Studies) in a fast and efficient manner. It implements regression using the linear, logistic or Cox proportional hazards models.
This version is mostly a bug fix release. The most important user-visible change is the fact that the ‘official’ name for the wrapper script that runs a GWAS over a range of chromosomes is now called probabel instead of probabel.pl. This change was induced by my attempts to get ProbABEL packaged in the Debian Linux repositories. One of the warnings that occurred during the package creation process was a Lintian warning that said that scripts with ‘language extensions’ are not allowed. There are several reasons for that, but the one I found most compelling was the fact that the user shouldn’t be concerned with the programming/scripting language we used to write it in. Moreover, being ‘agnostic’ in this matter also allows us to write such a script in a different language.
Of course, we have left the original name in place (via a symlink) in order not to disrupt any current pipelines. If the user runs the script with the old name a warning appears, urging him/her to start using the new name and that the old name will be deprecated in the future.
In the mean time, ProbABEL v0.4.1 has been accepted in Debian (unstable) and as of today it is also available in Debian ‘testing’. Lots of thanks to the Debian Med team that helped me a lot in preparing the .deb package. Note that the package has been split up in probabel (architecture-dependent files) and probabel-examples (with architecture independent files: the examples). See the Debian Package Tracking System page for ProbABEL for more details of the package.
From Debian the package has trickled down to Ubuntu as well (Launchpad page here), so it will be available by default in the next Ubuntu release (14.04, a.k.a. Trusty Tahr).
Last week I released v0.4.1 of ProbABEL, just a few days after releasing v0.4.0, which contained a small, but irritating bug.
This release took quite some time to create, but features quite a few bug fixes, including a major one: for the first time since the filevector format was introduced somewhere in 2009/2010, the Cox proportional hazards regression module works with filevector/DatABEL files. This is a major step forward, because up till now we had to maintain two branches of code: trunk, with all the regular updates and improvements, and the old branch that contained the Cox PH module that was only capable of reading text files.
Another notable change is the incorporation of 
--mmscore option. The implementation was relatively easy, because part of the code was still there from previous versions; it was disabled however, because it didn’t deal with missing genotype data. Now it does. Using the LRT is also easier in the case of the 2df (or genotypic) genetic model, where using the Wald test is not straightforward.
The third user-visible change was a change in the [code]probabel.pl[/code] script that hides some of the details (e.g. the location of the files with genotype data) of running a regression for the user. Previously, using the -o option meant that the output file name was constructed from the name of the phenotype file, the argument of the -o option and a fixed extension that depends on the model(s) being run. Starting with v.0.4.0 this behaviour has changed. If the -o option is specified its argument is used as the start of the output file name, with only the fixed extension appended to it. This allows users to specify output in a different directory than the one where the phenotype file was created.
Packages for Ubuntu Linux (or one of its derivatives and probably also Debian) can be found in the GenABEL PPA (personal package archive). Previously we also released pre-compiled Windows binaries, but I’ve stopped doing that. They were never tested anyway, and I think there isn’t much demand for them anyway. Most people who do genome-wide association studies use Linux servers anyway.
Development of ProbABEL (and other members of the GenABEL suite) takes place on this R-forge page. If you are in search of an open source project to contribute to, feel free to contact us!
User support for the GenABEL suite can be found at our forum.
On New Year’s day I released version 0.3.0 of ProbABEL, almost two months after the previous release.
This update contains a few small bug fixes, but the most important feature of this new release is that thanks to the work of Maarten Kooyman we have a four to five-fold speed increase for the types of GWAS we run at work. In his e-mail to the GenABEL developers list he explains what he did to achieve this. The take-home-message of it is that you should always look for a suitable library for important tasks of any program you write. The old ProbABEL was based on a self-written matrix class that handled things like matrix multiplication and matrix subsetting. In the new release we make use of the Eigen C++ template library, maintained and developed by people who know much more about fast implementations of linear algebra than we do.
For those of you running Ubuntu Linux (or one of its derivatives and probably also Debian) I have set up the GenABEL PPA (personal package archive) where you can download and install the ProbABEL .deb package and stay up to date with future updates.
ProbABEL is also available for MS Windows, although we don’t have much experience running it on that platform.
Development of ProbABEL (and other members of the GenABEL suite) takes place on this R-forge page. If you are in search of an open source project to contribute to, feel free to contact us!
User support for the GenABEL suite can be found at our forum.
One of the first things I tried on my Raspberry Pi was to compile ProbABEL and see if it runs. Since the Raspberry Pi has an ARM processor I wasn’t sure whether our code was portable to it. Apparently it is! Compiling ProbABEL (r.1027 from SVN) took 30 minutes (single threaded of course) compared to 34 seconds on my Desktop (4 threads on an Intel Core i3 processor), but hey, it worked :-). Surprisingly it also passed all the checks in make check.
Once I hook up some more storage to device I will try to run ProbABEL on some real data. It will be interesting to see how much time it takes to run a linear regression on e.g. chromosome 22 of HapMap3 imputated data for a few hundred samples…
Will the Raspberry Pi be the next platform for GWAS ;-)?
On November 7th I released version 0.2.2 of ProbABEL, a set of programs that allow scientists (usually geneticists and epidemiologists) to run Genome-wide association studies (GWAS) in a fast and efficient way, even on machines with low amounts of RAM.
ProbABEL is part of the GenABEL suite, wich is a set of open source package for statistical genomics. Its main developer is Yurii Aulchenko, my former supervisor at the Erasmus Medical Centre.
This update contains a few small bug fixes and an update of the probabel.pl wrapper script that enables the use of chunked imputation output files as input. For more detailed changes, check the announcement.
For those of you running Ubuntu Linux (or one of its derivatives and probably also Debian) I have set up the GenABEL PPA (personal package archive) where you can download and install the ProbABEL .deb package and stay up to date with future updates.
ProbABEL is also available for MS Windows, although we don’t have much experience running it on that platform.
Development of ProbABEL (and other members of the GenABEL suite) takes place on this R-forge page. If you are in search of an open source project to contribute to, feel free to contact us!
User support for the GenABEL suite can be found at our forum.
For the ProbABEL project I’m working on I wanted to test for the presence of the Eigen header files using . Eigen is a C++ template library for linear algebra. It basically consists of a bunch of header files. On my PC the Eigen files are installed in /usr/include/eigen3/ since I used the Debian/Ubuntu libeigen3-dev package.
I first tried to test for the headers by simply including
AC_CHECK_HEADERS([eigen3/Eigen/Dense])
in configure.ac, but that didn’t work:
... checking eigen3/Eigen/Dense usability... no checking eigen3/Eigen/Dense presence... no checking for eigen3/Eigen/Dense... no ...
It turns out that you have to add the following lines to configure.ac:
AC_LANG_PUSH([C++]) AC_CHECK_HEADERS([eigen3/Eigen/Dense]) AC_LANG_POP([C++])
Now I get the following output when running ./configure:
... checking how to run the C++ preprocessor... g++ -E checking eigen3/Eigen/Dense usability... yes checking eigen3/Eigen/Dense presence... yes checking for eigen3/Eigen/Dense... yes ...
To go into a bit more detail, these are the errors in config.log without the AC_LANG options:
...
configure:4877: checking eigen3/Eigen/Dense usability
configure:4877: gcc -c -g -O2 -Wall conftest.c >&5
In file included from /usr/include/eigen3/Eigen/Core:35:0,
from /usr/include/eigen3/Eigen/Dense:1,
from conftest.c:67:
/usr/include/eigen3/Eigen/src/Core/util/Macros.h:188:5: error: unknown type name 'namespace'
/usr/include/eigen3/Eigen/src/Core/util/Macros.h:188:21: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
In file included from /usr/include/eigen3/Eigen/Dense:1:0,
from conftest.c:67:
/usr/include/eigen3/Eigen/Core:98:12: error: expected identifier or '(' before string constant
In file included from /usr/include/eigen3/Eigen/Dense:1:0,
from conftest.c:67:
/usr/include/eigen3/Eigen/Core:144:18: fatal error: cerrno: No such file or directory
...
It seems to be the case that AC_LANG is set to C instead of C++ and consequently compilation of autoconf’s test programme fails. The AC_LANG_PUSH option forces autoconf to use C++.
Thanks to this post on Stack Overflow I could solve this problem.
Yesterday I was working on ProbABEL, an open source package for running GWAS (genome-wide association studies). We use R-Forge and they provide us with a Subversion (SVN) server for revision control.
Some time ago we created a branch in which one of the co-developers is doing some major refactoring of the code. In the mean time I have been fixing bugs and adding new features to trunk. Now that the work in the refactoring branch comes to an end I thought it was high time to integrate the changes in trunk with the changes in the branch so that we can later promote the branch to trunk.
Since I had never done this before I decided to try the merge in the doc directory first, because I knew that in that directory nothing had changes since the branch was created, so all changes from trunk should be imported. At first I followed the SVN book instructions so I went into the doc dir in the branch and ran
$ svn merge ^/pkg/ProbABEL/doc
Unfortunately that didn’t work out. For some reasons conflicts appeared as wel as files that weren’t supposed to be there at all.
Thanks to Google and this blog post I found a solution. It boils down to explicitly telling SVN which revisions to use for the merge.
First I used
$ svn log | grep -C3 branch
to find out at which revision I created the branch:
------------------------------------------------------------------------ r864 | lckarssen | 2012-03-27 17:38:05 +0200 (Tue, 27 Mar 2012) | 1 line Creating ProbABEL branch for code refactoring
Next I went to trunk and ran
$ svn update At revision 987.
to find out at which revision trunk currently was. Back in the doc directory in the brach I ran
svn merge -r 864:987 ^/pkg/ProbABEL/doc
to merge al the changes since the branch was split off and it worked like a charm! All changes in trunk applied cleanly.
I then id the same for the other directories which also had changes in the branch. It turns out that when SVN find a conflict it is easier to postpone resolving the conflict because Emacs has a great SVN merge minor mode called SMerge! It highlights your changes vs. the incoming ones and allows you to select a resolution and move to the next conflict with a few easy keystrokes. After all conflicts have been resolved Emacs automtically removes the intermediate files SVN created and you are ready to commit.
© 2024 Lennart's weblog
Theme by Anders Norén — Up ↑