Flofloflo

Dear Lazyweb,

I just spent 5 hours learning that if you supply features to GBrowse 2 via DAS (for example with my custom python based pyDasD server), their names must not start with a dash (’-').

Otherwise the (weird, backwards compatibility) heuristic in Bio::Graphics::Panel->_do_add_track will silently eat the first feature, because it thinks it’s an option.
Hooray for in band data transmission, I’d say.

So long,
Flo

OpenGrok is an open source soure code browser.
It creates and index over code and allows you to quickly find and browse the annotated source.
To use it, you’d need a java webserver, and of course some code to run it on
OpenGrok’s creators use it for the Open Solaris Source - incidentially a good example - try searching for the defintion of strcmp.

OpenGrok isn’t difficult to set up - if you have a running Tomcat with a Java runtime 1.5, that is. Now, that’s a bit unussual for the shop I work with, so I decided to implement a central code browser using VMWare (Player) and a virtual Debian.

Step 1: Get yourself a Debian vm
Either download the vm found on Vmware’s VM Applicances site, or get the Debian Netinstall. I went with the Netinstall (and VBX Creator), since I wanted my VM to have two virtual harddrives, one for the Debian system and one for our code. This will allow me easily to replace the code harddisk later on with a bigger one (this turned out to be necessary a few years later).

After answering the basic installations questions ( manual package selection, nothing ), I suggest apt-get install ssh, so you can access the vm from your favourite SSH client - I use Putty when on windows.

Step 2: Uprade to Debian Etch
Since Tomcat 5 isn’t in the current stable Debian (sarge) we’ll switch to testing (Etch) instead.
#first, my favourite editor…
apt-get install joe
joe /etc/apt/sources.list
add contrib…

Step 3: Java

Now, power off the VM and make a backup. During the package creation for Java, you’ll save some largish files that will enlarge your VM. I’ll only sketch the step here.
First, we’ll need the most current Java Runtime.
#Webbrowser
apt-get install w3m
w3m http://java.sun.com/javase/downloads/index.jsp
Fetch the java se runtime

Now, we’ll create an installable java package
apt-get install java-package
su nonRootUser
fakeroot fakeroot make-jpkg jre-1_5_0_06-linux-i586.bin
(obviously change the file name for your java runtime)

SCP the created .deb file to a different machine, throw away the VM and start your backup.
Install the .dev here.

Step 4: Install Tomcat
apt-get install tomcat 5

Step 5: Fetch and install openGrok
Download openGrok and move the source.jar to /var/lib/tomcat5/webapps
Move openGrok.jar and lib to /opt/openGrok
Restart Tomcat (/etc/init.d/tomcat 5 restart )

Step 6: Mount the second harddrive
fdisk /dev/hdb # one partition is enough
mkfs.ext3 /dev/hdb5
joe /etc/fstab
…
mount /var/lib/tomcat5/webapps/source/code/
mkdir /var/lib/tomcat5/webapps/source/code/src
mkdir /var/lib/tomcat5/webapps/source/code/data

Step 6: Configure Tomcat
If your’re paranoid like me, Tomcat Apps need explicit permissions.
So place the following in your /etc/tomcat5/policies/04webapps.policy

grant codeBase “file:${catalina.home}/webapps/source/-” {
  permission java.util.PropertyPermission “subversion.native.library”, “read”;
permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;
  permission java.lang.RuntimePermission “loadLibrary.libsvnjavahl-1″;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl”;
  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
  permission java.util.PropertyPermission “catalina.home”, “read”;
permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.lockdir”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.writeLockTimeout”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.commitLockTimeout”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.mergeFactor”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.minMergeDocs”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.*”, “read”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp”, “read”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp/*”, “write”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp/*”, “delete”;
};

grant codeBase “file:${catalina.home}/webapps/source/WEB-INF/lib/-” {
  permission java.util.PropertyPermission “subversion.native.library”, “read”;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;

  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
permission java.util.PropertyPermission “catalina.home”, “read”;
permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
};

grant codeBase “file:${catalina.home}/webapps/source/WEB-INF/classes/-” {
permission java.util.PropertyPermission “subversion.native.library”, “read”;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;
  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
  permission java.util.PropertyPermission “catalina.home”, “read”;
  permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
};

Restart Tomcat

Step 7: Fetch code
I use python and subversion (command line) to fetch our repositories each night, but then I already had a webbased repository list at hand.
apt-get install python subversion
Basically, I just call svn update on each working copy (or delete it and do a new checkout if that fails for some reason).
I could be using pysvn, but I never got it to work with Umlauts in the svn repository. Apperantly it simply ignores the locale I set…

Step 8: Create a shellscript and a cron
joe /opt/fetchAndIndexCode.sh
python /opt/fetchCode.py
java -jar /opt/openGrok.jar -s //var/lib/tomcat5/webapps/source/code/src /var/lib/tomcat5/webapps/source/code/data

chmod +x fetchAndIndexCode.sh
crontab -e

Step 9:
Run fetchAndIndexCode

Step 10: Visit http://yourip:8888/source

Step 11: Make tomcat accessible via Port 80
This is optional - for something more secure, I suggest using Apache and mod_proxy

I’d love to make it accessible from /, but apperantly there are some hardcoded paths in the webapp - and it’s not annyoing enough for me to work on that. I simple placed a meta redirect in /var/lib/tomcat5/webapps/ROOT/index.html

That’s it, I hope this quick guide has helped you.
I’d also love to get some comments, expieriences, improvements, etc.

So long,
Flo

Some of us have to write scholarly articles/papers/thesisis from time to time.

Here’s a quick German guide how to not only cite Wikipedia correctly, but how to cite a page as it was on the day you looked at it.

Thou art all fair, my love; there is no spot in thee.

finding naked people apperantly is a valid research topic.

They not only got it published in the
Proceedings of the 4th European Conference on Computer Vision-Volume II in 1996,
no, they even got grant money for it!

To Flo’s Blog.

You’re probably either here because I directed you here myself,
or because google thought I had a good article ;-).

So, exepect the usual… ramblings, tech, human interest, plain old weirdness.

Comments are on, but I’ll have to aprove your first comment.

To contact me, simply leave a comment on one of my blog entries.
It will get to my inbox.

 The logo image was created using

a photo of a butterfly © Michael Jastremski for openphoto.net which is under the CC:Attribution-ShareAlike licence
and another one - the ice cube/cherry creation - , gracefully provided to me by my favourite Punk Rock Princess Miri