Flofloflo

OpenGrok is an open source soure code browser.
It creates and index over code and allows you to quickly find and browse the annotated source.
To use it, you’d need a java webserver, and of course some code to run it on
OpenGrok’s creators use it for the Open Solaris Source - incidentially a good example - try searching for the defintion of strcmp.

OpenGrok isn’t difficult to set up - if you have a running Tomcat with a Java runtime 1.5, that is. Now, that’s a bit unussual for the shop I work with, so I decided to implement a central code browser using VMWare (Player) and a virtual Debian.

Step 1: Get yourself a Debian vm
Either download the vm found on Vmware’s VM Applicances site, or get the Debian Netinstall. I went with the Netinstall (and VBX Creator), since I wanted my VM to have two virtual harddrives, one for the Debian system and one for our code. This will allow me easily to replace the code harddisk later on with a bigger one (this turned out to be necessary a few years later).

After answering the basic installations questions ( manual package selection, nothing ), I suggest apt-get install ssh, so you can access the vm from your favourite SSH client - I use Putty when on windows.

Step 2: Uprade to Debian Etch
Since Tomcat 5 isn’t in the current stable Debian (sarge) we’ll switch to testing (Etch) instead.
#first, my favourite editor…
apt-get install joe
joe /etc/apt/sources.list
add contrib…

Step 3: Java

Now, power off the VM and make a backup. During the package creation for Java, you’ll save some largish files that will enlarge your VM. I’ll only sketch the step here.
First, we’ll need the most current Java Runtime.
#Webbrowser
apt-get install w3m
w3m http://java.sun.com/javase/downloads/index.jsp
Fetch the java se runtime

Now, we’ll create an installable java package
apt-get install java-package
su nonRootUser
fakeroot fakeroot make-jpkg jre-1_5_0_06-linux-i586.bin
(obviously change the file name for your java runtime)

SCP the created .deb file to a different machine, throw away the VM and start your backup.
Install the .dev here.

Step 4: Install Tomcat
apt-get install tomcat 5

Step 5: Fetch and install openGrok
Download openGrok and move the source.jar to /var/lib/tomcat5/webapps
Move openGrok.jar and lib to /opt/openGrok
Restart Tomcat (/etc/init.d/tomcat 5 restart )

Step 6: Mount the second harddrive
fdisk /dev/hdb # one partition is enough
mkfs.ext3 /dev/hdb5
joe /etc/fstab
…
mount /var/lib/tomcat5/webapps/source/code/
mkdir /var/lib/tomcat5/webapps/source/code/src
mkdir /var/lib/tomcat5/webapps/source/code/data

Step 6: Configure Tomcat
If your’re paranoid like me, Tomcat Apps need explicit permissions.
So place the following in your /etc/tomcat5/policies/04webapps.policy

grant codeBase “file:${catalina.home}/webapps/source/-” {
  permission java.util.PropertyPermission “subversion.native.library”, “read”;
permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;
  permission java.lang.RuntimePermission “loadLibrary.libsvnjavahl-1″;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl”;
  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
  permission java.util.PropertyPermission “catalina.home”, “read”;
permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.lockdir”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.writeLockTimeout”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.commitLockTimeout”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.mergeFactor”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.minMergeDocs”, “read”;
  permission java.util.PropertyPermission “org.apache.lucene.*”, “read”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp”, “read”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp/*”, “write”;
  permission java.io.FilePermission “/var/lib/tomcat5/temp/*”, “delete”;
};

grant codeBase “file:${catalina.home}/webapps/source/WEB-INF/lib/-” {
  permission java.util.PropertyPermission “subversion.native.library”, “read”;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;

  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
permission java.util.PropertyPermission “catalina.home”, “read”;
permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
};

grant codeBase “file:${catalina.home}/webapps/source/WEB-INF/classes/-” {
permission java.util.PropertyPermission “subversion.native.library”, “read”;
  permission java.lang.RuntimePermission “loadLibrary.svnjavahl-1″;
  permission java.util.PropertyPermission “disableLuceneLocks”, “read”;
  permission java.util.PropertyPermission “catalina.home”, “read”;
  permission java.util.PropertyPermission “java.io.tmpdir”, “read”;
};

Restart Tomcat

Step 7: Fetch code
I use python and subversion (command line) to fetch our repositories each night, but then I already had a webbased repository list at hand.
apt-get install python subversion
Basically, I just call svn update on each working copy (or delete it and do a new checkout if that fails for some reason).
I could be using pysvn, but I never got it to work with Umlauts in the svn repository. Apperantly it simply ignores the locale I set…

Step 8: Create a shellscript and a cron
joe /opt/fetchAndIndexCode.sh
python /opt/fetchCode.py
java -jar /opt/openGrok.jar -s //var/lib/tomcat5/webapps/source/code/src /var/lib/tomcat5/webapps/source/code/data

chmod +x fetchAndIndexCode.sh
crontab -e

Step 9:
Run fetchAndIndexCode

Step 10: Visit http://yourip:8888/source

Step 11: Make tomcat accessible via Port 80
This is optional - for something more secure, I suggest using Apache and mod_proxy

I’d love to make it accessible from /, but apperantly there are some hardcoded paths in the webapp - and it’s not annyoing enough for me to work on that. I simple placed a meta redirect in /var/lib/tomcat5/webapps/ROOT/index.html

That’s it, I hope this quick guide has helped you.
I’d also love to get some comments, expieriences, improvements, etc.

So long,
Flo