OpenGrok, Ubuntu and Linux source tree

After all, thanks to our very long New Year holidays, I had a chance to install OpenGrok search and cross reference engine. The main reason is to have a tool to be able to efficiently browse Linux kernel source code. I use OpenGrok quite frequently, especially when there is a need to dive into Solaris/OpenSolaris internals. Frankly speaking, I haven’t seen anything else that comes close to it. And you know what? It’s a trivial game to install and run OpenGrok on your favorite Linux distro, BSD, Mac OS X or Solaris. But since I use Ubuntu as my cloud server all further details will be provided having it in mind. I will refer to the default installation since in 99.9% cases that would be the choice (I guess).

  1. First things first. Lets download OpenGrok and install additional packages if they’re missing.
  2. wget http://hub.opensolaris.org/bin/download/Project+opengrok/files/opengrok-0.11.1.tar.gz
    sudo apt-get install tomcat6 git exuberant-ctags
    
  3. By default, OpenGrok would use /var/opengrok so I chose to clone Linux source tree into /var/opengrok/src/linux-stable/ but you’re absolutely free to pick what suffice you most:
  4. sudo mkdir -p /var/opengrok/bin
    sudo mkdir /var/opengrok/lib
    sudo mkdir /var/opengrok/src && cd /var/opengrok/src
    sudo git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git linux-stable
    
  5. Now it’s time to untar the archive and deploy OpenGrok.
  6. tar -zxf ./opengrok-0.11.1.tar.gz
    sudo cp ./opengrok-0.11.1/lib/opengrok.jar /var/opengrok/lib/
    sudo cp ./opengrok-0.11.1/bin/OpenGrok /var/opengrok/bin/    
    sudo /var/opengrok/bin/OpenGrok deploy
    
  7. Now you should have source.war copied to your webapps directory (in my case that was /var/lib/tomcat6/webapps/source.war).
  8. Start Tomcat if it’s not and run the indexer.
  9. /etc/init.d/tomcat6 start
    sudo /var/opengrok/bin/OpenGrok index /opt/opengrok/src
    Loading the default instance configuration ...
    WARNING: OpenGrok generated data path /var/opengrok/data doesn't exist
      Attempting to create generated data directory ... 
    WARNING: OpenGrok generated etc path /var/opengrok/etc  doesn't exist
      Attempting to create generated etc directory ... 
      Creating default /var/opengrok/logging.properties ...
    

There is however a small catch. The indexer is started with -Xmx option set to 2048m and this is done on purpose. Since I only had 512MB of RAM I tried to reduce Xmx down to 128, 256 and 512 megabytes but all the time I got “java.lang.OutOfMemoryError: Java heap space” error. So in the end I temporarily increased the sizing of my cloud server to 4GB and only after that the indexer had finished successfully.
I’m happy with the overall result because even with 512MB OpenGrok works blazingly fast – come in and check. I’ll do my best to update it whenever there is a new Linux release. Since OpenGrok supports incremental updates that shouldn’t be a problem:

cd /var/opengrok/src/linux-stable && git pull
sudo /var/opengrok/bin/OpenGrok update
Posted on January 7, 2013 at 12:31 am by sergeyt · Permalink
In: Linux

7 Responses

Subscribe to comments via RSS

  1. Written by Sharath
    on January 24, 2013 at 10:03 am
    Reply · Permalink

    Hi,
    Thanks for the nice howto. I am trying to set up opengrok for my projects (on ubuntu 12.10) and I am running into issues.
    In your howto I noticed 2 additions:
    1. source.war also has to be copied to the /var/opengrok/lib
    2. docs folder also has to be copied to /var/opengrok

    After adding these 2 steps, I am able to get the opengrok page. But indexing is still failing with the following java exception:

    sharath@GanakaYantra:/var/opengrok/bin$ sudo ./OpenGrok index ../src/
    Loading the default instance configuration …
    Creating default /var/opengrok/logging.properties …
    Exception: org/apache/commons/jrcs/rcs/ParseException
    00:56:08 SEVERE: Unexpected Exception
    java.lang.NoClassDefFoundError: org/apache/commons/jrcs/rcs/ParseException
    at org.opensolaris.opengrok.history.RepositoryFactory.(RepositoryFactory.java:41)
    at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:421)
    Caused by: java.lang.ClassNotFoundException: org.apache.commons.jrcs.rcs.ParseException
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    … 2 more

    Can you please help me figure out what is going wrong? Thanks in advance.

    • Written by sergeyt
      on January 24, 2013 at 10:48 am
      Reply · Permalink

      Hi Sharath,

      Just a wild guess. Try to run OpenGrok index not from /var/opengrok/bin where you copied it but right from its untared directory, i.e. /some/path/opengrok-0.11.1/bin/OpenGrok. Looks like it’s not just enough to copy OpenGrok shell script to /var/opengrok/bin but some java class it depends upon must also be moved to a proper place. Will double check that later today.

      Thank you.

    • Written by sergeyt
      on January 24, 2013 at 8:50 pm
      Reply · Permalink

      Hi Sharath,

      Just curios, what version of OpenGrok did you use? I found the following BUG description that looks pretty much the same what you had – http://defect.opensolaris.org/bz/show_bug.cgi?id=19233
      However, it’s been opened against OpenGrok 0.11, whilst the latest version is 0.11.1

  2. Written by Sharath
    on January 25, 2013 at 12:26 am
    Reply · Permalink

    Hi Sergey,
    Thanks for your response. Yes, the description in the bug report is probably the same as what I am getting. I am running the latest version 0.11.1.
    I am running tomcat7(I had to make a few changes to the open grok scripts) on ubuntu 12.10.
    Would this make any difference?
    And running directly from the untar’d directory had the same error :(.

    Thanks,
    Sharath

    • Written by sergeyt
      on January 25, 2013 at 8:38 am
      Reply · Permalink

      Hi Sharath,
      That’s a bummer. I will try to upgrade from 12.04 to 12.10 later. Hopefully I will get the same error ;-)

      Cheers,
      Sergey

    • Written by sergeyt
      on January 26, 2013 at 7:22 pm
      Reply · Permalink

      Hi Sharath,

      I’ve found how to fix this issue. Not sure what is the root cause since everything worked perfectly well on Ubuntu 11.10, however after upgrade to 12.04 it stopped.
      The error “java.lang.NoClassDefFoundError” means that a definition of the class could be found. See Java docs for more information.
      If you unzip opengrok.jar and look at ./META-INF/MANIFEST.MF you’d see the following:

      Manifest-Version: 1.0
      Ant-Version: Apache Ant 1.8.0
      Created-By: 1.6.0_26-b03 (Sun Microsystems Inc.)
      Main-Class: org.opensolaris.opengrok.index.Indexer
      Class-Path: lib/ant.jar lib/bcel-5.2.jar lib/jrcs.jar lib/servlet-api.
      jar lib/swing-layout-0.9.jar lib/lucene-core-3.0.2.jar lib/lucene-spe
      llchecker-3.0.2.jar lib/derbyclient.jar lib/derby.jar

      That means that you need to have lib/jrcs.jar and preferably all other jar files adjacent to opengrok.jar

      That worked for me and hope it would help you as well.

      Cheers,
      Sergey

  3. Written by kartikeyan
    on September 13, 2013 at 9:49 am
    Reply · Permalink

    The following worked for me.
    The howto has to be corrected
    ==
    sudo cp -r opengrok-0.11.1/lib/* /var/opengrok/lib/
    ==
    This copies source.war opengrok.jar and lib/*.jar to /var/opengrok/lib.

Subscribe to comments via RSS

Leave a Reply