Category Archives: Open Source

Code Maat and my first public Dockerfile

Last year I read the excellent book “Your Code as a Crime Scene” by Adam Tornhill. It is an excellent book that uses the work from research on how to find weak spots in a code base by mining version history in order to fix broken designs, maintenance issues, and team productivity bottlenecks. I can highly recommend it to everyone.

In addition to the book Adam wrote an open source tool, Code Maat, in Clojure which he uses in his book to explain the theories and show it in a practical way on how to find the hot spots. With Code Maat you can analyse version history log files from Git, Perforce, Subversion and Mercurial. To run Code Maat you need to install and setup a couple of things on your developer PC. Since I finally started to work practically with Docker I created a Dockerfile. You can find it on my Github repository https://github.com/peternorrhall/code-maat and how to use it.

 

Mondrian in Action – a great introduction to data warehouse modeling (and Mondrian of course)

I am a newbie to data warehouse modeling and business intelligence, but thanks to the book “Mondrian in Action” (ISBN 978-1-61729-098-5) I now have a thorough understanding of the concept and what to think. I am far from an expert of course, but I am now on my way to create business intelligence model to analyze the domain of software development. Using the Pentaho platform which includes Mondrian makes it possible to create a model that enables to visualize the history and to make conclusions of the evolution of the software in order to make better software in the future.

The authors also recommends “The Data Warehouse Toolkit: The Definitive Guide to Dimensional Modeling, 3rd Edition” if you want to dig into the word of data warehouse modeling. It contains more in-depth knowledge once you get started with dw-modeling, but “Mondrian in Action” gave me a better introduction to the subject.

In coming posts I will add my findings around data warehouse modeling for software repository mining.

My first contribution to the Open Source community

Open Source

For the last decade I have used a lot of great and sometimes less great Open Source software in my attempts to build systems and software products. Most of them are published to be used under no obligations and others under restricted license terms. I am so grateful for all these work that others have done to make my life easier.

Github

So, I decided it’s payback time to create my first Open Source project. I decided to use Github as repository for the software. Since my current main employer is Extenda I created a Organization account (http://www.github.com/extenda).

Software Package Data Exchange®

The first project is really about Open Source. At Extenda we use Open Source libraries from different sources and we think we do oblige to the different licenses regarding paying license fees, provides license texts, source code etc. However we really want to 100% sure and we have looked at different software and services and discuss with lawyers to find a way to improve our process and assurance. During that process I discovered the standard Software Package Data Exchange® (SPDX® - https://spdx.org) which goal is to provide a standard way to describe the different licenses a product/system is using with a RDF report.

spdx-maven-plugin

Looking at their site there were no real community tools to generate such report. There were only commercial tools. Since Extenda uses maven site to generate product documentation including the list of libraries and the licenses we are using it felt natural to create a maven site plugin that could as part of the site genration phase create an spdx report file. The result is an spdx-maven-plugin (https://github.com/extenda/spdx-maven-plugin) and it is “published” and pushed to Github under Apache License 2.0. The current version is 0.0.1 and will still need a lot of work before it complies with the standard. I need to understand more how to write tests for maven plugins of this kind. Stay tuned…

Github Pages

I felt using the README.md would not be enough and since Github provides Github Pages (http://pages.github.com/) I created a “site” for organization Extenda (http://extenda.github.io/) and the plugin project (http://extenda.github.io/spdx-maven-plugin/). Even though the documentation (https://help.github.com/categories/20/articles) is rather straight forward I manage to misunderstand it partly because of my lack of practical experience with and Git and Github. But, finally I got it right so you should be able to do it as well :).

GitHub+BuildHive=True

The maven plugin is of course built using maven and so far with a very basic test suite (“Hello World” more or less). The goal is to create a set of tests, but I need to learn how to create those maven plugin tests.
In order to verify from an external part I wanted to make sure the build was successful in a different environment than my own. Therefor I created an account at Cloudbee’s service BuildHive in order to build and run junit tests for the project (https://buildhive.cloudbees.com/job/extenda/).

GitHub and BuildHive works well together

BuildHive is free of use for public projects on GitHub and it is really easy setup a build project for a repository on GitHub. Once verified the connection between BuildHive and GitHub, using the GitHub credentials, you just select “Add Git Repository”

Add Git Repository

and you will se the list of your personal and your organization’s repositories on GitHub.

List of GitHub repositories

To add a build select Enable and the Jenkins project will automatically be created for you.

Enabled BuildHive project

If you select the link you will be redirected to the Jenkins project build. You will probably notice that the project is already built and and probably has failed. BuildHive tries to figure out how to build the project, but you will probably need to configure the project. Select Configuration and enter the Shell Script for your project. Since spdx-maven-plugin is built using maven I entered mvn package (other build management solutions are supported as well – magic).

Enter mvn package as Shell Script

Select save and now the projects can be built successfully.

Succesfull build

So far BuildHive has no support the artifact(s) to a repository of some kind. It is primaraily a build and verification service. An option is to create an account on CloudBees (DEV@Cloud) which is a paid service with a free entry level where you can build your open source projects. It allows to use private settings to build and deploy your project. BuildHive (and CloudBees) uses web hooks to be notified once a new push has been made to the repository on GitHub and a new build is started automatically.

It is fascinating how easy it is with these integrated services to setup a versioned controlled project, creating a project site and connect it to a build server.