Liquibase

I’ve been working to get Liquibase into our development process.  If you haven’t checked out Liquibase, you should.  Liquibase is a tool to track and manage database changes.  The idea here is that I want to treat database changes as first class code artifacts that can be revisioned, merged, and monitored.  I store my Java code under source control, where everyone can see, why should my SQL code be any different?  Liquibase addresses this concern as it tracks database change sets and facilitates database refactorings.  In this post, I’ll cover my setup and some of what I went through to get Liquibase in place.

My Setup

Our project is Maven based.  It has multiple Java modules that make up our application.  We have multiple database schemas that we control.  We have additional schemas that we depend on.

Given that Maven is our build tool, I natually wanted to incoporate the Liquibase Maven pluin into the process.  I took the path of dedicating Maven modules to our two schemas that we have direct control over.  This effectively creates two database “projects” which contain the necessary property files to allow Liquibase to work.  Additionally, these modules contain the database changelog structure that Liquibase will use.  I set up Maven profiles to activate these modules during build time.  If they are not present, nothing database related will happen.  By default, these profiles are turned off.

Build Life vs. Command Line

The Liquibase documentation recommends attaching a goal to the process-resources phase of the build.  This works fine in general, but I had a couple of special requirements not really addressed in the Liquibase docs.  First, I want to run Liquibase from the command line using the Maven plugin for any goal exposed by the plugin.  Second, I needed to protect my database credentials on my build server so that they are not encoded in property files that are checked into source control for all to see.

To address the first requirement, I set up my pom.xml as follows:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>foo-parent</artifactId>
<groupId>com.foo.bar</groupId>
<version>1.0-SNAPSHOT</version>
<relativePath>../</relativePath>
</parent>
<artifactId>db-foo-bar</artifactId>
<packaging>jar</packaging>
<name>db-foo-bar</name>
<profiles>
<profile>
<id>db-foo-bar</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<dependencies>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
</dependency>
<dependency>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-core</artifactId>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0-alpha-2</version>
<executions>
<execution>
<id>get-liquibase-location-props</id>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<!-- this property file contains a pointer to a file that contains
filter property values for the build environment. this is necessary to handle
different property locations (for example, between local and the build server). -->
<file>src/main/conf/${env}-liquibase-location.properties</file>
</files>
</configuration>
</execution>
<execution>
<id>read-liquibase-location-props</id>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${location}</file>
<file>src/main/resources/liquibase/${env}-liquibase.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>

<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<executions>
<execution>
<id>updateSQL</id>
<phase>process-resources</phase>
<goals>
<goal>updateSQL</goal>
</goals>

<configuration>
<promptOnNonLocalDatabase>false</promptOnNonLocalDatabase>
<verbose>true</verbose>
<changeLogFile>${changeLogFile}</changeLogFile>
<migrationSqlOutputFile>updateOutput.sql</migrationSqlOutputFile>
<driver>${database.driver}</driver>
<url>${database.url}</url>
<username>${database.username}</username>
<password>${database.password}</password>
<verbose>true</verbose>
<expressionVars>
<property>
<name>db-changelog-dir</name>
<value>${basedir}/src/main/db-changelog</value>
</property>
<property>
<name>schema.name</name>
<value>${schema.name}</value>
</property>
<property>
<name>tablespace.index</name>
<value>${tablespace.index}</value>
</property>
<property>
<name>tablespace.data</name>
<value>${tablespace.data}</value>
</property>
</expressionVars>
</configuration>
</execution>

<!-- FOR USE ON THE COMMAND LINE ONLY! -->
<execution>
<id>default-cli</id>
<configuration>
<promptOnNonLocalDatabase>false</promptOnNonLocalDatabase>
<driver>${database.driver}</driver>
<url>${database.url}</url>
<username>${database.username}</username>
<password>${database.password}</password>

<propertyFile>target/classes/liquibase/${env}-liquibase.properties</propertyFile>
<!--
<migrationSqlOutputFile>updateOutput.sql</migrationSqlOutputFile>
-->
<verbose>false</verbose>
<expressionVars>
<property>
<name>db-changelog-dir</name>
<value>${basedir}/src/main/db-changelog</value>
</property>
<property>
<name>schema.name</name>
<value>${schema.name}</value>
</property>
<property>
<name>tablespace.index</name>
<value>${tablespace.index}</value>
</property>
<property>
<name>tablespace.data</name>
<value>${tablespace.data}</value>
</property>
</expressionVars>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>
</profiles>
</project>

This pom.xml makes use of the Maven’s default-cli execution behavior to instruct an execution that it is coming from the command line and to honor any goals associated with that plugin.  This allows me to run all of the Liquibase goals directly from the command line without having to attach Liquibase to a phase in the Maven build lifecycle.  To run this from the command line, given the pom.xml described above, I need to issue something like the following:

mvn initialize liquibase:<liquibase goal> -P<profile-name> -Denv=<environment>

This gets me to a place where I can kick off Liquibase command, using Maven, directly from the command line.  This opens things up to other developers on my team and allows everyone on my team access to Liquibase from the command line.  Now, to tackle the next requirement, securing database credentials.

Securing DB Credentials

So, I need to protect my database credentials.  Storing them in source control isnt an option.  I need a way to enable my build process to read credentials without my source knowing about them directly.  Again, I leverage Maven to do the work of sourcing properties files that are visible to the build process (anthill in my case) but not to anyone else.  The setup here is that my build server has local directories permissioned such that it, and only it, can read files containing my database credentials.  How can Maven load these files so that it can tie them into Liquibase?  I used additional Maven plugins to do this for me.

If you notice in my pom.xml, I have two executions as part of the properties-maven-plugin.  The first execution loads a “location” property file that is keyed by environment.  Once we have the appropriate location property file for the given environment, a second execution runs that loads the discovered property file.  Additionally, it loads supplemental properties for the given environment that are then used as part of the Liquibase configuration.  As you can see in my pom.xml, the supplemental properties are jammed into expressionVars for use in my changesets.

In my next post, I’ll dig into more detail…