Maven Transitivity Hell - How Dependency Mediation Can Snap Your Neck ;-)

Lately I ran into a problem at work which at first puzzled me somewhat so I figured putting it here could help others.

We have a rather complex maven hierarchy in our project with a lot of artifacts on many dependency levels. After the rather common task of changing an artifacts dependencies to their new release versions the tests crashed with a NoSuchMethodError within one of the changed dependencies. At first I suspected duplicate libs in the main artifact to be the cause but all libs were present only once and in the correct versions.

To illustrate the actual problem imagine a dependency hierarchy like this:

main artifact
    base lib V1
        data lib V1
    specialized lib V1
        base lib V2
            data lib V2

It immediately becomes obvious that during dependency resolution maven has to decide which version of base lib and data lib have to be added to main artifact's libs... Dependency Mediation TO THE RESCUE!!!

Following the mediation algorithm maven decides to use V1 of base and data lib now if specialized lib uses classes or methods of base or data lib that were introduced in V2 those will not be available at runtime causing NoSuchMethodErrors or ClassNotFoundExceptions.

In the above example a fix should be easy by simply change the pom of main artifact to depend on the new version of base lib but often enough real world scenarios are not solved that easy. In our case we were lucky to have full controll over all involved artifacts so we could adjust the code as needed. But if you are using third party libraries which in turn depend on other libs that you are also using in your own code like e.g. apache commons or similar an upgrade of said libraries can cause a lot of trouble.

What if specialized lib was a third party lib of which we need to use the newest version because it contains some vital enhancement or bug fix but on the other hand including the new version of base lib would cause severe runtime errors, e.g. LinkageErrors because now a different version of class is used as the container provides (happened to us with QName in weblogic which was the reason why we used V1 of base lib in the first place)? Then you could end up between a rock and a hard place...

The same problem could also occure if base lib was not a direct dependency of main artifact but a transrient one of another lib which makes it even harder to discover thus discrepancies.

Unfortunately the only thing you can do about it is to check the dependencies of the libs you plan to include in your project and make sure they are not conflicting with each other which can become quite cumbersome with growing projects and sometimes even impossible (especially when using large frameworks or application servers etc.).

All of this got me thinking about the size and complexitiy of today's average JEE applications and how many of them may be only working by coincidence because some of their libs have been compiled against other libs with versions that are not available in the deployment unit. Those applications can still work if, applied to the example above, specialized lib does not use any features of base lib that are only available in V2. This can happen a lot since when starting a new project you usually use the newest version of a library available but you might only be using methods and classes that have already been available for several versions. Of course the methods might work differently or even have some bugs in them that would have been fixed in the new version so you might run on broken code without even knowing.

So you now might like to go and check your projects dependency hierarchies... ;-)