Devoxx UK 2016

Ambience and Background

Three years ago London saw the first installment of Devoxx UK which I had the pleasure to visit. This year I decided it was time to come back and see how the conference has evolved and if it's value for
money is still as good as it was.

To make it short, yes it is! For a very reasonable ticket price you get two days full of talks spread over five tracks a decent catering and free water supply. The Business Design Center is a nice
location and of course London is always worth a visit.

In order to keep the ticket prices low the organizers have to bring a lot of sponsors, so the lobby was packed with stands. IMO this is not necessarily bad, at least you cab get a lot of goodies ;-)

Another cool thing is, that all talks are taped on video and the videos are all uploaded to youtube. So if you want to watch the talks just go to the Devoxx UK Channel

Day 1

Dot Con - James Veitch

The opening talk was a great start to the conference. A very funny narrative of what happens when you actually reply to scammer emails (like Nigeria Connection etc.) but in a special way.

James gave a summary of this experiences when messing with the people behind those emails and by that almost driving them mad. Usually I am not a fan of harrassing people, but if it happens to
frauds that try to steal the savings of innocent people who do not really understand that there is a whole con industriy out there, I strongly approve!

I highly recommend you watch the video to this talk, as no summary would do justice to it. It was hilarious.

Unfortunately it seems this video was not uploaded to the channel.. real pitty there...

Embracing Failure - Mazz Mosely

I chose this talk, because I remembered Mazz from a talk at the first Devoxx which was petty good, so I gave her another shot. You noticed immediately that she was very nervous speaking in front of
a rather large audience but apart from that the talk was good. The topic itself was more about what bad management looks like, especially when the project is in a bad phase. She drew the picture of
the typical jerk boss stereotype (pressumably from her first job) who responds to delays and errors by putting on more pressure but taking all the credit for success.

The story seemd to take a turn for the better when she told us about a meeting where she, as a young developer, dared to speak up in a review meeting, suggesting some improvements. Against all
expectations the more expirienced team mates backed her up and it seemed that from there on all would turn out good. But no, afterwarsd she got snubbed by the manager and it was clear that
there would be no happy future for her in this job. To make everything even sadder, the manager got promoted for the work the team did and no credit was left for those who actually saved the project.

In the end her morale was that you should speak, you should try to make your work a better place, even if it is hard. But if it turns out, that there is no way to make progress and that you will not be
happy where you are, then move on and find people who share the same values as you do.

Where's my free Lunch? - Hadi Hariri

For me, this one, can be summed pretty short. There are a huge number of online services that are, at a first glance, free. You can use them and no one is charging you. Hadi shows in detail in what way
you actually are paying for things like Google search or Facebook, just to name a few big ones. The bottom line is, you should not just blindly use everything out there. But make sure you protect your
data and understand what the service you are using, is taking from you in return.

Dials to 11 - Moderne Exteme Programming - Benjí Weber

Cool talk, revisting the principiles of Extreme Programming and Agile Development plus a few ideas and principile bulding on that. The concept I found most intriguing was Mob Programming, using the
brain power of the complete team to code complex core modules sounds like a really good idea. Other concepts revovled around Continious Deployment and Test Driven Development which did not strike me
as too revolutionary. The final point that I had to agree with is, that projects have to be refactored on a regular basis, especially when maintenance, testing, monitoring etc are too complicated.

Cybercrime and the Developer. How to start defending against the darker side of the web - Steve Poole

Now that topic did sound interresting, I was looking forward to hear about best practices against cyber attacks, backgrounds of the business etc. But in the end this was just the same old story of
"Check your app for security issues" and "The bad guys are out there" combined with some numbers on cyber crime and links to checklists for security issues. Breaking news.. And the biggest wtf moment
was a short advertisment break for a sponsor. Well, I am not going to get that time back...

Arduino and Java with the Intel Galileo - Simon Ritter

For this one, I was probably not the intended audience. I think I would have got more from it if I had at least some practial expirience with Arduino. Over a while it felt like an advertisement for
the Intel Galileo but soon turned into a report of the issues the Azul guys encountered when trying to get Java working on it.

Overall the style of the talk was good and it was interresting to hear what problems can occure on embedded platforms.

Extreme Profiling: Digging into Hotspots - Nitsan Wakart

Okay, this one went right into the nerd stuff. Nitsan gave a great overview over different profiling tools like VisualVM or Java Mission Control compared to what FlameGraphs in Java 8 can do and how
to use perf to help find performance issues within the Java application. If you like this kind of stuff, just go and watch the video, it is really insightful.

On Polymer and Smileys... or Polysmileys - Carmen Popoviciu

Finally some hands on coding! Carmen introduced the different features of Polymer and how it enables you to create reusable webcomponents. This was one of the talks where you saw from the first
second that the speaker really loves the topic. Carmen's passion and enthusiasm were very catching and the code examples she showed were precise, clear and easy to follow. One of my favorite talks
of this conference.

Polymer is abstracting the browser differences for webcomponents, but of course the webcomponent standard is changing. So Polymer has to adjust to those changes once those are final.

Busy Java Developers Guide to Hacking in Java - Ted Neward

Another talk with 10/10 nerd points. Ted had prepared a variety of special topics concerning tweakings of the jvm and the jdk. Basically it was an in depth tour of some of the more exotic settings
you can use to alter compiler and runtime behaviours. I must admit, I can't restate them here, so again I recommend watching the video. Ted's expertise combined with his presentation skills and his
relaxed and easy attitude made this a very enjoyable talk.

Knowledge is Power: Getting out of trouble by understanding Git - Steve Smith

I must admit, Steve knows Git. He gave an indepth overview of the basic principles Git was built upon. This started with the structure of the .git directory and how objects are created/stored and did
end with the power that the reflog gives you. In essence, if you screw up your git project, check the reflog to find the has before it all went wrong and then revert to that. For the details how to do
that just watch the video.

One thing I do not agree with though, is his hate towards git-flow. It is true, that the schema looks highly complicated and weird, but once you understand the basic system, it is rather simple and
intuitivly most teams implement something very similar. I do agree with his opinion, that you should only add as much complexity to your development workflow as necessary.

Git in Practice - John Stevenson

The last slot of the day was another Git topic, this time a Bird of Feathers (more a guided group discussion). Steve threw in some rather generic questions about workflows and team structures and
the group shared their experiences and opinions. There were no real surprises there, teams usually have some git flow like development process with feature and release branches that get merged back to
master/develop. One thing shocked my though, someone explained how their team uses code reviews to ensure code quality before merging the branch and, as I expected, there was a lot of nodding and
agreement among the other participants. But one guy spoke up and said, that he considers that sad because "you obviously don't trust each other". That was a huge wtf moment for me, but I was relieved
that noone else shared this view. A team where this kind of attitude is common can just mean that there must be a highly inconsistent code base.. *shudder*..

Day 2

Microsoft and Open Source? Microsoft and Java? Really? - Giles Davies

Well, the title says it all, really. Like most IT guys I have some reservations when it comes to Microsoft, this is of course caused by their actions in the 80s, 90s and early 2000s. But seeing how
they are now trying to get to grips with IT reality (e.g. the development of Edge) I figured I give them a chance to convince me that they are not that bad anymore.

The picture Giles tried to draw was one of a "regular" IT company that uses different technologies to provide their customers with the best possible service. While that all sounded fair enough it was
more or less what was to be expected by this kind of talk. How much of this is really as good as it sounds only time can tell. But at least it is clear, that Microsoft tries to abandon it's old ways
and so they will probably again become a global player again who might rival with Apple or Google.

Composing Music in the Cloud - James Weaver

Mr JavaFX talking again. As always his experiences speaking style made the presentation itself worthwhile. The topic itself was nice but it actually did not blow my mind. Basically Jim did show how
he was using CloudFoundry to host a Spring-Boot application that can assist in different ways of music composition and how fast this can be done with modern technology.

If you are looking for some distraction after a work day with some kind of nerd flavour, this video is the way to go.

Java 9 Modularity in Action - Sander Mak, Paul Bakker

Oh yeah, some real bleeding edge stuff coming up. Sander and Paul gave a nice insight into the Java 9 modul concept that is intended to change how Java applications are written and run. One of the
main points for me is that this aweful classpath "thing" is meant to die with Java 9. Instead applications will be composed of moduels which expose certain classes and functions and can be combined
using provide and require declarations.

Does that sound familiar? Oh yes.. OSGI!!!! The concepts of OSGI are rather old, the last time I really had to do with that was back in 2003/2004. But so we meet again. Don't get me wrong, this is
nothing bad. I really like the concept of OSGI as it also allows you to have different versions of modules available at the same time etc. So I am excited to see how this works out when it comes to
the "regular" Java world.

Of course it is not so easy to upgrade to this new version, at least not when you want to follow the new paradigm. First of, the JDK and JRE are using the module structure too, so if your
application is doing something nasty with those resources, it is very lickely going to break with Java 9. And how do you migrate to the module structure? Every application has a lot of dependencies
and those have to be migrated as well. But that was taken care of, you can use jar files as so called automatic modules, that way you can upgrade your dependencies step by step.

The downside is, that in practice those upgrade will not happen soon at most companies, because they are expensive and risky. So we will have to wait and see when Java 9 will truely be taking over.

Refactoring to Java 8 - Trisha Gee

Not so much bleeding edge, but still a very interessting topic as Java 8 is, unfortunately, still new for most companies. In additon, a talk with Trisha is always worth listening to and so was this one.
What I liked most about her approach was, that she did not do any of that fanboy "omg, you have to use all the features asap because this is new and therefore better than, omg" stuff. Instead she
took some of the most popular Java 8 features and examined them closer in terms of: How easy is it to migrate? How much can go wrong? And how does the performance change?

The results were somewhat disillusioning, not only did some patterns prove pretty tricky to to migrate to but also the performance analysis showed that there was hardly any performance gain from
using the new features and in some cases peformance even dropped.

Bottom line is, you have to decide for each language feature and each use case independently if it worth to migrate and you always have to run performance tests on the code base to compare the
behaviour under high load. For everyone who is about to upgrade to Java 8, this video is a must watch!

Faster Java By Adding Structs (Sort Of) - Simon Ritter

And some more Java action! This time about the ObjectLayout project from the guys at Azul. The idea behind this is, to bring the performance advantages that C/C++ have compared to Java due to the use
of structs.

With a struct defintion the compiler knows exactly how large the struct object is and all objects do have the same size, as all elements are only pointers to the real data. That way the code can make
assumptions of where a certain object is within an array etc. and thus take short cuts during the execution.

This is what the ObjectLayout project tries to bring into Java. For details you better watch the video, as I am sure I will mix something up if I try to summarize it here. Pretty weird stuff.

Building multiplayer game using Reactive Streams - Michal Plachta

The last talk of the conference was again more hands on. Michal showed us, how Reactive Streams work and hwo those can be used to create a nice simple multiplayer game in a live coding session. Nice
topic and the coding was ratehr easy to follow, so for me this was a pretty good end to the conference.

Summary

As I said before, the value for money ratio on this conference is still great. Lots of talks and variety of topic to choose from with some very good and experienced speakers.

Of course there are some rather minor issues, or rather things that I would not need which I want to sum up just for completeness sake.

In comparission to the first year, there where less tables, seats and especially power plugs available. This makes it a but cumbersome for us techies with all our electronic devices. I understand
that the space is limited and the organizers preferred to put up more buffets for lunch but still it would be great to get some more set up in the future.

There are also more talk slots that I did not mention, for one the Ignite talks. A series of 5 Minutes talks about some small or humorous topics. I did not get much from them and I really don't
see the point of those. But obviously there is an audience for that. The other are 15 Minute talks during the lunch breaks. Usually you are in line waiting to get to the buffet and so don't have
time to listen to them. It seems to be pretty ungratifying to give such a talk. But again, this is in place since the first year, so some people seem to like it.

The one thing that really kinda bummed me out was the organization of the DevRoxx party. The party took place at the second evening, but it was not really advertised that much apart from the
closing talk of the conference. It as then that I took a closer look at the home page and finally found the section about it and then saw that you had to go to some of the sponsors to get a ticket
for the party. At that point it was of course to late, but we decided to give it a try and went to the location just to see that quite a lot of people did not know about the tickets. So it was
not just my problem ;-) So maybe that is something to improve for the years to come.

The last thing I want to criticize is that even during the closing talk not only the sponsor stands got removed but also the wifi was taken down, which can be a problem for foreigners who have no
data contract and who want to look up things, like e.g. the party location.

Even though this seems to be a lot to complain about I want to stress the point, that this is really good conference considering the cheap prizes and that I am very sure you will not regret coming
to this in the future.

Constant Tarfiles For Resumable Downloads

Our Situation

 

We have an Java based backend server providing daily changing content to mobile applications as a gzipped tar archive. The content can change over the course of the day and some applications need to download the complete content in the new version, while others need to get the diff between some versions. In addition old content from previous days has also to be available in all those variations. That means it is not really feasible to have all possible different tar files pregenerated and stored in our backend (yes Hadoop and friends stand ready to help but at the moment this is not an option for us for several reasons).

So what we do is, store the different versions of our content, calculate the needed tar files,  deliver them to the requesting app and put the file into a caching server to reduce load but of course we cannot cache everything permanently so the entries timeout from time to time.

 

Our Problem

 

Some applications reported downloading issues that we at first blamed on general mobile network issues but after a while it became evident that there was more to it. After some debugging on application side it seemed that in some cases the app was not able to resume an interrupted download due to the checksum of the file having changed.

 

The Underlying Cause

 

This issue was quite a surprise for us as we were sure that for every kind of request the corresponding file was created from exactly the same content every time. But by bypassing our cache the effect could be reproduced immediately by downloading the same file twice from our server. The mighty diff tool told me that the two archives differ even though their extracted contents did not (phew...).

My first suspect was gzip and a quick Google search brought up some hints about the gzip timestamp header that is contained in the first 10 bytes of an archive.So I fired up my hex editor to check the starting sequences of the two files only to be disappointed because they were exactly the same. Having a look at the libraries source code of GZipOutputStream (we use the Apache compressor lib) revealed that this implementation sets a fixed timestamp for all gzip archives (all fields are set to 0 whch I think is the right thing to do).

Looking at the differences of the archives in my hex editor and matching those against the tar rfc docs it became clear that the culprit was the mtime header for tar entries. In our code we construct the tar entries using the filepath and then add bytes from a stream to it. The invoked constructor sets the entry's mtime to the current time so it is different every time the archive is created. Eureka!!

 

The Solution

 

Now that was rather easy to fix, we now calculate a timestamp value based on the date the content was created at, with a few tweaks, that is constant for each single file and set that as mtime for each tar entry we create, et voilà :-)

 

Sometimes They Come Back

 

A few weeks later we got reports about some strange download issues again, this time with only with certain content packages and it was absolutely not obvious where those came from. We had a lot of red herrings to hunt and during one of those hunts I noticed that files that should be identical in some cases simply weren't. Again diff and hex editor came to the rescue and I found that for some entries the mtime headers again did not match!!! So we got down on the code again... Did I forget to handle any tar entries? Nope, everything was covered... Any issues with entries being added twice or changed in other methods? Nope.. nothing.. In addition after extracting the tar all files had the desired timestamp, no sign of the other timestamps at all.. Oh how I swore...

So back to checking the archive contents.. then the tar docs.. and again the archive contents.. finally I found a clue, a classic tar entry has only 100 bytes reserved for it's name field which contains not only the file name but also the complete path to the file within a directory hierarchy and the offending files lay within a rather deep directory hierarchy and had pretty long names itself. To change my suspicion to certainty I added a copy of a file with the correct timestamp to the content and altered it's name to exceed the 100 byte limit. In the resulting tar the copy again had the invalid timestamp and just then I noticed something else, for the copy's name there were two header entry's while the original file only had one.

So I did more research about long file names and header fields and dug into the code of the TarOutputStream class. To handle long file names tar can use additional posix or gnu headers and in both cases the TarOutputStream creates additional tar entries using the same constructor as we did in our code. So if you have long file names in our archive it will contain additional tar entries that always have their creation date as mtime header value!! Oh man, that was mean..

 

The Solution - For Real This Time

 

The only way to fix this was to copy the code of TarOutputStream to our project (and TarBuffer as that class is a dependency of the former with default scope) and adjust it so we can set a custom timestamp for the additonal entries.Not my favorite way of fixing things but unfortunately we had no other choice here... So beware of tars with long file names if you need to rely on the checksum..

Self Logging Exception Pattern

Error handling is an important and vital part of any serious application and usually a fair amount of application code addresses this topic. This is too some extend due to the fact that proper error handling requires to consider two aspects.

When ever an error occurs usually the control flow gets iterrupted and needs to take a different route. In Java this is done by throwing, catching, rethrowing and wrapping exceptions so that the application either aborts the current request to show an error message to the user or to recover from the error automatically.

Additionally those errors need to be logged to allow the support staff to see what is going on in the application and to be able to tell customers why their actions did not give the expected result and most importantly to find bugs.

This often leads to code blocks like the following:

UserTransaction transaction = null;
EntityManager em = null;
try {
  transaction = (UserTransaction) new 
      InitialContext().lookup("java:comp/UserTransaction");
  transaction.begin();
  em = emf.createEntityManager();
  em.merge(this.be);
  transaction.commit();
} catch (IllegalStateException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (RollbackException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (IllegalArgumentException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (TransactionRequiredException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (PersistenceException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (NamingException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (NotSupportedException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (SystemException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (SecurityException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (javax.transaction.RollbackException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (HeuristicMixedException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} catch (HeuristicRollbackException ex) {
  logger.error("Persist failed", ex);
  throw new ApplicationException("Persist failed", ex);
} finally {
  em.close();
}

Quite anoying, isn't it? Now there are certain possibilities to try to reduce this boiler plate. The first attempt that is still employed far too often is to just catch Exception. This pattern is higly discouraged because it swallows exceptions resulting from coding errors like the very common NullPointerException. Another way to make the code a little bit cleaner is to wrap the log statement and exception generation into a convinience method that might also be parameterized with a message to be used as log ouput and the exceptons detail message:

private void bail(String msg, Throwable ex){
  logger.error(msg, ex);
  throw new ApplicationException(msg, ex);
}

...
} catch (SystemException ex) {
  this.bail("Error message 1", ex);
} catch (SecurityException ex) {
  this.bail("Error message 2", ex);
} catch (javax.transaction.RollbackException ex) {
  this.bail("Error message 1", ex);
} catch (HeuristicMixedException ex) {
  this.bail("Error message 1", ex);
...

But what if you do not only want to control the exception text to be displayed but also the kind of exception to be thrown. One could swap the error message for an enum and then decided within the bail method which exception should be thrown or to avoid this additional boiler plate add the exception class to the enum and then generate the exception by using reflection.

But all these approaches still rely on the programmer to make sure that where ever he handles the original excpetion he has to do both, log the error and throw the exception. Even if this is wrapped in a utility class that might be used throughout the application. Don't get me wrong, this approach is usually pretty fine and works rather well, at least as long as all developers involved in the project are aware of this necessity and can be trusted not to forget it. But often enough even the most reliable developers are in a hurry or the project staff changes or...

So I thought it would be nice to have mechanism that makes sure that when ever an application sepcific exception is thrown the corresponding error will be logged. That is why I came up with this idea:

public class ApplicationException extends RuntimeException {

  private static final long serialVersionUID = -4599465800805791368L;
  private static final Logger logger = Logger.getLogger(ApplicationException.class);

  private MessageCode code;

  private List<Object> params;

  public ApplicationException (MessageCode code) {
    this(code, null, null);
  }

  public ApplicationException (MessageCode code, List<Object> params) {
    this(code, null, params);
  }

  public ApplicationException (MessageCode code, Throwable cause) {
    this(code, cause, null);
  }

  public ApplicationException (MessageCode code, Throwable cause, List<Object> params) {
    super(getResolvedMessage(code.getMsg(), params), cause);
    this.code = code;
    this.params = params;
    this.log();
  }

  private void log() {
    StackTraceElement ste = this.getStackTrace()[0];
    if (logger.isEnabledFor(this.code.getLevel())) {
      logger.log(this.code.getLevel(), "ApplicationException thrown at "
        + ste.getClassName()
        + "."
        + ste.getMethodName()
        + ":"
        + ste.getLineNumber()
        + " with message '"
        + getResolvedMessage(this.code.getMsg(),this.params) 
        + "'");
    }
  }

  public String getKey() {
    return this.code.getKey();
  }

  private static String getResolvedMessage(String msg, List<Object> params) {
    if (null != params) {
      for (int i = 0; i < params.size(); i++) {
        if (null != params.get(i)) {
          msg = msg.replaceAll("\\{" + i + "\\}", params.get(i).toString());
        }
      }
    }
    return msg;
  }
}


public enum MessageCode {

  // General
  CODING_ERROR("Error.CodingError", "This is very likely a bug"),

  // Persistence
  PERSISTANCE_SAVE_FAILED("Error.Persistance.SaveFailed", "Error during entity persitance."),

  // User
  USERNAME_ALREADY_TAKEN("Info.User.NameAlreadyTaken", "Username '{0}' already taken", Level.INFO), 
  INVALID_PASSWORD("Info.User.LoginFailed", "Login failed due to an invalid password for user '{0}'", Level.INFO);

  private String msg;

  private String key;

  private Level level;

  private MessageCode(String key, String msg) {
    this(key, msg, Level.ERROR);
  }

  private MessageCode(String key, String msg, Level level) {
    this.key = key;
    this.msg = msg;
    this.level = level;
  }

  public String getKey() {
    return this.key;
  }

  public Level getLevel() {
    return this.level;
  }

  public String getMsg() {
    return this.msg;
  }
}

Here the MessageCode class takes several arguments, a key that defines the property key to be used when displaying an error message to the user, a string containing information for developers that will be used as log statement and an optional level defining the log level for the message. The latter is useful since not everything that causes request abortion is an error like a user providing an invalid password.

The exception class wraps that MessageCode and creates a log statement within it's constructor. Of course this has a draw back, most logging frameworks like e.g. log4j allow to automatically augment the log message with the class name and line number where the message was logged. With the log statement being created within the exception that piece of informaton is lost. To get around this limitation I am using the exeptions own stacktrace by accessing it's first element that contains the class name and line number where the exception constructor was invoked.

That way it is possible to log an error and abort request execution with one statement in your application code and by incooperating this capability into the exception class itself you can create your application specific exception hierarchy that inherits this behavior.

So far I am only using this pattern in my private projects but I am thinking of also introducing it to my customers' projects.

As always I would be very interessted in what you guys think of this approach especially if you see any problems with it or have suggestions for improvements.