So far, I have published two pretty successful open source projects along with a few that are flying under the radar. In this post, I will share what I learned along the way with one of them.

How it started

When I joined trivago in 2016, a new central automation team was just formed (this did not turn out to be the best of decisions, but this is another story). This team was responsible for aligning and reorganizing all automated testing activities. A large part of that was also the creation of a new test framework as the old one proved not to be the best choice for our setup and use cases. This included adding new features, solidifying its architecture to make it resilient and maintainable and, of course, documenting it all.

Why Cucumber?

There is always some discussion about the Gherkin language in tests. Is it needed? Isn't it just overhead?

For us, it was a great idea for our end to end tests:

• It keeps the tests readable for all involved parties, even for non-technical stakeholders
• It is easy to implement
• It has lots of documentation
• Its community is very likable and helpful

Frankly, as it is often the case, we do not religiously follow the BDD (Behavior Driven Development) methodology. It is a great concept but just never fully worked for us. The ubiquitous language aspect, though, changed our tests for the better.

In case you are not familiar with Gherkin, here is a small example:

Feature: QA Meetup
    Scenario: Ticket distribution
        Given we have 49 registered people
        When I go through the registration process
        Then there should be 50 registered people

For each step, you need to implement code (in our case, Java code) to connect it to some programmatic functionality. This is an overhead compared to code-only tests but keeps the scenarios concise and understandable.

Reporting

In the beginning of our Cucumber based testing, we used pretty much the de facto standard of test reporting: cucumber-reporting. To make it clear, the following paragraph is not about bashing it or saying it is bad. It is just that it did not quite work for us since it shows a lot of information that we didn't need.

Here, you can see that a lot of the numbers in the bottom part distract from the important information of what scenarios failed and why.

The Hackathon

In 2017, I decided to try to create a custom reporting Maven plugin to solve this and have a solution that we can maintain and extend ourselves. Since I had already quite some experience with Java templating via my previous work involving JSF (Java Server Faces), plus the fact that Cucumber spits out easy to parse JSON files that include all relevant information about the tests, this did not seem like such a daunting task after all.

The tech

In Cluecumber, Google's Gson library is used to parse and pre-process the JSON files. For turning the deserialized data into HTML reports, I decided to go with Apache Freemarker. This is an open-source templating solution for Java that closely resembles JSF mentioned before.

The result

This is how the final product looked after the hackathon:

Here, the focus is on the state of each test scenario at first glance without distraction.

Cluecumber turns open source

The result of it all was Cluecumber, which had its very first open-source release in November 2017. This was done with full backing of trivago. Not only was this hackathon time basically sponsored by my company, but also the move to open-source was something that was fully endorsed and supported.

The goal here was to assess if this project could be valuable to other teams while still being in control of its further development. After all, it is still in use in a lot of internal test projects at trivago.

The pros

This company backing proved to be a major factor for Cluecumber's success. In my personal opinion (you can disagree here, of course), it just makes it a little more credible and motivates me to keep on maintaining and improving it. Also, it is beneficial for the company branding (after all, trivago is a tech company), and, frankly, also my personal branding. It is certainly nice to be able to mention that I am an OSS maintainer.

On contrast, my personal project Sandboy which is a reporting solution for Maven Surefire and Failsafe is completely flying under the radar and is not well-known at all...

The learnings

Dealing with people

Dealing with people is not easy. This is made even harder when you have to do that only in written form via reported GitHub issues, questions and feature requests. Often times, you get the same question over and over again. This is not necessarily a sign that you should add the answer to the project's documentation but more a sign that people are just lazy and don't read it. This often times applies to myself, too, so I can understand it. However, when you are "on the other side" of this, it can be draining.

Feature requests

Sometimes, there are feature requests (rarely, a pull request is coming in) which you have to assess and approve or reject. After all, Cluecumber is used in quite some testing teams worldwide and just adding any edge-case feature that is only applicable to a single use-case would just bloat the core product itself. This is not easy. Also, it is quite challenging to not come across as snobby when rejecting something. There should always be a good reason behind it.

Clean code

I am a clean code and code craftsmanship enthusiast. The thing is, that it can be time-consuming and quite boring to ensure that everything not only works but is also clean. Usually, there are two approaches when it comes to open source: release quickly and improve it over time or wait longer and release something clean from the start. I chose the latter approach. In my very personal view, I wouldn't want to push something to the public that I think is working but maybe a little flawed. Yes, this slows me down sometimes but I can sleep better at night.

Test coverage and documentation

In me experience there are very few people who enjoy writing documentation even though this is very important. Not only does this ensure smoother onboarding but also might answer questions right away that people may have (again, if they read it...). Also, unit tests are quite important, especially in an always evolving project like Cluecumber (which reached 77 releases in seven years with 236 issues closed). You want to be sure that existing functionality is not affected by anything you add to it.

Reinventing the wheel

I am sure some of you are thinking "Is it worth it? Couldn't you have just used the old reporting solution and ignore the small issues with it? Why reinventing the wheel?" and I get that. It takes time and effort to develop something from scratch that does something similar than other solutions.

The reasons for me to do it are:

• It was (and still is) fun to work on Cluecumber
• I learned a lot about the involved technologies
• We now have a solution that is completely open for any ideas we come up with
• It gives me great joy knowing that people enjoy using it outside of my company

If you are given time to research and try out things, why not reinvent the wheel?

You will learn a lot about wheels!