How I Learned to Love Testing Open Source

“Wait! You’re saying strangers will be touching the code?”

Just the thought of a developer I’ve never met changing code I’m responsible for testing was a nightmare. Imagine the stranger danger I felt when Datical transitioned to Liquibase, in name and in mission, moving from the proprietary code I had worked on for years to QAing our open source code.

The mental shift from testing enterprise code to open source

I was overwhelmed at the prospect of being responsible for testing the open source code coming in and, yes, I was scared I’d fail. The shift from testing enterprise products to testing an OSS project is scary but it can be done. Here are some realizations that helped me in the transition:

• Recognize that you are not alone.
  Behind every great OSS project is a community. They are strangers, true, but they are strangers who share your goal and want to work with you. My first experience with a Liquibase community member helping with testing came from GitHub user @MenschNestor (Christoph). He not only described the bug, but provided an example Maven project that demonstrated the issue. This made my job so much easier and although we’ve never spoken, I’d find it difficult to describe someone so helpful as a stranger. Thanks, Christoph!

• Being involved in the process helps.
  It’s important to be involved in guiding the process changes that you think need to change in your engineering department. It’s easy to feel overwhelmed when you don’t have clear priorities and processes. If there isn’t a discussion on process change, initiate one. At Liquibase, we needed to decide how to navigate communication and hand-off of Liquibase community issues into the internal engineering team. This included decisions on build processes, links between internal Jira and GitHub issues, and internal code reviews of community-provided fixes.
  
• Accept that there will always be bugs.
  Bugs will be found by the community and there is nothing you can do to prevent that. You couldn’t do that with proprietary product testing, either. Keep in mind that the difference is in visibility, not in you suddenly sucking at your job.

Summing it up

As Nathan Voxland, founder of Liquibase reminded me, for every bug report that’s submitted there are thousands of people using Liquibase without any problem. Testing OSS isn’t as structured or clear-cut as testing corporate products — but it is more fun. Unlike corporate products, where a company decides how a product is used, the community uses Liquibase in inventive, surprising ways. If corporate testing is classical music, community testing is jazz. Both are beautiful — but jazz offers a freedom to innovate that classical music precludes.

Take a deep breath, take stock of where automation can help, and start making new friends in the community! It really does take a village to keep Liquibase the awesome tool that it is. If you find a bug, report it! Better yet, see if you can help shape the solution by submitting a pull request.