We talk about it all the time - agility. At GoodData it’s in our DNA. We understand being agile is not necessarily easy. You need the right people, attitude and tools. And exactly as we’re trying to deliver the GoodData platform to help our customers bring agility into a BI arena, we need the very same tooling internally to sustain the pace of a 14-day release cycle. We bring our customers new features every two weeks - unheard of in the stodgy world of BI and analytics.

Let’s get some perspective since 14 days might not sound so difficult for agile developers. How do you stay confident that all 2,713 data marts, 1,344 dashboards and 19,086 reports (our internal count as of June 2010) are valid from release to release? For a long time, we performed very expensive and time consuming validation directly with our customers. But how many reports you can actually test? Where does the attention to every single detail end and become a boring chore?

It turns out it’s at the second report you have to validate. Then you have to add security and privacy concerns. We motivate our customer to be as self-service as possible, so why we should destroy this experience by a need to manual verification? GoodData is about the Cloud. And the cloud need automation to scale.

A few weeks back, we quietly and successfully deployed our Regression Testing Toolkit. Its purpose is simple: take two different code branches and select data mart(s) you want to compare and bang - results. All anonymized, with no data being exposed, and still providing enough information for our engineers to fix the possible problems. For me and you, the results might look a bit sci-fi but engineers love them, which is critical if they need to resolve the problems as fast as possible.

Here’s what it looks like:

For the rest of us, the final OK message is pretty much what we are looking for.
Then there are the unexpected benefits.

As the devops supporter, I’m glad we found a way to get our engineers more engaged in our production environment. We got also a new performance tracking tool as a by-product. For every single report, and as aggregate for whole data mart, we now have a runtime value that is easy to compare across the releases or different environment. That’s what I call getting the good business value from our engineering team!

No matter how passionate we’re about this tool, there’s more work needed. And hopefully soon you will be able to see validation of your data and model directly. No delays. Enable you to iterate through the changes in your data model faster.

Building confidence and trust for on-demand solution is a long run. Tools like our Regression Testing Toolkit helps us to go that extra mile and me, an ops guy, help to sleep better…