We hosted our first ever mabl hackday yesterday. It was voluntary, and we scheduled it on pretty short notice, needless to say there were a lot of unknowns. Is one day enough to do impactful work? Would many people participate? Would it be worth the effort?
The results exceeded all expectations. We had a lot of fun and learned a lot along the way. Here’s how we did it.
Let’s take a break from our (remote) routines
Most mablers work from our office in Boston every day, so everyone working remotely is quite an adjustment for us. After the first few days of “Quarantine 2020”, we realized that we’d need to be more intentional about some of the things that happen naturally in the office.
In particular, we wanted to create an opportunity for everyone to take a break from their routines, have fun, and create or deepen their connections to other mablers. We also wanted to create space for people to explore ideas and projects that inspire them. We were excited to see if any would live on as part of our product or operations. So we organized and announced our first hackday!
Bring your pet projects
About a week in advance, we created a Slack channel to brainstorm potential Hackday projects. We didn’t provide much guidance other than to suggest that our work should be related to mabl in some way. There were LOTS of initial ideas, and they were as diverse as the people who submitted them. Here are a few examples:
- An Engineer proposed converting our Trainer (test recorder) to TypeScript.
- A Product Manager proposed reworking our online documentation.
- A UX designer proposed refactoring the user experience for our Test Plans.
- A Customer Success Manager proposed that we implement a new support utility.
- A founder (who shall rename nameless) submitted ideas on behalf of customers, so that it wouldn’t look like he was pushing his own agenda.
You can see where this is going. About 100 pet project ideas made their way to the Slack channel in the span of a couple hours. It was a cathartic exercise. Once that was out of everyone’s systems, we created a worksheet and encouraged people to add their three best ideas. People started adding their best ideas. Most followed the guidelines. One added eight. No comment.
The thrill of competition
We’re not generally a competitive bunch, but we thought it would add to the fun to incorporate some feedback and prizes into the mix. To ensure a fair and impartial evaluation process, we recruited some “Friends of mabl” to help out. Four esteemed technical leaders agreed to join us as judges:
- Toby Smith - Engineering Manager working on AI products for Google Cloud.
- Erik Nordlander - General Partner at GV (formerly an engineering leader)
- Phil Jacob - Engineering Manager working on education-related products at Google
- Geoff Meyer - Test Architect for Dell EMC’s server Infrastructure division
Matchmaking is fun
You’ll be surprised to hear that we aren’t all natural connectors on the mabl Product team, so we needed some support to get everyone matched up on teams. We hosted a Hangout and gave people a chance to pitch their ideas to others. We created a sheet for people who wanted to participate but weren’t attached to a specific project yet. In short order, teams were being formed. Engineers were recruited and poached. People continued to push for their ideas behind the scenes. The intrigue!
And we’re off
We hosted a brief kickoff on Monday afternoon. In that call, we reiterated our stringent rules for the competition:
- Anyone can participate
- Project should be at least loosely related to mabl
- Don’t do anything illegal
- Don’t create a mess that someone else has to untangle
- Doesn’t have to be shipped product (but can!)
- Don’t sabotage other teams
- Be inclusive and supportive
- Have fun!
We also talked about the criteria that the judges would be using to evaluate teams, and mentioned that special prizes would be awarded. We used Surveymonkey to capture all feedback with this main rubric for judges:
We also asked the judges to capture informal feedback and notes on each presentation to promote reflection and learning outside of the competition. We also asked them to rank the teams in a separate survey at the end (the order was randomized to avoid any bias).
It’s working!
By Monday evening, most people had joined teams and a handful of Pull Requests were already in review. Team sizes ranged from a single engineer to a cross-functional group of five. The projects were generally related to our product. Several were in the realm of new features, one aimed to improve the efficiency of our visual testing service, a couple were UX enhancements, and another was an improvement to our Sandbox environment. I’ll let the reader guess which is which, but here are the team names:
- Jest in time
- Flow Rida
- Octo Harold
- mablTV
- Better than /giphy
- Trainer time
- Auto-heal sandbox
- OeX-men
The scope of each project was pretty ambitious for 24 hours. Then again, we have a pretty ambitious team. There were dozens of commits and pull requests put up over the course of the day. Teams collaborated on Slack, Zoom, and Hangout. And there may or may not have been a little smack-talking going on between teams.
Demo time
The teams, judges, and audience all gathered via Zoom Tuesday at 5 PM. After reviewing the ground rules and introducing our judges, we were off and running!
Each team had 5 minutes to present their project. They generally started with a few introductory slides outlining the business value and quality methods for their project (a critical focus, given mabl’s business). There were several live demos of features and a few recorded videos.
Nearly all of the teams had most of their work either in PR or merged and deployed to our staging environment. There were lots of great questions and comments from the audience. One of the judges commented that he was amazed by how supportive our team was, which was no surprise, since supportive is one of our core values.
And the winners are... all of us
We could not have been happier with the results of our first hackathon. Every one of the 7 teams presented work that would positively impact our customers, and we have high confidence that most of it will make its way to production in the coming weeks. Here are some of the most pointed comments from our judges:
- “I love seeing shift-left opportunities come to life”
- “I love the concept of shortening the communication cycle time between Dev and Test”
- “Visual debugging is really the best”
- “Anything to help Testers save time and re-allocate their efforts to higher-value!”
I’m pleased to say our most important goal was achieved, the teams had fun! In our post-event survey, we asked people to comment on their favorite part of the hackathon. The most common sentiment was that people appreciated the opportunity to collaborate with other mablers that they don’t normally get to work with. And of course everyone loved seeing the demos at the end.
We asked all the participants how often they think we should host hackathons. Most thought twice annually would be a good cadence.
The funniest “other” response was, “Quarterly (and monthly during captivity)” - which highlights the value of this type of activity during the current coronavirus quarantine.
Lessons learned
Shorter hackathons work, but 2-3 days might be better than 24 hours
In the end, we were shocked by how much the teams were able to accomplish in 24 hours, and this investment of time was a no-brainer from the company’s perspective. On the other hand, about two thirds of participants would prefer 2-3 days, and many asked for just a little more time on the current hackathon to finish up their projects.
Be as explicit as possible about how the event will work in advance
We weren’t as proactive as we should have been in communicating how teams would be formed, what types of projects would be best, and how the work would be evaluated. This uncertainty cost the teams some time that could have been better spent working on their projects. Next time we’ll have this organized and documented in advance.
Allocate 10 minutes for each team presentation
Squeezing a presentation, demo, and questions into 5-6 minutes is a nice way to keep people focused and engaged, but it felt a bit too rushed. We should allow a little more time for some healthy Q&A. Giving each team a full 10 minutes seems like it would be just the right balance.
Encourage small, cross-functional teams
Most of the teams had two or more engineers and at least one product manager, designer, or researcher. That seemed to work well. The smaller teams seemed to bite off appropriately-sized chunks of work. While the engineers did most of the development, the others played key roles in getting people organized, generating and refining ideas, preparing and delivering the presentations, and more.
Feel free to hack remotely
Having everyone working remotely wasn’t an issue at all. The teams made good use of the tools at their disposal and didn’t seem to miss a beat. Even the final presentations worked well, despite the fact that we had nearly 40 people tuning in via Zoom. If anything, the remote situation promoted focus.
Don’t be overly prescriptive about projects
Our natural tendency is to encourage self-directed product teams, so it wasn’t a surprise to us that the teams were excited to work on the projects that they believed would be most interesting and have the highest impact. As one team member put it, “Seeing how much we can get done when people are working on projects they're genuinely excited about is so much fun, and it's the real embodiment of Drive (another of our core values). All of the projects were not just impressive, but useful and making mabl a better product for our customers and for ourselves.”
Allocate time for quality and clean-up
The teams are so excited to solve the problem and get ready for the demo that they optimize for the “show”, which isn’t a problem unless everyone goes back to their regularly scheduled programming after the demo. Consider allocating at least half a day after the demos for the teams to wrap up their projects; remember that the same forces that prevented all this good work from happening outside the hackathon will be back in play immediately after, so it’s better to wrap them up while the teams have momentum.
We look forward to hosting our next hackathon in coming months. Of course I’d be remiss if I didn’t mention we’re hiring! If you’re looking for your next great adventure and want to join a talented product team, let us know!
