Engaging the team with Mixpanel through games!

Zanbato has been using Mixpanel as our business analytics platform for quite awhile now, but we struggled to understand how to use it correctly. Mixpanel tracks individual events from both our front-end and back-end. With that data, it provides powerful tools for aggregating and segmenting those events so we can understand how our users are using our platform to accomplish their goals. Our team is responsible for defining all of those events, and although this gives us tremendous flexibility, it also has been a major roadblock.

Without a built-in source of data, Mixpanel is a blank slate, and it’s hard to know how to structure events in a meaningful way in the system. This presented a chicken-and-egg situation: we needed data to learn how best to structure it, but we needed the structure to get it into the system. For months, we only slowly added events haphazardly without really figuring it out. Furthermore, we weren’t sure where event tracking fit into the development cycle: who was responsible for defining the events, and at what point were they added?

We were missing a holistic sense about the value of business analytics and how to implement it. To jumpstart that process, we blocked out an hour on the calendar for the team to do the ambiguously named “Mixpanel Activity.”

The Setup

Zanbato loves its board/video/athletic/trivia games, so we came up with a competition to get everyone using Mixpanel. We split up into 3 teams of 3 with the following scoring:

  • 1 point for a team name – I think every team competition is better with team names
  • 5 statistics questions about our production site usage from the Mixpanel tools. The first team to answer it gets 3 points, the second gets 2 points, and the third gets only 1 point. Answers were accepted in the form of showing the scorer the Mixpanel page with the answer. Here are a few examples of questions:
    • What percent of our users are using IE9 or less?
    • Over the last 30 days, on how many pages did we present the “unread notifications” icon on the screen? Out of those, how many times did they actually view them?
    • In September, how many times did a user navigate to analytics, and what was the breakdown in where they came from?
  • Come up with more interesting facts from looking at our data, where “interesting” is judged by the team. The first 2 interesting facts are worth 3 points, and each subsequent fact is worth 1 point. Present and discuss the facts with the team at the end
  • Come up with more events we should be tracking in our system and why they’re useful. Again, 3 point for the first 2, and 1 point for each additional fact. Again, present them at the end, where other teams could challenge if an event is already tracked
  • Come up with a new funnel for 5 points.

Each team was allowed to use only 1 computer (so that they were required to work together). I wasn’t so sure how long these tasks would actually take, so we were somewhat loose with that, but we ended up being roughly correct by budgeting an hour.

And of course, there was an incentive for winning in the form of Ghirardelli chocolate. Who wouldn’t want to win?

Results

IMG_1538

Overall, I was quite pleased by how the team received it. Everyone seemed engaged, and competitive spirits motivated the teams to work hard and quickly (and even strategizing to maximize points). We encountered a few hitches (like oddities with the scoring), but nothing major came up. Afterwards, we had a good, team discussion, and everyone got a better understanding of the value and how to use Mixpanel.

5 Lessons from the Mixpanel Activity

After the discussion around facts and new events, we had a meta-discussion about how the activity went and what we learned about using Mixpanel overall. Here were a few highlights.

1. Funnels are very nice.

Out of all of the different views and tools, we found the funnels the easiest to understand and derive insights. We attributed this to two major factors. First, funnels add structure to a variety of events, and that provides clear interpretations. Second, the funnel creator put deliberate thought into it, and their fore-thought makes it more likely to have value at all, especially compared to the mass of events otherwise.

2. Have a scheme for event names and properties.

The most frustrating part for the teams was trying to find a particular piece of information and not knowing what it was called or whether it was an event or property name. We had already spent a long time discussing different naming schemes and conventions abstractly. However, first-hand experience wrestling with the data made the concerns much more clear to everyone.

3. Clearly distinguish “mechanical events” and “intentional events” in our system.

We use Mixpanel to ask a lot of different questions, but there are 2 primary levels of analysis. First, we need to figure out if users are completing their goals: these are high level concepts like “send message” or “invite user” that can be accomplished in many ways. Although this may seem like the “best” way to group data meaningfully (especially if the interface or flow changes), we also need to work at a second, more mechanical level. This deals with specific physical actions taking place on the platform, like “open X popup” or “click Y button”. These judge whether users are able to achieve their goals at all. By grouping our events broadly into these categories, we can understand the data more broadly.

4. Everyone can contribute in what we should track and glean from the data.

I was surprised by the diversity of interesting facts and new events from the team. Hypothetically, the team could have converged low-hanging fruit, but we got events and facts from all around the site. Everyone is working on different tasks day-to-day, and we do have different priorities that we can measure. Although we had tasked 1 engineer and 1 designer as being our internal champions for business analytics, everyone has something to add in this activity.

5. Competitions can make tasks fun!

We had been struggling for a long time to get the team engaged with Mixpanel. Without any clear goals, we couldn’t really assign tasks for doing it. Beyond that, it was just a request during a meeting or a block of time set aside on the calendar that could easily be bypassed. Just a little bit of structure and friendly competition got our team excited and thinking about business analytics in a very different way.

Posted by: Kevin Leung, VP of Engineering