March 13, 2017

[Exercism - Behind the Scenes] That gamification question (again)

One of the consistent problems we've been struggling on Exercism is that a lot of people don't get feedback—at all. It's still possible to get a lot of value out of doing the exercises, and it's incredible how much you can learn by browsing other solutions, but we can do better.

The suggestion that has come up most frequently to try to address this, is to add gamification.

I've pushed back pretty hard on the idea. It's not that I think that gamification is bad, I think it's hard. It's hard to do it right, and easy to get it wrong, and if we get it wrong, it could destroy some of the fragile things that do work well on Exercism.

Last week Edward Anderson submitted an issue for discussion entitled Studying game design for increasing code review participation. In it, he summarizes the existing research on gamification and its effects on motivation. The TL;DR is that
[A] gamified system can preserve and enhance intrinsic motivation by aligning goals, and designing for autonomy through informational messaging, voluntary participation, and avoiding serious consequence.
After covering the basic science, Edward goes on to analyze the two main components of Exercism (exercise completion and code review), evaluating how well they fulfill the suggested criteria for achieving gameful design per some research on so-called skill atoms. Somewhat unsurprisingly, exercise completion does quite well in the analysis:
For exercise completion, every component necessary for a gameful skill atom is present.
Also, no surprise, code review does not:
The code review system, viewed through a skill atom lens, is poorly designed for amplifying intrinsic value.
The most exciting part of this isn't that someone knowledgable about the actual mechanics of gameful design has laid them out and then analyzed Exercism in light of this research. The most exciting part of this is that Edward goes on to describe a research experiment that he is going to run as part of his Master's degree program in Computer Science.

The experiment will do a one-week baseline test against the existing feature set, then test a set of experimental improvements for one week, and then finally we will remove the experimental features, which should provide a dataset that can speak to whether or not removing the gameful elements has a negative effect on people's intrinsic motivation to provide feedback.

Read all the background and details here:​

You Can Help!

We need help with copywriting and wordsmithing. Edward has written up a detailed issue describing the copy that we need, and has included both background and details about what might go into this copy.

If you don't feel up to writing a whole polished article, then even an outline would help. Or a rough first draft. We've noticed that once we have something in place, a lot more people have thoughts and suggestions about it.

Check out the issue here: Introduction to the benefits of reading and reviewing code