April 16, 2017

[Exercism - Behind the Scenes] Sustainability

I've known for years that Exercism was unsustainable. The first tip-off may have been when someone asked me how the project was going and I burst into tears and mumbled "I don't want to talk about it". That was early 2014. Since then I've tried a lot of things to keep afloat mentally and emotionally, and to reduce the amount of time I need to spend on the project every week. Unsuccessfully, mostly.

People have assured me that I don't owe anyone anything.

I have a colleague who works on an enormous open source project, used by millions of people. He once said that people tend to think that the competitors to his project are other projects that solve the same problem, but that's not true. The real competitors are video games and his dog.

I've considered just walking away. Playing video games, maybe getting a dog. But I haven't yet. Also I don't like dogs. Or video games, for that matter.

A few months ago I finally had a realization that let me think about this in a productive way: Exercism is unsustainable along two axes. The thing is, once I was able to untangle those two axes, I discovered that both of them are eminently fixable—separately. When I didn't see the difference between them, I made no progress.

User Experience

The first axis is the usability of the site. It's not a scaling problem, it's not a question of clean code or code craftsmanship. It's not about refactoring or decoupling or choosing the right backend language. It's about the basic ideas about what Exercism is and how it works, understanding what motivates people and how they learn and what holds them back. And then it's about building on top of that understanding to design a workflow and user interface that gets out of people's way. I wrote a bit about the design work in The Delightful Design of Exercism in January.

A suboptimal user experience is unsustainable because it generates lots and lots of issues. Support requests, bugs, confusion. A lot of people try to address each little issue (me included), but because the problem is so much deeper than that, we end up flailing around. Since we don't have a unified idea of Exercism's user experience, every single issue gets tackled as though it's an isolated problem, to be solved from scratch.

The solution is to develop a wholistic understanding of what Exercism is, and then design a site on top of that consistent understanding. That will reduce confusion, resulting in fewer support requests and fewer issues. It will also simplify the site, resulting in fewer bugs and better performance.

This in turn will let us focus on the more interesting questions of how we can support people's learning day-to-day, rather than just trying to mitigate people's frustration.

Maintaining Language Tracks

The second axis of unsustainability is the Exercism curriculum: we support almost 60 programming languages to some degree or another. This is a scaling problem, but not of code or performance. For the curriculum we need to scale the human side of things.

We know what goes into launching and maintaining a language track on Exercism. We don't know everything about it, but we know enough that if we had three or four people actively maintaining each track, no single person would have to make great sacrifices to keep the project running.

The solution here is to write down everything we know about maintaining a track, figure out what open questions there are, make decisions about how to handle those, and then actively recruit people to handle the tracks that are under-maintained.

Overlap Between User Experience and Curriculum

There are ways in which asking better questions about the user experience will impact the curriculum. A few weeks ago we started digging into the ideas of progression and completion as they pertain to Exercism's language tracks.

When are people done with an exercise? When are people done with a language track? If we have 200 exercises, do people need to do all of them to be done? (No, probably not, but some people are more completionist than others, and will feel an itch until they have done every single exercise). How do people know when they've learned something? Is progression linear? What about when some people want to dig deeper into a topic but others don't?

When we have good answers to these questions, it will likely impact the work that we do to implement and organize exercises in each track to some degree.

Legal Necessities

User experience doesn't design itself, and so I'm contracting with professional user experience folks to do the bulk of the work. I'm taking part as an advisor, but I don't have design skills. This costs money, so Exercism will be looking for sponsorships and partnerships to cover the costs of this.

I've worked with a lawyer to create a separate legal entity for Exercism. This will allow me to open a bank account for the project and handle things like hiring contractors, and it also offers legal protections. I probably should have done it long ago.

Other than being able to make contracts and accept sponsorships, I'll be signing over copyright to the entity. Other than that nothing will change noticeably.

Divesting Responsibilities

I've decided to explicitly fail at managing language tracks going forward. I won't be paying attention to undermaintained or unmaintained tracks. This means that some people will contribute and not hear back for a long time. I hate that, because it's a terrible user experience, but I don't have the capacity to keep these tracks going, so I'm going to knowingly disappoint people.

I'll be focusing on the more strategic questions for a while, trying to move everything towards sustainability, rather than keeping everything running unsustainably.