October 16, 2016

Keeping up with Exercism (is just not possible)

Trying to keep on top of everything that is going on with Exercism is pretty overwhelming. There are over 75 repositories, 50 language tracks in various states of development, hundreds of contributors, thousands of issues, and lots and lots of pull requests.

It's hard to make meaningful progress on anything when you're trying to keep up with a mountain of notifications. Lately there's been so much activity on the project, so many people contributing wonderful things, that I've found myself unable to keep up.

Failing to keep up has had me thinking pretty hard about how to help make it sustainable, and how to make progress.

In short: there are so many things that could be improved, and a lot of them require deep thought and hard work. In order to make room for that deep thought and hard work I'm going to focus not on the product, but on the project: on reducing the friction, sharing the responsibility, and spreading the work across more people.

This should free up resources that we can use to focus on higher-value / higher-impact activities like exploring this idea of using machine learning to group similar solutions of an exercise, which would let us show people conversations that happen on solutions that are similar to theirs, and also point them towards solutions that are very different from theirs. Both of these things would potentially be powerful learning experiences. Anyway. That's an interesting, hard problem that can't be solved while trying to process hundreds of email notifications.

I think the most important thing that we can do in this regard is to focus on the individual language tracks, making sure that every single track has three to four active maintainers: people who check in regularly, respond to issues, review pull requests, improve the exercises, and help define lots of "good first patches" that new contributors can come in and easily tackle.

The language tracks are a great starting point for a couple of reasons:

First, each track is independent. You can work on the exercises for the Haskell track without knowing anything about the Prolog track or the command-line client or the website. You just need to know a bit of Haskell.

Second, the work of maintaining a track is well-defined. We have almost 50 tracks (33 active ones), and each track consists of lots and lots of exercises. Most of the work involves improving individual exercises or reviewing changes to them, in addition to adding new exercises or reviewing these new additions. There's also some interesting work in adding tooling: continuous integration, linting, that sort of thing, and helping figure out what would make a better experience. Should we reorder the exercises? Deprecate an exercise? Reorder the tests? Delete some tests? Is there a concept in the language that we're missing an exercise for? Are the tests trying to decide too much? Too little? Is it idiomatic?

People have different reasons to get involved with maintaining a track, and different reasons to move on. We should constantly have people cycling into the project in order to make it easy for people to leave when they've gotten what they want out of it.

If you're learning programming, then getting involved in an open source project can be the first time that you're working on something that feels real. It's not a tutorial. It's not a guided tour. It's not a fake project that's just meant to teach you something. It's a project that other people use and appreciate—and that can be pretty exciting. It can also be a great way to develop skills, both programming and not-quite-programming-but-totally-related-and-definitely-useful skills, like using source control, submitting a pull request, reviewing code, discussing trade-offs, and just generally collaborating with people on GitHub. And lastly, doing this in the open means that people see your communication style and your skills, and this can be handy when looking for a job.

If you're already a programmer and you're using Exercism to ramp up in a new language, contributing to the language track itself can be a quick way to get more fluent in the syntax of the language and more familiar with the idioms. Reviewing pull requests and discussing with other contributors is a quick and powerful way of ramping up in a language, and adding new exercises is great way to get in some extra practice.

If you're already proficient in a language then contributing to a language track is a way to develop some of the skills that you need when mentoring new team members and untangling legacy code. The code review pushes you to think through trade-offs, and discussing these trade-offs helps you sharpen your understanding of various issues related to design, performance, simplicity, and readability.

It's not always clear why someone starts contributing.

If you've contributed I'd love to hear your thoughts about it. Why did you get started? What was hard or challenging or scary? What was surprising? What do you enjoy the most? What do you enjoy the least? What's frustrating? What's motivating? (Reply to this email! I read every email I get, even if I can't promise that I'll respond).

If you've moved on and no longer contribute, were there one or two things in particular that led you to stop? Or was it more of a general "did that, got the t-shirt" sort of feeling?

There are a few things I think we can do in order to make it easier for people to make their first contributions, and also to become more regular contributors and eventually perhaps maintainers.

Document language track maintenance

What are the things that maintainers do? How do they do it? How can it be done better?

I've started working on some documentation, and we've got bits and pieces in the contributing guide, but I think there's a lot more that we can do here. If you have thoughts and ideas about this, please join us in this discussion.

Create "content strategy" for issues

One thing that makes it easy for people to get involved is small, focused, well-documented and well-described issues. Each of the language tracks has a number of categories of things that people can contribute with, and very often it's in the form of "Do X for each of the existing exercises". In other words, for a given thing that needs to be done we can create a script that generates one issue for each exercise, label it with good first patch, and then make it easy to find these issues for people who want to get involved.

We've had good results with this in a couple of tracks, and I think we can be more deliberate about doing this more broadly in all of the language tracks.

Find potential new maintainers

One thing that we've not done at all is specifically go looking for new contributors at meetups and conferences, or in the various discussion forums for individual languages. I'd want to keep it classy. This isn't really marketing, just a way to make it clear that we welcome contributions, and it's fun and you learn a lot, and it's not hard to get started. If you have ideas about this, come share them in this issue.

Create tooling to identify active contributors

There are so many people involved in Exercism, and so many people doing amazing work. And I don't always see that work. There are people who are doing the work of a maintainer (discussing issues, responding in issues and pull requests, mentoring new contributors, performing code reviews), they just don't happen to have commit on the repo. I'd like a way to discover who's doing this work so that I can give them access to the repositories that they're effectively maintaining.

... and other things, I'm sure.

So that's it. Maybe if we get enough people involved, none of us will have to keep up in the first place.