Sharp Tools

Developers love borrowing concepts from other trades to describe their work. We especially enjoy to comparing ourselves to woodworkers. The phrase “sharp tools” brings to mind a chisel chopping out an oak mortise, a hatchet splitting a well seasoned timber, and a sawmill slicing a tree into boards. Programmers use the phrase “sharp tools” to refer to tradeoff of productivity and the bloodlust and gore that awaits the careless worker. I’m a sometimes woodworker and a full time programmer and I have used truly sharp tools for both.

There are two wood tools that I keep “shaving” sharp in my shop - chisels and plane blades. You probably know what a chisel is, but lots of people might be unfamiliar with a plane. It is designed to shave off bits of wood in impossibly small increments, the best can get wood shavings in 0.001 inch thicknesses. It makes more sense to see it, than to have it described:

The blade on this device is razor sharp (literally), and it has to be. You can’t see in the video, but this man, likely 180+ lbs, is putting his whole body weight to provide the forces necessary to separate the bonds and literally shave off a section of wood. If a tool is not sharp enough to shave arm hair, it’s going to be much harder to make whispy thin curls of wood shavings:

This is what sharp means to me. You can’t buy sharp, you have to create it and maintain it. Sharp is a constant state of maintenance and honing. These tools are a joy to use. The tool does it’s job remarkably well. A sharp tool is one that works with you, a dull tool constantly reminds you of its shortcomings and fights you every step along the way.

There is a fallacy common to “advanced” developers that tools designed to be “simple” or “easy” or “safe” are not sharp. They are “beginner’s” tools. Some devs see a large framework and think themselves above it. They see complex tasks wrapped up in simple APIs and ask “where is the blood?”. The association between effective devices and their carnage so interlinked that they cannot be separated.

While a jointer plane is a remarkably sharp device, I’ve never once cut myself while using it. This isn’t a statement of skill, but rather a comment on the design of the device. It does everything I will it to, and more. Its clever design means that this razor sharp edge is barely protruding from a safe surface. When using it your hands are on the opposite end side of the body. Here’s a view from the business end:

You can see the blade sticking through the “throat” of the plane. This device provides me with safety and a simple interface, it certainly isn’t “just for beginners”. This design is an evolution of planes over centuries of use. This safety does not diminish their usefulness to users of all skill levels. In fact, while a beginner might use this tool unaware of the full feature set of the tool, only a truly advanced user can appreciate the subtleties.

Just because there’s no blood doesn’t mean your tool isn’t sharp

Pain and difficulty of use do not indicate an “advanced” tool. Developers of all levels can appreciate and consume a good API. I’ve never met someone who complained because the documentation was too well thought out. I’ve never been upset about an error that was too helpful. A good tool guides and compliments the worker.

This isn’t to say that all sharp tools are also bloodless. On the woodworking forums there’s a number of posts reminding us that chisels can send you to the emergency room as fast as a power tool. Like a gun, don’t point the dangerous end at anything you’re attached to. Another tool I own is a “draw knife”. You use it by pointing the sharp end towards your body and pulling. It is probably the scariest thing in my shop:

This brings me to the second, somewhat contradictory fallacy of “advanced” developers - they think that if a tool is simple that it must inherently hiding painful secrets.

A common example here is an ORM. On the surface they can seem very simple; automating tedious tasks. If you’ve used one long enough you’ve been bitten by it. A recent example for me was bringing down a production database. How? I used the postgres int datatype instead of bigint for a primary key. I eventually created so many records that I ran out of numbers and the database started throwing errors. Migrating all the entries locked the database for a few hours:


Why did this happen? I used a DSL provided by my ORM for creating the schema and it wasn’t obvious that it was using int by default. I know what you’re thinking and no, it wasn’t Active Record. While the ORM was originally super fast in helping me define and migrate my schema, when I slipped with it, it wasn’t pleasant.

This was an extremely painful experience. As humans we learn from pain, we try to avoid it. The longer you’ve been programming the more adverse to specific pain points you become.

“Fool me once, shame on you, use an ORM again, shame on me.”

This type of mentality ends up with a massive case of NIH. The fallacy is equating the inherent pain caused to the ease of use with simplicity of the tool. The solution in the user’s mind is clear. If we make something more complex and explicit we wouldn’t have hit that pain. Either people will go to hand crafted artisanal SQL, or to a smaller home brew ORM. Unfortunately it’s usually too late when they realize their micro library has a SQL injection vulnerability, or another critical mistake. More blood.

If simplicity hides pain, and complexity also hides pain, what’s the solution?

To avoid pain…find where pain lives and take steps to avoid it.

While working in the shop I wear canvas apron and use a bench that is bolted to the floor. I clamp my wood securely to the table. My tool might slip, but it’s not compounded by additional failures.

In some cases you can modify a tool to your needs. For example you can use the drawknife for cutting corners on square stock. For this one off task you can attach bronze drawknife guides:

It won’t prevent all nicks and cuts, but it will provide an extra layer of safety when things do go wrong. It also helps us do this one task faster than we could without any protection at all. A good tool helps to guide our work.

In the case of the ORM, instead of giving up we can find the pain and fix it. For this case we can default foreign key columns to bigserial. This doesn’t make the ORM a “tool for beginners” it makes it a tool for everyone.

On a sidenote you can use pg:diagnose on Heroku to see if you’re using an integer primary key and are running out of numbers.

The difference between an “advanced” and a “beginner” programmer is not the tools we choose. The difference is that when a senior programmer experiences pain, they dig in with a retrospective. Why did we use that tool incorrectly? Was there documentation that explained that problem? Why not? Can we make this tool safer without sacrificing performance or usability? Yes? Let’s raise awareness by opening an issue. Fix it by opening a pull request. At the very least write some docs.

An example of this process could be my PR preventing destructive actions in production databases. If there’s not an easy technical solution, maybe you need to grab a soap box and raise awareness. A shoddy worker blames their tools, a good worker makes better tools.

There are times when we experience pain even with good documentation and safeguards. It doesn’t mean bloodshed has to be avoided at all costs. It’s natural to recoil when we hit pain. Seasoned developers can learn from that pain and reach deep into their tools to make them better. Their work isn’t always selfless or charitable. It’s not always about “giving back”, if anything they are “fixing forwards”. The next time they’re tired and frustrated they’ll be glad they spent the extra effort making sure that pointy corner was rounded.

The beauty of open source is that we can all learn from each other’s mistakes. The tragedy is that so many people fail to realize they aren’t alone and don’t share their lessons. Some have never used a truly powerful “sharp” interface that is also effective and safe. Some have, but the experience was so seamless the presence of the tool wasn’t even noticed. When such an interface is missing, they don’t spend much time thinking of how to create it.

While using woodworking is fun for me, there is a mantra in the shop. Be present or be hurt. You might be the best worker in the world but if you go into the shop tired, upset, hungry, or drunk you’ll be sorry. The same is true for programmers. We shrug exhaustion and wear it as a badge of honor. We drink beers at happy hour then get paged. We push our bodies and minds to such lengths that frustration doesn’t even begin to describe our mental state. When we do this, we are all beginners. Either we only use tools when we’re inspired and working at 100%, or we start accepting that sharp tools need to be forgiving of our mistakes. A good tool can be appreciated by the elite, a great tool can be appreciated by everyone.

Stay safe. Stay productive. Stay sharp.

If you like this post you might also enjoy Do you Believe in Programming Magic?. You can get these and other articles like them direct to your inbox.

Tips for Fixing Open Source bugs

Support isn’t sexy, but it’s necessary. How open source software is supported is just as important as how well it works. Given the choice between building awesome new features or carefully reading and responding to 10 bug reports, which would you choose? Which is more important? When you think of Open Source maintainers what do you see? I see issues. I see dozens of open bug reports that haven’t been responded to in days. I see a pile of feature requests waiting to be worked on. Now when I open those issues, I see maintainers spending most of their time trying to get the information they need. “What version are you using?”, “was it working before”, “can you give me an example app”? Would you rather maintainers spend time asking for minute in-bug reports or fixing issues?

This post originally published on

When I think of open source bug reports, I think of hospitals. A bug report is like a sick patient walking into an emergency room. No one knows exactly why they need to get better. They have self reported symptoms, but more information is needed. When someone goes into an ER, they don’t immediately go to a thoracic surgeon to get their blood pressure taken. Instead, they go to someone that records their vitals, take care of paperwork, and then makes sure the right doctor is assigned to the case. Hospitals call this process “triage”, from the French “separate out”. This person or team of people doesn’t need to know exactly what is wrong with you, they only need to know what information the doctor will need. Every minute someone spends on triage, is an extra minute someone else gets to dig into the root problem.

Triaging bugs is a necessary skill for any open source maintainers, whether they’re working on a newly minted library or helping out with a 10 year old framework. It’s also a skill that can be picked up relatively quickly without years of required programming knowledge. If you want to get started in Open Source but worried you’re not ready, triage can be a great first step.

Getting Started

First up, consider Open Source software you use. Think of the languages, and then think of any open source libraries you use. For example if you’re a Python programmer you might use django, the Requests library, and a whole lot of other packages. Pick a few that you think are very popular. These projects will have a larger existing team and you’ll get good experience from them, however you’ll also want to pick a few smaller projects. While you might get lost in the crowd with a larger project, 1 person can make a huge impact on a smaller one.

Once you’ve picked a few libraries, find their issue trackers. Try to think of a good trigger when you’ll be able to spend some time on open source. Maybe it’s every day in the middle of your morning coffee, or right after your once-a-week team standup, or maybe you can use the time right before your once-a-month community meet-up. I maintain an Open Source project that sends you issues to your inbox. Find a consistent schedule and pace that works with your life.

When you sit down to triage, make a goal. Maybe you want to find 5 issues to comment on. What exactly should you say? There are 3 states to an issue:

10) A bug is reported. 20) Extra information is gathered to reproduce the bug. 30) If the bug can’t be reproduced GOTO 20.

It’s your job to gather information and move towards a resolution. Let’s look at some common issue types and things you can do to help.

Stale Issues

Issues have a shelf life. If they’re not actively being worked on, they can go bad. Maybe the original reporter found a work around, or maybe the bug resolved itself.

To find issues, sort by “updated at” if the issue tracker allows for it. Go through a few of them and leave a comment:

Can you confirm this is still an issue?.

If the issue is really old and someone already did that, you can push for the inactive issue to be closed:

There is no activity here for the last 2 months, let’s close this issue for now and re-open if needed.

An issue is only good as helpful as the people working on it. If the issue isn’t important enough for the original reporter to care about, there are likely more pressing problems that maintainers should focus on. Closing an issue isn’t a finality, if the problem comes up again, the issue can be re-opened or referenced by a new issue. Often if a thread goes on for too long it can be difficult to easily scan all the conversations, sometimes starting fresh is the best way to move forward.


When someone’s program was working but after a version update no longer works, it’s a regression. These are usually the easiest bugs to work on as a maintainer. Usually these bugs will say something like “it worked before” but not give specifics, which is where you can come in:

What version are you using now? What version were you using before when it was working?

Usually these reports will have some kind of reproduction instructions. Once you find the version numbers, try to reproduce the problem. Time box it to maybe 5 or 10 minutes so you don’t sink too much time into the problem. Take notes. If you run into any problems or questions when trying to reproduce the issue, write them down. 99% of reports will not include enough information to reproduce the bug. To the reporter it’s almost impossible to NOT hit the bug, they will give you wide sweeping brush strokes and expect you to hit the bug just as they did. In reality they’re probably hitting a stress case or made an assumption that most other people don’t. Your job is to find that assumption.

If you can’t reproduce the problem, you’ve got two options. Either you can report back with detailed information about how you tried to reproduce the issue and how it didn’t work. Hopefully this will jog the reporter’s memory and they will point out a crucial step you missed. The second takes more time from the reporter but in my experience it will save everyone time in the long run.

Can you please attach a small example app that reproduces the problem?

If the original reporter makes a new repo with the minimal artifacts required to reproduce the problem, you might still need to have some back and forth with them, but it will be much less. If they don’t want to make an example app, tell them about how you couldn’t reproduce and remind them that normally if a bug cannot be reproduced it cannot be fixed. People triaging and fixing open source bugs have limited time, the faster it takes them to reproduce the problem, the faster they can fix it. If a problem is important enough to you to report, it is important enough to report well, which means going the extra mile and including an example app.

If they already included an app, try to reproduce the problem and report back if it’s valid or not. When a maintainer sees the issue if they know someone else reproduced the problem, they’ll know they can reproduce as well. It might not sound like much, but it’s really helpful.

Trust but Verify

Reading bug reports is a bit like reading a story written by an unreliable narrator. They all mean well, but submitting bug reports is as much a skill as triaging bugs. Half of your job is education and coaching bug reporters on what a good bug report looks like. Likewise, it can be tempting when you see a report that looks like a “Usual Suspect” to declare the underlying problem code without verification.

When someone comes along to look into fixing a bug, we want them to have a consistent narrative. One thing that can help with getting this information is a good rapport with the person submitting the bug. You can thank them for their time and their report. If they have follow up questions about the triage process “why exactly do I have to make an example app after I gave you reproduction instructions?”. Remember that this might be the first issue they’ve ever opened. We’re all on the same team and working towards the same goals of making our open source software better. Humans are a large part of software and they all come with emotions. When everyone is level headed and respectful is when the most gets accomplished.

Pull Requests

A PR is a bug report with a solution attached. Go through the same motions. Make sure you understand the problem, if there isn’t a linked issue ask for a reproduction case. In addition, see if the coding style matches the rest of the project. Does the PR come with tests or a CHANGELOG attached? What are base things that every new feature or bugfix to the project must have?

Not a Bug

Frequently people will open up bug reports that have nothing to do with the library or have very well documented fixes. Instead of replying “RTFM”, closing, and locking the issue, put yourself in their shoes. Maybe they saw the documentation but were confused. Maybe it could be documented somewhere else? It costs you nothing to be nice, and may earn a contributor in the future.

Here is an example from a comment I made in 2012. The behavior was unclear so I explained why the software worked as it did. I also linked to the relevant documentation and invited them for followup questions.

Every time an issue is closed you should ask yourself, why did they open that issue? There are lots of technically “not a bug” problems in software that are masked by lack of documentation, meaningful error messages, or properly written deprecations.

Sometimes issues are opened to ask questions “I want to do this " but i'm not sure how. This is a clear sign that documentation could be better. If you're looking for easy commits, look no further than writing documentation. The best stories are written by the people who experienced them. The best docs are written by users. If you don't have time to add docs, you can leave a note explaining where you think a doc change should have gone and ask the reporter to make the change. If your issue tracker is small, maybe open a new issue pointing out the documentation change. Often it's easier to just make the change.

Nothing to do

If you find a bug report that is active and has already been reproduced, you might not need to do anything. Great! Find another issue or take the time to read how they got from bug report to reproduction. Is there a consistent theme or phrase other maintainers use? Maybe they have their own techniques for triaging issues that you can use.

Historian - Code, culture, and conventions

Over time you’ll become more and more familiar with the projects you’re triaging. You’ll know which maintainers love fixing bugs, you’ll be able to predict which maintainer will ask to trim whitespace. Every project has their own conventions. For example, I’ve worked with projects that don’t accept any refactoring as they mess with the ability to blame. Some projects welcome feature proposals in issues, some don’t and ask for discussions to go to a mailing list. As you see more issues, you’ll see what the existing maintainers tell people who open issues. Over time you’ll notice an issue almost identical to a previous one. It’s good for you to provide context. If you know that another maintainer will ask for something, you can be pre-emptive and ask for it first.

Eventually you’ll start to see patterns in bugs, or documentation. While you’re only signing up to triage issues for now, it’s a very slippery slope to full blown bug fixing and code contribution. This is a good thing. If you already know what the other maintainers will say about an issue or a PR you open up, the process becomes a whole lot less scary.

Start at the Beginning

While this article is a bit long, the core ideas are simple. Maintainer time is valuable. Triaging issues takes a minute of your time and saves a minute of maintainers time. Triaging is a skill you can learn with common patterns that can be easily applied. If you’re still struggling with where to start I recommend going back to your list of libraries, find or add them to CodeTriage and subscribe.

Contributing to Open Source doesn’t have to be a full time job. If everyone chips in a little the contributions really add up. Your small contributions can make a big impact.

The 'Goal' of Performance Tuning

How do we make our programs faster? How do we make anything faster? My first co-op job was working in packaging. They had a small Industrial Engineering library and let me read from it at work. The first book I read changed my life and my way of thinking about problem solving: The Goal by Eliyahu M. Goldratt. Just about every time I work on performance, it’s impossible for me to not make comparisons to The Goal in my head. In this post, we’ll look at some stories from the book and how they apply to performance tuning in programming.

Keep Reading on Codeship

Saving Sprockets

What do you do when a maintainer leaves a project with over 51 million downloads? That is what we had to consider this year when Sprockets lost the developer responsible for more than 70% of the commits. We’ll explore this and more through my RailsConf 2016 talk and the transcript below.

I’ve spoken in 12 countries on 5 continents and this is my favorite talk to date. This talk was difficult. The ideas were in my head, but I couldn’t get the words right for the longest time. This was an emotional talk for me . Writing this talk turned into a soul searching journey through open source. What does it mean to be a maintainer? What does it mean to leave a project? What does it mean to respect and help a maintainer? The road was long and perilous but I’m very happy with the talk. Hopefully you’ll join me on this archeological expedition.

This version has intro music and closed captions. Transcript and slides are below.

Here’s what some people had to say about my talk.

  • “That was an epic talk on many levels” - [@tankard](

  • “Every single Ruby developer should watch @schneems’s [Saving Sprockets] talk” - @samphippen

  • “good talk man, every conf should have sth like that until people learn how this oss thing works :)” - @solnic

  • “Thoroughly enjoyed, would attend again, ⭐️⭐️⭐️⭐️⭐️ / ⭐️⭐️⭐️⭐️⭐️” - @chrisarcand

  • “I loved it […] but I am biased in favor of smart ideas full disclosure” - @searls

  • “Go home people, @schneems just had the best slide analogy of #RailsConf” - @cecycorrea

  • “Don’t quote me boy” - Eazy-E



This is the text of my talk with some minor adaptation to make sense when reading without slides. It’s about 5,000 words or the length of two of my normal blog posts.

We want to start out asking the question, Why does Sprockets need saving? For those of you who haven’t been around since May of 2011, it is the premiere feature of Rails 3.1, and Sprockets is the Asset Pipeline. Sprockets actually came first, before it was ever wrapped up into Rails. One thing I wanted to mention before we get too carried away, is that you don’t need to look like Indiana Jones in order to maintain open source, it just so happens that he is my Sprockets spirit animal.

So, from 2011 to 2016 Sprockets has had 51 million downloads, and I’d like to put that into perspective. Rails has had 65 million downloads, so Sprockets is pretty close, and, of that entire library, one developer is responsible for 2027 commits, which happens to be about 68% of Sprockets. That’s one person. Compared, in contrast, to a Ruby hero, Rafael Franca, who has over 5000 commits on Rails. This accounts for only 0.9% of Rails. 51 million downloads, one developer, and one day, Josh left. “I’m cutting it, I’m out, I’m gone.”

So, when something like this happens, what should we do? Should we, as a community, abandon Sprockets? There’s a lot of people who said “I don’t like Sprockets”, and “it’s got problems”. To them, I ask: what are the problems? Do you know what they are? Because we can’t fix what we can’t define, and if we want to attempt a re-write, then a re-write would assume that we know better. We still have the same need to do things with assets, so we don’t really know better.

I think we should stick with Sprockets and make it better. Assets are really the easy part of Sprockets. There’s a whole bunch of edge cases. Also, Sprockets has a really well-defined and established API.

Losing maintainers is inevitable, and it’s not always expected. Jim Weirich was the creator of a amazing library that we’ve all used called Rake, and in 2014, Jim passed away very suddenly. It wasn’t like anybody saw this coming. He wasn’t working with someone to pass on the software to for a long period of time. And so, whether a maintainer suddenly walks away or they pass away, it hurts. We, as library consumers, have to cope with it, and there’s a lot of different ways that we do that. We might go through a period of denial and say something like “they’re going to come back”. We are going to get this person back into our lives. You might be angry and say, “leaving is selfish”, or “that was such a jerk thing to do”. You might start bargaining and say “Maybe if we hire them, they’ll work on it full time and we can get them to come back”, and eventually, acceptance. “They’re not going to come back, who’s going to take this over?”

The number one rule, is that a maintainer does not owe you anything.

A maintainer does not owe you Anything

Not even an explanation. If you’re going to leave a project, or someone is leaving a project, it’s a very personal decision. I actually reached out to Josh and said, “Hey, man, let’s talk about this. I’m giving a talk at RailsConf. I need some content, gotta help me out,” and Josh didn’t want to talk about it, and I want to respect that wish, and I also want to respect what he’s done, which brings me on to the number two rule.

The number two rule, is that you owe a maintainer respect.

Some people will say things like, “Oh, but I really hate this project.” It is possible to critique software without demonizing the creator, and, as a matter of fact, I’m going to critique the crap out of Sprockets. Notice word choice was intentional here. I’m critiquing and I’m not criticizing. I aim to be productive with the words that I’m using. I want to find what is bad and then make it better. Originally, when Sprockets fell into my lap, and somebody said, “hey, do you want to be on Sprockets core?” I was like, “Sprockets, why did it have to be Sprockets?” You are not your software. Josh gave years of his life to the project. No matter what you think of the project, or what you think of how it was maintained, I want you to give a thank you to Josh.

Rule number three, is that words without actions are empty.

I want you to be actionable with your critiques and think about this. For example, we have a Hacker News comment which says, “unless they add this feature to Node, I see this as ugly and barely usable”

When I read that, that’s not going to make me want to go out and help them. Instead, they could’ve easily said, “Hey, this is great, this is amazing, I love it. It looks like they don’t have this thing I need, and, as a matter of fact, I can’t use it and here is my use case” and that’s actionable feedback. You can critique without criticizing. So, I want you to ask yourself, “is this comment adding anything?”

Hyperbole in comments and blog posts is good for laughs and fake internet points but it doesn’t help. I want you to be honest with your critiques. I want you to be productive. Here is the babeljs creator tweeting a screenshot, again, coincidentally, from Hacker News, and it reads, “Babel sucks. I never thought I could hate something so strongly.”

Wow, that’s really going to encourage that guy to go out and fix all of your problems. You might disagree and you might have very strong opinions, and those opinions might be very negative. Even if that’s the case, this software is in your life for a reason, and if you can figure out why those things hurt, why you are having those negative feelings, and you can enumerate that in a productive way then it helps everyone. Complaining by itself accomplishes nothing. When I started the talk, I wanted to touch on “how do we keep a maintainer longer?” Or if you’re maintaining software, how can you stick around longer? To do that we need to look at what maintainers want. We also need to do our homework and ask ourselves “is there any value in a maintainer sticking around?” All maintainers will one day leave and we can either have a maintainer that just mic drops and you never see them again, or we can have somebody who’s passing the torch and doing a graceful hand-off.

While I’m working on Sprockets, there’s so many times that I say “this is absolutely batshit insane. This makes no sense. I’m going to rip this all out. I’m going to completely redo all of this.” And then, six hours later, I say “wow, that was genius,” and I didn’t have the right context for looking at the code. Maintainers are really historians, and these maintainers, they help bring context. We try to focus on good commit messages and good pull requests. Changelog entries. Please keep a changelog, btw. But none of that compares to having someone who’s actually there. A story is worth 1000 commit messages. For example, you can’t exactly ask a commit message a question, like, “hey, did you consider trying to uh…” and the commit message is like, “uh, I’m a commit message.” It doesn’t store the context about the conversations around that.

So, maintainers are historians, and we can keep those maintainers longer by giving them what they want. Maintainers want respect. They want to be appreciated. They also want help, and I know all of you are thinking, “Ugh, this is the part where he’s going to be like, asking me to help, and I really don’t want to do that.” Or, maybe you already are helping. Maybe you’re saying, “I don’t have enough time”, or “ughh just fix all of the things for me it will be faster if you do it”. I’m here to say that if you have five minutes to snap-to-face-to-fours-tagram, then you have five minutes to help open source. You can contribute to docs. You can read the guides. You can fix typos. Maybe you found a really surprising behavior. Was that behavior documented? If not, then go ahead and add it to the guide. If you have five minutes to help, then you can submit a bug report. Seriously, the maintainers have no clue that things are broken. You might be thinking, “oh, there’s thousands of people using Rails and all of them have reported this thing.” No.

The question of “why is Sprockets bad?” I don’t know. Nobody actually gives me actionable bug reports. So, if you have five minutes to help, then please let us know what your problems are in a productive way.

Critique over criticism.

Another thing you can do to help is sign up for a service that I wrote and maintain called CodeTriage. You can go there and sign up for a project you care about and it will send you an issue in your inbox once a day. It’s a very actionable way to get started. When you get the issue, you can ask common questions like “what version were you running on?” or “Was this working previously?” Let’s step back, would you rather the maintainer of that project spent the time fixing bugs or would you rather they spent the time asking for insanely small minutiae on the issues? Anyone can ask those questions. And it might only seem like you’re giving a minute or two out of your day, how could that be impactful? If you give a minute, you are actually saving a minute of a maintainer’s time. A little bit of help can go a long way. And if you don’t help, then who will? It also has the benefit of exposing you to different parts of projects, which helps you grow as a developer.

If you have 10 minutes to help, include an example app to reproduce the problem. Example apps are amazing. I get all these bug reports that are like, “well, first I run rails new,” and then I go and try the instructions and come back an hour later “sorry, couldn’t reproduce,” and then they respond, “oh yeah, I forgot to add this other thing,” and then I try it, and couldn’t reproduce, and I waste hours of my life that I could be spending fixing bugs or writing new features. As the reporter, you waste hours of your life. Nobody’s happy. Instead, you can ask and say “here’s an application that is going to reproduce my problem.” Make a new project with the bare minimum to get the bug to show. Put it on<username>/ExampleApp if you don’t have that yet. You can even choose ExampleApp1 or ExampleApp2 as a repo name. I’m not picky. If you give a minute of your time, then you’re going to save a minute for a maintainer.

I personally challenge you, if you haven’t already, please try and make it your goal to produce one example app this year. It is so helpful.

If you have 30 minutes to help, you can try fixing a bug. Anybody’s bug or your bug. It’s not as hard as it sounds, just timebox it. Even if you don’t fix it, then you’re guaranteed to learn something. You’re guaranteed to read other people’s code. You’re going to be navigating and debugging other people’s code, which happen to be highly marketable skills. With all of this, I know you’re like, “Okay, uh, I don’t want to do that like, every time,” and that brings me back to club soda.

I drink club soda at home, and I don’t like putting the whole thing in my refrigerator. So instead, what I do is I put like three or four in to get cold. Then when I pull one out, I put one back in. However, sometimes I run out of club soda. How did this happen? Is somebody stealing my club soda? Is my dog drinking my club soda? The rule is one in, one out. It’s pretty simple. But clearly it’s not sustainable. Instead, what I found, is that if I put in two cans instead of just one can that I somehow end up with 3-4 cans in the refrigerator. Now I always have club soda. You got it? It all makes sense. No?

So what I’m saying is, you don’t always have to contribute to open source. You don’t always have to make an example app, but just every once in a while please go the extra mile.

These are all different ways that we can help a maintainer. Ways we can make their job a little bit easier. So how do we transition from one maintainer to another maintainer? Well, what is a maintainer? We’ve talked about this. A maintainer is somebody who knows the stories. A maintainer is someone who’s going to take 5, 10, 30 minutes out of their day to help. If a maintainer is somebody who helps, and the act of helping preserves history, then maybe the act of helping is the answer to keeping a maintainer. Also, the act of helping is the key to creating maintainers.

If you have people familiar with your code, whenever you actually go through that hand-off process, people aren’t just starting from zero.

The next question we have is, how can we foster a culture for helping? How can we get more people to help? If you are a maintainer, you want people in your project. If you’re using that project, you want more people helping and contributing because that makes it better. So, how do we foster that culture for helping? We talked about what maintainers want, but we never really talked about what the helpers want.

Well, helpers want documentation. They want sane code. They want what regular users want: good user experience. Non-magical code, backwards compatibility, good deprecations, reliable tests. These are all things that are interesting to them. So let’s look at one and compare it on the Sprockets chart.

Documentation. Sprockets has 73% documented methods. Seventy-three percent of all methods are documented. That’s a lot. That’s really up there. On a side-note, I think that method documents are kind of like unit tests. They are very focused on one part, and don’t necessarily tell the whole story, so it is possible for those comments to get a little bit out of sync with reality. I also highly recommend keeping a README. A README I see as something more like an integration test. It’s going to tell a little bit more of the whole story, and if we look at Sprockets, well, Sprockets has about 2,600+ words. That’s a pretty long blog post. That’s a pretty substantial README. That’s a pretty long story. If I’m here, telling you that helpers love docs, and I’m telling you that Sprockets has docs, why doesn’t Sprockets have anybody helping?

I put on my design research hat. I went to design research school and I learned about user stories. So, we are going to actually consider the people using our product. I want to introduce you to Pedro. Pedro enjoys long walks on the beach. Favorite food is bagel bites, and he’s building the next Uber for goldfish. Pedro is a Rails user, and Pedro cares about the Rails interface. This is going to be, how do I get one file to require another one? I want to know what I actually have to type into my project to get it to work. Now, I don’t care about all that other stuff. I just want the things I need now.

Next up is Pat. Pat is addicted to ES6. I know. Pat loves to fly fish and Pat is a plugin developer for Sprockets. By the way, did you know that Sprockets has plugins? Okay, well, they’re not called plugins, they’re called processors and transformers and compressors and like 20 other things, but it does have a plugin system. And Pat cares about the processor interface. They maintain one. They want to know that, whenever we pass the hash of things to them, what is going to be in it? And what can I do with it? What should I do with it? Pat wants this documented when Pat is working on their plugin.

Finally, Diana has a dog named Exception and hates mustard. Both of these are highly relevant to Diana’s job as a Rails developer, i.e. somebody actually developing Rails, somebody actually building an asset pipeline. Diana cares about the low-level interface. What does that mean? Diana cares about what are the classes she can use? What are the methods on those classes? All of them. If she wants to be able to disable gzip. Here you go.

These are all different people with very different needs who need different documentation. Don’t make them hunt down the documentation that they need. When I started working on Sprockets, somebody would ask, “is this expected?” and I would say honestly, “I don’t know, you tell me. Was it happening before?” And through doing that research, I put together some guides, and eventually we could definitively say what was expected behavior. The only way that I could make those guides make sense is if I split them out, and so, we have a guide for “building an asset processing framework”, if you’re building the next Rails asset pipeline, or “end user asset generation”, if you are a Rails user, or “extending Sprockets” if you want to make one of those plugins. It’s all right there, it’s kind of right at your fingertips, and you only need to look at the documentation that fits your use case, when you need it.

We made it easier for developers to find what they need. Also, it was a super useful exercise for me as well. One thing I love about these guides is that they live in the source and not in a wiki, because documentation is really only valid for one point in time. Otherwise, you end up in the wiki like, “if you’re using this version, do this, if you’re using this version, do this,” and it’s 20 versions and it’s no good. Helpers love contributing to docs, so you know what? We can make more docs. We can make our docs better. Those docs are going to be the gateway drug to code contributions.

The next thing I want to talk about is sane code and realtalk. Sprockets was designed to solve problems, and sometimes, when it’s putting out a fire, it kind of feels like it’s starting other fires. Maybe making additional problems that you didn’t see before, and you don’t know why it fails. The reason you don’t know, is because Sprockets isn’t talking to you. How does code talk? Code can speak to you through errors, and I’m not talking about “something broke”, or “no method error, on nil”. I’m talking about: I want my error to say, “this broke”. I want my error to say, “ID key is missing”, look here, “this is the thing that you’re missing”. Good errors are instructive, and Sprockets will have better errors. It doesn’t yet. I do care about this. I am the owner of a gem called, wait for it, Sprockets Better Errors. That gem was merged into Sprockets Rails, but yes, I had some good ideas for better errors in Sprockets itself.

The other way that code can speak to us is through deprecation. Now, deprecating something in a code comment is not enough. Right now, Sprockets is doing this. They have a little code comment, and they’re like, “by the way, this method is now deprecated.” We will just delete it, and it won’t be available, and you never knew that because A) No one is casually reading the method documentation, and B) Who has the time. It’s not as though every single time you upgrade every version of Sprockets, you’re going to read every method comment in the documentation. Are you going to do that? No. You cannot just sit back and break your API, especially when you have 51 million downloads. That is unacceptable. Since your code knows when somebody’s using a deprecated interface, we can yell at them.

We have these things called deprecations. I wrote a detailed guide on using Deprecations in your library.

Sprockets 3.X will have deprecations before we go to Sprockets 4.X. We have a branch. We’ve started working on this. If you’ve not implemented deprecations in your own project, it’s super simple. You say “hey, the thing you’re using is deprecated. Use this other thing that’s not deprecated, and here’s where you were using it.” It’s kind of like a three-step process. So deprecations are going to nudge people into the right behavior. They’re going to help get people to upgrade, and they also help with API design, because, if you can’t write a good deprecation, then guess what? The interface probably wasn’t the best.

In the talk I mention an hash key based API, turns out I was wrong. More information can be found at this PR. This just further underscores the importance of having a maintainer who knows what is up around to sanity check things for you.

This is my favorite section coming up. I hope you’re paying attention. Sprockets suffers from something that I like to call the god object problem. It has this one main class that has all of these concerns mixed in with it. It’s one object with 105 methods. It’s using a lot of them, and you ask yourself, “where did that method come from?” and you look at this source code, maybe it came from


This is my personal favorite,


Is mixed into


Which is then mixed into


Which is then included in


Which is inherited by


Which is then wrapped in cache by


It’s impossible to just glance at something and know how things are interacting. You change this one method that you thought was only being used in this one part of the project, and something else breaks.

For more information about how Sprockets work, I highly recommend you go to Rafael’s talk

What is the solution to god objects? We can move logic over to helper classes. For example I introduced the URITar class while I was adding new functionality. It takes an absolute path and trims it down to a relative path, or it can take a relative path, and make it an absolute path. We need this for storing things in the cache. The beautiful thing about this is it has a couple extra methods that make it a little bit cleaner that are not actually exposed to that god object API. We can expose only the things we need. So it’s going to minimize that god object API and it also, hopefully, produces small, easy-to-read files.

You can look at that file and say to yourself “I vaguely understand a tar utility can expand or compress a file; maybe it’s related to that, but for URIs”. Ideally, this produces readable code and readable code also attracts helpers who read code, believe it or not. As a side-note, I will say that Ruby is object oriented, if you’re not super comfortable with objects and classes, please spend a little bit of time there. It’s totally worth checking out. Sandi Metz has a book that I have totally not read, but she’s given a ton of conference talks, which I have seen, and you should watch the talks, and I’m sure the book is amazing as well. As well as Katrina Owen, who has done a ton of Refactoring talks. If you want to see how can we make this better, how can we make this more readable, as well as is an actual place where you can go and try out your skills. You actually refactor things there. It’s pretty cool. They’re also working on a book together which I’ll actually read when it comes out.

Helping takes commitment, and we do need to respect that. How are different ways that we can respect that commitment? When somebody gives a pull request, even if it’s not the best pull request, we, as maintainers, can say “thanks for submitting this”. That person cared enough, and you can help them to help you. You can explain the reasons why you’re not merging it, or help them to get to a place where you can merge it. You can also help guide them and say “I don’t really care about that thing. How about you look at this other thing which I really care about?” And it’s a way to get them on board. If you just close an issue, dismiss it, and then lock it: that is not how you attract people to help you.

What else do people want? People want recognition. Rails has this great leader board. That’s actually the reason why I had my first commit under Rails, I wanted to just be on the board, period. Maybe you don’t have a leader board, but you can still give recognition.

Maybe your helpers want pizza.

There’s a fun story where, when I introduced a feature, I actually broke Windows on a minor release of Sprockets, and I had a developer come to me tell me that I broke the build on Windows. I had no idea what to do. How do I fix this? He helped explain the problem, we worked through it, we pushed a release. Later I went to him and I said, “okay, obviously you care about Sprockets, you care about Windows, you have a Windows machine. Can you help me get the Sprockets tests running on Windows?” He was hesitant to commit that much time to a pretty thankless task. I wanted to show how much I would appreciate it so I offered to buy him a pizza for his efforts. I’m not joking. That’s not like, hyperbole. “I will actually order delivery for you for a pizza to your home”. Well, a couple weeks later, he did it, and he did not happen to, in those couple of weeks, reach out to me and mention “hey, by the way, I live in Germany”. But he did live in Germany. I now know lots about ordering pizzas for delivery in Germany and explaining to credit card companies what open source is. I’m happy to do things for people who help me. The cost of the pizza was trivial compared to the time spent on the feature, but the gesture was worth more. We mentioned acknowledgement. Well, thank you, thank you, Daniel, for doing this, and Sprockets is now tested on a Windows CI server via appveyor.

All of these things we’ve talked about: good docs, clean code, those are ideals we can strive for. What happens when you actually have the scenario where you inherited a project? Where the precious maintainer might not have done all of these things? They just mic-dropped? What are you going to do?

You can start by finding something that needs fixing. I call this bug-driven development. I keep on talking about example apps because the only way to get started is with an example app that allows you to reproduce the problem. If I didn’t mention, example apps are amazing, and you should probably make it a goal to make one this year.

Once you have an example app, reproduce the problem and then repeat. Every single bug that you fix, you’re going to learn a little bit more about the codebase, and eventually you’re going to start seeing non-bug problems. Eventually you’re going to be a lot more comfortable. An example of this for me was Source Maps. Source Maps are a thing in Sprockets 4, and it was half-finished when Josh stepped away, and when I got this project, I had no idea what they were. Somebody would report a bug. I would try and fix that bug and it made the tests break, and I had to step back and ask if the tests are even reliable? I’ve got no clue. So where do we start? I put on my archaeologist hat, which was totally the inspiration for this talk, and I started research. I looked at the Mozilla RFC. I got out evernote and I started taking notes, and I learned a whole lot about source maps. I eventually was able to take those notes and actually turn them into guides. I took all of that information and said “I don’t know this, so other people probably don’t know this. Let’s not make them work as hard”. I made those notes into a guide and I put it in the Sprockets source tree. If you’re interested in reading it, it’s totally rough, but it works. I’ve even used it for my own reference more than a few times. So if you want to know what a source map is, then I can tell you, go read my Source Maps guide. In this process I ended up having to borrow from some other projects, I actually used other projects from the technology which shall not be named (NPM).

I used uglifyjs to verify my encoding and make sure that my encoding tests were valid. I used source-map to verify that my decoding tests were correct, and then I got the tests to a place where they could pass. So, is Source Maps finished? No. I need more bug reports that are actionable.

We’ve got to wrap up soon. With all of that being said, where do we go? Because maintainers won’t be around forever. I won’t be around forever. So, I need help. I need help maintaining the history of Sprockets. I don’t need you to know everything. I don’t need you to go out there and fix all of the problems. You know what? Sometimes I might just need you to help not say bad things about me on the internet. If somebody’s like, hey, trash-talking me ask them to clarify their positions into a critique.

We can preserve these stories by getting involved, and if you don’t get involved, then who will?

It’s open source.

You can say, well, maybe somebody else will. Guess what? That somebody else is you, and we all need to step up.

We can take five minutes. We can just read those guides. We can write some docs. We can open some issues. We can create example apps, which I’ve totally not mentioned before.

It’s only



I invite you to join me, and together we can become maintainers, we can become helpers, and together, we can Save Sprockets.

The Straight Dope on Deprecations

The road to stability is paved with good deprecations. A deprecation is a warning message that tells a user they’re using some piece of code or interface that will go away soon. In this post, we’ll peel back the seemingly simple veneer of deprecations, and we will learn when and how to use deprecations effectively.

Keep Reading on Codeship