Rate Limiting, Rate Throttling, and how they work Together

In the beginning, there were API requests, and they were good. But then some jerk went and made too many requests too fast and brought the server crashing to its knees. Enter: Rate limiting.

Keep Reading


Lies, Damned Lies, and Averages: Perc50, Perc95 explained for Programmers

I got a customer ticket the other day that said they weren’t worried about response time because “New Relic is showing our average response time to be sub 200ms”. Sounds good, right? Well, when it comes to performance - you can’t use the average if you don’t know the distribution. It’s usually best to use the median, which is also perc50, though you’ll also want to look at your long tail of responses. If you’re not following, then this post is for you.

Keep Reading


The Curious Case of the Table-Locking UPDATE Query

I maintain an internal-facing service at Heroku that does metadata processing. It’s not real-time, so there’s plenty of slack for when things go wrong. Recently I discovered that the system was getting bogged down to the point where no jobs were being executed at all. After hours of debugging, I found the problem was an UPDATE on a single row on a single table was causing the entire table to lock, which caused a lock queue and ground the whole process to a halt. This post is a story about how the problem was debugged and fixed and why such a seemingly simple query caused so much harm.

Keep Reading


Why does my App's Memory Use Grow Over Time?

Why on earth does my memory consumption chart look like that? It’s a question I hear every week. To help answer that question, I wrote a Web server request simulator to model how Ruby uses memory over time, though it applies to other languages as well. We will use the output of that project to dissect why a web app’s memory would be expected to look like this:

Keep Reading


Puma 4: Hammering Out H13s—A Debugging Story

For quite some time we’ve received reports from our larger customers about a mysterious H13 - Connection closed error showing up for Ruby applications. Curiously it only ever happened around the time they were deploying or scaling their dynos. Even more peculiar, it only happened to relatively high scale applications. We couldn’t reproduce the behavior on an example app. This is a story about distributed coordination, the TCP API, and how we debugged and fixed a bug in Puma that only shows up at scale.

Keep Reading


Puma 4: New I/O 4 Your Server

Here’s the setup: You are a web server named Puma. You need to accept incoming connections and give them to your thread pool, but before we can get that far, you’ll have to make sure all of the request’s packets have been received so that it’s ready to be passed to a Rack app. This sounds like the job for a Reactor!

Keep Reading


Blockwalk for Beto: Enter to win my Technical Time

Today I have an unusual proposition for you. I’m spending a bunch of time to try to get Beto elected to Texas Senate, so I’ve not been able to write as much technical content. Rather than slow down on my door knocking, I’m looking to pick up the pace, and I want you to do it with me. Starting today, I’m offering anyone who phone banks or “block walks” (knocks on doors) the opportunity to win some of my technical time. Here’s how it’s going to work.

Keep Reading


Cache Invalidation Complexity: Rails 5.2 and Dalli Cache Store

Rails applications that use ActiveRecord objects in their cache may experience an issue where the entries cannot be invalidated if all of these conditions are true:

Keep Reading


Pair With Me: Rubocop Cop that Detects Duplicate Array Allocations

You might know rubocop as the linter that helps enforce your code styles, but did you know you can use it to make your code faster? In this post, we’ll look at static performance analysis and then at the end there’s a video of me live coding a PR that introduces a new performance cop to rubocop.

Keep Reading


Rails Asset Pipeline Directory Traversal Vulnerability (CVE-2018-3760)

All previously released versions of Sprockets, the software that powers the Rails asset pipeline, contain a directory traversal vulnerability. This vulnerability has been assigned CVE-2018-3760.

Keep Reading


Subscribe to my Newsletter 😻 🤠

Join thousands of developers who get new code, writing, and programming links from me delivered to their inboxes.