The After Effects render crisis

The After Effects plugin has one job that matters: take an animation somebody built in After Effects and get it out as a Lottie file. That sounds like a conversion and it is closer to a negotiation. Two systems describe motion differently, and the thing they agree on has to render the same way everywhere afterwards.

None of that happens on my machine. It happens inside a copy of After Effects I have never seen, on a project I have never opened, on a laptop with its own opinions about graphics. When it goes wrong it goes wrong in the field, and what comes back to me is a person saying it did not work.

The render error rate sat at ~4%, and there was no account of what was inside that number.

That absence is the actual problem, and it is worth being precise about why. A failure rate like that is not one bug happening often. It is an unknown number of distinct failures with distinct causes, arriving one at a time as individual reports. Every one of those reports is fixable. Fixing them one at a time does not move the rate, because the next report is a different failure, and nothing in front of you says whether it is the same thing you fixed last week or the two hundredth of its kind.

Live production pressure pulls hard in exactly the wrong direction here. Somebody is blocked right now, the fix is an afternoon, and the afternoon is available. Do that for a month and you have a lot of closed reports and the same number on the dashboard.

The other half of the constraint is that the pipeline crosses boundaries I did not own. An export starts inside a host application somebody else ships, passes through a rendering step, and ends in a file format that has to survive being played back by other people's software. Each of those has its own failure modes, and from the outside all of them look identical: it did not work.

Stop treating reports as bugs. Start treating them as samples.

That reframe is the whole study, and everything after it is a consequence of it.

The first consequence is that instrumentation comes before fixing. A render that dies has to come back saying enough about where and how it died to be sorted with the others. Without that, every report is a one-off by construction, and no amount of diligence will tell you otherwise.

The second is sorting: a class is a group of failures with one cause and one fix. Naming the classes is most of the work and none of the glamour. It is reading failures until the shape appears, discovering that three reports that looked unrelated are one thing, and that one report that looked routine is three.

The third is that you rank by volume and take the top of the list, not the loudest report. The loudest report is a real person and it is genuinely hard to deprioritize them. It is also how a month disappears.

And the rule that held the whole thing together: a fix that closes a report and leaves the class open does not count as done. That is the only discipline that keeps this from collapsing back into firefighting the moment something urgent arrives, and something urgent always arrives.

Instrumentation first means the number does not move while you build it. There is a stretch where the honest status update is that the render error rate is exactly where it was after weeks of work on the render error rate, and somebody is entitled to ask about that. Answering it plainly, more than once, without flinching into a quick fix just to have something to show, is part of the job and it is the part I found hardest.

Class elimination is also front-loaded, and the curve is not kind. The first few classes are most of the population, and the last stretch down to ~0.3% costs far more per point than everything before it, because the tail is where the one-offs live. Somewhere in there is a real decision about when to stop, and it is a business decision wearing an engineering costume.

An average also hides people. A user whose project reliably trips what is left is not seeing a small failure rate, they are seeing a broken plugin every single time, and the fleet average is no comfort at all. The rate is the right number for deciding what to work on and the wrong number for deciding whether somebody is having a good time.

And the taxonomy is now a thing to maintain. Classified failures need a scheme, a scheme goes stale, and a stale scheme quietly sorts new failures into old boxes.

The render error rate went from ~4% to ~0.3%.

The number is the evidence. The durable part is that the pipeline can now say what went wrong. A spike arrives as a named class on the first day instead of as a number somebody has to go and interpret, which is the difference between an incident that takes months and one that takes a week.

If a failure rate has no breakdown underneath it, the rate is not the problem yet. The missing breakdown is.

That is the thing I would take anywhere. It is not a story about a clever fix, and I could not tell you which individual change moved the number most. It is a story about refusing to work on the report in front of me until I knew what family it belonged to.