Fixing TypeScript Performance Problems: A Case Study

On a recent TypeScript project, we ended up in a situation where editor performance had degraded and the TypeScript compiler (and language server) was increasingly struggling with some specific areas of the codebase resulting in sluggish intellisense, long type-checking times, occasional stale type information, and a lot of frustration.

This project is a monorepo with 7 TypeScript packages. To the credit of the original creators, it is already using project references (which requires the composite setting), and incremental compilation but it also has some heavy TypeScript dependencies like prisma, kysely, ts-pattern, and hotscript.

Diagnosing The Problem #

When doing any sort of troubleshooting, my first port of call is the docs. To my knowledge, there isn’t a dedicated page on https://www.typescriptlang.org/ for this, but a quick search leads to the GitHub wiki page on Performance which has a bunch of sage advice and useful suggestions.

It’s wise to begin with the usual suspects:

• make sure your editor is up to date including plugins and extensions
• disable all editor extensions to eliminate them from the list of suspects
• make sure no other processes are hogging system resources
• use the latest version of TypeScript that you can use (many releases include performance fixes and optimizations)
• update dependencies to the latest version whenever possible (including any @types declarations your dependencies need)
• use strict compilation unless you have a good reason not to and you know what you’re doing (it allows the compiler to check for more kinds of potential problems and also to use certain more performant algorithms like faster variance checks)

From there, you can start to dive into additional information that the TypeScript compiler can provide:

Validating Source Files Inclusion #

Ensure that only the minimal set of source files are being processed by the compiler.

$ tsc --listFilesOnly

Why?: fewer files = fewer types = less work for the compiler.

If you see files that you don’t expect in the compilation, you can ask the compiler to tell you why they are included:

$ tsc --explainFiles > explanations.txt

Tweak your tsconfig’s include/exclude/types/typeRoots/paths as necessary if adjustments are needed to omit unneeded files from the compilation

Performance Metrics and Measurement #

Performance tuning requires careful planning and execution. In order for two compilations to be meaningfully comparable, it’s important to make sure they’re run on the same hardware and under the same conditions (system load, battery/power, etc.). Even under carefully managed conditions, outliers can skew results so it’s wise to run the compilation several times in order to get a good sense of variance. A rigorous approach would be to use statistical analysis to calculate significance based on standard deviations, but if compilation times are long then that can be prohibitively time-consuming.

Establishing a baseline is crucial when doing any sort of optimization work in order to assess whether subsequent changes have a meaningful positive impact. To do this, the TypeScript compiler offers a flag that outputs detailed metrics about the compilation: tsc --extendedDiagnostics. When you run that command, you’ll get an output that looks like the following (you may see more than one such report if you have multiple TypeScript projects in your compilation):

Of the many data points this analysis reports, the most useful metrics to look at are the number of files and types, amount of memory used, and the amount of time spent doing I/O, parsing, and type-checking. If I/O times are high or the number of files/lines are high look at Validating Source Files Inclusion.

Deeper Insights with Compiler Traces #

If the compiler is spending a lot of time in the parse, bind, or check phases, then capturing a trace can help identify which parts of your program are contributing most to this expense. The compiler offers a flag that instruments everything it does which produces a dataset that can be mined for deeper insights: tsc --generateTrace <output_dir> (available since TypeScript 4.1). If this command is giving you trouble, make sure you’re not doing an incremental build by passing in the -f argument (for build mode) or --incremental false (for regular compilation).

Running this command will output some files into the given directory. The outputs will look a bit different depending whether you’re running tsc in build mode (-b) or not, but either way there will be one or more JSON files for types and trace. Look for the largest of these files as a starting point or check these docs for a more complete explanation.

These traces can be analyzed with Chrome’s trace viewer (which should be available at about://tracing in Chromium-based browsers including Arc). You can also try the newer Perfetto UI but I have had better luck with the older method.

Issues I ran into: #

tsc fails with a Heap Out of Memory error when running extended diagnostics and/or traces

By default node processes (since version 12) have a maximum heap size that depends on the system’s available memory. On many modern systems this defaults to ~2GB. If you run into errors related to memory, you can bump up the maximum allowed memory usage by passing a flag to node (note this command runs both the extended diagnostics and generates trace files during one compilation run):

$ node --max-old-space-size=8192 ./node_modules/.bin/tsc -b --extendedDiagnostics --generateTrace ./ts-trace

Trace Files can’t be analyzed by trace viewer

Trace files can be quite large — so much so that tools like about://tracing and Perfetto UI will refuse to process them. That was the case for some but not all of the trace files for me and I ended up relying on @typescript/analyze-trace to understand them.

The recommended process-tracing script outputs nothing

The Performance Tracing docs suggest using a process-tracing script to sample extra large traces in order to produce smaller traces that are easier to work with. In practice, this script just output empty files when I tried it. YMMV.

Identifying the Root Cause(s) #

In our specific case, @typescript/analyze-trace was the key to identify the most problematic application code. Because of the monorepo structure of this project, there were 6 or 7 trace files to consider. Not all of them contained obvious issues, but eventually I stumbled upon this one and was immediately intrigued.

$ npx analyze-trace ./ts-trace

Analyzed /<client>/<project>/packages/<package>/tsconfig.json (trace.12493-5.json)
Hot Spots
├─ Check file [35m/<client>/<project>/packages/<package>/src/tasks/extractions/common.ts (80609ms)
│  └─ Check deferred node from (line 10, char 10) to (line 29, char 4) (80608ms)
│     └─ Check expression from (line 11, char 12) to (line 28, char 7) (80607ms)
│        └─ Check expression from (line 11, char 15) to (line 28, char 6) (80607ms)
│           ├─ Check expression from (line 13, char 7) to (line 27, char 8) (51716ms)
│           │  └─ Check expression from (line 14, char 9) to (line 26, char 12) (51711ms)
│           │     ├─ Check expression from (line 24, char 18) to (line 25, char 69) (38423ms)
│           │     │  └─ Check expression from (line 25, char 13) to (line 25, char 69) (38422ms)
│           │     │     └─ Check expression from (line 25, char 44) to (line 25, char 68) (14922ms)
│           │     └─ Check expression from (line 14, char 9) to (line 24, char 17) (13287ms)
│           │        └─ Check expression from (line 14, char 9) to (line 23, char 12) (13287ms)
│           │           └─ Check expression from (line 14, char 9) to (line 17, char 21) (13262ms)
│           │              └─ Check expression from (line 14, char 9) to (line 16, char 42) (13261ms)
│           │                 └─ Check expression from (line 16, char 19) to (line 16, char 41) (4364ms)
│           └─ Check expression from (line 12, char 7) to (line 12, char 32) (28891ms)
└─ Check file /<client>/<project>/packages/<package>/src/tasks/transforms/operation.ts (712ms)
   └─ Check expression in /<client>/<project>/packages/<package>/src/tasks/extractions/operation.ts from (line 46, char 50) to (line 46, char 57) (611ms)
      └─ Check expression from (line 48, char 17) to (line 191, char 10) (502ms)
         └─ Check expression from (line 48, char 17) to (line 190, char 19) (502ms)
            └─ Check expression from (line 48, char 17) to (line 190, char 13) (500ms)
               └─ Check expression from (line 48, char 17) to (line 188, char 4) (500ms)
                  └─ Check expression from (line 168, char 10) to (line 187, char 6) (322ms)
                     └─ Check expression from (line 169, char 5) to (line 187, char 6) (322ms)
                        └─ Check expression from (line 169, char 15) to (line 186, char 59) (321ms)
                           └─ Check expression from (line 170, char 7) to (line 186, char 59) (321ms)
                              └─ Check expression from (line 170, char 7) to (line 186, char 18) (320ms)
                                 └─ Check expression from (line 170, char 7) to (line 185, char 63) (320ms)
                                    └─ Check expression from (line 170, char 7) to (line 185, char 18) (320ms)
                                       └─ Check expression from (line 170, char 7) to (line 184, char 10) (320ms)
                                          └─ Check expression from (line 170, char 7) to (line 180, char 18) (319ms)
                                             └─ Check expression from (line 170, char 7) to (line 179, char 10) (319ms)
                                                └─ Check expression from (line 170, char 7) to (line 174, char 18) (317ms)
                                                   └─ Check expression from (line 170, char 7) to (line 173, char 55) (317ms)
                                                      └─ Check expression from (line 170, char 7) to (line 173, char 18) (316ms)
                                                         └─ Check expression from (line 170, char 7) to (line 172, char 40) (316ms)

It’s not that easy to see but there is a file in this trace that took 80,609ms (80 seconds) to type-check. That’s a long time! Why did it take so long?

The next line shows that in this exceptional case, the top-level check is a deferred node Check deferred node from (line 10, char 10) to (line 29, char 4) (80608ms) which gives us some line and column numbers to check out. In this context deferred just means that the compiler does not have enough information yet to resolve the type and it must continue the compilation in order to gather more information (often other inferred types) so that it can fully resolve this deferred node.

Here’s what I discovered when I opened that file and looked at line 10, char 10 (the details here have been obfuscated):

import type { Db } from '@lib/kysely/db';
import type { ExpressionBuilder, ExpressionWrapper } from 'kysely';

export const existsValidThing = <
  const T extends keyof Db,
  EB extends ExpressionBuilder<Db, keyof Db>,
>(
  thingIdRef: ExpressionWrapper<Db, T, string | null>,
) => {
  return ({ eb, or, exists }: EB) => {
    return or([
      eb(thingIdRef, 'is', null),
      exists(
        eb
          .selectFrom('thing as t')
          .select(eb.lit(1).as('exists'))
          .innerJoin('category', 'category', 't.category_id')
          .where('t.id', '=', thingIdRef),
      ),
    ]);
  };
};

It’s a bit awkward, right? Line 10, char 10 is the beginning of the anonymous lambda function that is returned from this existsValidThing helper function. So why is the type of this helper function so difficult for TypeScript to resolve?

There are a few reasons but ultimately it comes down to:

1. Db is a large interface filled with a mapping of ~30 database tables and their many fields (dozens per table)
2. kysely is built in such a way that heavily relies on type inference and distributing across large unions (i.e. the tables in a database, or the fields in a table, one GitHub commenter casually floated the idea that the complexity was on the order of O(n^x) in some edge cases)

Even though there was an explicit type declared for the argument to the lambda (EB, coming from the generic), TypeScript still has to infer the return type of this lambda which is itself a complex series of nested function calls each requiring inference to determine their return types.

The things kysely lets you do with strong typing are fairly magical. However, trying to split those things off into re-usable helpers can produce unexpected type-checking bottlenecks with sufficiently large databases.

The Fix(es) #

There were a bunch of things we tweaked to improve performance but the largest gains came from just deleting these problematic kysely helper functions and inlining their queries where they were being used.

Other things we did that helped:

1. fixed circular dependencies (see madge)
2. removed unused types
   1. prisma-zod-generator generated types
   2. uninstalled unused dependencies
   3. deduplicated packages
3. clean up and removed barrel files
4. upgraded packages to newer versions
5. upgraded node and ensured everything was on the same version
6. added syncpack and set up monorepo linting rules

Results #

Performance matters. Many factors impact the developer experience. Even when a team’s values and culture place adequate importance on maintaining and improving performance, still other elements may conspire to threaten daily productivity.

The tale for this specific project is one of triumph. By the numbers it was a bloodbath.

Most Significant Reductions: #

• I/O Write time: -85.0% (0.40s → 0.06s)
• Tracing time: -83.1% (2.72s → 0.46s)
• Check time: -83.1% (226.19s → 38.23s)
• Emit time: -80.5% (2.61s → 0.51s)
• Print time: -80.4% (2.60s → 0.51s)
• Build time: -79.0% (374.32s → 78.55s)

Resource Usage: #

• Memory usage: -50.2% (7.1GB → 3.5GB)
• Files processed: -28.6% (14,628 → 10,445)
• TypeScript lines: -56.6% (205K → 89K)

Overall Impact: #

The optimization reduced the total build time from 6.2 minutes to 1.3 minutes, representing a 79% improvement in compilation speed.

That’s nice and all but we’re rarely running a fresh, full compilation with no cache. The real impact is a responsive language server and instant intellisense.

If you know your tools, have a bit of time and patience, and are motivated to make things better there’s a lot you can do even when faced with what feels like an impossible task.