Keeping JSON Endpoints in Check with json-weight

json-weight helps identify what is making a JSON endpoint too heavy.

Performance audits are an integral part of our development process. Good data is essential when doing this kind of work. It's too easy to neglect true bottlenecks; to chase after the wrong problems.

I've run into a couple of situations where JSON API endpoints were larger than expected. Without a solid understanding of the shape of a response, I've found it can be difficult to determine where to focus attention.

To help solve this problem, I wrote a script called json-weight.

Introducing json-weight

json-weight visualizes the weight of JSON endpoints. It reports the size of each branch in a JSON response as it steps through the tree of key/value pairs. This generates a helpful analysis of the composition of the response.

Check it out on Github. More information on usage follows.

Installing json-weight

json-weight is published on npm. Assuming you have NodeJS (really any version) installed, picking it up is simple:

$ npm install -g json-weight

Using it

I've added a fairly large JSON response as a static file on the Github project for the script. It clocks in at around 430kb. Pretty hefty.

Let's figure out why:

$ curl -s | json-weight 2

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
1: 67.54kb (30.55kb gzip)
  id: 0.03kb (0.05kb gzip)
  name: 0.02kb (0.05kb gzip)
  bio: 1.38kb (1.03kb gzip)
  friends: 66.09kb (29.85kb gzip)

#... truncated for brevity

Here I'm piping the API response to json-weight and telling it to report 2 branches deep. From the report we can already identify that the friends field contributes the most weight.

But why? Let's dig deeper.

$ curl -s | json-weight 4

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
    0: 2.02kb (1.40kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.02kb (0.05kb gzip)
      bio: 1.99kb (1.36kb gzip)

    # ...

    39: 1.53kb (1.16kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.01kb (0.05kb gzip)
      bio: 1.49kb (1.10kb gzip)

Cool. We can see that quite a few friends are being pulled in for each person. 40 in this instance. There are a couple of natural follow-up questions:

  1. How are we using the friends field?
  2. Do we really need to include the full bio?
  3. Should we consider paginating the friends field?

These are good follow-up steps; we've moved past "why is the page is slow". We can talk productively about the ramifications of changing this endpoint, and what kind of benefits it might yield.

Check it out for yourself

It's a simple script, but it hasn't let me down yet. Check it on Github:

Nate is a senior developer in our Durham, NC office, where he focuses on client-side application development. Most days, you can find him neck-deep in JavaScript working with clients such as The Nature Conservancy and the Wildlife Conservation Society.

More posts by Nate