Keeping JSON Endpoints in Check with json-weight

Nate Hunzaker, Former Development Director

Article Category: #Code

Posted on

json-weight helps identify what is making a JSON endpoint too heavy.

Performance audits are an integral part of our development process. Good data is essential when doing this kind of work. It's too easy to neglect true bottlenecks; to chase after the wrong problems.

I've run into a couple of situations where JSON API endpoints were larger than expected. Without a solid understanding of the shape of a response, I've found it can be difficult to determine where to focus attention.

To help solve this problem, I wrote a script called json-weight.

Introducing json-weight #

json-weight visualizes the weight of JSON endpoints. It reports the size of each branch in a JSON response as it steps through the tree of key/value pairs. This generates a helpful analysis of the composition of the response.

Check it out on Github. More information on usage follows.

Installing json-weight #

json-weight is published on npm. Assuming you have NodeJS (really any version) installed, picking it up is simple:

$ npm install -g json-weight

Using it #

I've added a fairly large JSON response as a static file on the Github project for the script. It clocks in at around 430kb. Pretty hefty.

Let's figure out why:

$ curl -s http://code.viget.com/json-weight/people.json | json-weight 2

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
1: 67.54kb (30.55kb gzip)
  id: 0.03kb (0.05kb gzip)
  name: 0.02kb (0.05kb gzip)
  bio: 1.38kb (1.03kb gzip)
  friends: 66.09kb (29.85kb gzip)

#... truncated for brevity

Here I'm piping the API response to json-weight and telling it to report 2 branches deep. From the report we can already identify that the friends field contributes the most weight.

But why? Let's dig deeper.

$ curl -s http://code.viget.com/json-weight/people.json | json-weight 4

0: 69.14kb (30.95kb gzip)
  id: 0.03kb (0.06kb gzip)
  name: 0.02kb (0.06kb gzip)
  bio: 1.76kb (1.25kb gzip)
  friends: 67.31kb (30.03kb gzip)
    0: 2.02kb (1.40kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.02kb (0.05kb gzip)
      bio: 1.99kb (1.36kb gzip)

    # ...

    39: 1.53kb (1.16kb gzip)
      id: 0.00kb (0.03kb gzip)
      name: 0.01kb (0.05kb gzip)
      bio: 1.49kb (1.10kb gzip)

Cool. We can see that quite a few friends are being pulled in for each person. 40 in this instance. There are a couple of natural follow-up questions:

  1. How are we using the friends field?
  2. Do we really need to include the full bio?
  3. Should we consider paginating the friends field?

These are good follow-up steps; we've moved past "why is the page is slow". We can talk productively about the ramifications of changing this endpoint, and what kind of benefits it might yield.

Check it out for yourself #

It's a simple script, but it hasn't let me down yet. Check it on Github:

https://github.com/vigetlabs/json-weight.

Related Articles