Modular packages in AWS SDK for JavaScript

As of December 15th, 2020, the AWS SDK for JavaScript, version 3 (v3) is generally available.

On October 19th, 2020, we published the Release Candidate (RC) of the AWS SDK for JavaScript, version 3 (v3). One of the major changes in the JavaScript SDK v3 is modularized packages. This blog post explains why we decided to publish modular packages, and gives an example of performance improvement due to bundle size reduction.

Motivation

The v2 of AWS SDK for JavaScript is published as a single package. If you import the entire SDK even if your application uses just a subset of SDK’s functionalities, there are performance implications noticeable in resource-constrained environments, like IoT devices or browsers on low-end mobile devices.

You can reduce the bundle size of your application with dead-code elimination, commonly known as tree shaking, using tools like webpack or Rollup. Tree shaking prunes unused code paths from your application, but it requires knowledge and expertise of the tools. Rather than installing the entire SDK and subsequently prune dead code, it is better to only import parts of the JavaScript SDK that are used by your application.

Usage

In v3 of AWS SDK for JavaScript, we achieved modularity by breaking the JavaScript SDK core into multiple packages and publishing each service as its own package. These packages are published under @aws-sdk/ scope on NPM to make it easy to identify packages that are part of the official AWS SDK for JavaScript.

Importing a service client

The service clients are prefixed with client- followed by service name. In v2, the S3 client can be created using single monolithic package:

const AWS = require("aws-sdk");

const s3Client = new AWS.S3({});
await s3Client.createBucket(params);

The service can also be imported as sub-module from in v2, which provides some benefit of bundle size reduction.

const S3 = require("aws-sdk/clients/S3");

const s3Client = new S3({});
await s3Client.createBucket(params);

In v3, you can create the modular aggregated S3 client by importing @aws-sdk/client-s3:

const { S3 } = require("@aws-sdk/client-s3");

const s3Client = new S3({});
await s3Client.createBucket(params);

The v3 also lets you import a modular bare-bones client with specific commands that help further reduce application bundle size!

const { S3Client, CreateBucketCommand } = require("@aws-sdk/client-s3");

const s3Client = new S3Client({});
await s3Client.send(new CreateBucketCommand(params));

Importing a high level operation

This modularization also applies to high-level operations, which are no longer a part of the client package. These high-level operations share lib- prefix followed by the operation name. For example, the S3 Multipart Upload is a part of S3 client in v2:

const AWS = require("aws-sdk");
const multipartUpload = new AWS.S3.ManagedUpload({
  params: {Bucket: 'bucket', Key: 'key', Body: stream}
});

In v3, the high-level operation is moved into the new package @aws-sdk/lib-storage:

const { S3Client } = require("@aws-sdk/client-s3");
const { Upload } = require("@aws-sdk/lib-storage");

const multipartUpload = new Upload({
    client: new S3Client({}),
    params: {Bucket: 'bucket', Key: 'key', Body: stream},
});

Importing a utility function

The v3 also publishes utility packages that have a util- prefix followed by the utility name. For example, we publish marshall and unmarshall operations for DynamoDB in @aws-sdk/util-dynamodb to convert JavaScript object into DynamoDB record and vice-versa.

const { DynamoDB } = require("@aws-sdk/client-dynamodb");
const { marshall, unmarshall } = require("@aws-sdk/util-dynamodb");

const client = new DynamoDB(clientParams);
const putParams = {
  TableName: "Table",
  Item: marshall({
    HashKey: "hashKey",
    NumAttribute: 1,
    BoolAttribute: true,
    ListAttribute: [1, "two", false],
    MapAttribute: { foo: "bar" },
    NullAttribute: null,
  }),
};

await client.putItem(putParams);

const getParams = {
  TableName: "Table",
  Key: marshall({
    HashKey: "hashKey",
  }),
};

const { Item } = await client.getItem(getParams);
unmarshall(Item);

Implementation

Even though v3 has 300+ packages published on npm, it is managed in a single multi-package repository (also called a monorepo) on GitHub. We use lerna and yarn workspaces to split the large codebase into separate independently versioned packages.

Metrics

To measure bundle size reduction, we created a self-guided workshop that provides step-by-step instructions to migrate a simple note taking application from using JavaScript SDK v2 to v3. The application manages notes in a DynamoDB table using AWS SDK for JavaScript in Node.js in a lambda backend, and manages files in S3 using the JavaScript SDK in the browser on the frontend.

Backend

In the workshop README for backend, we import the entire v2 which results in lambda bundle size for each of the create, read, update, delete, list operations to be ~817 kB.

We update the code to use submodules in v2, followed by importing entire client in v3 and importing bare-bones client with commands. The final code results in each lambda bundle size to reduce to ~23 kB!

To examine performance benefits of reduction in bundle size during cold start, we wrote a cloudwatch event to trigger both lambdas every 20 minutes for 18 values over 6 hours. The data for AWS::Lambda::Function (in ms) shows ~40% improvement in function executions times in our experiment.

Frontend

In the workshop README for frontend, we import the entire v2 which emits main chunk of size ~395 KB in the production bundle.

File sizes after gzip:

  395.2 KB  build/static/js/2.9a081e7a.chunk.js
  2.88 KB   build/static/js/main.9af70d78.chunk.js
  792 B     build/static/js/runtime-main.64ddd279.js

We update the code to use submodules in v2, followed by importing entire client in v3 and importing bare-bones client with commands. Modular imports now allow us to use code-splitting, which reduces the amount of code needed during the initial load. This reduces the main chunk size to ~48 KB in the production bundle!

File sizes after gzip:

  47.81 KB  build/static/js/1.7e51cbd2.chunk.js  
  46.96 KB  build/static/js/4.818586d4.chunk.js  
  7.85 KB   build/static/js/0.6a9c1fc3.chunk.js  
  3.02 KB   build/static/js/6.c7a500e3.chunk.js  
  2.5 KB    build/static/js/5.12e58bc3.chunk.js  
  1.72 KB   build/static/js/7.3d0fbc81.chunk.js  
  1.33 KB   build/static/js/8.074d72d1.chunk.js  
  1.24 KB   build/static/js/runtime-main.fb721bd4.js  
  525 B     build/static/js/main.ad4e136c.chunk.js

To examine performance benefits of reduction in bundle size in the frontend, we serve a production bundle over localhost and hard reload it by simulating “Fast 3G” under network tab in Chrome Developer Tools.

The frontend bundle created by importing entire v2 takes ~17 seconds to fire Load event in our experiment.

Where as the frontend bundle which imports bare-bones client with commands in v3 and uses code splitting takes less than 3 seconds to fire Load event!

This note taking application code clearly shows the performance improvement due to bundle size reduction using modular packages in AWS SDK for JavaScript v3 in a real-world application, both in the frontend as well as in the backend.

Feedback

We value your feedback, so please tell us what you like and don’t like by opening an issue on GitHub on AWS SDK for JavaScript v3 repository and workshop repository.