Package size is not optimized for lambda; this library is not useable in production. #23

Overview

When my team installed this dependency to get SignalFX goodness for our lambdas, we noticed that our zipped lambda package size ballooned from 620 KB to 4.2 MB. Moreover, the lambda that received this package never actually ran because it continuously ran up against timeout on cold start (tested lambda default 3s up to 4x at 12s; typically this lambda completes in 200ms). Therefore this package is not useable is production until it is optimized for lambda.

Regardless, SignalFX could make some simple changes to get the package size moving in right direction. I don't think my team will consider using it until it gets down to 200~300 KB.

tl;dr

Get rid of huge-but-little-used dependencies, like request
Get rid of huge-and-hidden dependencies, like lodash

Issues

So the main issue here is that, despite the package name, this is not at all an optimized product for lambda. If you look at the project dependencies, it's clear that SignalFX has brought in the entirety of its server solution and just thrown a lambda wrapper on it. While storage is widely available on server-based solutions, for FaaS (i.e lambdas) package size is a huge premium. Package size affects cost in at least two ways: run time cost and storage cost. Moreover, as noted above, using this package even prevented our lambda from running at all because it continuously timed out on cold start.

Closer examination reveals that the SignalFX base package for Node JS (signalfx-nodejs) is relying on huge dependencies that could easily be interchanged with smaller packages or even some native Node JS.

signalfx-nodejs deps

 "dependencies": {
    "base64-js": "^1.1.2",
    "bignumber.js": "^2.3.0",
    "pako": "^1.0.4",
    "promise": "^7.0.4",
    "protobufjs": "^6.0.0",
    "request": "^2.61.0",
    "sse.js": "^0.4.1",
    "text-encoding": "^0.6.4",
    "winston": "^1.0.1",
    "ws": "^7.1.2"
  },

For example, unzipped request comes in at 729 KB. From what I could find, it is only used to make basic GET and POST requests. https://github.com/ethanent/phin).

promise adds... I'm not sure what? Promises are natively supported in Node JS (signalfx-nodejs uses Node 8.0). Unzipped this package comes in at 209 KB.

winston is a nice logger to be sure, but unzipped comes in at 569 KB.

pako is 800+ KB unzipped.

I could go on, but the point is that in FaaS, dependency size is a premium, and much of these dependencies are not worth their weight in KB.

Last but not least, something somewhere in the signalfx-nodejs dependency tree has committed what our team refers to as the Cardinal Sin of Lambda and included lodash. Unzipped, lodash costs a whopping 5.1 MB.

It is clear that signalfx/lambda-nodejs is not a lambda-specific and optimized implementation, but short of a total rewrite, there are some definite short term opportunities that could be taken to make this package appropriate for test usage.