The difficulty with Node.js

I originally wanted to call this post “I hate Node.js,” but that seemed a little too provocative. Also it’s not quite accurate–Node.js is a very useful product. As a fast, efficient JavaScript runtime, it does exactly what’s on the tin and fulfills a dream of being able to use the same programming language on both clients and servers of web applications. The real problem I have with Node.js is the package ecosystem. Npm is a complete mess.

To illustrate the problem, let me start with a seemingly straightforward example:

1
2
npm init
npm install gulp

It’s not unusual for me to install gulp first thing with a new project, as it’s a common tool that I like to use for the initial build pipeline for a JavaScript-based web application. This was fresh on my mind since I use gulp for building my professional web site, havisullivan.com, which I was working on the past few days. After installing just that one explicit dependency, here’s what my node_modules directory looks like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
ansi-gray/             graceful-fs/              object.map/
ansi-regex/ gulp/ object.pick/
ansi-styles/ gulplog/ object-visit/
ansi-wrap/ gulp-util/ once/
archy/ has-ansi/ orchestrator/
array-differ/ has-gulplog/ ordered-read-streams/
array-each/ has-value/ os-homedir/
array-slice/ has-values/ parse-filepath/
array-uniq/ homedir-polyfill/ parse-passwd/
array-unique/ inflight/ pascalcase/
arr-diff/ inherits/ path-parse/
arr-flatten/ ini/ path-root/
arr-union/ interpret/ path-root-regex/
assign-symbols/ is-absolute/ posix-character-classes/
atob/ is-accessor-descriptor/ pretty-hrtime/
balanced-match/ isarray/ process-nextick-args/
base/ is-buffer/ readable-stream/
beeper/ is-data-descriptor/ rechoir/
brace-expansion/ is-descriptor/ regex-not/
braces/ isexe/ repeat-element/
cache-base/ is-extendable/ repeat-string/
chalk/ is-extglob/ replace-ext/
class-utils/ is-glob/ resolve/
clone/ is-number/ resolve-dir/
clone-stats/ isobject/ resolve-url/
collection-visit/ is-odd/ ret/
color-support/ is-plain-object/ safe-buffer/
component-emitter/ is-relative/ safe-regex/
concat-map/ is-unc-path/ semver/
copy-descriptor/ is-utf8/ sequencify/
core-util-is/ is-windows/ set-value/
dateformat/ kind-of/ sigmund/
debug/ liftoff/ snapdragon/
decode-uri-component/ lodash/ snapdragon-node/
defaults/ lodash._basecopy/ snapdragon-util/
define-property/ lodash._basetostring/ source-map/
deprecated/ lodash._basevalues/ source-map-resolve/
detect-file/ lodash.escape/ source-map-url/
duplexer2/ lodash._getnative/ sparkles/
end-of-stream/ lodash.isarguments/ split-string/
escape-string-regexp/ lodash.isarray/ static-extend/
expand-brackets/ lodash._isiterateecall/ stream-consume/
expand-tilde/ lodash.keys/ string_decoder/
extend/ lodash._reescape/ strip-ansi/
extend-shallow/ lodash._reevaluate/ strip-bom/
extglob/ lodash._reinterpolate/ supports-color/
fancy-log/ lodash.restparam/ through2/
fill-range/ lodash._root/ tildify/
find-index/ lodash.template/ time-stamp/
findup-sync/ lodash.templatesettings/ to-object-path/
fined/ lru-cache/ to-regex/
first-chunk-stream/ make-iterator/ to-regex-range/
flagged-respawn/ map-cache/ unc-path-regex/
for-in/ map-visit/ union-value/
for-own/ micromatch/ unique-stream/
fragment-cache/ minimatch/ unset-value/
gaze/ minimist/ urix/
get-value/ mixin-deep/ use/
glob/ mkdirp/ user-home/
glob2base/ ms/ util-deprecate/
global-modules/ multipipe/ v8flags/
global-prefix/ nanomatch/ vinyl/
glob-stream/ natives/ vinyl-fs/
globule/ object-assign/ which/
glob-watcher/ object-copy/ wrappy/
glogg/ object.defaults/ xtend/

Let that listing sink in for a moment. That’s 198 dependencies installed (a total of 5.5 MB of code assets), and so far all I have is my task runner. I haven’t even started to write actual application code that will need real dependencies.


Yeah, I’ve just been waiting to use this meme one day…

This is just a tiny bit insane to me, not just because of the size of the dependencies, but also because of how ridiculously niche a lot of them are. Let’s pick one at random: path-root. (With apologies to the author, Jon Schlinkert: I’m not doing this to call him out. His code was just a convenient example.) What does this do? According to its README: “Get the root of a posix or windows filepath.” Okay, seems straightforward. Then I look at the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/*!
* path-root <https://github.com/jonschlinkert/path-root>
*
* Copyright (c) 2016, Jon Schlinkert.
* Licensed under the MIT License.
*/

'use strict';

var pathRootRegex = require('path-root-regex');

module.exports = function(filepath) {
if (typeof filepath !== 'string') {
throw new TypeError('expected a string');
}

var match = pathRootRegex().exec(filepath);
if (match) {
return match[0];
}
};

That’s literally the whole “library.” A single function that matches a string to a regular expression. Even more crazy, the regular expression itself does not exist in this module; it’s in a nested dependency, path-root-regex. What. The. Heck.

This raises several questions in my mind:

  • Why is there an entire module dependency dedicated to what could be boiled down to one line of code?
  • Why is that module in turn dependent on a completely different module that basically acts as a data source for the first module?
  • Why did the developers of other modules resort to importing these dependencies? (Point of order: it’s not gulp, it’s actually like four dependencies deep.)

I bring up this particular example because it’s very indicative of how Node.js modules are built and distributed: as tiny pieces with an arcane and unfathomable dependency chain. You want A, and you end up getting B, C, D, etc. whether you wanted them or not, and the value/provenance of those dependencies is suspect. In my own searches of looking for tools in the npm repository, I’ve stumbled upon dozens of these types of micro-modules that someone wrote, single-function and seemingly pointless. Did I really make myself more productive by importing two modules into my project instead of writing a single-line regular expression match?

I’m not steeped in the culture of Node.js/npm by any stretch of the imagination, but we use Node.js for quite a few web applications at my workplace, and I also have taken to using them on my personal web sites as a means of learning more about them. This way of writing code seems insane to me, but I also came from a background in monolithic application software written using the .NET Framework. I think this is where I get lost. Instead of the Node.js environment providing a standard library of common features that developers can build on, we’re instead left to our own devices, writing whatever we think might be useful and posting it onto a central repository for everyone to pull into their software, probably with varying degrees of review. The flaws in a completely democratic system can be self-evident.

It’s also not uncommon when trying to find something that does what you want, you’ll find at least a half-dozen implementations from various sources (often with confusingly similar names), in various states of maintenance. There is a lot of abandonware in npm, and also it’s not always obvious if there are tons of hidden bugs you’re inheriting that the author never found and/or never intends to fix. It’s very much a “buyer beware” scenario.

Of course, I’m not novel in recognizing this. While I was researching for this post, I found a nice post by Mateusz Morszczyzna that reaches pretty much the same conclusions I did. He even used the same joke image I picked out. Oh well.

At this point, I’d almost be willing to go back to PHP–while the language has its flaws, at least it had a well-maintained, well-documented set of standard extensions to cover all the common scenarios where you’d want to take on dependencies. I feel like npm’s design actively encourages dependency hell and obfuscation, and it results in people pulling in first-level dependencies unnecessarily and second-level dependencies unwittingly.