Turning "hard private" into "soft private"

[jwalton]

In which we provide a fairly trivial way to bypass "hard private" protections
on third party libraries, in practice.

Is this really "hard private"?

It seems like a lot of decisions in this proposal concerning private properties have been made specifically to support "hard private" properties. There was a long discussion here about whether this was a good idea or not, which didn't really seem to come to a consensus. TC39 is never the less proceeding with "hard private", and I see a decisions being justified using the argument "the alternative wouldn't really be hard private." This is stange to me, because the current proposal isn't hard private (at least in practice), and I suspect creating hard private properties is impossible in JavaScript.

What is "hard-private" anyways?

To quote @ljharb:

If it's accessible, it's public. "soft-private" is no different than underscores, because both are a convention (albeit, the former would be a convention baked into the language).

If a user can do it, they will, and if you break them, you broke them, even if they did something unwise.

(We can have a whole argument here between the "programming by contract" people, and the "language purist" people, about whether that second statement is correct or not - in fact we did in tc39/proposal-private-fields#33, so let's not have it again. 😉)

Obviously, nothing is really hard-private. I mean, the contents of that private variable are somewhere in memory, and I'm sure I can find some interesting clever way to get at them. I can write my own javascript engine, or write a node.js native module that peeks at memory, or I can use some esoteric attack like rowhammer to exfiltrate that data. What we really mean by "hard private" is that, in a "normal" javascript environment (not a debugger, and maybe in a browser), you can't get access to the contents of a private member without doing something extraordinarily.

The motivation for hard private seems mainly (?) to be for library authors; so let's say if a user can access private members in your library "easily", it's not hard private.

So, how "hard private" is this proposal?

PoC||GTFO

Bob writes an npm module:

export default class MyClass {
    #secret = "You can't see me!";
}

Alice is using Bob's npm module, and realizes she wants access to that #secret variable. I mean, it's named with a hashtag, and hashtags are for sharing, right? (And, let's be honest, this is a pretty boring NPM module. What else is Alice going to do with it?)

const inst = new MyClass();
console.log(inst.#secret);  // No worky worky.  Sad tombone.

In order to get around this, Alice adds the following to her .babelrc.js:

module.exports = {

  ...

  overrides: [{
    test: ["./node_modules/bobs-package"],
    plugins: ["private-class-fields-to-public"]
  }],
}

babel-plugin-private-class-fields-to-public is a Babel plugin which transforms Bob's package to look like this:

export default class MyClass {
    #secret = "You can't see me!";
    get _secret() { return this.#secret; }
    set _secret(secret) { this.#secret = secret; }
}

Now Alice can write:

const inst = new MyClass();
console.log(inst._secret);

Mischief managed!

Wait wait. Isn't this cheating? Babel isn't a "normal javascript environment!"

Between @babel/core and babel-core, babel was downloaded around 11 million times in the past week. Not too many javascript programs these days don't go through babel. You could even run babel in a browser, and transpile third party modules on the fly. babel-plugin-private-class-fields-to-public plugin was written in typescript, which for a while was a big babel competitor, but even that plugin was compiled with babel. Seems like a normal javascript environment to me.

Even if you don't buy that argument, look at this from a practical standpoint:

In Python, private members are about as soft-private as you can get. Private members are members that start with an _. To get around this, you need to type the name of the variable, and maybe feel bad for a few moments.

Java, on the other hand, is still soft-private by the "if it's accessible" definition above, but you have to do some work to access a private member. You have to go look up the refletion package and figure out how it works again (since the last time you used it was probably when you last had to get around a pesky visability problem). Then, when it doesn't work, you need to go to stack overflow to find out you forgot to call setAccessible().

With this proposal, as it stands, accessing private members in JavaScript falls somewhere between these two; you need to install a babel plugin, but after that it's basically "type _ and feel slightly guilty". Let's put it somewhere between Python and Java; maybe not as "hard private" as all that.

[ljharb]

Hard private is not “defeated” if you edit the source - either manually or programmatically. The concern is about at runtime - if the code shipped to the engine has encapsulation, then it will be reserved (just like with variables inside a closure).

[jwalton]

Really? I admit that installing a babel plugin might be, as you put it "unwise", but if a user can do it, they will, and here they certainly can. At the end of the day, JavaScript is an interpreted language; if you want to share a module with me, you need to share the source, and I can infer the existence of private variables by reading that source. So can my program. I can even write a loader that reads in your module and does this at execution time.

(Anyways, hopefully I at least got a grin out of you with my hashtag comment. ;)

[ljharb]

If they’ve done that, they’re not actually using the feature. Actually using the feature - shipping the code using private fields that are not intentionally exposed - results in “hard private”, which is what matters.

[jwalton]

Isn't this is like saying "If you type otherObject._something in python, you're not using the features in PEP 8 - actually using the feature, and not typing the _, results in hard private"?

I mean, the "feature" here is a feature for library developers to prevent library users from accessing private members in their library. It's not really a feature for library consumers at all. If there's a simple way for library consumers to get around that protection, I'd argue that it's not "hard private".

[ljharb]

A source rewriter isn’t a simple way, and most importantly, it doesn’t work at runtime.

[jwalton]

People will have the source rewriter installed if they even want to use a module that uses private variables, at least for the next several months.

And why do you think it doesn't work at runtime? Babel is just a javascript program; it's as simple as replacing your import with:

    const moduleSource = await fetch('https://domain.com/bobsPackage.js');
    const transformed = babel.transform(source, {
        plugins: [privateToPublic],
    });
    const bobsPackage = eval(transformed);
    console.log(bobsPackage._secret);

It even works in a browser.

[robpalme]

@jwalton Thanks for the entertaining insight. It's good to highlight this.

My view is that it all comes down to boundaries. If you don't define boundaries, then you can make arguments that any encapsulation is not truly "hard" (not a true Scotsman) unless it's on an air-gapped machine that can only be communicated with via unequally-laden carrier pigeons.

The boundary of ECMAScript is the source text ingestion. So it's fair to say that details are not abstracted away from anyone that can tamper with code prior to ingestion. Therefore any judgement of whether something is truly encapsulated by ECMAScript should be assessed post-ingestion.

A secondary and lesser point: Real-life use of fields will tend to go through minifiers that will mangle private names in ways that (underscore-prefixed) properties cannot safely be mangled. This will act as a deterrent to casual/widespread use of the babel-plugin-private-class-fields-to-public approach because mangled names are harder to deduce and can change frequently.

A final and least important point (because it all just flows back to top-level philosophy): as an electrical OEM who manfactures computer PSUs locked by uncommon star-shaped screws and covered with "danger of death" stickers, changing the internals of the PSU is a reasonable thing to do. If a user gets inside and has a bad experience because they didn't understand the updated internals, we all know who is at fault. If that same PSU just has the stickers but no physical lock, it would be more of a grey area and some (in places with consumer regulations) would argue it's an accident waiting to happen. The Babel plugin is an invasive power-tool akin to a star-shaped screwdriver IMO.

[jwalton]

@robpalme Thanks for the insightful comments. These are all fair points.

And, one deterrent to the use of babel-plugin-private-class-fields-to-public that you missed, at least in the short term, is that most libraries published to npm have already been through babel once, and don't have their original es6 source published; if your private fields have already been turned into a WeakMap, then there's no way for my little babel plugin to find them.

But, in reference to your last point; I would point out that in, say, Java, once you break out the reflection API, you're pretty clearly tearing through "danger of death" stickers. You have very definitely violated the contract put in place by the author of the module you're using, and you know that if bad things happen later on, it's totally your fault. So I would call Java a "hard private" language, if this is where we're drawing the line. (Is Python like one of those cheap "Warranty Void If Removed" stickers that's already started to peel up at the edges when you take the product out of the box?)

But we software types (as a generality, and like all good generalities it's not always true) tend toward the tinkerer type - the sort of people who think up a pretty funny joke about hashtags and private members, and then spend an hour learning how to write a babel plugin just so they can make that joke. Folks like us are going to break out the power tools and star shaped screw drivers from time to time.

Ultimately, I guess, the point of this issue is, if we're going to make design decisions and say "we will make this extra complicated in service of the very important goal of hard privacy," but then we don't actually get the benefits hard privacy, was that complexity worth while? The answer to that question - where and what tradeoffs are "worth it" - is going to be different for everyone, of course, and maybe it is worth it. But hopefully I've given people something to think about over the holidays; once this makes it to stage 4, it's forever.

[littledan]

I'd say that running a Babel transform like this is a way of forking your dependency, which is a reliable way to get at private state. The key is, when forking a dependency, you are taking on the burden of maintaining that fork. By contrast, many large JS projects have had to roll back changes when there were too many compatibility complaints from people accessing "internal" APIs. I'd say this is a qualitative difference.

[jwalton]

@littledan I understand the point you're making, but I think the line you're drawing here is a little murky, and is only going to get less clear as time goes on. Babel added the overrides keyword to their config specifically to make it easy to transpile dependencies.

In our product, we're already transpiling the very popular debug library because they now publish code with es6 features to npm. Our toolchain for an older part of our product has an ancient i18n string extractor, written by a former co-op, which is totally unmaintained and breaks on the const keyword, so we transpile debug all the way down to ES5, so it doesn't crash our i18n preprocessor. We don't even rely on debug directly - our dependencies rely on debug, but that's enough to get the code into our bundle. (Wow, I really need to fix that. That's something I should work on while I'm at my in-laws for the holidays.)

But my point is, there are a lot of advantages to transpiling dependencies, and it's only going to get more popular to do so as time goes on.

And again, to look at this from a purely practical standpoint; if there's a library foo, and foo's users constantly violate it's privacy restrictions, that's a pretty good indicator that the author of foo is perhaps being a bit too restrictive in what they expose. If, in order to be useful, users of foo need to violate privacy restrictions, then they will figure out a way to do so. Life finds a way. If people are violating the privacy restrictions en-masse, and the package maintainer makes a change but it breaks the Internet, Wreck-it-Ralph style, then on that day when we are all huddled in our cold, dark homes, unable to control our IoT thermostats and lights, when Alexa no longer fills our orders for milk and we are unable to feed ourselves, will it really matter if we can say "This is our own fault, because we didn't have hard private variables in JavaScript, and someone posted a popular stack overflow answer that told people to use foo._widget!" Or will we somehow be saved if we can say "We gave the world beautiful perfect hard privacy. It's totally not our fault that someone posted a popular stack overflow answer that told people to install a babel plugin and /then/ to use foo._widget"?

[superamadeus]

Unintentionally depending on a library's "soft-private" api is easier than you'd think. For example:

My software depends on library@1.0.0.

// node_modules/library@1.0.0

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }
}

// index.js

class MyLibrary extends Library {
  getBooksByAuthor(author) {
    return this._getBooksWithFilter(b => b.author === author);
  }

  _getBooksWithFilter(filterFn) {
    return this.getBooks().filter(filterFn);
  }
}

Ooh, Library@1.1.0 was released! I'll upgrade because features I want have been added in a non-breaking way (according to semantic versioning).

// node_modules/library@1.1.0

class BookFilter {
  applyFilter(books) {
    return books.filter(/* ... */);
  }
}

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }

  _getBooksWithFilter(filter) {
    return filter.applyFilter(this.getBooks());
  }
}

Suddenly my software is not working as expected. How is this possible? It was a minor version change! This library is unpleasant!

[zenparsing]

@superamadeus This is a good use case for symbols.

[superamadeus]

@zenparsing Alternatively, private fields.

Edit:

// node_modules/library@1.1.0

class BookFilter {
  applyFilter(books) {
    return books.filter(/* ... */);
  }
}

class Library {
  /* ... */

  getBooks() {
    return [ /* ... */ ]
  }

  #getBooksWithFilter = (filter) => {
    return filter.applyFilter(this.getBooks());
  }
}

[ljharb]

Note that the semver spec is a bit ambiguous about whether that change can be semver-minor - are symbols and underscores part of the documented API or not? If by “documented” you mean “what i can see in the repl”, yes - if you mean “what’s written in prose in another place” then probably not. I interpret it in the former way, to protect my users from any unintentional breakage; some authors are less cautious. Private fields make this unambiguous.

[zenparsing]

In my opinion symbols are a better fit for the library use case.

Why?

• They interoperate better with existing language features (like Proxy and Reflect).
• They provide enough friction to discourage most "mucking about with" scenarios.