Allow to create in-memory hypercores

[Frando]

What do you think about adding a { persist: false } option to corestore.get? It would create an in-memory hypercore that behaves just the same as stored hypercores, with regard to both key derivation (from the master key and namespaces) and replication.

The dat-sdk currently has some code to "patch in" in-memory cores into corestore's replication logic, using internal methods. This could be removed then. There's several usecases for replicated in-memory cores I think.

[mafintosh]

Can you describe the use case for this a bit more? Makes things inside the corestore much more complex in terms of lifecycle when storage isn't consistent

[RangerMauve]

I use it in applications where I want to load data temporarily and throw it out after I'm done while using the same corestore to manage persistent data.

[RangerMauve]

Flow could be something like:

• Create persistent hyperdrive
• Load a core in-memory to fetch a bit of data from it
• Place data in hyperdrive
• Close core because I don't need it anymore

[mafintosh]

You can use hypercore(ram) for that with no added complexity currently. Not sure what’s gained by adding that to the corestore.

[RangerMauve]

If you use hypercore(ram) you need to set up the swarming for it separately from the stuff you're already doing for the corestore. 😁 It can be nice to leverage connection reuse / reduce overall network traffic.

[andrewosh]

Adding this to corestore would definitely simplify getting the temporary cores hooked up to networking, but as @mafintosh said it'd come with a complexity cost. I think the need for this motivates one or more of four things:

1. A corestore method that temporarily inserts an externally-managed core into a corestore's caches + replication streams.
2. A userland module that wraps a single-corestore networking module (like corestore-swarm-networking) and makes it easy to push other cores into its replication streams.
3. A corestore deletion API (which I know you're keen on @RangerMauve, but which we've been hesitant to do until a good approach to key backups is available).
4. A corestore persist: false option.

The persist: false option seems the least desirable because of its complexity -- the corestore API surface/internals should be as small/simple as possible, and we're already pushing it IMO!

I'd personally opt for (2) and/or (3). (2) because it's userland which is always nicer, and (3) because it's something we need to tackle anyway, so we'd get many benefits elsewhere.

[RangerMauve]

For what it's worth (3) could suit the needs of temporary cores, and it doesn't need any special APIs inside corestore. You could do feed.destroyStorage() which'd do the trick. 🤷 I think it'd require developers to be aware that they should clean up cores they don't want stored anymore.