Error: Cannot find module 'env'

[EdgeKing810]

🐛 Bug description

After running a wasm-pack build --release --target nodejs, when importing the generated js file from the pkg/ folder in a NodeJS app, I'm getting the following error:

node:internal/modules/cjs/loader:933
  const err = new Error(message);
              ^

Error: Cannot find module 'env'

🤔 Expected Behavior

This line should not be in the generated js file:

imports['env'] = require('env');

I do not know if this should be replaced by something else but it sure doesn't work by just removing this line.

👟 Steps to reproduce

Easiest way to reproduce this would be to execute the following commands:

git clone https://github.com/EdgeKing810/kinesis-db
cd kinesis-db/
bash build-wasm.sh
cd api/
node index.js

Full error when doing a import pkg from '../pkg/kinesis_db.js'; in my NodeJS app:

node:internal/modules/cjs/loader:933
  const err = new Error(message);
              ^

Error: Cannot find module 'env'
Require stack:
- /home/edgeking810/Documents/Rust/kinesis-db/pkg/kinesis_db.js
    at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
    at Function.Module._load (node:internal/modules/cjs/loader:778:27)
    at Module.require (node:internal/modules/cjs/loader:999:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/home/edgeking810/Documents/Rust/kinesis-db/pkg/kinesis_db.js:2:18)
    at Module._compile (node:internal/modules/cjs/loader:1097:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1149:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/translators:190:29) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [ '/home/edgeking810/Documents/Rust/kinesis-db/pkg/kinesis_db.js' ]
}

Node.js v17.2.0

I tried using other flags such as --dev or --debug but I face the same problem. Building without the --target nodejs flag is not feasible for me since I'll be using this wasm build in a NodeJS backend app, not for a frontend application with a bundler such as Webpack.

🌍 My environment

# cargo --version
cargo 1.57.0 (b2e52d7ca 2021-10-21)
# rustc -V
rustc 1.57.0 (f1edd0429 2021-11-29)

Tried using the latest nightly Rust build too.

Additional

This issue has initially been raised in #743, I'm just creating a new issue because this is a really blocking problem. I tried other solutions such as rustwasmc too but they all have some kind of issues.

Here's the Cargo.toml file I'm using as well in case that helps:

[package]
name = "kinesis-db"
description = "A secure, portable and fast database written in Rust for Kinesis API"
repository = "https://github.com/EdgeKing810/kinesis-db"
license = "MIT"
version = "0.1.0"
edition = "2018"

[lib]
crate-type = ["cdylib", "rlib"]

[dependencies]
regex = "1"
magic-crypt = "3.1.9"
wasm-bindgen = "0.2.74"
fastrand = "1.5.0"
blowfish = { version = "0.8", features = ["bcrypt"] }
base64 = { version = "0.13", default-features = false }
parking_lot_core = "=0.8.0"

[profile.release]
lto = true
opt-level = "z"

FYI, I've tried tweaking the profile.release section multiple times by adding/removing the lto = true line, and 'unsetting' the opt-level variable or by giving it the value "s" + testing with multiple flags.

Happy New Year 🌃 in advance and I hope we get a fix soon for this issue 🤞. Very grateful if anyone could help on this, thanks in advance!

[EdgeKing810]

Update: I tried copying the exact same content of the Cargo.toml file in another project with every other file EXCLUDING those in the src/ file in another project and building. This works flawlessly with no weird require('env') in the generate js file. I'm still investigating what in my code is causing this line to be added, still blurry but surely progressing.

[EdgeKing810]

Okay, found where this originates. It was really painful to troubleshoot and find this. Turns out it comes from the bcrypt package. It has a hash function used to verify passwords. Commenting the following line effectively solved my problem:

 user.password = hash(password.trim(), DEFAULT_COST).unwrap().to_string();

I'm trying to patch this myself, and eventually use another solution if I am not able to get this working. Will update soon.

[EdgeKing810]

Update: Tried replacing bcrypt with argonautica and rust-argon2. The latter works and compiles but I get the same exact error that I did initially. For the former, well, it doesn't even compile to wasm and I didn't dig deeper than that, I just assumed it doesn't support building to wasm and abandoned it.

Just check out the implementation that I did in my 2 latest commits if that interests you.
https://github.com/EdgeKing810/kinesis-db/commits/master

• https://github.com/bcmyers/argonautica (17b7937a02b88198f3778de0bac17b0e32fd22aa)
• https://github.com/sru-systems/rust-argon2 (52511987d61d40723bafc3141ae86ed67da3ea99)

[simlay]

Hey so I had this issue with this (was ~8 months ago) but the issue was around parking_lot or a dependency that used parking_lot. Amanieu/parking_lot#269 might be relevant or rustwasm/wasm-bindgen#2215.

[EdgeKing810]

Hey @simlay, thanks for the info but I really don't believe I'm currently facing this issue because of the parking_lot crate since there is absolutely no mention of it in my Cargo.lock file. If it was coming from this crate, it should've been a dependency technically and be listed in that file ://

[EdgeKing810]

Update: Got an idea upon waking up to use the wasm bridge to call the hashing and verifying function for storing passwords from JS instead. It was either that or implementing my own hashing algorithm (aka bad idea). So I did that, completely removed the rust-argon2 crate and updated functions calls to call the following functions from Rust instead in the following fashion:

#[wasm_bindgen]
extern "C" {
    // #[wasm_bindgen(method, js_name = hash)]
    fn hash(s: &str, salt: &str) -> String;
    // #[wasm_bindgen(method, js_name = verify)]
    fn verify(s: &str, hash: &str) -> usize;
}

After compiling the code down to wasm, I found that I still had the initial error and I am sure that it is linked to the functions related to hash/verify passwords itself since upon removing calls to parts of the code doing this, I ceased to have initial issue with the 'env' dependency. Further on, calling Rust functions from the NodeJS code with the wasm bridge caused even more issues that I couldn't fix. A simple [Rust] function with the goal of reading a file then outputting its contents didn't return anything in JS. After a while of fighting against this, I just realized that it just wasn't worth it using wasm. Here's one of the common errors I got while calling Rust functions from JS.

wasm://wasm/002f3a3a:1


RuntimeError: unreachable
    at wasm://wasm/002f3a3a:wasm-function[300]:0x49c84
    at wasm://wasm/002f3a3a:wasm-function[776]:0x5b4bd
    at wasm://wasm/002f3a3a:wasm-function[887]:0x5d54d
    at wasm://wasm/002f3a3a:wasm-function[509]:0x541ce
    at wasm://wasm/002f3a3a:wasm-function[904]:0x5d832
    at wasm://wasm/002f3a3a:wasm-function[91]:0x2fc0a
    at wasm://wasm/002f3a3a:wasm-function[234]:0x41e99
    at wasm://wasm/002f3a3a:wasm-function[302]:0x49ff6
    at module.exports.save_users (/home/edgeking810/Documents/Rust/kinesis-db/pkg/kinesis_db.js:417:10)
    at file:///home/edgeking810/Documents/Rust/kinesis-db/api/index.js:41:18

Node.js v17.2.0

In conclusion, I find Rust to be already powerful enough to do what I want it to do and I think I've got the time to learn and build the API in Rust itself. I was initially against this since I have more knowledge in building APIs in JS than Rust but at this point, I can't trust the wasm bridge enough to have the peace of mind of running a prod API using this technology. It is going to be fun learning about Rocket nonetheless and I hope to have a really robust API by the end. If anyone wants to try out and troubleshoot what I did, I made a branch just about it: https://github.com/EdgeKing810/kinesis-db/tree/wasm. It only needs running the build-wasm.sh script and going in the api/ directory and doing a node index.js, The code that have been commented in order to prevent the 'env' issue from popping up are just 2 blocks inside of src/bindings/bindings_user.rs.

[MatthewHerbst]

@EdgeKing810 do you have any tips on how to identify code that might be causing env to be imported? I'm having a similar issue but bcrypt is not in my dependency tree at all. Thanks!

[EdgeKing810]

@EdgeKing810 do you have any tips on how to identify code that might be causing env to be imported? I'm having a similar issue but bcrypt is not in my dependency tree at all. Thanks!

@MatthewHerbst Unfortunately, you just have to comment out parts of the code block by block and running a wasm-pack build until that require('env') line thingy is not present anymore. When you identify the problem, you can find an alternative.

[trevyn]

For tracking down the offending imports, you can use wasm-dis, as in rustwasm/wasm-bindgen#2570 (comment)

wasm-dis target/wasm32-unknown-unknown/debug/deps/wasm_pack-xxxx.wasm | grep \"env\"