Skip to content

Commit 4104f0b

Browse files
NathanWalkerNathanWalker
committed
feat: HMR dev-sessions, ESM resolver hardening, dev-mode runtime globals
Adds the Hot Module Replacement runtime layer plus the supporting ESM resolver hardening and dev-session globals that make hot reload viable on Android. * `import.meta.hot`: `data`, `accept`, `dispose`, `prune`, `decline`, `invalidate`, `on`/`off`/`send` event surface. * Dev-session globals (`__nsStartDevSession`, `__nsReloadDevApp`, `__nsInvalidateModules`, `__nsRunHmrDispose`, `__nsRunHmrPrune`, `__nsHasDeclinedModule`, `__nsKickstartHmrPrefetch`, `__nsGetLoadedModuleUrls`, `__nsApplyStyleUpdate`, `__nsConfigureDevRuntime`/`__nsConfigureRuntime`, `__nsTerminateAllWorkers`). * Speculative HTTP module prefetch (opt-in) with canonical-key normalization so `__ns_hmr__/v<N>` and `__ns_boot__/b<N>` tag prefixes share `hot.data` identity across reload cycles. * ESM resolver hardening in `ModuleInternalCallbacks.cpp` to: - Preserve synthetic-namespace identity (`ns-vendor://`, `optional:`, `node:`, `blob:`) — these are NOT filesystem paths. - Handle HTTP/HTTPS module URLs end-to-end (resolution, fetch, canonical-key collapse, dynamic import). - Compile `.json` imports into synthetic ES modules. * Android runtime-dex support for `.extend()` classes created during HMR that the static binding generator can't see at build time: runtime DEX generation with `$`/`_` inner-class normalization (`DexFactory`), dev/HMR class-resolution fallback (`ClassResolver`), and dev-flag / `logScriptLoading` plumbing (`AppConfig`, `DevFlags`).
1 parent c9d41e6 commit 4104f0b

23 files changed

Lines changed: 5306 additions & 252 deletions

‎README.md‎

Lines changed: 108 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ Contains the source code for the NativeScript's Android Runtime. [NativeScript](
77

88
- [Main Projects](#main-projects)
99
- [Helper Projects](#helper-projects)
10+
- [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation)
1011
- [Architecture Diagram](#architecture-diagram)
1112
- [Build Prerequisites](#build-prerequisites)
1213
- [How to build](#how-to-build)
@@ -29,9 +30,115 @@ The repo is structured in the following projects (ordered by dependencies):
2930

3031
## Helper Projects
3132

32-
* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code.
33+
* [**android-static-binding-generator**](android-static-binding-generator) - build tool that generates bindings based on the user's javascript code. See [SBG vs Runtime Dex Generation](#sbg-vs-runtime-dex-generation) for the production vs HMR-dev split.
3334
* [**project-template**](build-artifacts/project-template-gradle) - this is an empty placeholder Android Application project, used by the [NativeScript CLI](https://github.com/NativeScript/nativescript-cli) when building an Android project.
3435

36+
## SBG vs Runtime Dex Generation
37+
38+
The Android runtime turns every JavaScript `.extend('com.tns.Foo', Bar, { ... })`
39+
(or `Bar.extend({ ... })`) call into a real Java subclass with a dispatching
40+
proxy. There are **two** code paths that produce that subclass — a build-time
41+
path and a runtime path — and which one runs depends on whether the
42+
`.extend(...)` call site was visible to the Static Binding Generator (SBG)
43+
when the APK was built.
44+
45+
### Build-time path (production / classic CLI)
46+
47+
[**android-static-binding-generator**](android-static-binding-generator) (SBG)
48+
runs over the bundled JS once during `nativescript build android`:
49+
50+
1. SBG parses every JS file the bundle includes and finds every
51+
`.extend('com.tns.X', BaseClass, { ... })` call (and the modern
52+
`BaseClass.extend({ ... })` shorthand).
53+
2. For each call it asks
54+
[**android-binding-generator**](test-app/runtime-binding-generator)'s
55+
`ProxyGenerator`/`Dump` to emit a `.dex` for a Java class named
56+
`com.tns.gen.<BaseFqn-with-$-as-_>` (or, for `@JavaProxy(...)`-style
57+
explicit names, the user-chosen name).
58+
3. The resulting dex files are packaged into the APK alongside the JS
59+
bundle and listed in metadata so the runtime can find them via the
60+
classloader.
61+
62+
In production this means the Java class is already present and loadable
63+
the very first time the JS `.extend(...)` call runs. The runtime never
64+
generates a fresh dex.
65+
66+
### Runtime path (HMR-dev / dynamic `.extend`)
67+
68+
When SBG can't see the call at build time, the runtime generates the dex
69+
on demand. This happens in two common shapes:
70+
71+
* **HMR/Vite dev workflow** — the build-time bundle (`bundle.mjs`) is just
72+
the HMR bootstrap; modules like
73+
`@nativescript/core/ui/frame/fragment.android.ts` are fetched over HTTP
74+
at runtime. SBG never sees those `.extend(...)` calls, so no pre-baked
75+
dex exists.
76+
* **`unnamed-extend` use cases** — `eval`-generated extends, or extends
77+
whose first argument is computed at runtime, escape the SBG scan even
78+
in production builds.
79+
80+
The runtime path is wired through
81+
[`com.tns.ClassResolver`](test-app/runtime/src/main/java/com/tns/ClassResolver.java)
82+
→ [`com.tns.DexFactory`](test-app/runtime/src/main/java/com/tns/DexFactory.java):
83+
84+
1. `ClassResolver.resolveClass` first tries `classStorageService.retrieveClass(name)`.
85+
In production this hits the SBG-generated dex and we're done.
86+
2. On `LookedUpClassNotFound` (typical for HMR), if a `baseClassName` is
87+
present, `ClassResolver` falls back to `DexFactory.resolveClass(...)`
88+
which runs the same `ProxyGenerator`/`Dump` pipeline SBG uses — only
89+
it does it at runtime, writes the dex into the app's per-thumb cache
90+
under `<dexDir>/<name>.dex`, wraps it in a `.jar`, and loads it via
91+
`DexClassLoader` (or `BaseDexClassLoader` injection when the
92+
`injectIntoParentClassLoader` flag is on).
93+
3. Generated proxy class names normalize JVM inner-class `$` to `_`
94+
(both in `Dump`'s class signature and in `DexFactory`'s
95+
`loadClass(...)` arg). The actual JVM lookup name is always
96+
`com.tns.gen.<base-with-$-as-_>` — never `com.tns.gen.<base>$<nested>`.
97+
98+
### Edge cases worth knowing about
99+
100+
* **`$` vs `_` mismatches** — `Class.forName(baseClassName)` requires JVM
101+
`$` inner-class syntax (`android.app.Application$ActivityLifecycleCallbacks`),
102+
but `classLoader.loadClass(generatedName)` requires the `_`-normalized
103+
sibling (`com.tns.gen.android.app.Application_ActivityLifecycleCallbacks`).
104+
Both `DexFactory.resolveClass` and `DexFactory.findClass` apply the
105+
normalization in the loadable-name path while preserving `$` for the
106+
reflective base-class lookup. Removing either normalization
107+
reintroduces the "Didn't find class
108+
`com.tns.gen.android.app.Application$ActivityLifecycleCallbacks`"
109+
ClassNotFoundException for runtime-generated proxies.
110+
* **Cache invalidation** — the runtime dex cache is keyed on a per-build
111+
`dexThumb` (the runtime regenerates it whenever the JS bundle changes).
112+
Stale dex from a previous boot is purged in
113+
`DexFactory.updateDexThumbAndPurgeCache`. Don't bypass the thumb — a
114+
dex generated against an older base class can crash at method dispatch
115+
time when the base API drifts.
116+
* **Parent-classloader injection** — Android framework code that calls
117+
`Class.forName(name)` searches the app's `PathClassLoader`, *not* an
118+
isolated `DexClassLoader`. When a generated proxy needs to be visible
119+
to framework reflection (e.g. `FragmentFactory`, `Activity` resolution
120+
from `AndroidManifest.xml`), construct the `DexFactory` with
121+
`injectIntoParentClassLoader=true` so its `injectDexIntoClassLoader`
122+
helper splices the generated jar's `dexElements` onto the parent's
123+
`pathList`.
124+
* **Don't normalize synthetic module keys** — module registry keys like
125+
`ns-vendor://...`, `optional:...`, `node:...`, `blob:...` are not
126+
filesystem paths and must be preserved verbatim through invalidation
127+
+ reload. They are NOT the same kind of "synthetic name" as the
128+
`com.tns.gen.<...>` class names above and don't go through this dex
129+
pipeline.
130+
131+
### What to look at when this breaks
132+
133+
* `ClassResolver.java` — the fallback decision between
134+
`classStorageService` and `DexFactory`.
135+
* `DexFactory.java` — runtime dex generation, cache, and the
136+
`$`→`_` normalization.
137+
* `ProxyGenerator.java` + `Dump.java` (under `runtime-binding-generator`)
138+
— what the generated class actually looks like in bytecode.
139+
* `android-static-binding-generator` — what SBG sees (and crucially
140+
what it *doesn't* see) at build time.
141+
35142
## Architecture Diagram
36143
The NativeScript Android Runtime architecture can be summarized in the following diagram.
37144

‎test-app/app/src/main/assets/app/mainpage.js‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,3 +75,5 @@ require('./tests/testQueueMicrotask');
7575
require("./tests/testConcurrentAccess");
7676

7777
require("./tests/testESModules.mjs");
78+
require("./tests/testHmrHotDataExt.mjs");
79+
require("./tests/testNodeBuiltinsAndOptionalModules.mjs");

‎test-app/app/src/main/assets/app/tests/esm/hmr/hot-data-ext.js‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
// HMR hot.data test module (.js).
2+
//
3+
// INTENTIONAL twin of hot-data-ext.mjs. Two physical files with
4+
// different extensions are required so the HMR canonical-key
5+
// extension-collapse path is actually exercised by tests that import
6+
// BOTH variants (see testHmrHotDataExt
7+
// "should share hot.data across .mjs and .js variants"). Each file
8+
// MUST own its own `import.meta.hot` reference — re-exporting from the
9+
// sibling would defeat the test, because `dataMjs === dataJs` would
10+
// then hold trivially via function identity instead of validating the
11+
// runtime's canonical-key normalization.
12+
//
13+
// Keep the body in lock-step with `hot-data-ext.mjs`.
14+
15+
export function getHot() {
16+
return (typeof import.meta !== "undefined" && import.meta) ? import.meta.hot : undefined;
17+
}
18+
19+
export function getHotData() {
20+
const hot = getHot();
21+
return hot ? hot.data : undefined;
22+
}
23+
24+
export function setHotValue(value) {
25+
const hot = getHot();
26+
if (!hot || !hot.data) {
27+
throw new Error("import.meta.hot.data is not available");
28+
}
29+
hot.data.value = value;
30+
return hot.data.value;
31+
}
32+
33+
export function getHotValue() {
34+
const hot = getHot();
35+
return hot && hot.data ? hot.data.value : undefined;
36+
}
37+
38+
export function testHotApi() {
39+
const hot = getHot();
40+
const result = {
41+
ok: false,
42+
hasHot: !!hot,
43+
hasData: !!(hot && hot.data),
44+
hasAccept: !!(hot && typeof hot.accept === "function"),
45+
hasDispose: !!(hot && typeof hot.dispose === "function"),
46+
hasDecline: !!(hot && typeof hot.decline === "function"),
47+
hasInvalidate: !!(hot && typeof hot.invalidate === "function"),
48+
hasPrune: !!(hot && typeof hot.prune === "function"),
49+
};
50+
51+
try {
52+
if (hot && typeof hot.accept === "function") {
53+
hot.accept(function () {});
54+
}
55+
if (hot && typeof hot.dispose === "function") {
56+
hot.dispose(function () {});
57+
}
58+
if (hot && typeof hot.decline === "function") {
59+
hot.decline();
60+
}
61+
if (hot && typeof hot.invalidate === "function") {
62+
hot.invalidate();
63+
}
64+
result.ok =
65+
result.hasHot &&
66+
result.hasData &&
67+
result.hasAccept &&
68+
result.hasDispose &&
69+
result.hasDecline &&
70+
result.hasInvalidate &&
71+
result.hasPrune;
72+
} catch (e) {
73+
result.error = (e && e.message) ? e.message : String(e);
74+
}
75+
76+
return result;
77+
}
78+
79+
console.log("HMR hot.data ext module loaded (.js)");

‎test-app/app/src/main/assets/app/tests/esm/hmr/hot-data-ext.mjs‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
// HMR hot.data test module (.mjs).
2+
//
3+
// INTENTIONAL twin of hot-data-ext.js. Two physical files with
4+
// different extensions are required so the HMR canonical-key
5+
// extension-collapse path is actually exercised by tests that import
6+
// BOTH variants (see testHmrHotDataExt
7+
// "should share hot.data across .mjs and .js variants"). Each file
8+
// MUST own its own `import.meta.hot` reference — re-exporting from the
9+
// sibling would defeat the test, because `dataMjs === dataJs` would
10+
// then hold trivially via function identity instead of validating the
11+
// runtime's canonical-key normalization.
12+
//
13+
// Keep the body in lock-step with `hot-data-ext.js`.
14+
15+
export function getHot() {
16+
return (typeof import.meta !== "undefined" && import.meta) ? import.meta.hot : undefined;
17+
}
18+
19+
export function getHotData() {
20+
const hot = getHot();
21+
return hot ? hot.data : undefined;
22+
}
23+
24+
export function setHotValue(value) {
25+
const hot = getHot();
26+
if (!hot || !hot.data) {
27+
throw new Error("import.meta.hot.data is not available");
28+
}
29+
hot.data.value = value;
30+
return hot.data.value;
31+
}
32+
33+
export function getHotValue() {
34+
const hot = getHot();
35+
return hot && hot.data ? hot.data.value : undefined;
36+
}
37+
38+
export function testHotApi() {
39+
const hot = getHot();
40+
const result = {
41+
ok: false,
42+
hasHot: !!hot,
43+
hasData: !!(hot && hot.data),
44+
hasAccept: !!(hot && typeof hot.accept === "function"),
45+
hasDispose: !!(hot && typeof hot.dispose === "function"),
46+
hasDecline: !!(hot && typeof hot.decline === "function"),
47+
hasInvalidate: !!(hot && typeof hot.invalidate === "function"),
48+
hasPrune: !!(hot && typeof hot.prune === "function"),
49+
};
50+
51+
try {
52+
if (hot && typeof hot.accept === "function") {
53+
hot.accept(function () {});
54+
}
55+
if (hot && typeof hot.dispose === "function") {
56+
hot.dispose(function () {});
57+
}
58+
if (hot && typeof hot.decline === "function") {
59+
hot.decline();
60+
}
61+
if (hot && typeof hot.invalidate === "function") {
62+
hot.invalidate();
63+
}
64+
result.ok =
65+
result.hasHot &&
66+
result.hasData &&
67+
result.hasAccept &&
68+
result.hasDispose &&
69+
result.hasDecline &&
70+
result.hasInvalidate &&
71+
result.hasPrune;
72+
} catch (e) {
73+
result.error = (e && e.message) ? e.message : String(e);
74+
}
75+
76+
return result;
77+
}
78+
79+
console.log("HMR hot.data ext module loaded (.mjs)");

‎test-app/app/src/main/assets/app/tests/testHmrHotDataExt.mjs‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
// HMR import.meta.hot.data sharing tests.
2+
//
3+
// These tests exercise the canonical-key extension-collapse path:
4+
// importing the *same* logical module under `.mjs` and `.js` extensions
5+
// MUST yield the same `import.meta.hot.data` object identity, so that
6+
// state written from one variant is observable in the other.
7+
//
8+
// The two fixture files under `tests/esm/hmr/` MUST remain independent
9+
// (no re-export of one from the other) — see the comment header in
10+
// each fixture for why.
11+
//
12+
// HTTP-loader variants of these tests (live-tagged, boot-tagged, and
13+
// /ns/core bridge URLs) live in HttpEsmLoaderTests on iOS. They depend
14+
// on a dev-server harness that Android does not currently stand up,
15+
// and are intentionally not ported here. The local twin-file path
16+
// below still exercises the core canonical-key normalization.
17+
18+
describe("HMR hot.data", function () {
19+
it("exposes the import.meta.hot API surface", async function () {
20+
const mod = await import("~/tests/esm/hmr/hot-data-ext.mjs");
21+
expect(mod).toBeTruthy();
22+
expect(typeof mod.testHotApi).toBe("function");
23+
24+
const result = mod.testHotApi();
25+
expect(result).toBeTruthy();
26+
if (!result.hasHot) {
27+
pending("import.meta.hot not available (release build?)");
28+
return;
29+
}
30+
31+
expect(result.hasData).toBe(true);
32+
expect(result.hasAccept).toBe(true);
33+
expect(result.hasDispose).toBe(true);
34+
expect(result.hasDecline).toBe(true);
35+
expect(result.hasInvalidate).toBe(true);
36+
expect(result.hasPrune).toBe(true);
37+
expect(result.ok).toBe(true);
38+
});
39+
40+
it("should share hot.data across .mjs and .js variants", async function () {
41+
const [mjs, js] = await Promise.all([
42+
import("~/tests/esm/hmr/hot-data-ext.mjs"),
43+
import("~/tests/esm/hmr/hot-data-ext.js"),
44+
]);
45+
46+
const hotMjs = mjs && typeof mjs.getHot === "function" ? mjs.getHot() : null;
47+
const hotJs = js && typeof js.getHot === "function" ? js.getHot() : null;
48+
if (!hotMjs || !hotJs) {
49+
pending("import.meta.hot not available (release build?)");
50+
return;
51+
}
52+
53+
const dataMjs = mjs.getHotData();
54+
const dataJs = js.getHotData();
55+
expect(dataMjs).toBeDefined();
56+
expect(dataJs).toBeDefined();
57+
58+
const token = "tok_" + Date.now() + "_" + Math.random();
59+
mjs.setHotValue(token);
60+
expect(js.getHotValue()).toBe(token);
61+
62+
// Canonical hot key strips common script extensions, so these must share identity.
63+
expect(dataMjs).toBe(dataJs);
64+
});
65+
});

0 commit comments

Comments
 (0)