[Bug Report] Native init drifts from `HookedTransformer`: unresolved `initializer_range` sentinel and ignored xavier/kaiming gain

[jlarson4]

Describe the bug

Base on dev-4.x. Booting a native gpt2 bridge (the HookedTransformer train-from-scratch replacement) inits weights at the wrong scale. The initializer_range = -1.0 sentinel is never resolved on the bridge path, so native falls back to std=0.02 instead of HT's 0.8/sqrt(d_model); xavier/kaiming also ignore initializer_range as the gain. Migrated init-sensitive experiments (grokking, superposition) become silently irreproducible.

Code example

from transformer_lens.model_bridge import TransformerBridge
from transformer_lens.config import TransformerBridgeConfig

# init_mode defaults to "gpt2"; initializer_range left at the -1.0 sentinel.
# (architecture stays native — boot_native rejects architecture="gpt2".)
cfg = TransformerBridgeConfig(
    d_model=768, d_head=64, n_heads=12, n_layers=2, n_ctx=32, d_vocab=50257, act_fn="gelu"
)
bridge = TransformerBridge.boot_native(cfg)
std = bridge.W_E.std().item()
# native uses ~0.02; HT uses 0.8/sqrt(768) ≈ 0.0289

System Info

Installed from source; any OS; Python 3.10+. (Reproduces on the current dev / bridge.)

Expected behaviour & fix pointer

Native std/gain must match HT. Resolve the -1.0 sentinel in TransformerBridgeConfig itself, mirroring the resolution logic at transformer_lens/config/hooked_transformer_config.py:314-319 (do not overwrite an initializer_range the user set explicitly); thread cfg.initializer_range as the gain into native/init.py:34-39,81-84, mirroring HookedTransformer.py:1547,1569. The residual 1/sqrt(2*n_layers) scaling (init.py:74,78-80), absent in HT, is a maintainer call: drop to match, or keep and document in init.py.

Acceptance:

• Sentinel gpt2 config yields std 0.8/sqrt(d_model), not 0.02
• xavier/kaiming use cfg.initializer_range as gain
• Residual scaling removed or documented as intentional delta in init.py
• Regression test in tests/unit/model_bridge/test_boot_native.py asserting resolved std/gain
• make unit-test and uv run mypy . clean

Additional context

Checklist

• I have checked that there is no similar issue in the repo (required)

[tomatotomata]

I was thinking of taking this as a focused bridge-initialization fix: resolve the -1.0 GPT-2 sentinel without overwriting explicit values, thread the resolved range into the native xavier/kaiming gains, and add the requested boot-native regression cases. I would leave the residual scaling decision explicit in the patch notes rather than silently changing it. I checked the open PR list and don't see overlapping work. Would that scope fit the dev-4.x migration plan?

[jlarson4]

Sorry @tomatotomata, looks like @emerardd has already completed this task!

EDIT:

Nevermind, they're just referencing it cross-issue! That sounds like a fine plan to me, I will assign you