[jonasschaefer]

Summary

DEW (Dual-Embedding Watermarking) is a semantic watermarking method. At each
generation step it biases the token logits by the similarity between keyed
random projections of (a) the preceding context, embedded by a sentence
encoder, and (b) the candidate token embeddings. Both sides are projected into
a shared latent space with secret-key-seeded random matrices, and the per-token
score is standardized so that detection reduces to a one-sided Gaussian test on
the mean score.

Paper: https://arxiv.org/abs/2606.31602

What's included

• watermark/dew/ — DEWConfig, DEWUtils, DEWLogitsProcessor, DEW,
  following the existing MarkLLM algorithm interfaces.
• config/DEW.json — default configuration.
• Registered in the AutoWatermark and AutoConfig mappings.
• Token-side whitening (StandardScaler + PCA) is fit on demand from the
  generation model's input embeddings and cached to disk (the cache directory
  is gitignored, so no large binaries are committed).
• requirements.txt — adds scikit-learn and joblib for the whitening
  pipeline.
• README.md — DEW added to the "Currently Supported Algorithms" table.
• test/test_method.py — DEW accepted by the test harness.

Testing

Tested on a GPU (RTX A5000, CUDA 12.4) with facebook/opt-1.3b and the
sentence-transformers/paraphrase-multilingual-mpnet-base-v2 context encoder:

• watermarked text detected at p ≈ 9e-17 over 228 tokens;
• unwatermarked and human/natural text not flagged (p ≈ 0.04 and 0.84);
• the whitening pipeline auto-fits, caches to disk, and reloads on the next run;
• existing algorithms are unaffected (verified end-to-end with KGW).

[GaryFuzheyu]

The watermarking algorithm DEW you proposed is very valuable. Thank you for your interest in the MarkLLM repository and for your pull request. However, there is one improvement that we would like you to consider, and we hope you can make the following modification: DEWLogitsProcessor.call crashes with RuntimeError: scatter(): Expected self.dtype to be equal to src.dtype when the base model runs in fp16/bf16 (reproduced on OPT-1.3B). The projection matrices R_T / R_C are fp32 so the bias tensor is fp32; the mixed-precision add is upcast to fp32 and then fails to scatter into the fp16/bf16 new_scores. Casting back with .to(new_scores.dtype) before scatter_ fixes it. Verified end-to-end in fp16 (p=1e-8 vs 0.065) and bf16 (p=3e-11 vs 0.9).

[jonasschaefer]

Thanks for catching this, and sorry for the late response. I was busy improving the DEW preprint for the EMNLP 2026 camera-ready. I’ve now reproduced the fp16/bf16 issue and fixed it by casting the biased logits back to the model’s dtype before scatter_.