Title: 8-BIT QUANTIZATION PROVIDES NO PRIVACY BEN-EFIT AGAINST TRAINING-FREE EMBEDDING INVER-SION FARS
PDF: c788d080-f041-43af-9986-ab3e4f51b8f2.pdf
Score: 4.8
Verdict: Reject
Confidence: 0.72
Elapsed: 229.7s

Strengths:
1. Utility-matched experimental design (§3.4): calibrating Gaussian noise σ=0.05 to match quantized nDCG@10 (0.5434 vs 0.5441, Δ=0.0007) isolates privacy effects from utility confounds. This is a methodologically sound contribution to the evaluation paradigm for embedding privacy defenses.
2. Geometric mechanistic analysis (§4.3, Table 2): quantifies why quantization fails — 0.9° angular deviation and ρ=0.9999 pairwise correlation preservation vs 41.5° and ρ=0.656 for noise. This goes beyond empirical observation to provide a causal explanation tied to ZSInvert's cosine-similarity optimization objective.
3. Honest negative-result reporting with practical clarity: the paper clearly states quantization provides negligible privacy benefit and does not oversell the result. The Limitations section (§5) explicitly acknowledges single encoder/dataset/attack/quantization scheme, which is appropriate transparency.
4. Stage 3 ablation (§4.4): demonstrates the correction model contributes zero exact matches across all conditions, cleanly identifying cosine-similarity-guided beam search (Stage 2) as the sole driver of attribute recovery — strengthening the geometric mechanism argument.

Weaknesses:
1. Geometrically trivial core masked by 'surprising' framing: 8-bit quantization preserves cosine similarity by construction (that is its design purpose for retrieval), and ZSInvert optimizes cosine similarity — so the result that quantization fails to defend is a near-tautology, not a surprise. The ρ=0.9999 preservation is exactly what absmax int8 quantization produces. The paper over-packages this as 'surprising' (used 3× in abstract/intro) when it is geometrically inevitable.
2. No statistical significance tests: Canary-EM 0.060±0.004 vs 0.064±0.010 for raw vs quantized — the confidence intervals overlap substantially, and no t-test, bootstrap, or permutation test is reported. The 6% relative reduction claim is statistically indistinguishable from zero. This is a hard fail for empirical rigor (HF_NO_SIGNIFICANCE).
3. 55× L2-norm confound in noise baseline: Table 2 shows noise perturbation has L2=1.383±0.034 vs quantization L2=0.025±0.002 — a 55× magnitude difference. Utility-matching controls nDCG@10 but not perturbation magnitude. The privacy comparison is partially confounded: noise achieves more privacy partly because it perturbs embeddings far more aggressively, and utility-matching selects the noise level that coincidentally preserves nDCG@10 despite massive geometric disruption. The comparison is 'same utility, different perturbation magnitude' not 'same perturbation type, different privacy effect.'
4. Extremely narrow experimental scope: single encoder (Contriever-768), single dataset (SciFact), single attack (ZSInvert with Qwen2.5-7B), single quantization scheme (absmax int8). No other quantization methods (product quantization, binary, 4-bit), no other encoders (BERT-based, OpenAI), no other attacks (Vec2Text, GEIA), no other domains. The generalizability claim in the title ('8-Bit Quantization Provides No Privacy Benefit') is far stronger than the evidence supports.
5. Noise is itself an explicit privacy defense, not a 'free' baseline: the paper frames noise as a comparison showing quantization fails, but adding calibrated Gaussian noise to stored embeddings is itself an active privacy mechanism requiring explicit deployment. The practical implication ('efficiency and privacy require separate, explicit mechanisms') is already well-understood — the paper confirms common sense rather than advancing knowledge.

Must Fix Items:
1. Add statistical significance tests (paired t-test, bootstrap CI, or permutation test) for all pairwise Canary-EM comparisons — the 6% relative reduction claim is unsupported without them.
2. Acknowledge the L2-norm confound explicitly: discuss whether the privacy difference between quantization and noise is explained by perturbation magnitude rather than perturbation type, and consider a magnitude-matched noise condition as an additional control.
3. Reduce 'surprising' framing: the result is geometrically expected from the design properties of absmax quantization and ZSInvert's cosine-similarity objective. Frame as empirical confirmation of a geometrically predicted outcome rather than a surprise.

Runs:
- run=1 score=4.8 verdict=Reject confidence=0.72 error=None