Standard dense alignment methods force LLMs to collapse conflicting cultural values into a generic average (Mean Collapse), producing responses that fail to resonate with any specific group. We attribute this to Cultural Sparsity — human values cluster into distinct, conflicting modes that a single set of parameters cannot simultaneously capture.

CuMA resolves this via demographic-aware routing: it conditions expert selection on both semantic content and the user's demographic profile, learning a Latent Cultural Topology that explicitly disentangles conflicting gradients into specialized parameter subspaces. The figure below shows the architecture of a CuMA transformer block. CuMA freezes the backbone weights and injects $N$ LoRA experts, using a demographic-conditioned router to select the Top-$k$ experts per token.

• Mean collapse mitigation: dense baselines show high prediction entropy and lower response diversity, while CuMA sharpens per-profile alignment and preserves cross-profile diversity.
• Routing precision beats scale: CuMA ($r=8$, 1.53% trainable parameters) outperforms larger semantic-only MoE baselines such as HydraLoRA (2.31%).
• Distributional fidelity: CuMA improves WVB accuracy while reducing EMD, indicating that it models the full shape of human value distributions rather than only predicting the majority mode.
• Holistic alignment: on Qwen3-8B, CuMA reaches 78.2% / 76.8% GRPO win-rates on CA / PRISM generation tasks.
• Zero-shot transfer: the learned latent topology supports held-out demographic profile generalization, with sample-weighted zero-shot WVB accuracy of 48.10%, outperforming the strongest baseline at 36.2%.

The project uses pyproject.toml for dependency management. We recommend using uv for fast installation:

uv sync

models/          # CuMA model, demographic encoder, and baseline models (HydraLoRA, MixLoRA)
train/           # Training scripts for CuMA (SFT/DPO/GRPO) and all baselines
evaluate/        # Evaluation (Acc, F1, EMD, Win-Rate)
utils/           # Dataset loaders and utilities
process_datasets/# Data preprocessing
scripts/         # Shell scripts for training, evaluation, and ablation studies

• WorldValuesBench (WVB): Download from WorldValuesBench and place contents in WorldValuesBench/.
• PRISM & Community Alignment (CA): Run the download script:

  python download_datasets.py

python process_datasets/process_datasets.py

CuMA on WVB (SFT):

bash scripts/train_cuma_wvs_sft.sh

CuMA on CA (SFT → RL):

bash scripts/train_cuma_facebook_sft_dpo.sh   # SFT → DPO

CuMA on PRISM (SFT → RL):

bash scripts/train_cuma_prism_sft_dpo.sh      # SFT → DPO

Baselines (examples):

bash scripts/train_lora_wvs_sft.sh          # LoRA
bash scripts/train_fft_wvs_sft.sh           # Full Fine-Tuning
bash scripts/train_hydralora_wvs_sft.sh     # HydraLoRA
bash scripts/train_mixlora_wvs_sft.sh       # MixLoRA
bash scripts/train_prefix_wvs_sft.sh        # P-Tuning v2

Llama-3.1-8B:

bash scripts/train_cuma_wvs_sft_llama3.1_r64_lr5e-6.sh

Ablation studies:

bash scripts/train_cuma_wvs_sft_r64_a128_lr5e6_ablation_minimal.sh     # w/o Demo & Balance
bash scripts/train_cuma_wvs_sft_r64_a128_lr5e6_ablation_soft_routing.sh # Soft routing
bash scripts/train_cuma_wvs_sft_wo_demo.sh                              # w/o Demographic

bash scripts/evaluate_wvs.sh         # WVB: Acc / Macro-F1 / EMD
bash scripts/evaluate_dpo.sh         # CA/PRISM: GPT-4o Win-Rate

GRPO training is also supported via train/train_cuma_grpo.py with GPT-4o as the demographic-aware reward judge.

If you find this work useful, please cite:

@misc{sun2026cuma,
    title={CuMA: Aligning LLMs with Sparse Cultural Values via Demographic-Aware Mixture of Adapters},
    author={Ao Sun and Xiaoyu Wang and Zhe Tan and Yu Li and Jiachen Zhu and Yuheng Jia and Shu Su},
    year={2026},
    eprint={2601.04885},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2601.04885},
}

This project is licensed under the Apache License 2.0.