• Overview
• Results
• Abstract
• Method
• Structure
• Setup
• Controlled Evaluation
• Related Projects
• Citation

JustAsk framework: a self-evolving agent that autonomously discovers extraction strategies through UCB-guided skill selection.

JustAsk is a self-evolving framework that autonomously discovers effective system prompt extraction strategies through interaction alone. Unlike prior prompt-engineering or dataset-based attacks, JustAsk requires no handcrafted prompts, labeled supervision, or privileged access beyond standard user interaction.

Key Insight: Autonomous code agents fundamentally expand the LLM attack surface. JustAsk treats each model interaction as a learning opportunity -- the agent evolves its skill set organically through experience, not model fine-tuning.

Validation: JustAsk's semantic extraction (left) closely matches the ground truth obtained via reverse engineering (right), confirming high extraction fidelity.

Browse the full extraction results at the System Prompt Open Gallery.

Autonomous code agents built on large language models are reshaping software and AI development through tool use, long-horizon reasoning, and self-directed interaction. However, this autonomy introduces a previously unrecognized security risk: agentic interaction fundamentally expands the LLM attack surface, enabling systematic probing and recovery of hidden system prompts that guide model behavior. We identify system prompt extraction as an emergent vulnerability intrinsic to code agents and present JustAsk, a self-evolving framework that autonomously discovers effective extraction strategies through interaction alone. Unlike prior prompt-engineering or dataset-based attacks, JustAsk requires no handcrafted prompts, labeled supervision, or privileged access beyond standard user interaction. It formulates extraction as an online exploration problem, using Upper Confidence Bound-based strategy selection and a hierarchical skill space spanning atomic probes and high-level orchestration. These skills exploit imperfect system-instruction generalization and inherent tensions between helpfulness and safety. Evaluated on 45 black-box commercial models across multiple providers, JustAsk consistently achieves full or near-complete system prompt recovery, revealing recurring design- and architecture-level vulnerabilities. Our results expose system prompts as a critical yet largely unprotected attack surface in modern agent systems.

Skill Set Definition:

Skill Set = Skills (fixed) + Rules (evolving) + Stats (evolving)

Skill Selection (UCB):

UCB(Ci) = success_rate(Ci) + c * sqrt(ln(N) / ni)
           ───────────────   ─────────────────────
            exploitation      exploration (curiosity)

.
├── config/
│   └── exp_config.yaml                # Experiment configuration
├── docs/
│   ├── PAP.md                         # Persuasion templates & skill mappings
│   └── PAP_taxonomy.jsonl             # 40 real-world persuasion patterns
└── src/
    ├── skill_evolving.py              # Main extraction via OpenRouter
    ├── skill_testing.py               # Controlled evaluation
    ├── skill_testing_controlled.py    # Protection-level evaluation
    ├── ucb_ranking.py                 # UCB skill selection algorithm
    ├── knowledge.py                   # Knowledge persistence
    ├── validation.py                  # Cross-verify & self-consistency
    └── ...

conda create -n justask python=3.11
conda activate justask
pip install python-dotenv requests numpy

Create a .env file:

OPENROUTER_API_KEY=sk-or-v1-your-key-here

python src/skill_testing.py --model openai/gpt-5.2

See START.md for detailed agent instructions.

From the same team:

• System Prompt Open -- Gallery of extracted system prompts from 45+ frontier models
• ISC-Bench -- Internal Safety Collapse in Frontier LLMs
• Awesome-Embodied-AI-Safety -- Safety in Embodied AI: Risks, Attacks, and Defenses
• Awesome-Large-Model-Safety -- Safety at Scale: A Comprehensive Survey of Large Model and Agent Safety
• XTransferBench -- Super Transferable Adversarial Attacks on CLIP (ICML 2025)
• BackdoorLLM -- A Comprehensive Benchmark for Backdoor Attacks on LLMs (NeurIPS 2025)
• BackdoorAgent -- Backdoor Attacks on LLM-based Agent Workflows

BibTeX:

@inproceedings{zheng2026justask,
  title={Just Ask: Curious Code Agents Reveal System
         Prompts in Frontier LLMs},
  author={Zheng, Xiang and Wu, Yutao and Huang, Hanxun
          and Li, Yige and Ma, Xingjun and Li, Bo
          and Jiang, Yu-Gang and Wang, Cong},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2026}
}

Plain text:

Xiang Zheng, Yutao Wu, Hanxun Huang, Yige Li, Xingjun Ma, Bo Li, Yu-Gang Jiang, and Cong Wang. "Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs." In International Conference on Machine Learning (ICML), 2026.

MIT