ChartMaster is a new framework for chart-to-code generation, enabling Multimodal Large Language Models (MLLMs) to convert chart images into executable code. This task is essential for automated data analysis, report generation, and intelligent question answering, but faces two key challenges: dataset diversity and visual consistency.

Previous datasets mainly use synthetic seeds to prompt GPT models, resulting in homogeneous samples and limited generalization. To address this, we propose ReChartPrompt, a large-scale automatically constructed dataset based on 30,071 real arXiv papers, yielding 240K diverse chart-image/code/instruction triplets. These real-world charts cover rich design styles and research fields, greatly enhancing model robustness.

The second challenge is that traditional supervised fine-tuning (SFT) improves code understanding but cannot guarantee visual fidelity. We introduce ChartSimRL, a reinforcement learning algorithm based on Group Relative Policy Optimization (GRPO), guided by a novel chart similarity reward. This reward combines attribute similarity (matching layout, color, text, and values) and visual similarity (comparing CNN-extracted features), ensuring generated charts align closely with originals in both semantics and appearance.

By integrating ReChartPrompt and ChartSimRL, our ChartMaster model achieves state-of-the-art results among open-source 7B-parameter models, rivaling GPT-4o performance across multiple benchmarks. All code, datasets, and models are released to facilitate further research.