This repository contains materials related to our study on visualization analogies, including charts, datasets, analysis scripts, participant sketches, and survey questions.

Associated with the paper: From Reality to Recognition: Evaluating Visualization Analogies for Novice Chart Comprehension.

This folder contains the analogy and baseline charts we presented in the paper and used in the Qualtrics survey.

• Analogy Chart:
  

• Baseline Chart:
  

For more analogy charts, visit our website: Visualization Analogies

• Contains detailed JSON files for chart creation dataset and D3.js code for the creation of baseline charts.

This folder holds de-identified datasets collected from our study.

• performanceAnalysis.csv → Contains visualization novices' scores.
• 128.csv → Includes VARK questionnaires, self-rated preferences, NASA-TLX data, and more from 128 participants.
• data.ipynb → Used to analyze participant ratings.
• performance.ipynb → Used to analyze participant performance.

Create \annotation folder under dataAnalysis/ To view the images of sketches, run:

python extract_image.py
python overlay_images.py

After running the scripts, \annotation contains participant sketches. 📌 Note: You may need to change sketch_dir= in extract_image.py (line 48) to the local finalSketch path. You can download the finalSketch folder and unzip it under dataCollected/participantsSketches/finalSketch.zip/

This folder holds example participant sketches that are already overlayed from the previous step and de-identified think-aloud audio transcriptions.

Due to file size limitations, download and unzip selectedSketches.zip to access the selected participant's dataset.

• Contains de-identified transcriptions of think-aloud audio data.
• Each CSV file corresponds to a chart used in the study.

This folder contains data related to expert reviews conducted as part of our study. The expert evaluations focus on assessing candidate visualizations based on predefined design criteria.

• Contains the raw data of expert reviews.
• Includes expert comments on each of the four design criteria for all 10 candidate charts.

• Summarizes expert feedback on which charts require modifications or refinements.
• Includes reasons provided by experts for suggested changes.

• Contains the rubric used by experts to rate visualizations.
• Ratings are based on a scale from 1 to 5.

Includes the question solutions, grading rubric and study questions from Qualtrics, available in both:

This folder contains JavaScript scripts for integrating a custom sketching interface into Qualtrics' Signature Question type. The interface allows participants to draw directly on charts as part of the study.

Watch a demonstration of the sketching interface in action:

1. Upload the necessary JavaScript files into Qualtrics' "Look & Feel" > "Custom JavaScript" section.
2. Set the Signature Question to accept sketch input.
3. Link the appropriate script to the specific chart question.
4. Ensure Qualtrics settings allow participants to submit drawings along with responses.

• Solution for the study questions
• Rubric for the study questions (each out of 5)
• The folder contains .md files to each of the chart type. Each chart type is a json file in the format of Question Category, Question body, Detailed Rubric. For example, below is one of the question for Waterfall Chart:

{
      "Category": "Summary",
      "Question": [
        "What do you think this chart is about? Can you describe what each section represents?",
        "What overall pattern or strategy do you notice in the character's journey through this chart?"
      ],
      "Answer": [
        "The chart is a depiction of a video game level. Each section or icon shows different events: Starting Point, Mushroom Boost, Goomba Attack, Coin Bonus, and Piranha Plant obstacle.",
        "The pattern shows that the character gains points from bonuses and loses points from obstacles. The strategy involves balancing the positive and negative impacts to reach the highest score."
      ],
      "Detailed Rubric": {
        "0": "No response or completely off-topic. The student fails to mention any of the key events or the overall strategy.",
        "1": "Partial identification of the chart's theme. The student may mention one event or provide a very vague description of the chart with no clear strategy.",
        "2": "Basic understanding with at least two correct event identifications. The response shows minimal detail and does not adequately explain the overall pattern or strategy.",
        "3": "Correctly identifies the main theme and at least three events. The student provides a basic explanation of the overall pattern but lacks depth or insight into the strategy.",
        "4": "Provides a comprehensive description of all main events and general purpose. The student explains most events clearly and outlines the balance between positive and negative impacts, though some details might be missing.",
        "5": "Thorough explanation including all events (Starting Point, Mushroom Boost, Goomba Attack, Coin Bonus, and Piranha Plant obstacle) along with insightful observations about the strategy and pattern. The response is detailed and demonstrates full understanding of the chart."
      }
}

• QSF format
• For importing into Qualtrics
• Qualtrics Survey Exported Questions and Charts