ReFinE is a Figma plugin and FastAPI backend for turning UI screenshots into structured design context, retrieving research-backed implications, and generating actionable design suggestions.

This codebase is unified on Google's SDK:

• Generative model: gemini-2.0-flash
• Text embedding model: text-embedding-004

Set your API key before running the backend:

export GEMINI_API_KEY="your-api-key"

backend/   FastAPI server and research-recommendation logic
frontend/  Figma plugin UI built with Svelte

The paper-submission version does not bundle research assets such as .npy files or proprietary paper corpora. Instead, place the required data locally and point the backend at it with environment variables.

The recommendation pipeline expects these local assets:

1. paper_index.json Maps each paper_id to six precomputed entity embeddings aligned with: who, domain, modality, pain_point, client, metric.

   Example:

   {
     "paper_001": [[0.1, 0.2], [0.3, 0.4], [0.5, 0.6], [0.7, 0.8], [0.2, 0.1], [0.9, 1.0]]
   }

2. implications/<paper_id>.json One JSON file per paper. Each file should contain a list of implication records with:

   • design_implication
   • original_paragraph
   • design_implication_embedding

   Example:

   [
     {
       "design_implication": "Provide a lightweight progress indicator for multi-step tasks.",
       "original_paragraph": "Participants reported feeling uncertain during longer workflows...",
       "design_implication_embedding": [0.11, 0.42, 0.73]
     }
   ]

3. entities/<paper_id>.json Optional but recommended metadata used to contextualize summaries. Expected keys:

   • who
   • domain
   • modality
   • pain_point
   • client
   • metric

By default the backend looks for these files under backend/data/. You can override the paths with:

export REFINE_DATA_ROOT="/absolute/path/to/data"
export REFINE_PAPER_INDEX_PATH="/absolute/path/to/data/paper_index.json"
export REFINE_IMPLICATIONS_DIR="/absolute/path/to/data/implications"
export REFINE_ENTITIES_DIR="/absolute/path/to/data/entities"

The paper describes an offline preprocessing pipeline that:

1. converts raw paper PDFs into TEI XML with GROBID
2. extracts six design-context dimensions from the XML
3. extracts design implications from the XML
4. embeds the context dimensions and implications for retrieval

A matching scaffold is included in backend/preprocessing.py. It expects a running GROBID service:

export GROBID_URL="http://<grobid-host>:<grobid-port>"

The preprocessing module is offline-oriented and is not required for serving the plugin at runtime once the paper index and per-paper JSON artifacts have already been prepared.

Install the backend dependencies from backend/requirements.txt, then start the API from backend/.

pip install -r requirements.txt
uvicorn main:app --reload

From frontend/:

npm install
npm run dev

Use npm run build to generate the production plugin bundle.