This project is a small, production-minded prototype of a Healthcare Claims Validation Agent implemented in Python.
It validates a medical claim JSON, runs deterministic checks, and uses an LLM to propose structured fixes and a human-readable summary.

• Deterministic validation tools:
  • Schema validation using Pydantic.
  • Required fields checks.
  • Date consistency rules.
  • Amount consistency rules.
  • Basic code format rules (ICD-10, CPT, NPI).
• LLM-assisted suggestions:
  • Uses OpenAI Chat Completions with function calling (tools).
  • LLM generates structured SuggestedFix objects, summary text, and a risk score.
• Strict schemas & guardrails:
  • All inputs/outputs modeled with Pydantic.
  • Final ValidationReport enforced by schema.
  • Retries and fallback behavior if LLM output doesn’t validate.
• Artifacts & logging:
  • Each run has a unique run_id.
  • Input claim, tool outputs, and final report saved under artifacts/<run_id>/.
• Simple UI + CLI:
  • Streamlit app for interactive validation.
  • Optional CLI runner for headless use.
• Basic evaluation script:
  • Runs over sample claims and reports:
    • Percent valid.
    • Average issues per claim.
    • Severity distribution.
    • Percent of reports that required retry.

  app.py                      # Streamlit UI
  cli.py                      # CLI runner
  requirements.txt
  README.md
  .env.example
  src/
    __init__.py
    config.py                 # env loading
    schemas.py                # Pydantic models
    tools.py                  # deterministic checks
    llm_client.py             # OpenAI client wrapper + retries
    agent.py                  # orchestration
    logging_utils.py          # structured logging setup
    artifact_store.py         # save inputs/outputs
  data/
    sample_claims/
      claim_valid_01.json
      ...
  tests/
    test_tools.py
    test_schemas.py
    test_agent_smoke.py
  scripts/
    run_eval.py
  artifacts/
    ... created at runtime ...

• Python: 3.11+ recommended. From the project root:

python -m venv .venv
source .venv/bin/activate  # On Windows: .venv\Scripts\activate
pip install -r requirements.txt

Create a .env file:

cp .env.example .env

Edit .env and set:

• OPENAI_API_KEY
• Optionally OPENAI_MODEL (default: gpt-4.1-mini).

From the project root:

streamlit run app.py

The UI lets you:

• Paste or edit claim JSON.
• Click Validate to run the agent.
• View issues and suggested fixes.
• Inspect the full ValidationReport JSON.
• Download the report.
• See the run_id and artifact folder location.

python cli.py --input data/sample_claims/claim_valid_01.json --output report.json

• --input (required): path to a claim JSON file.
• --output (optional): save final report JSON to this path.

CLI prints:

• Run status.
• Run ID.
• Artifact directory.

python scripts/run_eval.py

This will:

• Load all JSON files under data/sample_claims/.
• Run the agent for each.
• Print aggregate stats to stdout.
• Save evaluation results under artifacts/eval_<timestamp>.json.

pytest

Tests include:

• Schema tests: basic instantiation & validation.
• Tool tests: deterministic validation rules.
• Agent smoke test: runs the agent with a fake LLM and checks that the ValidationReport schema validates.

{
  "claim_id": "CLM12345",
  "patient": {
    "patient_id": "PAT001",
    "dob": "1980-01-15",
    "gender": "F"
  },
  "provider": {
    "npi": "1234567890",
    "name": "Dr. Alice Smith"
  },
  "service": {
    "date_of_service": "2024-05-10",
    "place_of_service": "11"
  },
  "codes": {
    "icd10": ["E11.9"],
    "cpt": ["99213"]
  },
  "amounts": {
    "billed_amount": 150.0,
    "allowed_amount": 120.0,
    "paid_amount": 100.0,
    "patient_responsibility": 20.0
  },
  "payer": {
    "name": "Acme Health",
    "member_id": "M123456"
  },
  "notes": "Follow-up visit"
}

• The LLM is not allowed to invent new issues; it only proposes fixes to issues discovered by deterministic tools.
• If LLM output fails validation after retries, the system falls back to a deterministic-only report, with an explanatory summary, still conforming to the ValidationReport schema.