Add OnlineTraceScoringProcessor that orchestrates online scoring:
- Fetches traces since last checkpoint
- Applies sampling to select scorers per trace
- Runs scoring in parallel using ThreadPoolExecutor
- Logs assessments back to traces
- Updates checkpoint after processing

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Verify that metadata.mlflow.sourceRun IS NULL filter is applied
to exclude traces generated during evaluation runs.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Move the EvalItem import inside _execute_scoring() to avoid pulling
in pandas at module load time. This allows OnlineTraceScoringProcessor
to be exported from __init__.py without breaking the skinny client.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Move evaluation module imports inside _execute_scoring method to
prevent pulling in pandas at module load time, which would break
the skinny client.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Update test patches to target mlflow.genai.evaluation.harness module
directly instead of trace_processor module, since imports are now lazy.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

When traces from multiple filters have overlapping timestamps, they are
sorted alphabetically by trace_id at each timestamp. Updated test to
expect correct interleaved ordering rather than sequential blocks.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Calculate overlap pairs and checkpoint values based on MAX_TRACES_PER_JOB
constant rather than hardcoding values. The test now works correctly
regardless of the MAX_TRACES_PER_JOB value.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Replace hardcoded timestamp ranges with dynamic calculations:
- Staging filter starts at 0, covers MAX_TRACES_PER_JOB range
- Prod filter starts at 0.8*MAX_TRACES_PER_JOB, creating 20% overlap
- All expected values computed from these fractions

Test now works correctly regardless of MAX_TRACES_PER_JOB value.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Changed loop to iterate over tasks.items() instead of tasks.values()
to access trace_id for better debugging when a task has no trace.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Add exclusive flag support to the job decorator to prevent duplicate
job execution for the same parameters. Add run_online_trace_scorer_job
that uses OnlineTraceScoringProcessor to score individual traces.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

…ng job

Signed-off-by: dbczumar <corey.zumar@databricks.com>

…ent module

Signed-off-by: dbczumar <corey.zumar@databricks.com>

- Add unit test for _compute_exclusive_lock_key in test_utils.py
- Add integration tests for exclusive job behavior in test_jobs.py

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Verify the job correctly instantiates OnlineTraceScoringProcessor
and calls process_traces.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

The custom JSON serializer was failing when Huey tried to serialize lock
values (plain strings) because it expected all data to be Message objects
with _asdict(). Updated serializer to handle both Message objects and
plain data, and improved deserializer to reconstruct Messages only when
appropriate.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

…lain data"

This reverts commit 11f8272.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

The JsonSerializer needs to handle two types of data:
1. Message objects (task data) with ._asdict() method
2. Plain data like lock values (e.g., '1' for exclusive locks)

Without this fix, exclusive jobs fail when Huey tries to serialize
lock values during lock acquisition.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Add integration test that verifies run_online_trace_scorer_job uses
exclusive=["experiment_id"] correctly. Test submits two jobs with same
experiment_id but different scorers and verifies only one runs while the
other is canceled due to the exclusive lock.

Also simplify test helper by inlining OnlineScorer dict creation using
asdict() instead of maintaining a helper function.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Move duplicated test utilities (_setup_job_runner, wait_job_finalize, etc.)
from test_jobs.py and test_online_scoring_jobs.py into a shared helpers.py
module for better code reuse across the test suite.

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>

Signed-off-by: dbczumar <corey.zumar@databricks.com>