During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/workspace/.venv/lib/python3.11/site-packages/data_designer/interface/data_designer.py", line 168, in create
builder.build(num_records=num_records, buffer_size=self._buffer_size)
File "/workspace/.venv/lib/python3.11/site-packages/data_designer/engine/dataset_builders/column_wise_builder.py", line 107, in build
self._run_batch(generators, batch_mode="batch", group_id=group_id)
File "/workspace/.venv/lib/python3.11/site-packages/data_designer/engine/dataset_builders/column_wise_builder.py", line 181, in _run_batch
raise DatasetGenerationError(f"🛑 Failed to process {column_error_str}:\n{e}")
data_designer.engine.dataset_builders.errors.DatasetGenerationError: 🛑 Failed to process column 'sql':
|-- Data generation was terminated early due to error rate exceeding threshold.
|-- The summary of encountered errors is:
{
"failure_threshold": 1.0,
"completed_count": 256,
"success_count": 230,
"early_shutdown": true,
"error_count": 26,
"task_errors": {
"ModelContextWindowExceededError": 19,
"ModelGenerationValidationFailureError": 7
}
}
Math should check out. Should not terminate the workflow with threshold set to 1.0
Priority Level
Critical (Total blocker)
Describe the bug
The entire workflow occasionally gets killed when we hit generation failures, despite using a failure threshold of 1.0. In the example below, this happened when working with the
LLMCodeColumnConfigcolumn.Steps/Code to reproduce bug
Expected behavior
Math should check out. Should not terminate the workflow with threshold set to 1.0
Additional context
No response