Requirements

• Users may create a custom inspector with a model ID and a model config + policy
• Custom inspectors can be added when creating a team agent
• An easier option to choose ready-made general inspectors (type: ready-made, custom, etc.)
• Inspector spec (type + ID + config + policy) is wrapped in a dict and sent to the agentification backend via payload

Interface Example

inspector = InspectorFactory.create_from_model(
    name="custom inspector",
    model_id="abcdefg",
    model_config={"group": "PII"},
    policy="adaptive"
)

inspector_auto = InspectFactory.create_from_auto(
    name="sth",
    auto="alignment"
)

team = TeamAgent(
    name="team"
    agents=agents,
    description="team description",
    llm_id="xyz",
    use_mentalist=True,
    inspectors=[inspector, inspector_auto],
    inspector_targets=["steps", "output"],
)

Implementation Notes

Module: ModelWithParams (abstract class)

• Model ID + (undefined) extra parameters

Module: Inspector(ModelWithParams)

• New fields: model config, policy, auto

Factory: InspectorFactory

• New arg: inspectors: List[Inspector]
• Removed: use_inspector, num_inspectors (these are inferred from inspectors)

Next Steps

• Custom inspector spec is sent to the custom asset backend to have a new asset ID (except policy)