-
Notifications
You must be signed in to change notification settings - Fork 4.5k
Description
Leverage ILogger with Azure Monitor (AppInsights) to send custom telemetry for the following:
-
Ask (disabled by default) – the actual Ask (text) coming into the Kernel from the application (example: “Create an email to my boss with a summarization of this ......” this will be important in Dev and maybe UAT/test environments. But for PII in production it should be disabled
-
Planner (enabled by default) – this is the planner step (either using one of our planners or if the developer hard coded in a plan
a. Ideal information to store here: Execution time, Model used for planner, success state (Success / Failed), Plan returned, total tokens used -
Steps executed for the returned plan (enabled by default)
a. Ideal information to store here: Plugin executed, execution time, success state (Success / Failed), Tokens used, Model used, Plan this was run from -
Kernel Errors throughout the process
a. Critical, Failures, Warnings – (enabled by default)
b. Non-fatal (disabled by default)
### Tasks
- [x] Telemetry for Ask
- [x] Telemetry for Planner
- [x] Telemetry for plan execution
- [x] Telemetry for Kernel errors
- [x] Samples and demos with telemetry usage
Metadata
Metadata
Assignees
Labels
Type
Projects
Status