Skip to main content

Add auto qa config

Prompt Configuration

Configure the prompt and AI model settings. Test your prompt using the action button below.

Available variables for prompt generation:

- requirement: Description of the QA item
- docs: Documentation related to QA (loaded from docs/docs/QA directory)
- last_config: Configuration from the last iteration
- best_config: Configuration from the best performing iteration
- baseline_config: Baseline configuration of the QA item
- last_performance: Performance metric from the last iteration
- best_performance: Best performance metric achieved so far
- baseline_performance: Baseline performance metric
- baseline_failure_data: List of failure data points from current iteration
- last_failure_data: List of failure data points from last iteration
- best_failure_data: List of failure data points from best iteration

Your prompt code should assign a string value to the 'prompt' variable that will be used for QA item generation.
The language model to use for this step.
Temperature for model output randomness. Only used for models that support it (not used for O1).

Settings

Basic configuration and iteration control settings.
The maximum number of iterations for the automation.
Threshold for continuing iterations. Below threshold: retry up to max_iterations. Above threshold: stop. If not set: run all max_iterations.
Whether to automatically deploy the QA item to production when the best iteration is found and mark the is_manual field to False.

Timestamps

Jan. 29, 2026, 10:06 p.m.
-
-