Skip to content

Geval refusal

GEval Refusal Metric.

This metric is used to predict if the question and expected response is a refusal response using GEval.

GEvalRefusalMetric(name=None, evaluation_params=None, model=DefaultValues.MODEL, criteria=None, evaluation_steps=None, rubric=None, model_credentials=None, model_config=None, threshold=0.5, additional_context=None, batch_status_check_interval=DefaultValues.BATCH_STATUS_CHECK_INTERVAL, batch_max_iterations=DefaultValues.BATCH_MAX_ITERATIONS, num_judges=DefaultValues.NUM_JUDGES, aggregation_method=DefaultValues.AGGREGATION_METHOD, max_concurrent_judges=None, strict_mode=False)

Bases: DeepEvalGEvalMetric

GEval Refusal Metric.

This metric is used to predict if the question and expected response is a refusal response.

Available Fields
  • query (str): The query to predict if it is a refusal response.
  • expected_response (str): The expected response to predict if it is a refusal response.
Scoring
  • [0, 1] (Continuous): Normalized score range. Stored native 0-1 rubric value in rubric_score field. Higher score is better (higher_is_better=True).
Cookbook Example

Please refer to example_geval_refusal.py in the gen-ai-sdk-cookbook repository.