langchain.evaluation.schema.AgentTrajectoryEvaluator

class langchain.evaluation.schema.AgentTrajectoryEvaluator[source]

Interface for evaluating agent trajectories.

Attributes

requires_input

Whether this evaluator requires an input string.

requires_reference

Whether this evaluator requires a reference label.

Methods

__init__()

aevaluate_agent_trajectory(*, prediction, ...)

Asynchronously evaluate a trajectory.

evaluate_agent_trajectory(*, prediction, ...)

Evaluate a trajectory.

__init__()
async aevaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) dict[source]

Asynchronously evaluate a trajectory.

Parameters
  • prediction (str) – The final predicted response.

  • agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.

  • input (str) – The input to the agent.

  • reference (Optional[str]) – The reference answer.

  • kwargs (Any) –

Returns

The evaluation result.

Return type

dict

evaluate_agent_trajectory(*, prediction: str, agent_trajectory: Sequence[Tuple[AgentAction, str]], input: str, reference: Optional[str] = None, **kwargs: Any) dict[source]

Evaluate a trajectory.

Parameters
  • prediction (str) – The final predicted response.

  • agent_trajectory (List[Tuple[AgentAction, str]]) – The intermediate steps forming the agent trajectory.

  • input (str) – The input to the agent.

  • reference (Optional[str]) – The reference answer.

  • kwargs (Any) –

Returns

The evaluation result.

Return type

dict

Examples using AgentTrajectoryEvaluator