langchain.smith.evaluation.string_run_evaluator.LLMStringRunMapper

class langchain.smith.evaluation.string_run_evaluator.LLMStringRunMapper[source]

Bases: StringRunMapper

Extract items to evaluate from the run object.

Create a new model by parsing and validating input data from keyword arguments.

Raises ValidationError if the input data cannot be parsed to form a valid model.

__call__(run: Run) Dict[str, str]

Maps the Run to a dictionary.

Parameters

run (Run) –

Return type

Dict[str, str]

map(run: Run) Dict[str, str][source]

Maps the Run to a dictionary.

Parameters

run (Run) –

Return type

Dict[str, str]

serialize_chat_messages(messages: List[Dict]) str[source]

Extract the input messages from the run.

Parameters

messages (List[Dict]) –

Return type

str

serialize_inputs(inputs: Dict) str[source]
Parameters

inputs (Dict) –

Return type

str

serialize_outputs(outputs: Dict) str[source]
Parameters

outputs (Dict) –

Return type

str

property output_keys: List[str]

The keys to extract from the run.