cli.evaluate

cli.evaluate

CLI to run evaluation on a model.

Functions

Name	Description
do_cli	Parses axolotl config, CLI args, and calls do_evaluate.
do_evaluate	Evaluates a transformers model by first loading the dataset(s) specified in the

do_cli

cli.evaluate.do_cli(config=Path('examples/'), **kwargs)

Parses axolotl config, CLI args, and calls do_evaluate.

Parameters

Name	Type	Description	Default
config	Union[Path, str]	Path to axolotl config YAML file.	Path('examples/')
kwargs		Additional keyword arguments to override config file values.	{}

do_evaluate

cli.evaluate.do_evaluate(cfg, cli_args)

Evaluates a transformers model by first loading the dataset(s) specified in the axolotl config, and then calling axolotl.evaluate.evaluate, which computes evaluation metrics on the given dataset(s) and writes them to disk.

Parameters

Name	Type	Description	Default
cfg	DictDefault	Dictionary mapping axolotl config keys to values.	required
cli_args	TrainerCliArgs	CLI arguments.	required