CometLogger

class torchtune.training.metric_logging.CometLogger(api_key: Optional[str] = None, workspace: Optional[str] = None, project: Optional[str] = None, experiment_key: Optional[str] = None, mode: Optional[str] = None, online: Optional[bool] = None, experiment_name: Optional[str] = None, tags: Optional[List[str]] = None, log_code: bool = True, **kwargs: Dict[str, Any])

Logger for use w/ Comet (https://www.comet.com/site/). Comet is an experiment tracking tool that helps ML teams track, debug, compare, and reproduce their model training runs.

For more information about arguments expected by Comet, see https://www.comet.com/docs/v2/guides/experiment-management/configure-sdk/#for-the-experiment.

Parameters:

• api_key (Optional[str]) – Comet API key. It’s recommended to configure the API Key with comet login.

• workspace (Optional[str]) – Comet workspace name. If not provided, uses the default workspace.

• project (Optional[str]) – Comet project name. Defaults to Uncategorized.

• experiment_key (Optional[str]) – The key for comet experiment to be used for logging. This is used either to append data to an Existing Experiment or to control the ID of new experiments (for example to match another ID). Must be an alphanumeric string whose length is between 32 and 50 characters.

• mode (Optional[str]) –

  Control how the Comet experiment is started.

  • "get_or_create": Starts a fresh experiment if required, or persists logging to an existing one.

  • "get": Continue logging to an existing experiment identified by the experiment_key value.

  • "create": Always creates of a new experiment, useful for HPO sweeps.

• online (Optional[bool]) – If True, the data will be logged to Comet server, otherwise it will be stored locally in an offline experiment. Default is True.

• experiment_name (Optional[str]) – Name of the experiment. If not provided, Comet will auto-generate a name.

• tags (Optional[List[str]]) – Tags to associate with the experiment.

• log_code (bool) – Whether to log the source code. Defaults to True.

• **kwargs (Dict[str, Any]) – additional arguments to pass to comet_ml.start. See https://www.comet.com/docs/v2/api-and-sdk/python-sdk/reference/Experiment-Creation/#comet_ml.ExperimentConfig

Example

>>> from torchtune.training.metric_logging import CometLogger
>>> logger = CometLogger(project_name="my_project", workspace="my_workspace")
>>> logger.log("my_metric", 1.0, 1)
>>> logger.log_dict({"my_metric": 1.0}, 1)
>>> logger.close()

Raises:

ImportError – If comet_ml package is not installed.

Note

This logger requires the comet_ml package to be installed. You can install it with pip install comet_ml. You need to set up your Comet.ml API key before using this logger. You can do this by calling comet login in your terminal. You can also set it as the COMET_API_KEY environment variable.

close() → None

Close log resource, flushing if necessary. Logs should not be written after close is called.

log(name: str, data: Union[Tensor, ndarray, int, float], step: int) → None

Log scalar data.

Parameters:

• name (str) – tag name used to group scalars

• data (Scalar) – scalar data to log

• step (int) – step value to record

log_config(config: DictConfig) → None

Logs the config

Parameters:

config (DictConfig) – config to log

log_dict(payload: Mapping[str, Union[Tensor, ndarray, int, float]], step: int) → None

Log multiple scalar values.

Parameters:

• payload (Mapping[str, Scalar]) – dictionary of tag name and scalar value

• step (int) – step value to record