MONAI_Metric_Design_Discussion

Telemetry Design

Targets

BenM Others please ratify / correct this list

We have identified the following telemetry sinks as priorities for implementation

• Tensorboard
• Jupyter notebook / lab

Additionally, we have identified the following telemetry sinks as nice to have

• D3

Tensorboard

The two most popular ways to write to tensorboard from torch are tensorboard.utils and tensorboardX. TensorboardX is currently used by Ignite examples, and has a slightly richer interface than tensorboard.utils.

Jupyter notebook / lab

There are a lot of useful examples in Eric's ptproto code that show how a rich set of wrappers could be designed to help people use MONAI with jupyter. We should consider all of the following:

• Creating SummaryWriter like wrappers for outputting to jupyter
• Providing interactive visualisation capabilities (often a pain point for people unfamiliar with jupyter)

Metrics

Ignite comes with a metric mechanism that we should consider as a baseline to extend from, if extension is required. The core of this is the Metric class. The key activity we need to go through is identifying use-cases that it doesn't readily fit, so that we can determine whether an extension is required.