changes to incorporate larger resnet

benchmark/MLPerfTiny_Rules.adoc

@@ -121,6 +121,15 @@ benchmarks.
 The implementation should not encode any information about the content of the
 input dataset in any form.
 
+=== Progressively Larger Submissions Required
+
+For benchmarks with multiple sizes (as of spring 2026, only Image Classification),
+if an organization submits one size of the benchmark, it must also submit all
+smaller sizes.  So for example, you may submit IC01 (smaller image classification)
+and IC02 (larger image classification), or only IC01, but you may not submit IC02
+without also submitting IC01.  This is to maximize the availability of direct comparisons
+in the results.
+
 === Audit Process
 
 In depth audits will not be conducted in this version (v1.2) of MLPerf Tiny

benchmark/README.md

@@ -3,7 +3,8 @@ The current version of the MLPerf Tiny benchmark suite consists four benchmarks:
 |:--------------------:|:-----------------------------------------------:|:-------------------------:|:----------------:|:-------------------:|
 |   Keyword Spotting   |        Small vocabulary keyword spotting        |      Speech Commands [[1]](#1)                |      [DS-CNN](https://github.com/mlcommons/tiny/blob/master/benchmark/training/keyword_spotting/keras_model.py)   | 90% (Top 1)
 |   Visual Wake Words  |           Binary image classification           | Visual Wake Words Dataset [[2]](#2)           |     [MobileNet](https://github.com/mlcommons/tiny/blob/master/benchmark/training/visual_wake_words/vww_model.py)    | 80% (Top 1)
-| Image Classification |            Small image classification           |          Cifar10 [[3]](#3)                    |      [ResNet](https://github.com/mlcommons/tiny/blob/master/benchmark/training/image_classification/keras_model.py)      | 85% (Top 1)
+| Image Classification |            Image classification           |          Cifar10 [[3]](#3)                    |      [ResNet](https://github.com/mlcommons/tiny/blob/master/benchmark/training/image_classification/keras_model.py)      | 85% (Top 1)
+| Image Classification 2 |            Larger image classification           |          Cifar10 [[3]](#3)                    |      [ResNet](https://github.com/mlcommons/tiny/blob/master/benchmark/training/image_classification/trained_models/pretrainedResnet_large.h5)      | 91% (Top 1)
 |   Anomaly Detection  | Detecting anomalies in machine operating sounds |          ToyADMOS [[4]](#4)[[5]](#5)[[6]](#6) | [Deep AutoEncoder](https://github.com/mlcommons/tiny/blob/master/benchmark/training/anomaly_detection/keras_model.py) | 0.85 (AUC)
 
 ## Divisions

benchmark/reference_submissions/image_classification/README.md

@@ -14,3 +14,17 @@ Compile using Mbed CLI tools
 Deploy on Nucleo board
 
     cp ./BUILD/NUCLEO_L4R5ZI/GCC_ARM/image_classification.bin /Volumes/NODE_L4R5ZI/
+
+### IC02 -- Larger ResNet
+To compile the larger image classification model, convert the tflite file to C using `xxd` and 
+replace the contents of `ic_model_quant_data.cc` with the resulting data.  Then in `ic/api/submitter_implemented.h` 
+change the line
+
+```
+#define TH_MODEL_VERSION EE_MODEL_VERSION_IC01
+```
+to 
+```
+#define TH_MODEL_VERSION EE_MODEL_VERSION_IC02
+```
+

benchmark/reference_submissions/image_classification/api/internally_implemented.h

@@ -31,6 +31,7 @@ The file name has been changed and some functions removed.
 #define EE_MODEL_VERSION_VWW01 "vww01"
 #define EE_MODEL_VERSION_AD01 "ad01"
 #define EE_MODEL_VERSION_IC01 "ic01"
+#define EE_MODEL_VERSION_IC02 "ic02"
 
 typedef enum { EE_ARG_CLAIMED, EE_ARG_UNCLAIMED } arg_claimed_t;
 typedef enum { EE_STATUS_OK = 0, EE_STATUS_ERROR } ee_status_t;

benchmark/runner/tests_accuracy.yaml

@@ -15,6 +15,14 @@ ic01:
     - loop: 
       - download
       - infer 1 0
+ic02:
+  name: image_classification
+  model: ic01
+  truth_file: y_labels.csv
+  script:
+    - loop: 
+      - download
+      - infer 1 0
 kws01:
   name: keyword_spotting
   model: kws01

benchmark/runner/tests_energy.yaml

@@ -14,6 +14,14 @@ ic01:
     - loop 5: 
       - download
       - infer 20 0
+ic02:
+  name: image_classification
+  model: ic01
+  truth_file: y_labels.csv
+  script:
+    - loop 5: 
+      - download
+      - infer 20 0
 kws01:
   name: keyword_spotting
   model: kws01

benchmark/runner/tests_performance.yaml

@@ -14,6 +14,14 @@ ic01:
     - loop 5:
       - download
       - infer 20 5
+ic02:
+  name: image_classification
+  model: ic01
+  truth_file: y_labels.csv
+  script:
+    - loop:
+      - download
+      - infer 20 5
 kws01:
   name: keyword_spotting
   model: kws01