From 17d6eff105fae6492aaee5dd22428a741bb9c9ea Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 21:13:40 +0000 Subject: [PATCH 1/4] Add DICOM, image de-identification and imaging data management value sets Value sets derived from "Toward AI-Ready Medical Imaging Data" (Nikolov et al., Bridge2AI consortium), covering the DICOM data lifecycle from extraction through validation and de-identification. medical/dicom.yaml - DICOMModalityEnum: PS3.16 CID 29 acquisition modality codes, mapped to NCIT where a term exists and to DCM codes throughout - DICOMValueRepresentationEnum: the 34 VRs of PS3.5 Section 6.2 - DICOMAttributeTypeEnum: attribute types 1, 1C, 2, 2C, 3 - DICOMTransferSyntaxEnum: current PS3.6 transfer syntaxes with UIDs - DICOMNetworkServiceEnum: DIMSE-C/N services and DICOMweb services - DICOMSoftwareToolEnum: toolkits, validators, servers and anonymizers medical/imaging_deidentification.yaml - DICOMDeidentificationMethodEnum: PS3.15 Annex E profile and options (CID 7050) - DICOMDeidentificationActionEnum: PS3.15 Table E.1-1a action codes - ImageDeidentificationTargetEnum, DeidentificationRegulatoryFrameworkEnum - DefacingMethodEnum and DefacingToolEnum from the paper's Table I medical/imaging_data_management.yaml - ImagingDataLifecycleStageEnum: the paper's six-stage framework - ImagingDataSourceTypeEnum, DICOMValidationIssueTypeEnum - ImageQualityMetricEnum, ImagingInteroperabilityStandardEnum - Bridge2AIDataGenerationProjectEnum All ontology CURIEs verified against OLS; DICOM codes verified against the current DICOM standard tables. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JqKeWA6q8xdJhgpuzHE4Ym --- src/valuesets/schema/medical/dicom.yaml | 1054 +++++++++++++++++ .../medical/imaging_data_management.yaml | 371 ++++++ .../medical/imaging_deidentification.yaml | 490 ++++++++ src/valuesets/schema/valuesets.yaml | 3 + 4 files changed, 1918 insertions(+) create mode 100644 src/valuesets/schema/medical/dicom.yaml create mode 100644 src/valuesets/schema/medical/imaging_data_management.yaml create mode 100644 src/valuesets/schema/medical/imaging_deidentification.yaml diff --git a/src/valuesets/schema/medical/dicom.yaml b/src/valuesets/schema/medical/dicom.yaml new file mode 100644 index 00000000..e00a3a99 --- /dev/null +++ b/src/valuesets/schema/medical/dicom.yaml @@ -0,0 +1,1054 @@ +name: dicom +title: DICOM Standard Value Sets +description: >- + Value sets drawn from the Digital Imaging and Communications in Medicine (DICOM) + standard (ISO 12052), the internationally recognized format for storing, transmitting + and managing medical imaging data. Covers the acquisition modality codes used in the + Modality (0008,0060) attribute, the Value Representations (VRs) that govern attribute + encoding, the attribute requirement types (1, 1C, 2, 2C, 3) used in Information Object + Definitions, the registered transfer syntaxes that define byte ordering and pixel data + compression, the DIMSE and DICOMweb network services used to query and retrieve images + from PACS, and the open-source toolkits commonly used to read, validate and serve DICOM + files. + + These value sets support the medical imaging data lifecycle described in "Toward AI-Ready + Medical Imaging Data" (Nikolov et al., Bridge2AI consortium), in particular the data + extraction, conformance checking and metadata tag validation stages, where VR, VM and + attribute type rules are checked with tools such as dciodvfy, dcentvfy, DVTk and pydicom. + + Codes that come from the DICOM standard itself (modality codes, VRs, action codes) are + kept in their standard form rather than being upper-cased or expanded. +id: https://w3id.org/valuesets/medical/dicom +imports: +- linkml:types +prefixes: + linkml: https://w3id.org/linkml/ + valuesets: https://w3id.org/valuesets/ + NCIT: http://purl.obolibrary.org/obo/NCIT_ + DCM: http://dicom.nema.org/resources/ontology/DCM/ + orcid: https://orcid.org/ + valuesets_meta: https://w3id.org/valuesets/meta/ +default_prefix: valuesets +slots: + dicom_modality: + description: The DICOM acquisition modality code of an imaging instance or series + range: DICOMModalityEnum + dicom_value_representation: + description: The DICOM Value Representation (VR) of an attribute + range: DICOMValueRepresentationEnum + dicom_attribute_type: + description: The DICOM attribute requirement type of an attribute within an IOD module + range: DICOMAttributeTypeEnum + dicom_transfer_syntax: + description: The DICOM transfer syntax used to encode a data set or its pixel data + range: DICOMTransferSyntaxEnum + dicom_network_service: + description: The DICOM network service used to query, retrieve or store instances + range: DICOMNetworkServiceEnum + dicom_software_tool: + description: A software toolkit, validator, server or library used to process DICOM data + range: DICOMSoftwareToolEnum +enums: + DICOMModalityEnum: + title: DICOM Acquisition Modality + description: >- + Acquisition modality codes from DICOM PS3.16 Context ID 29 (Acquisition Modality), + the defined terms for the Modality (0008,0060) attribute. The permissible value is the + DICOM code itself; the title is the NCI Thesaurus label where a mapping exists and the + DICOM code meaning is carried as an alias when it differs. Waveform modalities (ECG, + EEG, etc.) are defined in CID 34 and are not included here. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_29.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + AR: + title: Autorefraction + description: Automated measurement of refractive error of the eye + meaning: NCIT:C176330 + exact_mappings: + - DCM:AR + BI: + title: Biomagnetic Imaging + description: Imaging based on magnetic fields produced by the body, such as magnetoencephalography + meaning: DCM:BI + BMD: + title: Bone Mineral Densitometry + description: Measurement of bone mineral content and density, such as DXA + meaning: NCIT:C190514 + exact_mappings: + - DCM:BMD + CR: + title: Computed Radiography + description: X-ray imaging using a phosphor imaging plate read out to a digital image + meaning: NCIT:C190521 + exact_mappings: + - DCM:CR + CT: + title: Computed Tomography + description: Cross-sectional X-ray imaging reconstructed by computer + meaning: NCIT:C17204 + exact_mappings: + - DCM:CT + CFM: + title: Confocal Microscopy + description: Laser-scanning microscopy that rejects out-of-focus light + meaning: NCIT:C17753 + exact_mappings: + - DCM:CFM + DMS: + title: Dermoscopy + description: Non-invasive microscopic examination of the skin surface + meaning: NCIT:C116478 + exact_mappings: + - DCM:DMS + DG: + title: Diaphanography + description: Transillumination imaging of tissue, historically used for the breast + meaning: DCM:DG + DX: + title: Digital Radiography + description: Projection X-ray imaging acquired directly with a digital detector + meaning: NCIT:C18001 + exact_mappings: + - DCM:DX + ES: + title: Endoscopic Procedure + description: Imaging from an endoscope inserted into a body cavity or organ, including laryngoscopy and video endoscopy + meaning: NCIT:C16546 + exact_mappings: + - DCM:ES + aliases: + - Endoscopy + XC: + title: External-camera Photography + description: Visible-light photography of the patient with an external camera + meaning: DCM:XC + GM: + title: Microscopy + description: General microscopy not otherwise classified + meaning: NCIT:C16853 + exact_mappings: + - DCM:GM + aliases: + - General Microscopy + IO: + title: Intraoral Radiography + description: Dental X-ray imaging with the detector inside the mouth + meaning: NCIT:C190548 + exact_mappings: + - DCM:IO + aliases: + - Intra-oral Radiography + IVOCT: + title: Intravascular Optical Coherence Tomography + description: Catheter-based optical coherence tomography of blood vessels + meaning: NCIT:C190550 + exact_mappings: + - DCM:IVOCT + IVUS: + title: Intravascular Ultrasound + description: Catheter-based ultrasound imaging of blood vessels + meaning: NCIT:C99535 + exact_mappings: + - DCM:IVUS + KER: + title: Keratometry + description: Measurement of the curvature of the anterior corneal surface + meaning: NCIT:C190551 + exact_mappings: + - DCM:KER + LS: + title: Laser surface scan + description: Surface geometry acquired with a laser scanner + meaning: DCM:LS + LEN: + title: Lensometry + description: Measurement of the optical properties of spectacle lenses + meaning: DCM:LEN + MR: + title: Magnetic Resonance Imaging + description: Imaging using radiofrequency pulses in a strong magnetic field + meaning: NCIT:C16809 + exact_mappings: + - DCM:MR + aliases: + - Magnetic Resonance + MG: + title: Mammography + description: Low-dose X-ray imaging of the breast + meaning: NCIT:C16818 + exact_mappings: + - DCM:MG + NM: + title: Radionuclide Imaging + description: Gamma camera imaging of an administered radiotracer, including planar and SPECT + meaning: NCIT:C62667 + exact_mappings: + - DCM:NM + aliases: + - Nuclear Medicine + OAM: + title: Ophthalmic Axial Measurements + description: Measurement of axial dimensions of the eye, such as axial length + meaning: DCM:OAM + OPM: + title: Ophthalmic Mapping + description: Topographic or thickness maps of ocular structures + meaning: DCM:OPM + OP: + title: Ophthalmic Photography + description: Photography of the eye, including fundus and slit lamp photography + meaning: NCIT:C190559 + exact_mappings: + - DCM:OP + OPT: + title: Ophthalmic Tomography + description: Optical coherence tomography of the eye, including retinal OCT B-scans + meaning: NCIT:C190561 + exact_mappings: + - DCM:OPT + OPTBSV: + title: Ophthalmic Tomography B-scan Volume Analysis + description: Volume analysis derived from ophthalmic OCT B-scans + meaning: DCM:OPTBSV + OPTENF: + title: En-face Optical Coherence Tomography + description: Transverse (en face) images derived from ophthalmic OCT volumes + meaning: NCIT:C190563 + exact_mappings: + - DCM:OPTENF + aliases: + - Ophthalmic Tomography En Face + OPV: + title: Ophthalmic Visual Field + description: Perimetry results describing the visual field + meaning: DCM:OPV + OCT: + title: Optical Coherence Tomography + description: Interferometric imaging using near-infrared light, used outside ophthalmology + meaning: NCIT:C20828 + exact_mappings: + - DCM:OCT + OSS: + title: Optical Surface Scanner + description: Surface geometry acquired with an optical (non-laser) scanner + meaning: DCM:OSS + PX: + title: Panoramic X-Ray + description: Dental panoramic radiography of the jaws + meaning: DCM:PX + PA: + title: Photoacoustic Imaging + description: Imaging of ultrasonic waves generated by optical absorption of pulsed light + meaning: NCIT:C116749 + exact_mappings: + - DCM:PA + aliases: + - Photoacoustic + PT: + title: Positron Emission Tomography + description: Tomographic imaging of a positron-emitting radiotracer + meaning: NCIT:C17007 + exact_mappings: + - DCM:PT + RF: + title: Fluoroscopy + description: Real-time X-ray imaging, including radiofluoroscopy + meaning: NCIT:C16588 + exact_mappings: + - DCM:RF + aliases: + - Radiofluoroscopy + RG: + title: X-Ray Imaging + description: Conventional film or screen radiographic imaging + meaning: NCIT:C38101 + exact_mappings: + - DCM:RG + aliases: + - Radiographic imaging + RTIMAGE: + title: RT Image + description: Radiotherapy portal or setup image + meaning: DCM:RTIMAGE + SM: + title: Slide Microscopy + description: Whole slide imaging of microscope slides + meaning: DCM:SM + SRF: + title: Subjective Refraction + description: Refraction measured with patient feedback + meaning: DCM:SRF + TG: + title: Thermography + description: Imaging of body surface temperature + meaning: NCIT:C17194 + exact_mappings: + - DCM:TG + US: + title: Ultrasound Imaging + description: Imaging using high-frequency sound waves, including static images and cine loops + meaning: NCIT:C17230 + exact_mappings: + - DCM:US + aliases: + - Ultrasound + BDUS: + title: Ultrasound Bone Densitometry + description: Quantitative ultrasound estimation of bone mineral density + meaning: NCIT:C190516 + exact_mappings: + - DCM:BDUS + VA: + title: Visual Acuity + description: Measurement of the sharpness of vision + meaning: NCIT:C87149 + exact_mappings: + - DCM:VA + XA: + title: X-ray Angiography + description: X-ray imaging of blood vessels with contrast, including digital subtraction angiography + meaning: NCIT:C20080 + exact_mappings: + - DCM:XA + + DICOMValueRepresentationEnum: + title: DICOM Value Representation + description: >- + The Value Representations (VRs) defined in DICOM PS3.5 Section 6.2, which specify the + data type and format of the value of a data element. Tag validation tools check that + each attribute is encoded with the VR required by the data dictionary. Permissible + values are the two-letter DICOM VR codes. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part05/sect_6.2.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + AE: + title: Application Entity + description: A string identifying an Application Entity; 16 bytes maximum + annotations: + category: string + AS: + title: Age String + description: Age in the format nnnD, nnnW, nnnM or nnnY (days, weeks, months, years) + annotations: + category: string + AT: + title: Attribute Tag + description: An ordered pair of 16-bit unsigned integers that is the value of a data element tag + annotations: + category: binary + CS: + title: Code String + description: A string identifying a controlled concept; uppercase letters, digits, space and underscore, 16 bytes maximum + annotations: + category: string + DA: + title: Date + description: A date in the format YYYYMMDD + annotations: + category: date_time + DS: + title: Decimal String + description: A string representing a fixed point or floating point number + annotations: + category: string + DT: + title: Date Time + description: A concatenated date-time string of the form YYYYMMDDHHMMSS.FFFFFF&ZZXX + annotations: + category: date_time + FL: + title: Floating Point Single + description: Single precision IEEE 754 binary32 floating point value + annotations: + category: binary + FD: + title: Floating Point Double + description: Double precision IEEE 754 binary64 floating point value + annotations: + category: binary + IS: + title: Integer String + description: A string representing a base-10 integer + annotations: + category: string + LO: + title: Long String + description: A character string of up to 64 characters + annotations: + category: string + LT: + title: Long Text + description: A character string that may contain one or more paragraphs, up to 10240 characters + annotations: + category: text + OB: + title: Other Byte + description: An octet stream whose encoding is specified by the negotiated transfer syntax + annotations: + category: binary + OD: + title: Other Double + description: A stream of IEEE 754 binary64 values + annotations: + category: binary + OF: + title: Other Float + description: A stream of IEEE 754 binary32 values + annotations: + category: binary + OL: + title: Other Long + description: A stream of 32-bit words + annotations: + category: binary + OV: + title: Other 64-bit Very Long + description: A stream of 64-bit words + annotations: + category: binary + OW: + title: Other Word + description: A stream of 16-bit words; commonly used for Pixel Data + annotations: + category: binary + PN: + title: Person Name + description: A character string encoded using a five-component convention (family, given, middle, prefix, suffix) + annotations: + category: string + SH: + title: Short String + description: A character string of up to 16 characters + annotations: + category: string + SL: + title: Signed Long + description: Signed 32-bit two's complement integer + annotations: + category: binary + SQ: + title: Sequence of Items + description: A sequence of zero or more items, each of which is a nested data set + annotations: + category: sequence + SS: + title: Signed Short + description: Signed 16-bit two's complement integer + annotations: + category: binary + ST: + title: Short Text + description: A character string that may contain one or more paragraphs, up to 1024 characters + annotations: + category: text + SV: + title: Signed 64-bit Very Long + description: Signed 64-bit integer + annotations: + category: binary + TM: + title: Time + description: A time in the format HHMMSS.FFFFFF + annotations: + category: date_time + UC: + title: Unlimited Characters + description: A character string of unlimited length + annotations: + category: string + UI: + title: Unique Identifier (UID) + description: A string of numeric components separated by periods, up to 64 characters, used for UIDs such as SOP Instance UIDs and transfer syntax UIDs + annotations: + category: string + UL: + title: Unsigned Long + description: Unsigned 32-bit integer + annotations: + category: binary + UN: + title: Unknown + description: An octet stream whose encoding of the contents is unknown + annotations: + category: binary + UR: + title: Universal Resource Identifier or Universal Resource Locator (URI/URL) + description: A string identifying a URI or URL as defined in RFC 3986 + annotations: + category: string + US: + title: Unsigned Short + description: Unsigned 16-bit integer + annotations: + category: binary + UT: + title: Unlimited Text + description: A character string that may contain one or more paragraphs, of unlimited length + annotations: + category: text + UV: + title: Unsigned 64-bit Very Long + description: Unsigned 64-bit integer + annotations: + category: binary + + DICOMAttributeTypeEnum: + title: DICOM Attribute Type + description: >- + Attribute requirement types defined in DICOM PS3.5 Section 7.4, which state whether an + attribute must be present in a data set and whether it may have a zero-length value. + Conformance checkers report missing Type 1 and Type 2 attributes as errors. These types + also determine which de-identification action (D, Z or X) may be applied to an attribute. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part05/sect_7.4.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + TYPE_1: + title: Type 1 (Required) + description: The attribute shall be present with a valid non-zero-length value + aliases: + - "1" + TYPE_1C: + title: Type 1C (Conditionally Required) + description: The attribute shall be present with a valid value when a specified condition is met, and shall not be present otherwise + aliases: + - 1C + TYPE_2: + title: Type 2 (Required, Empty if Unknown) + description: The attribute shall be present but may have a zero-length value if the value is unknown + aliases: + - "2" + TYPE_2C: + title: Type 2C (Conditionally Required, Empty if Unknown) + description: The attribute shall be present, possibly with zero length, when a specified condition is met + aliases: + - 2C + TYPE_3: + title: Type 3 (Optional) + description: The attribute is optional and may be absent or present with or without a value + aliases: + - "3" + + DICOMTransferSyntaxEnum: + title: DICOM Transfer Syntax + description: >- + Transfer syntaxes registered in DICOM PS3.6 Annex A that define the byte ordering, VR + encoding and pixel data compression of a DICOM data set. Pixel data validation includes + confirming that decompression from a lossy or lossless transfer syntax yields the + expected bit depth and that no unintended data loss occurred. Video transfer syntaxes + (MPEG-2, H.264, HEVC) are used for endoscopy and ultrasound cine acquisitions. + Fragmentable variants of the MPEG transfer syntaxes and retired JPEG processes are + omitted; the full registry is at the see_also link. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part06/chapter_A.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + IMPLICIT_VR_LITTLE_ENDIAN: + title: Implicit VR Little Endian + description: Default transfer syntax for DICOM; VRs are looked up from the data dictionary rather than encoded + annotations: + uid: 1.2.840.10008.1.2 + compression: none + EXPLICIT_VR_LITTLE_ENDIAN: + title: Explicit VR Little Endian + description: Uncompressed encoding with VRs explicitly encoded in each data element + annotations: + uid: 1.2.840.10008.1.2.1 + compression: none + ENCAPSULATED_UNCOMPRESSED_EXPLICIT_VR_LITTLE_ENDIAN: + title: Encapsulated Uncompressed Explicit VR Little Endian + description: Uncompressed pixel data encapsulated in fragments, one per frame + annotations: + uid: 1.2.840.10008.1.2.1.98 + compression: none + DEFLATED_EXPLICIT_VR_LITTLE_ENDIAN: + title: Deflated Explicit VR Little Endian + description: Explicit VR Little Endian data set compressed as a whole with the deflate algorithm + annotations: + uid: 1.2.840.10008.1.2.1.99 + compression: lossless + EXPLICIT_VR_BIG_ENDIAN: + title: Explicit VR Big Endian (Retired) + description: Big endian byte ordering with explicit VRs; retired but still encountered in legacy archives + annotations: + uid: 1.2.840.10008.1.2.2 + compression: none + retired: 'true' + JPEG_BASELINE_PROCESS_1: + title: JPEG Baseline (Process 1) + description: Default transfer syntax for lossy JPEG 8-bit image compression + annotations: + uid: 1.2.840.10008.1.2.4.50 + compression: lossy + JPEG_EXTENDED_PROCESS_2_4: + title: JPEG Extended (Process 2 & 4) + description: Default transfer syntax for lossy JPEG 12-bit image compression (Process 4 only) + annotations: + uid: 1.2.840.10008.1.2.4.51 + compression: lossy + JPEG_LOSSLESS_PROCESS_14: + title: JPEG Lossless, Non-Hierarchical (Process 14) + description: Lossless JPEG compression using any predictor + annotations: + uid: 1.2.840.10008.1.2.4.57 + compression: lossless + JPEG_LOSSLESS_PROCESS_14_SV1: + title: JPEG Lossless, Non-Hierarchical, First-Order Prediction (Process 14 [Selection Value 1]) + description: Default transfer syntax for lossless JPEG image compression + annotations: + uid: 1.2.840.10008.1.2.4.70 + compression: lossless + JPEG_LS_LOSSLESS: + title: JPEG-LS Lossless Image Compression + description: Lossless compression using the JPEG-LS (ISO 14495) algorithm + annotations: + uid: 1.2.840.10008.1.2.4.80 + compression: lossless + JPEG_LS_NEAR_LOSSLESS: + title: JPEG-LS Lossy (Near-Lossless) Image Compression + description: Near-lossless JPEG-LS compression with a bounded per-pixel error + annotations: + uid: 1.2.840.10008.1.2.4.81 + compression: lossy + JPEG_2000_LOSSLESS_ONLY: + title: JPEG 2000 Image Compression (Lossless Only) + description: JPEG 2000 wavelet compression restricted to reversible (lossless) mode + annotations: + uid: 1.2.840.10008.1.2.4.90 + compression: lossless + JPEG_2000: + title: JPEG 2000 Image Compression + description: JPEG 2000 wavelet compression, lossy or lossless + annotations: + uid: 1.2.840.10008.1.2.4.91 + compression: lossy or lossless + JPEG_2000_MULTICOMPONENT_LOSSLESS_ONLY: + title: JPEG 2000 Part 2 Multi-component Image Compression (Lossless Only) + description: JPEG 2000 Part 2 multi-component transform, reversible mode only + annotations: + uid: 1.2.840.10008.1.2.4.92 + compression: lossless + JPEG_2000_MULTICOMPONENT: + title: JPEG 2000 Part 2 Multi-component Image Compression + description: JPEG 2000 Part 2 multi-component transform, lossy or lossless + annotations: + uid: 1.2.840.10008.1.2.4.93 + compression: lossy or lossless + JPIP_REFERENCED: + title: JPIP Referenced + description: Pixel data referenced via a JPEG 2000 Interactive Protocol URL rather than encoded in the data set + annotations: + uid: 1.2.840.10008.1.2.4.94 + compression: referenced + JPIP_REFERENCED_DEFLATE: + title: JPIP Referenced Deflate + description: JPIP referenced pixel data with the remaining data set deflated + annotations: + uid: 1.2.840.10008.1.2.4.95 + compression: referenced + MPEG2_MAIN_PROFILE_MAIN_LEVEL: + title: MPEG2 Main Profile / Main Level + description: MPEG-2 video compression for standard definition video + annotations: + uid: 1.2.840.10008.1.2.4.100 + compression: lossy + media: video + MPEG2_MAIN_PROFILE_HIGH_LEVEL: + title: MPEG2 Main Profile / High Level + description: MPEG-2 video compression for high definition video + annotations: + uid: 1.2.840.10008.1.2.4.101 + compression: lossy + media: video + MPEG4_AVC_H264_HIGH_PROFILE_LEVEL_4_1: + title: MPEG-4 AVC/H.264 High Profile / Level 4.1 + description: H.264 video compression for high definition video + annotations: + uid: 1.2.840.10008.1.2.4.102 + compression: lossy + media: video + MPEG4_AVC_H264_BD_COMPATIBLE_HIGH_PROFILE_LEVEL_4_1: + title: MPEG-4 AVC/H.264 BD-compatible High Profile / Level 4.1 + description: H.264 video compression constrained for Blu-ray Disc compatibility + annotations: + uid: 1.2.840.10008.1.2.4.103 + compression: lossy + media: video + MPEG4_AVC_H264_HIGH_PROFILE_LEVEL_4_2_2D: + title: MPEG-4 AVC/H.264 High Profile / Level 4.2 For 2D Video + description: H.264 video compression for 2D video at higher frame rates and resolutions + annotations: + uid: 1.2.840.10008.1.2.4.104 + compression: lossy + media: video + MPEG4_AVC_H264_HIGH_PROFILE_LEVEL_4_2_3D: + title: MPEG-4 AVC/H.264 High Profile / Level 4.2 For 3D Video + description: H.264 video compression for 3D (stereoscopic) video + annotations: + uid: 1.2.840.10008.1.2.4.105 + compression: lossy + media: video + MPEG4_AVC_H264_STEREO_HIGH_PROFILE_LEVEL_4_2: + title: MPEG-4 AVC/H.264 Stereo High Profile / Level 4.2 + description: H.264 stereo high profile for stereoscopic video + annotations: + uid: 1.2.840.10008.1.2.4.106 + compression: lossy + media: video + HEVC_H265_MAIN_PROFILE_LEVEL_5_1: + title: HEVC/H.265 Main Profile / Level 5.1 + description: HEVC video compression with 8-bit samples + annotations: + uid: 1.2.840.10008.1.2.4.107 + compression: lossy + media: video + HEVC_H265_MAIN_10_PROFILE_LEVEL_5_1: + title: HEVC/H.265 Main 10 Profile / Level 5.1 + description: HEVC video compression with 10-bit samples + annotations: + uid: 1.2.840.10008.1.2.4.108 + compression: lossy + media: video + JPEG_XL_LOSSLESS: + title: JPEG XL Lossless + description: JPEG XL compression restricted to lossless mode + annotations: + uid: 1.2.840.10008.1.2.4.110 + compression: lossless + JPEG_XL_JPEG_RECOMPRESSION: + title: JPEG XL JPEG Recompression + description: Lossless recompression of existing JPEG codestreams using JPEG XL + annotations: + uid: 1.2.840.10008.1.2.4.111 + compression: lossless + JPEG_XL: + title: JPEG XL + description: JPEG XL compression, lossy or lossless + annotations: + uid: 1.2.840.10008.1.2.4.112 + compression: lossy or lossless + HTJ2K_LOSSLESS_ONLY: + title: High-Throughput JPEG 2000 Image Compression (Lossless Only) + description: High-throughput JPEG 2000 (Part 15) restricted to lossless mode + annotations: + uid: 1.2.840.10008.1.2.4.201 + compression: lossless + HTJ2K_RPCL_LOSSLESS_ONLY: + title: High-Throughput JPEG 2000 with RPCL Options Image Compression (Lossless Only) + description: Lossless high-throughput JPEG 2000 with resolution-position-component-layer progression for progressive decoding + annotations: + uid: 1.2.840.10008.1.2.4.202 + compression: lossless + HTJ2K: + title: High-Throughput JPEG 2000 Image Compression + description: High-throughput JPEG 2000, lossy or lossless + annotations: + uid: 1.2.840.10008.1.2.4.203 + compression: lossy or lossless + RLE_LOSSLESS: + title: RLE Lossless + description: Run-length encoded lossless compression, widely used for ultrasound + annotations: + uid: 1.2.840.10008.1.2.5 + compression: lossless + SMPTE_ST_2110_20_UNCOMPRESSED_PROGRESSIVE_VIDEO: + title: SMPTE ST 2110-20 Uncompressed Progressive Active Video + description: Uncompressed progressive video streamed per SMPTE ST 2110-20, used in real-time video communication + annotations: + uid: 1.2.840.10008.1.2.7.1 + compression: none + media: video + SMPTE_ST_2110_20_UNCOMPRESSED_INTERLACED_VIDEO: + title: SMPTE ST 2110-20 Uncompressed Interlaced Active Video + description: Uncompressed interlaced video streamed per SMPTE ST 2110-20 + annotations: + uid: 1.2.840.10008.1.2.7.2 + compression: none + media: video + + DICOMNetworkServiceEnum: + title: DICOM Network Service + description: >- + Network services used to query, retrieve and store DICOM instances between imaging + systems such as PACS, modalities and research archives. Includes the classic DIMSE + (DICOM Message Service Element) services of PS3.7 and the RESTful DICOMweb services of + PS3.18. Cataloguing which services a source repository supports is part of the data + landscape assessment that precedes extraction. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part07/chapter_7.html + - https://dicom.nema.org/medical/dicom/current/output/chtml/part18/PS3.18.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + C_ECHO: + title: C-ECHO + description: DIMSE verification service used to test connectivity between two application entities + annotations: + protocol: DIMSE + C_STORE: + title: C-STORE + description: DIMSE storage service that pushes a composite instance to a peer + annotations: + protocol: DIMSE + C_FIND: + title: C-FIND + description: DIMSE query service that matches attributes against a peer's database at patient, study, series or instance level + annotations: + protocol: DIMSE + C_MOVE: + title: C-MOVE + description: DIMSE retrieve service that instructs a peer to send matching instances to a named destination via C-STORE + annotations: + protocol: DIMSE + C_GET: + title: C-GET + description: DIMSE retrieve service that returns matching instances on the same association + annotations: + protocol: DIMSE + N_EVENT_REPORT: + title: N-EVENT-REPORT + description: DIMSE-N notification service used to report events on a normalized SOP instance + annotations: + protocol: DIMSE + N_GET: + title: N-GET + description: DIMSE-N service that retrieves attribute values of a normalized SOP instance + annotations: + protocol: DIMSE + N_SET: + title: N-SET + description: DIMSE-N service that modifies attribute values of a normalized SOP instance + annotations: + protocol: DIMSE + N_ACTION: + title: N-ACTION + description: DIMSE-N service that requests an action on a normalized SOP instance, such as storage commitment + annotations: + protocol: DIMSE + N_CREATE: + title: N-CREATE + description: DIMSE-N service that creates a normalized SOP instance + annotations: + protocol: DIMSE + N_DELETE: + title: N-DELETE + description: DIMSE-N service that deletes a normalized SOP instance + annotations: + protocol: DIMSE + QIDO_RS: + title: QIDO-RS + description: DICOMweb RESTful query service (Query based on ID for DICOM Objects) for searching studies, series and instances + annotations: + protocol: DICOMweb + WADO_RS: + title: WADO-RS + description: DICOMweb RESTful retrieve service (Web Access to DICOM Objects) for retrieving studies, series, instances, frames, metadata and rendered images + annotations: + protocol: DICOMweb + STOW_RS: + title: STOW-RS + description: DICOMweb RESTful store service (Store Over the Web) for uploading instances + annotations: + protocol: DICOMweb + WADO_URI: + title: WADO-URI + description: Legacy DICOMweb URI-based retrieve service for single instances + annotations: + protocol: DICOMweb + UPS_RS: + title: UPS-RS + description: DICOMweb RESTful worklist service for Unified Procedure Step management + annotations: + protocol: DICOMweb + CUSTOM_API: + title: Custom API + description: A vendor- or institution-specific interface that is not a standard DICOM network service + + DICOMSoftwareToolEnum: + title: DICOM Software Tool + description: >- + Software toolkits, libraries, validators, servers and platforms commonly used to read, + write, validate, anonymize and serve DICOM data in research data management pipelines. + The category annotation distinguishes general-purpose toolkits from conformance + validators, archive servers and metadata anonymizers. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + DCMTK: + title: DCMTK + description: OFFIS DICOM Toolkit; C/C++ libraries and command-line utilities implementing DICOM network services and file handling + annotations: + category: toolkit + language: C++ + license: BSD + url: https://dicom.offis.de/en/dcmtk/ + PYDICOM: + title: pydicom + description: Pure Python library for reading, modifying and writing DICOM files, including pixel data access + annotations: + category: toolkit + language: Python + license: MIT + url: https://github.com/pydicom/pydicom + GDCM: + title: GDCM + description: Grassroots DICOM; C++ library with Python and other bindings for DICOM file and image codec handling + annotations: + category: toolkit + language: C++ + license: BSD + url: https://sourceforge.net/projects/gdcm/ + DCM4CHE: + title: dcm4che + description: Java DICOM toolkit and the basis of the dcm4chee archive + annotations: + category: toolkit + language: Java + license: MPL/GPL/LGPL + url: https://www.dcm4che.org/ + ITK: + title: ITK + description: Insight Toolkit; C++ image processing library with DICOM readers built on GDCM + annotations: + category: toolkit + language: C++ + license: Apache-2.0 + url: https://itk.org/ + DVTK: + title: DVTk + description: DICOM Validation Toolkit; validates object conformance and network behaviour against the standard + annotations: + category: validator + language: C# + license: LGPL + url: https://www.dvtk.org/ + DICOM3TOOLS: + title: dicom3tools + description: David Clunie's command-line utilities for creating, modifying, dumping and validating DICOM files + annotations: + category: validator + language: C++ + license: BSD + url: http://www.dclunie.com/dicom3tools.html + DCIODVFY: + title: dciodvfy + description: dicom3tools utility that verifies a file against the Information Object Definition for its modality, reporting missing required attributes, incorrect VRs and values outside allowed ranges + annotations: + category: validator + part_of: dicom3tools + DCENTVFY: + title: dcentvfy + description: dicom3tools utility that checks consistency of entity-level attributes across multiple files, such as all instances in a series sharing the same Series Instance UID + annotations: + category: validator + part_of: dicom3tools + PIXELMED: + title: PixelMed + description: PixelMed Java DICOM toolkit, including the DicomCleaner metadata anonymizer + annotations: + category: toolkit + language: Java + license: BSD + url: https://www.pixelmed.com/ + RSNA_CTP: + title: RSNA CTP + description: RSNA Clinical Trial Processor; pipeline application with a configurable DICOM anonymizer + annotations: + category: anonymizer + language: Java + url: https://mircwiki.rsna.org/index.php?title=CTP-The_RSNA_Clinical_Trial_Processor + ORTHANC: + title: Orthanc + description: Lightweight open-source DICOM server with a REST API and DICOMweb plugin + annotations: + category: server + language: C++ + license: GPL-3.0 + url: https://www.orthanc-server.com/ + DCM4CHEE: + title: dcm4chee + description: Open-source DICOM archive and image manager built on dcm4che + annotations: + category: server + language: Java + url: https://www.dcm4che.org/ + XNAT: + title: XNAT + description: Extensible Neuroimaging Archive Toolkit; open-source imaging informatics platform for managing, storing and sharing imaging data + annotations: + category: platform + language: Java + license: BSD + url: https://www.xnat.org/ + TCIA_UTILS: + title: tcia_utils + description: Python utilities from The Cancer Imaging Archive for querying, downloading and inventorying DICOM metadata + annotations: + category: toolkit + language: Python + url: https://github.com/kirbyju/tcia_utils + DICOM_CLEANER: + title: DicomCleaner + description: PixelMed graphical tool for metadata de-identification and blackout of burned-in text + annotations: + category: anonymizer + language: Java + part_of: PixelMed + HOROS: + title: Horos + description: Open-source macOS DICOM viewer forked from OsiriX with built-in anonymization + annotations: + category: viewer + license: LGPL-3.0 + url: https://horosproject.org/ + OSIRIX: + title: OsiriX + description: Commercial macOS DICOM viewer and workstation + annotations: + category: viewer + license: commercial + url: https://www.osirix-viewer.com/ + SLICER_3D: + title: 3D Slicer + description: Open-source platform for medical image visualization, segmentation and analysis with DICOM import + annotations: + category: viewer + license: BSD-style + url: https://www.slicer.org/ + ITK_SNAP: + title: ITK-SNAP + description: Open-source tool for manual and semi-automatic segmentation of 3D medical images + annotations: + category: viewer + license: GPL + url: http://www.itksnap.org/ diff --git a/src/valuesets/schema/medical/imaging_data_management.yaml b/src/valuesets/schema/medical/imaging_data_management.yaml new file mode 100644 index 00000000..4215b6fb --- /dev/null +++ b/src/valuesets/schema/medical/imaging_data_management.yaml @@ -0,0 +1,371 @@ +name: imaging_data_management +title: Medical Imaging Data Management Value Sets +description: >- + Value sets for managing medical imaging data through its research lifecycle, from + extraction out of clinical repositories through validation, quality assessment and + de-identification to AI-ready datasets. Covers the lifecycle stages, the types of source + systems that hold imaging data, the categories of structural and semantic problems found + by DICOM validation tools, the image quality metrics computed during pixel data + validation, the interoperability standards used to link images to clinical records, and + the NIH Bridge2AI data generation projects whose imaging datasets motivated the framework. + + Based on the six-stage data lifecycle framework in "Toward AI-Ready Medical Imaging Data" + (Nikolov et al., Bridge2AI Standards, Practices and Quality Assessment Working Group). +id: https://w3id.org/valuesets/medical/imaging_data_management +imports: +- linkml:types +prefixes: + linkml: https://w3id.org/linkml/ + valuesets: https://w3id.org/valuesets/ + NCIT: http://purl.obolibrary.org/obo/NCIT_ + orcid: https://orcid.org/ + valuesets_meta: https://w3id.org/valuesets/meta/ +default_prefix: valuesets +slots: + imaging_data_lifecycle_stage: + description: The stage of the medical imaging data management lifecycle + range: ImagingDataLifecycleStageEnum + imaging_data_source_type: + description: The type of system from which imaging data is extracted + range: ImagingDataSourceTypeEnum + dicom_validation_issue_type: + description: The category of problem found when validating a DICOM file or collection + range: DICOMValidationIssueTypeEnum + image_quality_metric: + description: A metric computed to assess image pixel data quality + range: ImageQualityMetricEnum + imaging_interoperability_standard: + description: A standard used to represent or link imaging data with clinical data + range: ImagingInteroperabilityStandardEnum + bridge2ai_data_generation_project: + description: The Bridge2AI data generation project that produced a dataset + range: Bridge2AIDataGenerationProjectEnum +enums: + ImagingDataLifecycleStageEnum: + title: Imaging Data Lifecycle Stage + description: >- + Sequential stages of DICOM data management for preparing FAIR, AI-ready medical imaging + datasets. Stages after extraction form the data reliability workflow that checks that + imaging data is complete, standardized and biologically plausible before sharing. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + DATA_EXTRACTION_AND_METADATA_CHARACTERIZATION: + title: Data extraction and metadata characterization + description: Extraction of imaging data from clinical repositories such as PACS, preceded by a landscape assessment of source locations, database systems, modalities and acquisition devices + meaning: NCIT:C44285 + annotations: + stage_number: 1 + FILE_INTEGRITY_VERIFICATION: + title: File integrity verification + description: Computation and periodic re-verification of cryptographic checksums such as SHA-256 to detect silent corruption from network errors, media decay or system failure + meaning: NCIT:C142477 + annotations: + stage_number: 2 + DATA_COMPLETENESS_AND_CONFORMANCE_CHECKS: + title: Data completeness and conformance checks + description: Detection of structurally invalid DICOM files (truncated, wrong VR encoding, missing required tags) and checks of internal coherence such as identifier uniqueness and demographic consistency across sites + annotations: + stage_number: 3 + METADATA_TAG_VALIDATION: + title: Metadata tag validation + description: Validation of DICOM attributes against the standard's VR, VM and type rules and against biological plausibility, and inventory of private tags + meaning: NCIT:C142500 + annotations: + stage_number: 4 + IMAGE_QUALITY_AND_PIXEL_DATA_VALIDATION: + title: Image quality and pixel data validation + description: Verification that pixel data is readable and plausible, that decompression preserves bit depth, and computation of image quality metrics + annotations: + stage_number: 5 + DEIDENTIFICATION: + title: Pixel-level and metadata de-identification + description: Removal of protected health information from headers and of facial features from pixel data before sharing and reuse + meaning: NCIT:C45970 + annotations: + stage_number: 6 + + ImagingDataSourceTypeEnum: + title: Imaging Data Source Type + description: >- + Types of systems in a hospital or research network that hold medical imaging data and + from which it may be extracted. A landscape assessment records, for each source, the + platform vendor, software version, supported query mechanisms, patient identifier + handling, retention policy and anonymization capabilities. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + PACS: + title: Picture Archiving and Communication System + description: Clinical system for storing, retrieving and distributing medical images, typically the central radiology archive + meaning: NCIT:C17624 + VENDOR_NEUTRAL_ARCHIVE: + title: Vendor neutral archive + description: Enterprise archive that stores images from multiple departments and PACS vendors in a standard format + aliases: + - VNA + DEPARTMENTAL_IMAGING_ARCHIVE: + title: Departmental imaging archive + description: Local archive maintained by a clinical department such as cardiology, ophthalmology or endoscopy outside the central PACS + RESEARCH_IMAGING_PLATFORM: + title: Research imaging platform + description: Research-specific imaging informatics database such as XNAT used to manage, store and share imaging data + MODALITY_WORKSTATION: + title: Modality workstation + description: Acquisition device or attached workstation that exports images through vendor-specific software + ELECTRONIC_HEALTH_RECORD: + title: Electronic Health Record + description: Clinical record system that references or embeds imaging studies and provides encounter linkage + meaning: NCIT:C142529 + PUBLIC_IMAGING_REPOSITORY: + title: Public imaging repository + description: Openly accessible imaging data resource such as The Cancer Imaging Archive or OpenNeuro + CLOUD_IMAGING_ARCHIVE: + title: Cloud imaging archive + description: Cloud-hosted object storage or managed imaging service holding DICOM data + + DICOMValidationIssueTypeEnum: + title: DICOM Validation Issue Type + description: >- + Categories of problems detected when validating DICOM files and collections for + completeness, conformance, metadata consistency, pixel data integrity and plausibility. + Used to classify findings from tools such as dciodvfy, dcentvfy, DVTk and pydicom + during data reliability workflows. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + TRUNCATED_FILE: + title: Truncated file + description: The file is incomplete, for example a truncated transfer syntax or missing pixel data at the end of the file + annotations: + category: structural + INVALID_VR_ENCODING: + title: Invalid VR encoding + description: An attribute is encoded with a Value Representation that does not match the data dictionary or the transfer syntax + annotations: + category: structural + MISSING_REQUIRED_ATTRIBUTE: + title: Missing required attribute + description: A Type 1 or Type 2 attribute required by the IOD, such as StudyInstanceUID or PixelData, is absent + annotations: + category: conformance + VALUE_MULTIPLICITY_VIOLATION: + title: Value multiplicity violation + description: An attribute has more or fewer values than permitted by its Value Multiplicity + annotations: + category: conformance + VALUE_OUT_OF_RANGE: + title: Value out of range + description: An attribute value is outside the enumerated or defined range allowed by the standard + annotations: + category: conformance + INVALID_UID_SYNTAX: + title: Invalid UID syntax + description: A UID does not conform to DICOM UID syntax, for example non-numeric components or excess length + annotations: + category: conformance + DUPLICATE_UID: + title: Duplicate UID + description: The same SOP Instance UID or other UID is reused across instances that should be distinct + annotations: + category: consistency + INCONSISTENT_ENTITY_ATTRIBUTES: + title: Inconsistent entity attributes + description: Attributes that should be identical across instances of the same entity differ, such as instances in one series carrying different Series Instance UIDs + annotations: + category: consistency + DATE_INCONSISTENCY: + title: Date inconsistency + description: Dates are mutually inconsistent, such as a study date after the patient's death date or a series date before the study date + annotations: + category: plausibility + IMPLAUSIBLE_VALUE: + title: Implausible value + description: A syntactically valid value that is biologically implausible, such as a patient age of 150 years + annotations: + category: plausibility + NONSTANDARD_PRIVATE_ATTRIBUTE: + title: Nonstandard private attribute + description: A private tag is present that is not documented in the site's private tag dictionary + annotations: + category: documentation + IDENTIFIER_LINKAGE_ERROR: + title: Identifier linkage error + description: Patient or study identifiers do not link correctly to clinical records, or demographics disagree across linked records + annotations: + category: consistency + UNREADABLE_PIXEL_DATA: + title: Unreadable pixel data + description: The pixel data cannot be decoded, typically due to acquisition or packaging problems + annotations: + category: pixel data + DECOMPRESSION_MISMATCH: + title: Decompression mismatch + description: Decompressed pixel data has an unexpected bit depth or differs from the uncompressed original beyond the expected loss + annotations: + category: pixel data + CHECKSUM_MISMATCH: + title: Checksum mismatch + description: The file's cryptographic hash no longer matches the value recorded at ingest, indicating corruption or alteration + annotations: + category: integrity + PROTECTED_HEALTH_INFORMATION_PRESENT: + title: Protected health information present + description: Identifying information remains in the header, private tags or pixel data after de-identification + annotations: + category: privacy + + ImageQualityMetricEnum: + title: Image Quality Metric + description: >- + Metrics computed during image quality and pixel data validation to document that images + are readable and of adequate fidelity for AI applications. Simple metrics can be + extended to check conformance with FDA technical performance guidance for quantitative + imaging devices where applicable. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + PIXEL_READABILITY: + title: Pixel readability + description: Whether the pixel array can be decoded without error + INTENSITY_HISTOGRAM: + title: Intensity histogram + description: Distribution of pixel intensities, checked for extreme outliers, clipping or empty images + SIGNAL_TO_NOISE_RATIO: + title: Signal to Noise Ratio + description: Ratio of signal in a region of interest to the standard deviation of background noise + meaning: NCIT:C94983 + aliases: + - SNR + CONTRAST_TO_NOISE_RATIO: + title: Contrast to noise ratio + description: Difference in signal between two regions relative to background noise + aliases: + - CNR + SHARPNESS: + title: Sharpness + description: Measure of edge definition or high-frequency content, such as Laplacian variance + BIT_DEPTH: + title: Bit depth + description: Number of bits per pixel stored and allocated, checked for consistency after decompression + PIXEL_SPACING: + title: Pixel spacing + description: Physical distance between pixel centres, checked for presence and plausibility + SLICE_THICKNESS: + title: Slice thickness + description: Nominal thickness of each slice in a volumetric acquisition, checked for presence and plausibility + COMPRESSION_FIDELITY: + title: Compression fidelity + description: Agreement between compressed and original pixel data, for example comparing a JPEG-compressed instance with its uncompressed source + ARTIFACT_PRESENCE: + title: Artifact presence + description: Detection of motion, metal, aliasing or other acquisition artifacts + + ImagingInteroperabilityStandardEnum: + title: Imaging Interoperability Standard + description: >- + Standards and data models used to represent medical imaging metadata and to link + images with clinical records, electronic health records and multimodal research + datasets. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + DICOM: + title: DICOM + description: Digital Imaging and Communications in Medicine (ISO 12052), the standard for storing, transmitting and managing medical imaging data + annotations: + url: https://www.dicomstandard.org/ + DICOMWEB: + title: DICOMweb + description: RESTful web services for DICOM (QIDO-RS, WADO-RS, STOW-RS) defined in DICOM PS3.18 + annotations: + url: https://www.dicomstandard.org/using/dicomweb + HL7_FHIR_IMAGINGSTUDY: + title: HL7 FHIR ImagingStudy + description: FHIR resource representing a DICOM study and its series and instances, used to integrate imaging metadata into FHIR-based systems + annotations: + url: https://www.hl7.org/fhir/imagingstudy.html + OMOP_CDM_IMAGING_EXTENSION: + title: OMOP CDM Imaging Extension + description: OHDSI Observational Medical Outcomes Partnership Common Data Model extension for imaging-based observational research, used to link DICOM studies to clinical encounters + annotations: + url: https://github.com/OHDSI/OmopImaging + IHE_PROFILES: + title: IHE integration profiles + description: Integrating the Healthcare Enterprise profiles for image sharing and identifier management, such as XDS-I and PIX + annotations: + url: https://www.ihe.net/ + BIDS: + title: Brain Imaging Data Structure + description: Community standard for organizing and describing neuroimaging datasets, commonly used after conversion from DICOM + annotations: + url: https://bids.neuroimaging.io/ + NIFTI: + title: NIfTI + description: Neuroimaging Informatics Technology Initiative file format for volumetric images, the usual target of DICOM conversion in neuroimaging pipelines + annotations: + url: https://nifti.nimh.nih.gov/ + + Bridge2AIDataGenerationProjectEnum: + title: Bridge2AI Data Generation Project + description: >- + Data generation projects (Grand Challenges) of the NIH Bridge to Artificial Intelligence + (Bridge2AI) program, which creates standardized, annotated, ethically sourced AI-ready + datasets across a diverse set of modalities. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://bridge2ai.org/ + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + CHORUS: + title: CHoRUS + description: Collaborative Hospital Repository Uniting Standards; the AI/ML for Clinical Care Grand Challenge, linking ICU imaging (MRI, CT, ultrasound, X-ray) with physiologic and clinical data + aliases: + - Clinical Care + annotations: + url: https://bridge2ai.org/data-chorus/ + AI_READI: + title: AI-READI + description: Artificial Intelligence Ready and Equitable Atlas for Diabetes Insights; the Salutogenesis Grand Challenge, including ophthalmology retinal imaging + aliases: + - Salutogenesis + annotations: + url: https://bridge2ai.org/people-ai-readi/ + VOICE: + title: Voice as a Biomarker of Health + description: The Precision Public Health Grand Challenge, connecting voice recordings with laryngoscopy video, brain MRI and CT, and omics data + aliases: + - Precision Public Health + annotations: + url: https://bridge2ai.org/people-voice/ + CM4AI: + title: CM4AI + description: Cell Maps for Artificial Intelligence; the Functional Genomics Grand Challenge, mapping cellular architecture with imaging and proteomics + aliases: + - Functional Genomics + annotations: + url: https://cm4ai.org/ diff --git a/src/valuesets/schema/medical/imaging_deidentification.yaml b/src/valuesets/schema/medical/imaging_deidentification.yaml new file mode 100644 index 00000000..3251e25d --- /dev/null +++ b/src/valuesets/schema/medical/imaging_deidentification.yaml @@ -0,0 +1,490 @@ +name: imaging_deidentification +title: Medical Image De-identification Value Sets +description: >- + Value sets for de-identifying medical imaging data before sharing and reuse. Covers the + DICOM PS3.15 Annex E de-identification profile and options (with their DCM codes from + Context ID 7050), the action codes that specify how each attribute is treated, the parts of + an image object that can carry identifying information, the regulatory frameworks that + define what must be removed, and the pixel-level "de-facing" methods and tools used to + remove reconstructable facial features from head CT and MRI volumes. + + Defacing only addresses pixel data. A complete de-identification pipeline also strips or + pseudonymizes protected health information in the DICOM header, as recommended by the + Medical Image De-Identification (MIDI) Task Group (Clunie et al., 2025), and anticipates + adversarial re-identification, for example face-recognition matching of surface renderings + of MRI head scans (Schwarz et al., 2019). + + Drawn from "Toward AI-Ready Medical Imaging Data" (Nikolov et al., Bridge2AI consortium), + which reviews the de-facing tools and methods listed here. +id: https://w3id.org/valuesets/medical/imaging_deidentification +imports: +- linkml:types +prefixes: + linkml: https://w3id.org/linkml/ + valuesets: https://w3id.org/valuesets/ + NCIT: http://purl.obolibrary.org/obo/NCIT_ + DCM: http://dicom.nema.org/resources/ontology/DCM/ + orcid: https://orcid.org/ + valuesets_meta: https://w3id.org/valuesets/meta/ +default_prefix: valuesets +slots: + dicom_deidentification_method: + description: A DICOM de-identification profile or option applied to an image object + range: DICOMDeidentificationMethodEnum + dicom_deidentification_action: + description: The de-identification action applied to a DICOM attribute + range: DICOMDeidentificationActionEnum + image_deidentification_target: + description: The part of an imaging object from which identifying information is removed + range: ImageDeidentificationTargetEnum + deidentification_regulatory_framework: + description: The regulation or standard under which data is de-identified + range: DeidentificationRegulatoryFrameworkEnum + defacing_method: + description: The method used to remove facial features from image pixel data + range: DefacingMethodEnum + defacing_tool: + description: The software tool used to remove facial features from image pixel data + range: DefacingToolEnum +enums: + DICOMDeidentificationMethodEnum: + title: DICOM De-identification Method + description: >- + The Basic Application Level Confidentiality Profile and its options defined in DICOM + PS3.15 Annex E, as coded in PS3.16 Context ID 7050 (De-identification Method). These + codes are recorded in the De-identification Method Code Sequence (0012,0064) to document + what was done to an instance. The "Clean" options remove additional identifying content; + the "Retain" options preserve information that the basic profile would otherwise remove. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part16/sect_CID_7050.html + - https://dicom.nema.org/medical/dicom/current/output/chtml/part15/chapter_E.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + BASIC_APPLICATION_CONFIDENTIALITY_PROFILE: + title: Basic Application Confidentiality Profile + description: The baseline profile that removes or replaces all attributes known to carry identifying information, including UIDs, dates and descriptive text + meaning: DCM:113100 + annotations: + kind: profile + CLEAN_PIXEL_DATA_OPTION: + title: Clean Pixel Data Option + description: Burned-in identifying text and annotations in the pixel data are removed + meaning: DCM:113101 + annotations: + kind: clean option + CLEAN_RECOGNIZABLE_VISUAL_FEATURES_OPTION: + title: Clean Recognizable Visual Features Option + description: Recognizable visual features such as the face are removed from pixel data; this is the option under which de-facing is recorded + meaning: DCM:113102 + annotations: + kind: clean option + CLEAN_GRAPHICS_OPTION: + title: Clean Graphics Option + description: Identifying information in graphic annotations, overlays and presentation states is removed + meaning: DCM:113103 + annotations: + kind: clean option + CLEAN_STRUCTURED_CONTENT_OPTION: + title: Clean Structured Content Option + description: Identifying information in structured report content items is removed + meaning: DCM:113104 + annotations: + kind: clean option + CLEAN_DESCRIPTORS_OPTION: + title: Clean Descriptors Option + description: Free-text descriptors such as Study Description are cleaned of identifying content rather than removed + meaning: DCM:113105 + annotations: + kind: clean option + RETAIN_LONGITUDINAL_TEMPORAL_INFORMATION_FULL_DATES_OPTION: + title: Retain Longitudinal Temporal Information Full Dates Option + description: Dates and times are retained unmodified to preserve the temporal relationship between studies + meaning: DCM:113106 + annotations: + kind: retain option + RETAIN_LONGITUDINAL_TEMPORAL_INFORMATION_MODIFIED_DATES_OPTION: + title: Retain Longitudinal Temporal Information Modified Dates Option + description: Dates and times are shifted consistently so that intervals between studies are preserved + meaning: DCM:113107 + annotations: + kind: retain option + RETAIN_PATIENT_CHARACTERISTICS_OPTION: + title: Retain Patient Characteristics Option + description: Physical characteristics such as age, sex, height and weight are retained + meaning: DCM:113108 + annotations: + kind: retain option + RETAIN_DEVICE_IDENTITY_OPTION: + title: Retain Device Identity Option + description: Device identifying attributes such as manufacturer, model and serial number are retained + meaning: DCM:113109 + annotations: + kind: retain option + RETAIN_UIDS_OPTION: + title: Retain UIDs Option + description: Original UIDs are retained rather than replaced + meaning: DCM:113110 + annotations: + kind: retain option + RETAIN_SAFE_PRIVATE_OPTION: + title: Retain Safe Private Option + description: Private attributes known not to contain identifying information are retained + meaning: DCM:113111 + annotations: + kind: retain option + RETAIN_INSTITUTION_IDENTITY_OPTION: + title: Retain Institution Identity Option + description: Institution identifying attributes such as institution name and address are retained + meaning: DCM:113112 + annotations: + kind: retain option + + DICOMDeidentificationActionEnum: + title: DICOM De-identification Action Code + description: >- + Action codes from DICOM PS3.15 Table E.1-1a that specify how a de-identifier treats each + attribute under the Basic Application Level Confidentiality Profile. Compound codes + (for example Z/D) indicate that the first action applies unless the attribute's type + requires the second to maintain IOD conformance. Permissible values use the DICOM code + letters, with slashes replaced by underscores in compound codes. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + see_also: + - https://dicom.nema.org/medical/dicom/current/output/chtml/part15/chapter_E.html + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + D: + title: Replace with dummy value + description: Replace with a non-zero length value that may be a dummy value and consistent with the VR + Z: + title: Replace with zero-length or dummy value + description: Replace with a zero length value, or a non-zero length value that may be a dummy value and consistent with the VR + X: + title: Remove + description: Remove the attribute, and if the attribute is a sequence, remove all sequence items and their contained attributes + K: + title: Keep + description: Keep unchanged for non-sequence attributes; cleaned for sequences + C: + title: Clean + description: Replace with values of similar meaning known not to contain identifying information and consistent with the VR + U: + title: Replace UID + description: Replace with a non-zero length UID that is internally consistent within a set of instances + Z_D: + title: Z unless D required + description: Z unless D is required to maintain IOD conformance (Type 2 versus Type 1) + aliases: + - Z/D + X_Z: + title: X unless Z required + description: X unless Z is required to maintain IOD conformance (Type 3 versus Type 2) + aliases: + - X/Z + X_D: + title: X unless D required + description: X unless D is required to maintain IOD conformance (Type 3 versus Type 1) + aliases: + - X/D + X_Z_D: + title: X unless Z or D required + description: X unless Z or D is required to maintain IOD conformance (Type 3 versus Type 2 versus Type 1) + aliases: + - X/Z/D + X_Z_U: + title: X unless Z or U required + description: X unless Z or replacement of contained instance UIDs (U) is required to maintain IOD conformance (Type 3 versus Type 2 versus Type 1 sequences containing UID references) + aliases: + - X/Z/U* + + ImageDeidentificationTargetEnum: + title: Image De-identification Target + description: >- + The components of a medical imaging object that can carry identifying information and + therefore need to be addressed by a de-identification workflow. A workflow first + assesses whether an image requires pixel-level (face or head) de-identification or only + metadata de-identification. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + HEADER_METADATA: + title: Header metadata + description: Standard DICOM attributes such as patient name, identifiers, birth date and study dates + PRIVATE_ATTRIBUTES: + title: Private attributes + description: Vendor-specific private tags that may contain identifying information and are not covered by the standard attribute list + BURNED_IN_ANNOTATION: + title: Burned-in annotation + description: Text or graphics rendered into the pixel data, common in ultrasound, secondary capture and screenshots + FACIAL_FEATURES: + title: Facial features + description: Facial surface anatomy reconstructable from volumetric head CT or MRI pixel data, or visible in photographs and video + UNIQUE_IDENTIFIERS: + title: Unique identifiers + description: Study, series and instance UIDs and accession numbers that can link an instance back to the source system + STRUCTURED_CONTENT: + title: Structured content + description: Identifying content in structured reports, overlays, presentation states and embedded documents + + DeidentificationRegulatoryFrameworkEnum: + title: De-identification Regulatory Framework + description: >- + Regulations and standards that define when medical imaging data is considered + de-identified. Under HIPAA, full-face photographs and comparable images are direct + identifiers; under the GDPR, facial images are biometric personal data requiring + special handling. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + HIPAA_SAFE_HARBOR: + title: HIPAA Safe Harbor + description: US HIPAA Privacy Rule method requiring removal of 18 specified identifier types, including full-face photographs and comparable images + annotations: + jurisdiction: United States + citation: 45 CFR 164.514(b)(2) + HIPAA_EXPERT_DETERMINATION: + title: HIPAA Expert Determination + description: US HIPAA Privacy Rule method in which a qualified expert determines that the risk of re-identification is very small + annotations: + jurisdiction: United States + citation: 45 CFR 164.514(b)(1) + GDPR_ANONYMISATION: + title: GDPR anonymisation + description: Irreversible processing such that the data subject is no longer identifiable, taking the data outside the scope of the EU General Data Protection Regulation + meaning: NCIT:C142392 + annotations: + jurisdiction: European Union + GDPR_PSEUDONYMISATION: + title: GDPR pseudonymisation + description: Processing so that data can no longer be attributed to a subject without additional information kept separately, as defined in GDPR Article 4(5) + meaning: NCIT:C142654 + annotations: + jurisdiction: European Union + DICOM_PS3_15_CONFIDENTIALITY_PROFILE: + title: DICOM PS3.15 Attribute Confidentiality Profile + description: The DICOM standard's own de-identification profile and options, designed to satisfy known regulations + annotations: + jurisdiction: international + MIDI_TASK_GROUP_RECOMMENDATIONS: + title: MIDI Task Group best practices + description: Best practices and recommendations of the Medical Image De-Identification (MIDI) Task Group (Clunie et al.) + annotations: + jurisdiction: international + + DefacingMethodEnum: + title: Defacing Method + description: >- + Approaches to pixel-level de-identification that remove or obscure facial features in + head imaging. Skull-stripping removes all non-brain tissue and may discard useful + anatomy; face-specific methods aim to remove only facial features while preserving as + much of the head volume as possible. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + SKULL_STRIPPING: + title: Skull stripping + description: Removal of all non-brain tissue including scalp, skull and face, for example with FSL BET or AFNI 3dSkullStrip + aliases: + - brain extraction + annotations: + preserves_skull: 'false' + TEMPLATE_BASED_MASKING: + title: Template-based masking + description: Registration of the image to a standard brain template such as MNI152 followed by application of a predefined binary face mask that zeros out facial voxels + annotations: + preserves_skull: 'true' + example_tools: PyDeface, FreeSurfer mri_deface, mydeface + SURFACE_BLURRING: + title: Surface blurring + description: Diffusion or blurring of face surface voxels to obscure identity while preserving head shape, as in Milchenko and Marcus (2013) + annotations: + preserves_skull: 'true' + SHEARING_PLANE: + title: Shearing plane + description: Removal of the front of the head by computing a plane through the head and discarding voxels in front of it, as in QuickShear + aliases: + - cropping plane + annotations: + preserves_skull: partial + DEEP_LEARNING_SEGMENTATION: + title: Deep learning segmentation + description: Use of a trained neural network such as a 3D U-Net to segment facial features (eyes, ears, nose) and mask or blur them + annotations: + preserves_skull: 'true' + example_tools: DeepDefacer, Asan Defacer + REFACING: + title: Refacing + description: Replacement of the subject's face with an average or synthetic face so that images retain a realistic head surface + annotations: + preserves_skull: 'true' + example_tools: AFNI refacer + MANUAL_MASKING: + title: Manual masking + description: Interactive painting or erosion of a face mask in an image editor such as 3D Slicer or ITK-SNAP, followed by zeroing or blurring of masked voxels + annotations: + preserves_skull: 'true' + FACE_DETECTION_AND_BLURRING: + title: Face detection and blurring + description: Frame-by-frame detection of faces in 2D images or video followed by blurring, pixelation or masking, for example with OpenCV Haar cascades + annotations: + applicable_to: photographs, video + MANUAL_CROP_OR_BLUR: + title: Manual crop or blur + description: Manual cropping or blurring of the face region in photographs or video with an image or video editor + annotations: + applicable_to: photographs, video + + DefacingToolEnum: + title: Defacing Tool + description: >- + Software tools and pipelines used for pixel-level de-identification of head imaging and + photographs, with their supported modalities, method and licensing. Metadata-only + anonymizers are included for completeness because they are commonly paired with + defacing tools, but they do not remove facial features. + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + permissible_values: + PYDEFACE: + title: PyDeface + description: Aligns a T1-weighted MRI to a template with FSL FLIRT and zeros out voxels in a predefined facial mask + annotations: + modality: MRI (T1w) + method: template-based masking + license: BSD + url: https://github.com/poldracklab/pydeface + FREESURFER_MRI_DEFACE: + title: FreeSurfer mri_deface + description: Template mask defacing using affine registration to fit a generic face mask, distributed with FreeSurfer + annotations: + modality: MRI (T1w) + method: template-based masking + license: FreeSurfer + AFNI_REFACER: + title: AFNI @afni_refacer_run + description: AFNI template-based defacing and refacing tool, often combined with skull stripping + annotations: + modality: MRI (T1w) + method: refacing + license: AFNI (open source) + QUICKSHEAR: + title: QuickShear + description: Computes a shearing plane through the head and removes the front of the head + annotations: + modality: MRI (T1w) + method: shearing plane + license: open source + DEEPDEFACER: + title: DeepDefacer + description: 3D U-Net trained to generate a facial mask from T1 MRI scans + annotations: + modality: MRI (T1w, T2w) + method: deep learning segmentation + license: open source + ASAN_DEFACER: + title: Asan Defacer + description: 3D U-Net that segments eyes, ears and nose and masks them, applicable to MRI and CT + annotations: + modality: MRI, CT + method: deep learning segmentation + license: open source + MYDEFACE: + title: mydeface + description: Defacing utility similar to PyDeface using an FSL FLIRT-registered mask + annotations: + modality: MRI (T1w, FLAIR) + method: template-based masking + license: BSD + url: https://github.com/neurolabusc/mydeface + MASK_FACE: + title: mask_face + description: Surface blurring tool from Milchenko and Marcus that obscures surface anatomy in volumetric data + annotations: + modality: MRI, CT + method: surface blurring + license: open source + FSL_BET: + title: FSL BET + description: FMRIB Software Library Brain Extraction Tool; removes all non-brain tissue + annotations: + modality: MRI, CT, PET + method: skull stripping + license: FSL (open source) + AFNI_3DSKULLSTRIP: + title: AFNI 3dSkullStrip + description: AFNI brain extraction program; removes all non-brain tissue + annotations: + modality: MRI, CT, PET + method: skull stripping + license: AFNI (open source) + ITK_SNAP: + title: ITK-SNAP + description: Interactive segmentation tool used to manually paint or erode a face mask in any 3D volume + annotations: + modality: any 3D volume + method: manual masking + license: GPL + SLICER_3D: + title: 3D Slicer + description: Image computing platform used to manually paint a face mask over a region of interest + annotations: + modality: any 3D volume + method: manual masking + license: BSD-style + IMAGEJ_FIJI: + title: ImageJ / Fiji + description: General image analysis tools used to manually blur or crop the face region in 2D or 3D images + annotations: + modality: 2D and 3D images + method: manual crop or blur + license: open source + OPENCV: + title: OpenCV + description: Computer vision library used for face detection (for example Haar cascades) followed by blurring or pixelation in video and 2D images + annotations: + modality: video, 2D images + method: face detection and blurring + license: Apache-2.0 + OSIRIX_HOROS_PLUGIN: + title: OsiriX / Horos plugin + description: Viewer plugins offering built-in anonymization with face removal options for multi-modality DICOM + annotations: + modality: DICOM (multi-modality) + method: template-based masking + license: commercial / free + PIXELMED_DICOM_ANONYMIZER: + title: PixelMed DICOM Anonymizer + description: Metadata anonymization only; does not mask faces + annotations: + modality: DICOM files + method: metadata anonymization + license: BSD + MANUAL_PHOTO_VIDEO_EDITING: + title: Manual photo or video editing + description: Cropping or blurring faces in photographs and videos with general-purpose editors + annotations: + modality: photographs, video + method: manual crop or blur diff --git a/src/valuesets/schema/valuesets.yaml b/src/valuesets/schema/valuesets.yaml index e39ada7f..1ca19751 100644 --- a/src/valuesets/schema/valuesets.yaml +++ b/src/valuesets/schema/valuesets.yaml @@ -114,6 +114,9 @@ imports: - industry/safety_colors - medical/clinical - medical/neuroimaging +- medical/dicom +- medical/imaging_deidentification +- medical/imaging_data_management - medical/family_history - medical/pediatric_oncology/diagnosis_categories - medical/pediatric_oncology/iccc3 From 9bc95b8cbd22f37bbc40aa4274bbaec3115624c6 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 21:52:11 +0000 Subject: [PATCH 2/4] Drop source paper citations from imaging schema descriptions The value sets stand on the DICOM standard and de-identification practice; the descriptions no longer cite the review article that seeded them. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JqKeWA6q8xdJhgpuzHE4Ym --- src/valuesets/schema/medical/dicom.yaml | 7 +++---- .../schema/medical/imaging_data_management.yaml | 5 +---- .../schema/medical/imaging_deidentification.yaml | 9 ++------- 3 files changed, 6 insertions(+), 15 deletions(-) diff --git a/src/valuesets/schema/medical/dicom.yaml b/src/valuesets/schema/medical/dicom.yaml index e00a3a99..0ca78819 100644 --- a/src/valuesets/schema/medical/dicom.yaml +++ b/src/valuesets/schema/medical/dicom.yaml @@ -11,10 +11,9 @@ description: >- from PACS, and the open-source toolkits commonly used to read, validate and serve DICOM files. - These value sets support the medical imaging data lifecycle described in "Toward AI-Ready - Medical Imaging Data" (Nikolov et al., Bridge2AI consortium), in particular the data - extraction, conformance checking and metadata tag validation stages, where VR, VM and - attribute type rules are checked with tools such as dciodvfy, dcentvfy, DVTk and pydicom. + These value sets support DICOM data extraction, conformance checking and metadata tag + validation, where VR, VM and attribute type rules are checked with tools such as dciodvfy, + dcentvfy, DVTk and pydicom. Codes that come from the DICOM standard itself (modality codes, VRs, action codes) are kept in their standard form rather than being upper-cased or expanded. diff --git a/src/valuesets/schema/medical/imaging_data_management.yaml b/src/valuesets/schema/medical/imaging_data_management.yaml index 4215b6fb..69cd4ff5 100644 --- a/src/valuesets/schema/medical/imaging_data_management.yaml +++ b/src/valuesets/schema/medical/imaging_data_management.yaml @@ -7,10 +7,7 @@ description: >- systems that hold imaging data, the categories of structural and semantic problems found by DICOM validation tools, the image quality metrics computed during pixel data validation, the interoperability standards used to link images to clinical records, and - the NIH Bridge2AI data generation projects whose imaging datasets motivated the framework. - - Based on the six-stage data lifecycle framework in "Toward AI-Ready Medical Imaging Data" - (Nikolov et al., Bridge2AI Standards, Practices and Quality Assessment Working Group). + the NIH Bridge2AI data generation projects that produce AI-ready imaging datasets. id: https://w3id.org/valuesets/medical/imaging_data_management imports: - linkml:types diff --git a/src/valuesets/schema/medical/imaging_deidentification.yaml b/src/valuesets/schema/medical/imaging_deidentification.yaml index 3251e25d..888ba444 100644 --- a/src/valuesets/schema/medical/imaging_deidentification.yaml +++ b/src/valuesets/schema/medical/imaging_deidentification.yaml @@ -9,13 +9,8 @@ description: >- remove reconstructable facial features from head CT and MRI volumes. Defacing only addresses pixel data. A complete de-identification pipeline also strips or - pseudonymizes protected health information in the DICOM header, as recommended by the - Medical Image De-Identification (MIDI) Task Group (Clunie et al., 2025), and anticipates - adversarial re-identification, for example face-recognition matching of surface renderings - of MRI head scans (Schwarz et al., 2019). - - Drawn from "Toward AI-Ready Medical Imaging Data" (Nikolov et al., Bridge2AI consortium), - which reviews the de-facing tools and methods listed here. + pseudonymizes protected health information in the DICOM header, and anticipates adversarial + re-identification such as face-recognition matching of surface renderings of head scans. id: https://w3id.org/valuesets/medical/imaging_deidentification imports: - linkml:types From 28aea757027c333374baf71305a9b25db512052e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 8 Sep 2026 22:00:21 +0000 Subject: [PATCH 3/4] Add FBbi dynamic value sets and FBbi mappings for DICOM modalities bio/biological_imaging_methods.yaml defines thirteen dynamic enums populated from the Biological Imaging Methods Ontology (FBbi): one per top-level branch (imaging method, sample preparation, visualization, illumination, detection, imaged parameter, contrast enhancement, resolution enhancement, source of contrast) plus the microscopy, light microscopy, fluorescence microscopy and fixation method subtrees. DICOMModalityEnum gains FBbi exact mappings for CT, CFM, GM, OCT and RG. FBbi is registered in the validator's ontology adapter configuration. Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JqKeWA6q8xdJhgpuzHE4Ym --- .../bio/biological_imaging_methods.yaml | 297 ++++++++++++++++++ src/valuesets/schema/medical/dicom.yaml | 6 + src/valuesets/schema/valuesets.yaml | 1 + src/valuesets/validators/oak_config.yaml | 1 + 4 files changed, 305 insertions(+) create mode 100644 src/valuesets/schema/bio/biological_imaging_methods.yaml diff --git a/src/valuesets/schema/bio/biological_imaging_methods.yaml b/src/valuesets/schema/bio/biological_imaging_methods.yaml new file mode 100644 index 00000000..c998fded --- /dev/null +++ b/src/valuesets/schema/bio/biological_imaging_methods.yaml @@ -0,0 +1,297 @@ +name: biological_imaging_methods +title: Biological Imaging Methods Value Sets +description: >- + Dynamic enumerations populated from the Biological Imaging Methods Ontology (FBbi), which + describes the methods involved in producing a biological image. FBbi is organized as nine + top-level branches under "method involved in biological imaging": sample preparation, + visualization (stains, probes and labels), imaging method (microscopy, radiography, + tomography and portrayed images), illumination, detection, imaged parameter, contrast + enhancement, resolution enhancement and source of contrast. Each branch is exposed here as + a value set, with additional value sets for the microscopy, light microscopy and + fluorescence microscopy subtrees that are most often needed as controlled vocabularies in + imaging metadata. +id: https://w3id.org/valuesets/bio/biological_imaging_methods +imports: +- linkml:types +prefixes: + linkml: https://w3id.org/linkml/ + valuesets: https://w3id.org/valuesets/ + FBbi: http://purl.obolibrary.org/obo/FBbi_ + orcid: https://orcid.org/ + valuesets_meta: https://w3id.org/valuesets/meta/ +default_prefix: valuesets +slots: + biological_imaging_method: + description: The method used to record or portray a biological image + range: BiologicalImagingMethod + biological_microscopy_method: + description: The microscopy method used to acquire an image + range: MicroscopyMethod + light_microscopy_method: + description: The light microscopy method used to acquire an image + range: LightMicroscopyMethod + fluorescence_microscopy_method: + description: The fluorescence microscopy method used to acquire an image + range: FluorescenceMicroscopyMethod + imaging_sample_preparation_method: + description: How the specimen was prepared before imaging + range: ImagingSamplePreparationMethod + tissue_fixation_method: + description: The fixation method applied to the specimen before imaging + range: TissueFixationMethod + imaging_visualization_method: + description: The stain, probe or label-based method used to visualize structures in the specimen + range: ImagingVisualizationMethod + imaging_illumination_method: + description: How the specimen was illuminated during image acquisition + range: ImagingIlluminationMethod + imaging_detection_method: + description: How the signal from the specimen was detected during image acquisition + range: ImagingDetectionMethod + imaged_parameter: + description: The physical parameter of the specimen that the image records + range: ImagedParameter + contrast_enhancing_method: + description: The optical or computational method used to enhance image contrast + range: ContrastEnhancingMethod + resolution_enhancing_method: + description: The super-resolution method used to exceed the diffraction limit + range: ResolutionEnhancingMethod + source_of_image_contrast: + description: The physical basis of contrast in the image + range: SourceOfImageContrast +enums: + BiologicalImagingMethod: + title: Biological Imaging Method + description: >- + Any imaging method from FBbi, covering recorded images (macroscopy, microscopy, + radiography, tomography) and portrayed images (diagrams, graphic illustrations, montages) + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000222 + include_self: false + relationship_types: + - rdfs:subClassOf + MicroscopyMethod: + title: Microscopy Method + description: >- + Any microscopy method from FBbi, including X-ray microscopy, microscopy with lenses + (light and electron microscopy) and scanning probe microscopy + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000241 + include_self: false + relationship_types: + - rdfs:subClassOf + LightMicroscopyMethod: + title: Light Microscopy Method + description: >- + Light microscopy methods from FBbi, including bright-field, phase contrast, differential + interference contrast, polarization, fluorescence and optical coherence tomography + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000345 + include_self: false + relationship_types: + - rdfs:subClassOf + FluorescenceMicroscopyMethod: + title: Fluorescence Microscopy Method + description: >- + Fluorescence microscopy methods from FBbi, including wide-field, confocal, multi-photon, + light sheet and total internal reflection fluorescence microscopy + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000246 + include_self: false + relationship_types: + - rdfs:subClassOf + ImagingSamplePreparationMethod: + title: Imaging Sample Preparation Method + description: >- + Sample preparation methods from FBbi, describing the state of the specimen at imaging + time such as embedded, sectioned, whole mounted or permeabilized tissue, dispersed or + adherent cells, and fixation methods + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000001 + include_self: false + relationship_types: + - rdfs:subClassOf + TissueFixationMethod: + title: Tissue Fixation Method + description: Fixation methods from FBbi, such as aldehyde, alcohol, freeze and heat fixation + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00001012 + include_self: false + relationship_types: + - rdfs:subClassOf + ImagingVisualizationMethod: + title: Imaging Visualization Method + description: >- + Visualization methods from FBbi, the largest branch of the ontology, covering + visualization of labels conjugated to probes, of contiguous regions, and by chemical + attribute (histological stains, fluorescent dyes and fluorescent proteins) + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000031 + include_self: false + relationship_types: + - rdfs:subClassOf + ImagingIlluminationMethod: + title: Imaging Illumination Method + description: >- + Illumination methods from FBbi, characterized by the probe (photons, electrons, + acoustic waves, neutrons, ions), field (widefield, narrowfield, nearfield, farfield), + coherence, timing and geometry + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000268 + include_self: false + relationship_types: + - rdfs:subClassOf + ImagingDetectionMethod: + title: Imaging Detection Method + description: >- + Detection methods from FBbi, characterized by what is detected (photons, electrons, + acoustic waves, induced current), field of detection and detector geometry (spot or + area detector) + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000269 + include_self: false + relationship_types: + - rdfs:subClassOf + ImagedParameter: + title: Imaged Parameter + description: >- + Physical parameters of the specimen recorded by an image, from FBbi, such as + absorption, elastic and inelastic scattering, refractive index, retardance, + fluorescence emission, electron density and elevation + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000270 + include_self: false + relationship_types: + - rdfs:subClassOf + ContrastEnhancingMethod: + title: Contrast Enhancing Method + description: Optical and computational contrast-enhancing methods from FBbi, such as phase contrast, DIC and deconvolution + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000271 + include_self: false + relationship_types: + - rdfs:subClassOf + ResolutionEnhancingMethod: + title: Resolution Enhancing Method + description: Super-resolution methods from FBbi, such as structured illumination, point-localization methods, STED and GSD + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000321 + include_self: false + relationship_types: + - rdfs:subClassOf + SourceOfImageContrast: + title: Source of Image Contrast + description: >- + Physical bases of image contrast from FBbi, such as differences in chemical composition, + distribution of an epitope, protein or nucleic acid sequence, stain binding, refractive + index boundaries, fluorescence lifetime and optical anisotropy + status: DRAFT + contributors: + - orcid:0000-0002-6601-2165 + - https://github.com/anthropics/claude-code + instantiates: + - valuesets_meta:ValueSetEnumDefinition + reachable_from: + source_ontology: obo:fbbi + source_nodes: + - FBbi:00000590 + include_self: false + relationship_types: + - rdfs:subClassOf diff --git a/src/valuesets/schema/medical/dicom.yaml b/src/valuesets/schema/medical/dicom.yaml index 0ca78819..0c54d53e 100644 --- a/src/valuesets/schema/medical/dicom.yaml +++ b/src/valuesets/schema/medical/dicom.yaml @@ -25,6 +25,7 @@ prefixes: valuesets: https://w3id.org/valuesets/ NCIT: http://purl.obolibrary.org/obo/NCIT_ DCM: http://dicom.nema.org/resources/ontology/DCM/ + FBbi: http://purl.obolibrary.org/obo/FBbi_ orcid: https://orcid.org/ valuesets_meta: https://w3id.org/valuesets/meta/ default_prefix: valuesets @@ -93,12 +94,14 @@ enums: meaning: NCIT:C17204 exact_mappings: - DCM:CT + - FBbi:00001002 CFM: title: Confocal Microscopy description: Laser-scanning microscopy that rejects out-of-focus light meaning: NCIT:C17753 exact_mappings: - DCM:CFM + - FBbi:00000251 DMS: title: Dermoscopy description: Non-invasive microscopic examination of the skin surface @@ -133,6 +136,7 @@ enums: meaning: NCIT:C16853 exact_mappings: - DCM:GM + - FBbi:00000241 aliases: - General Microscopy IO: @@ -233,6 +237,7 @@ enums: meaning: NCIT:C20828 exact_mappings: - DCM:OCT + - FBbi:00000371 OSS: title: Optical Surface Scanner description: Surface geometry acquired with an optical (non-laser) scanner @@ -269,6 +274,7 @@ enums: meaning: NCIT:C38101 exact_mappings: - DCM:RG + - FBbi:00001001 aliases: - Radiographic imaging RTIMAGE: diff --git a/src/valuesets/schema/valuesets.yaml b/src/valuesets/schema/valuesets.yaml index 1ca19751..323247db 100644 --- a/src/valuesets/schema/valuesets.yaml +++ b/src/valuesets/schema/valuesets.yaml @@ -18,6 +18,7 @@ imports: - bio/genome_features - bio/bio_entities - bio/structural_biology +- bio/biological_imaging_methods - bio/protein_structure_features - bio/biosafety - bio/insdc_missing_values diff --git a/src/valuesets/validators/oak_config.yaml b/src/valuesets/validators/oak_config.yaml index 8e5510b2..64b3e3c8 100644 --- a/src/valuesets/validators/oak_config.yaml +++ b/src/valuesets/validators/oak_config.yaml @@ -23,6 +23,7 @@ ontology_adapters: MSIO: sqlite:obo:msio GENO: sqlite:obo:geno CHMO: sqlite:obo:chmo + FBbi: sqlite:obo:fbbi # Data format and computation ontologies EDAM: sqlite:obo:edam From 29d07edb8f1909a8b29391973ae09fc6e1691e2d Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 13 Sep 2026 00:59:37 +0000 Subject: [PATCH 4/4] Validate mapping slots and reachable_from source nodes; fix label mismatches The enum evaluator previously checked only the `meaning` of each permissible value. It now also resolves every CURIE in exact/close/broad/narrow/related mappings and every `reachable_from.source_nodes` entry of a dynamic enum, reporting ERROR for configured prefixes and INFO otherwise, and counts them in the mappings-checked total. Unit tests cover both paths. Schema fixes from validating the new files against OLS: - NCIT terms broader than the lifecycle stage they annotated (Data Extraction, Data Integrity Verification, Data Validation) move from meaning to broad_mappings - GDPR anonymisation/pseudonymisation take the NCIT label as title with the GDPR wording as an alias - FBbi mappings on CT and RG become close_mappings and on GM broad_mappings; CFM and OCT remain exact - DICOMModalityEnum documents the NCIT-or-DCM meaning rule and the non-acquisition Modality values it excludes - DCM is registered in oak_config.yaml as a deliberately unvalidated prefix Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01JqKeWA6q8xdJhgpuzHE4Ym --- src/valuesets/schema/medical/dicom.yaml | 11 ++- .../medical/imaging_data_management.yaml | 9 +- .../medical/imaging_deidentification.yaml | 8 +- src/valuesets/validators/enum_evaluator.py | 90 +++++++++++++++++-- src/valuesets/validators/oak_config.yaml | 2 + tests/validators/test_enum_evaluator.py | 80 +++++++++++++++++ 6 files changed, 185 insertions(+), 15 deletions(-) diff --git a/src/valuesets/schema/medical/dicom.yaml b/src/valuesets/schema/medical/dicom.yaml index 0c54d53e..1927d675 100644 --- a/src/valuesets/schema/medical/dicom.yaml +++ b/src/valuesets/schema/medical/dicom.yaml @@ -55,8 +55,12 @@ enums: Acquisition modality codes from DICOM PS3.16 Context ID 29 (Acquisition Modality), the defined terms for the Modality (0008,0060) attribute. The permissible value is the DICOM code itself; the title is the NCI Thesaurus label where a mapping exists and the - DICOM code meaning is carried as an alias when it differs. Waveform modalities (ECG, - EEG, etc.) are defined in CID 34 and are not included here. + DICOM code meaning is carried as an alias when it differs. Where an NCI Thesaurus term + exists it is the meaning and the DCM code is an exact mapping; otherwise the DCM code is + the meaning, so every value carries its DCM code in one of the two fields. Waveform + modalities (ECG, EEG, etc.) are defined in CID 34 and are not included here, nor are + the non-acquisition values that also appear in Modality (0008,0060) such as SR, PR, SEG, + KO, DOC, OT and the RT objects. status: DRAFT contributors: - orcid:0000-0002-6601-2165 @@ -94,6 +98,7 @@ enums: meaning: NCIT:C17204 exact_mappings: - DCM:CT + close_mappings: - FBbi:00001002 CFM: title: Confocal Microscopy @@ -136,6 +141,7 @@ enums: meaning: NCIT:C16853 exact_mappings: - DCM:GM + broad_mappings: - FBbi:00000241 aliases: - General Microscopy @@ -274,6 +280,7 @@ enums: meaning: NCIT:C38101 exact_mappings: - DCM:RG + close_mappings: - FBbi:00001001 aliases: - Radiographic imaging diff --git a/src/valuesets/schema/medical/imaging_data_management.yaml b/src/valuesets/schema/medical/imaging_data_management.yaml index 69cd4ff5..e5141fa7 100644 --- a/src/valuesets/schema/medical/imaging_data_management.yaml +++ b/src/valuesets/schema/medical/imaging_data_management.yaml @@ -54,13 +54,15 @@ enums: DATA_EXTRACTION_AND_METADATA_CHARACTERIZATION: title: Data extraction and metadata characterization description: Extraction of imaging data from clinical repositories such as PACS, preceded by a landscape assessment of source locations, database systems, modalities and acquisition devices - meaning: NCIT:C44285 + broad_mappings: + - NCIT:C44285 annotations: stage_number: 1 FILE_INTEGRITY_VERIFICATION: title: File integrity verification description: Computation and periodic re-verification of cryptographic checksums such as SHA-256 to detect silent corruption from network errors, media decay or system failure - meaning: NCIT:C142477 + broad_mappings: + - NCIT:C142477 annotations: stage_number: 2 DATA_COMPLETENESS_AND_CONFORMANCE_CHECKS: @@ -71,7 +73,8 @@ enums: METADATA_TAG_VALIDATION: title: Metadata tag validation description: Validation of DICOM attributes against the standard's VR, VM and type rules and against biological plausibility, and inventory of private tags - meaning: NCIT:C142500 + broad_mappings: + - NCIT:C142500 annotations: stage_number: 4 IMAGE_QUALITY_AND_PIXEL_DATA_VALIDATION: diff --git a/src/valuesets/schema/medical/imaging_deidentification.yaml b/src/valuesets/schema/medical/imaging_deidentification.yaml index 888ba444..a373a1cd 100644 --- a/src/valuesets/schema/medical/imaging_deidentification.yaml +++ b/src/valuesets/schema/medical/imaging_deidentification.yaml @@ -260,15 +260,19 @@ enums: jurisdiction: United States citation: 45 CFR 164.514(b)(1) GDPR_ANONYMISATION: - title: GDPR anonymisation + title: Anonymization description: Irreversible processing such that the data subject is no longer identifiable, taking the data outside the scope of the EU General Data Protection Regulation meaning: NCIT:C142392 + aliases: + - GDPR anonymisation annotations: jurisdiction: European Union GDPR_PSEUDONYMISATION: - title: GDPR pseudonymisation + title: Pseudonymization description: Processing so that data can no longer be attributed to a subject without additional information kept separately, as defined in GDPR Article 4(5) meaning: NCIT:C142654 + aliases: + - GDPR pseudonymisation annotations: jurisdiction: European Union DICOM_PS3_15_CONFIDENTIALITY_PROFILE: diff --git a/src/valuesets/validators/enum_evaluator.py b/src/valuesets/validators/enum_evaluator.py index 3a316d24..7b50aa06 100644 --- a/src/valuesets/validators/enum_evaluator.py +++ b/src/valuesets/validators/enum_evaluator.py @@ -2,7 +2,9 @@ Enum evaluator for validating ontology mappings in LinkML schemas. This module validates that ontology term mappings (meanings) in enum definitions -match the expected labels from the ontology. +match the expected labels from the ontology, and that terms referenced from +mapping slots (exact/close/broad/narrow/related_mappings) and from the +reachable_from.source_nodes of dynamic enums resolve in their ontology. Uses OAK (Ontology Access Kit) as the abstraction layer for all ontology access. """ @@ -416,24 +418,91 @@ def extract_aliases(self, pv: PermissibleValue, value_name: str) -> Set[str]: return aliases + MAPPING_SLOTS = ( + "exact_mappings", + "close_mappings", + "broad_mappings", + "narrow_mappings", + "related_mappings", + ) + + def _is_skipped_prefix(self, curie: str) -> bool: + """True if the CURIE's prefix is configured with an empty adapter (deliberately unvalidated).""" + prefix = curie.split(":")[0] if ":" in curie else None + return bool(prefix and prefix.lower() in self._oak_config and not self._oak_config[prefix.lower()]) + + def _check_resolvable(self, curie: str, enum_name: str, value_name: str, + context: str) -> Optional[ValidationIssue]: + """ + Check that a CURIE resolves to a label in its ontology. + + Used for mapping slots and reachable_from source nodes, where only + existence of the term is required (no label match against the value). + Returns an issue if the term cannot be resolved, else None. + """ + if self._is_skipped_prefix(curie): + logger.debug(f"Skipping validation for {curie} (empty adapter string in config)") + return None + if self.get_ontology_label(curie) is not None: + return None + prefix = curie.split(":")[0] if ":" in curie else None + if prefix and self.is_prefix_configured(prefix): + severity = "ERROR" + message = f"Could not retrieve label for configured ontology term {curie} in {context}" + else: + severity = "INFO" + message = f"Could not retrieve label for {curie} in {context}" + return ValidationIssue( + enum_name=enum_name, + value_name=value_name, + severity=severity, + message=message, + meaning=curie + ) + + def validate_reachable_from(self, enum_def: EnumDefinition, enum_name: str) -> List[ValidationIssue]: + """Validate that every reachable_from source node of a dynamic enum resolves.""" + issues = [] + rq = enum_def.reachable_from + if not rq or not rq.source_nodes: + return issues + for node in rq.source_nodes: + issue = self._check_resolvable(str(node), enum_name, "", "reachable_from.source_nodes") + if issue: + issues.append(issue) + return issues + def validate_enum(self, enum_def: EnumDefinition, enum_name: str) -> List[ValidationIssue]: """ Validate a single enum definition. + + Checks three kinds of ontology reference: + - ``meaning`` on each permissible value: must resolve and its label must + match the value name, title or an alias + - mapping slots (exact/close/broad/narrow/related_mappings) on each + permissible value: must resolve + - ``reachable_from.source_nodes`` on dynamic enums: must resolve """ - issues = [] + issues = self.validate_reachable_from(enum_def, enum_name) if not enum_def.permissible_values: return issues for value_name, pv in enum_def.permissible_values.items(): + # Mapping slots only need to resolve + for slot in self.MAPPING_SLOTS: + for curie in (getattr(pv, slot, None) or []): + issue = self._check_resolvable(str(curie), enum_name, value_name, slot) + if issue: + issues.append(issue) + # Check if there's a meaning (ontology mapping) meaning = pv.meaning if not meaning: continue # Check if this prefix has an empty adapter string (skip validation) - prefix = meaning.split(":")[0] if ":" in meaning else None - if prefix and prefix.lower() in self._oak_config and not self._oak_config[prefix.lower()]: + if self._is_skipped_prefix(meaning): logger.debug(f"Skipping validation for {meaning} (empty adapter string in config)") continue @@ -500,17 +569,22 @@ def validate_schema(self, schema_path: Path) -> ValidationResult: for enum_name, enum_def in sv.all_enums().items(): result.total_enums_checked += 1 + if enum_def.reachable_from and enum_def.reachable_from.source_nodes: + result.total_mappings_checked += len(enum_def.reachable_from.source_nodes) + if enum_def.permissible_values: result.total_values_checked += len(enum_def.permissible_values) - # Count mappings + # Count mappings (meaning plus mapping slots) for pv in enum_def.permissible_values.values(): if pv.meaning: result.total_mappings_checked += 1 + for slot in self.MAPPING_SLOTS: + result.total_mappings_checked += len(getattr(pv, slot, None) or []) - # Validate the enum - issues = self.validate_enum(enum_def, enum_name) - result.issues.extend(issues) + # Validate the enum (dynamic enums are checked even without permissible values) + issues = self.validate_enum(enum_def, enum_name) + result.issues.extend(issues) except Exception as e: logger.error(f"Error validating schema {schema_path}: {e}") diff --git a/src/valuesets/validators/oak_config.yaml b/src/valuesets/validators/oak_config.yaml index 64b3e3c8..22e8ad11 100644 --- a/src/valuesets/validators/oak_config.yaml +++ b/src/valuesets/validators/oak_config.yaml @@ -24,6 +24,8 @@ ontology_adapters: GENO: sqlite:obo:geno CHMO: sqlite:obo:chmo FBbi: sqlite:obo:fbbi + # DICOM controlled terminology (DCM) has no OAK adapter; codes are checked by hand against PS3.16 + DCM: # Data format and computation ontologies EDAM: sqlite:obo:edam diff --git a/tests/validators/test_enum_evaluator.py b/tests/validators/test_enum_evaluator.py index 5aec4b29..fbaafdd7 100644 --- a/tests/validators/test_enum_evaluator.py +++ b/tests/validators/test_enum_evaluator.py @@ -211,3 +211,83 @@ def test_cache_behavior(): label1 = evaluator.get_ontology_label("TEST:123") label2 = evaluator.get_ontology_label("TEST:123") assert mock_adapter.label.call_count == 2 # Called twice + + +def _evaluator_with_mock(tmp_path, labels): + """Evaluator whose NCIT adapter is a mock returning labels from the given dict.""" + config = ValidationConfig(oak_adapter_string="dummy:", cache_dir=tmp_path / "cache") + evaluator = EnumEvaluator(config=config) + mock_adapter = Mock() + mock_adapter.label = Mock(side_effect=lambda curie: labels.get(curie)) + # NCIT is a configured (strict) prefix in oak_config.yaml; inject before any lookup + evaluator._per_prefix_adapters['ncit'] = mock_adapter + evaluator._prefix_caches['ncit'] = {} + evaluator._per_prefix_adapters['_default'] = mock_adapter + return evaluator + + +def test_validate_reachable_from_source_nodes(tmp_path): + """Dynamic enums have their source nodes resolved even with no permissible values.""" + from linkml_runtime.linkml_model.meta import EnumDefinition, ReachabilityQuery + + evaluator = _evaluator_with_mock(tmp_path, {"NCIT:C17204": "Computed Tomography"}) + + good = EnumDefinition( + name="GoodDynamic", + reachable_from=ReachabilityQuery(source_nodes=["NCIT:C17204"]), + ) + assert evaluator.validate_enum(good, "GoodDynamic") == [] + + bad = EnumDefinition( + name="BadDynamic", + reachable_from=ReachabilityQuery(source_nodes=["NCIT:C999999999"]), + ) + issues = evaluator.validate_enum(bad, "BadDynamic") + assert len(issues) == 1 + assert issues[0].severity == "ERROR" + assert issues[0].value_name == "" + assert issues[0].meaning == "NCIT:C999999999" + + +def test_validate_mapping_slots(tmp_path): + """exact/close/broad mappings must resolve; a resolvable one needs no label match.""" + from linkml_runtime.linkml_model import EnumDefinition, PermissibleValue + + evaluator = _evaluator_with_mock(tmp_path, { + "NCIT:C17204": "Computed Tomography", + "NCIT:C38101": "X-Ray Imaging", + }) + + enum_def = EnumDefinition( + name="Modality", + permissible_values={ + "CT": PermissibleValue( + text="CT", + title="Computed Tomography", + meaning="NCIT:C17204", + close_mappings=["NCIT:C38101"], # label differs from title: allowed for mappings + exact_mappings=["NCIT:C999999999"], # does not resolve + ), + }, + ) + issues = evaluator.validate_enum(enum_def, "Modality") + assert len(issues) == 1 + assert issues[0].severity == "ERROR" + assert issues[0].meaning == "NCIT:C999999999" + assert "exact_mappings" in issues[0].message + + +def test_unconfigured_prefix_mapping_is_info(tmp_path): + """Unresolvable mappings with an unconfigured prefix are INFO, not ERROR.""" + from linkml_runtime.linkml_model import EnumDefinition, PermissibleValue + + evaluator = _evaluator_with_mock(tmp_path, {}) + enum_def = EnumDefinition( + name="Tools", + permissible_values={ + "X": PermissibleValue(text="X", exact_mappings=["TESTONT:1"]), + }, + ) + issues = evaluator.validate_enum(enum_def, "Tools") + assert len(issues) == 1 + assert issues[0].severity == "INFO"