From ad6e9aecb06313802634f9d86414090848197d78 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Fri, 4 Sep 2026 14:28:39 +0100 Subject: [PATCH 01/13] Start documenting our nexus files... --- doc/Nexus.md | 200 ++++++++++++++++++ doc/conf.py | 2 + doc/index.md | 3 +- doc/nexus/nexus_instrument.md | 34 +++ .../nexus_instrument/nexus_instrument_dae.md | 54 +++++ .../nexus_instrument_dae_vetos.md | 13 ++ .../nexus_instrument_moderator.md | 13 ++ .../nexus_instrument_source.md | 22 ++ doc/nexus/nexus_measurement.md | 30 +++ doc/nexus/nexus_periods.md | 149 +++++++++++++ doc/nexus/nexus_sample.md | 49 +++++ doc/nexus/nexus_user.md | 14 ++ 12 files changed, 582 insertions(+), 1 deletion(-) create mode 100644 doc/Nexus.md create mode 100644 doc/nexus/nexus_instrument.md create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_dae.md create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_vetos.md create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_moderator.md create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_source.md create mode 100644 doc/nexus/nexus_measurement.md create mode 100644 doc/nexus/nexus_periods.md create mode 100644 doc/nexus/nexus_sample.md create mode 100644 doc/nexus/nexus_user.md diff --git a/doc/Nexus.md b/doc/Nexus.md new file mode 100644 index 0000000000..89f8bde0ba --- /dev/null +++ b/doc/Nexus.md @@ -0,0 +1,200 @@ +# NeXus files + +IBEX writes out scientific data as {external+nexus:doc}`NeXus files `. The NeXus format documents the *general* +layout of a NeXus file; this documentation documents the *specific* datasets written in practice by IBEX, +including any quirks or facility-specific details of their interpretation. + +The NeXus files conform, where possible, to the {external+nexus:doc}`TOFRaw` definition. + +## Data types + +Except where otherwise noted: +- Strings are ASCII-encoded, null terminated. +- Little-endian datatypes are used. +- 32-bit signed integers or 32-bit floats are used + +{#nexus_raw_data_1} +## `raw_data_1` + +An ISIS NeXus file always contains a top-level `raw_data_1` group, which is an {external+nexus_manual:doc}`classes/base_classes/NXentry`. +It may contain the datasets and groups listed below: + +{#nexus_definition} +### `raw_data_1/definition` + +**On a Neutron instrument**: ASCII string containing `TOFRAW`. See {external+nexus_manual:doc}`classes/applications/NXtofraw` for the application +definition this references. + +Attributes: +- `version`: `"1.0"` +- `url`: `"http://definition.nexusformat.org/instruments/TOFRAW?version=1.0"` + +**On a Muon instrument**: ASCII string containing `pulsedTD`. + +Attributes: +- `version`: `"2.0"` +- `url`: `"http://definition.nexusformat.org/instruments/pulsedTD?version=2.0"` + +{#nexus_definition_local} +### `raw_data_1/definition_local` + +`"ISISTOFRAW"`. See {external+nexus_manual:doc}`classes/applications/NXtofraw` for the application +definition this references. + +Attributes: +- `version`: `"1.0"` +- `url`: `"http://svn.isis.rl.ac.uk/instruments/ISISTOFRAW?version=1.0"` + +**This dataset is not written on a Muon instrument**. + +{#nexus_program_name} +### `raw_data_1/program_name` + +`"ISISICP.EXE"` on a DAE2/DAE3 instrument, or the name of the file-writer executable on a data-streaming instrument. + +{#nexus_run_number} +### `raw_data_1/run_number` + +The run number, as a 32-bit signed integer. For example, for run `POLREF12345`, the run number is `12345`. + +{#nexus_run_cycle} +### `raw_data_1/run_cycle` + +The ISIS cycle identifier, as a string. For example `24_5` for cycle 2024/05. + +{#nexus_title} +### `raw_data_1/title` + +The user-provided run title as a string. + +For example `"Quiet Counts after DAE fix 24/5"`. + +{#nexus_notes} +### `raw_data_1/notes` + +The user-provided notes as a string. Commonly empty and written as a zero-length string in that case. Set in IBEX via +Experiment Details -> Sample -> Comments. + +{#nexus_name} +### `raw_data_1/name` + +The instrument name, for example `POLREF` or `MERLIN`. + +Attributes: +- `short_name`: a shortened, typically 3-character, representation of the instrument name. For example, `MERLIN`'s short +name is `MER`. + +{#nexus_beamline} +### `raw_data_1/beamline` + +The instrument name, for example `POLREF` or `MERLIN`. The instrument name is identical to the {ref}`nexus_name` dataset. +Unlike {ref}`nexus_name`, this dataset does **not** contain a `short_name` attribute. + +{#nexus_start_time} +### `raw_data_1/start_time` + +The time when data collection started, as an ISO8601 string. For example `2025-03-27T10:56:13`. + +Attributes: +- `units`: `"ISO8601"` + +{#nexus_end_time} +### `raw_data_1/end_time` + +The time when data collection ended, as an ISO8601 string. For example `2025-03-27T10:56:13`. + +Attributes: +- `units`: `"ISO8601"` + +{#nexus_duration} +### `raw_data_1/duration` + +The duration of the measurement, as a float32. Identical to {ref}`nexus_collection_time`. This is not necessarily equal to the difference between {ref}`nexus_start_time` and +{ref}`nexus_end_time`, because time spent paused or under run control is not included. + +Attributes: +- `units`: `"second"` + +{#nexus_collection_time} +### `raw_data_1/collection_time` + +The duration of the measurement, as a float32. Identical to {ref}`nexus_duration`. This is not necessarily equal to the difference between {ref}`nexus_start_time` and +{ref}`nexus_end_time`, because time spent paused or under run control is not included. + +Attributes: +- `units`: `"second"` + +{#nexus_proton_charge} +### `raw_data_1/proton_charge` + +Accumulated "good" proton charge (i.e. the proton charge corresponding to {ref}`nexus_good_frames`) during the run, as a float32. + +Attributes: +- `units`: `"uAh"` + +{#nexus_proton_charge_raw} +### `raw_data_1/proton_charge_raw` + +Accumulated total proton charge (i.e. the proton charge corresponding to {ref}`nexus_raw_frames`) during the run, as a float32. + +Attributes: +- `units`: `"uAh"` + +{#nexus_good_frames} +### `raw_data_1/good_frames` + +Number of good frames during the run, as an int32. + +{#nexus_raw_frames} +### `raw_data_1/raw_frames` + +Number of good and vetoed frames during the run, as an int32. + +{#nexus_experiment_identifier} +### `raw_data_1/experiment_identifier` + +RB number of the run, formatted as a string. For example `"12345678"` + +{#nexus_seci_config} +### `raw_data_1/seci_config` + +This represents the **IBEX** configuration loaded at the time of the run. The name `seci_config` is historical, as +SECI was the previous control system used before IBEX. IBEX configuration names are user-specified, within a set of rules. +For example, `"Horizontal_HCM_with_Helmholtz_coils"` + + +### `raw_data_1/measurement_id` + +A copy of {ref}`nexus_measurement_id`. + +### `raw_data_1/measurement_subid` + +A copy of {ref}`nexus_measurement_subid`. + +### `raw_data_1/measurement_type` + +A copy of {ref}`nexus_measurement_type`. + +### `raw_data_1/measurement_first_run` + +A copy of {ref}`nexus_measurement_first_run`. + +### `raw_data_1/measurement_label` + +A copy of {ref}`nexus_measurement_label`. + +### `raw_data_1/script_name` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Script Name. + +--- + +The top-level {ref}`nexus_raw_data_1` {external+nexus_manual:doc}`classes/base_classes/NXentry` also contains the following groups, +which are documented on their own pages: + +```{toctree} +:glob: +:titlesonly: + +nexus/* +``` diff --git a/doc/conf.py b/doc/conf.py index f133f91d53..f7c007a5dc 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -117,4 +117,6 @@ # page layouts changed. "sysadmin": ("https://shadow.nd.rl.ac.uk/ibex_sysadmin_manual/", "sysadmin_manual.inv"), "mantid": ("https://docs.mantidproject.org/", None), + "nexus": ("https://www.nexusformat.org/", None), + "nexus_manual": ("https://manual.nexusformat.org", None), } diff --git a/doc/index.md b/doc/index.md index 7c25c38a82..0ad96f978f 100644 --- a/doc/index.md +++ b/doc/index.md @@ -17,13 +17,14 @@ Glossary ``` ```{toctree} -:caption: User Interfaces & Scripting +:caption: User Interfaces, files & Scripting :maxdepth: 1 :titlesonly: Client Script-Generator Scripting +Nexus Web ``` diff --git a/doc/nexus/nexus_instrument.md b/doc/nexus/nexus_instrument.md new file mode 100644 index 0000000000..978abca545 --- /dev/null +++ b/doc/nexus/nexus_instrument.md @@ -0,0 +1,34 @@ +# `raw_data_1/instrument` + +This is a {external+nexus_manual:doc}`classes/base_classes/NXinstrument`. + +{#nexus_instrument_name} +## `raw_data_1/instrument/name` + +The instrument name as a string. This contains the same data as {ref}`nexus_name`, including the `short_name` attribute. + +Attributes: +- `short_name`: a shortened, typically 3-character, representation of the instrument name. For example, MERLIN’s short name is MER. + +## `raw_data_1/instrument` - `instrument.xml` groups + +:::{caution} +TODO: Special logic here - MARI specific? +::: + +## `raw_data_1/instrument` - `instrument_components` groups + +:::{caution} +TODO: Special logic here - MARI specific? +::: + +--- + +The instrument group also contains the following groups: + +```{toctree} +:glob: +:titlesonly: + +nexus_instrument/* +``` diff --git a/doc/nexus/nexus_instrument/nexus_instrument_dae.md b/doc/nexus/nexus_instrument/nexus_instrument_dae.md new file mode 100644 index 0000000000..583f150581 --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_dae.md @@ -0,0 +1,54 @@ +# `raw_data_1/instrument/dae` + +This has an `NX_class` of `IXdae`; this is not a class specified by upstream NeXus definitions. + +{#nexus_instrument_dae_detector_table_file} +### `raw_data_1/instrument/dae/detector_table_file` + +This is a filepath to a 'detector table', from a location accessible by the data acquisition system. A detector table +specifies metadata, for example flight path length or two theta, for each detector pixel. + +For example, `"C:/Instrument/Settings/config/DEMO/configurations/tables/RCPTT_detector128.dat"` + +NeXus consumers should treat this as metadata, and should not assume that they will be able to read the file pointed at. + +{#nexus_instrument_dae_spectra_table_file} +### `raw_data_1/instrument/dae/spectra_table_file` + +This is a filepath to a 'spectra table', from a location accessible by the data acquisition system. A spectra table +defines the mapping between a physical hardware pixel, and a 'spectrum', which may represent one or more pixels grouped +together. + +For example, `"C:/Instrument/Settings/config/DEMO/configurations/tables/RCPTT_spectra128.dat"` + +NeXus consumers should treat this as metadata, and should not assume that they will be able to read the file pointed at. + +{#nexus_instrument_dae_wiring_table_file} +### `raw_data_1/instrument/dae/wiring_table_file` + +This is a filepath to a 'wiring table', from a location accessible by the data acquisition system. A wiring table +defines a mapping between an electronics channel/crate/card/location, to a pixel identifier. + +For example, `"C:/Instrument/Settings/config/DEMO/configurations/tables/RCPTT_wiring128.dat"` + +NeXus consumers should treat this as metadata, and should not assume that they will be able to read the file pointed at. + +{#nexus_instrument_dae_type} +### `raw_data_1/instrument/dae/type` + +A string identifying the type of data acquisition hardware in use; for example: +- `"ISIS_DAE3"` +- `"ISIS_DAE2"` + +Future acquisition systems may use different type identifiers; this list should not be treated as exhaustive. + +--- + +The `raw_data_1/instrument/dae` group also contains the following groups: + +```{toctree} +:glob: +:titlesonly: + +nexus_instrument_dae/* +``` diff --git a/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_vetos.md b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_vetos.md new file mode 100644 index 0000000000..576a31daa9 --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_vetos.md @@ -0,0 +1,13 @@ +{#nexus_instrument_dae_vetos} +# `raw_data_1/instrument/dae/vetos` + +This is an `IXvetos` class, which is a non-standard NeXus class. + +### `raw_data_1/instrument/dae/` + +There are one or more of these datasets, each describing a single veto called ``. + +The dataset contains the number of frames in the run vetoed by this specific veto, as an int32. + +Attributes: +- `enabled` (int32) - `1` if the veto was enabled, or `0` if it was disabled at the end of the run. Note that veto enablement may have changed mid-run. diff --git a/doc/nexus/nexus_instrument/nexus_instrument_moderator.md b/doc/nexus/nexus_instrument/nexus_instrument_moderator.md new file mode 100644 index 0000000000..9e5b54fe78 --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_moderator.md @@ -0,0 +1,13 @@ +# `raw_data_1/instrument/moderator` + +This is a {external+nexus_manual:doc}`classes/base_classes/NXmoderator`. + +## `raw_data_1/instrument/moderator/distance` + +This is written as the **negative** of the L1 flight path distance, as a float32. It is set in IBEX +under Experiment Details -> Beamline Parameters -> Primary Flight Path (L1) + +For an instrument with an L1 of 12.3 m, this dataset would be written as `-12.3`. + +Attributes: +- `units`: `"metre"` \ No newline at end of file diff --git a/doc/nexus/nexus_instrument/nexus_instrument_source.md b/doc/nexus/nexus_instrument/nexus_instrument_source.md new file mode 100644 index 0000000000..0849a1baaf --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_source.md @@ -0,0 +1,22 @@ +# `raw_data_1/instrument/source` + +This is a {external+nexus_manual:doc}`classes/base_classes/NXsource`. + +{#nexus_instrument_source_name} +## `raw_data_1/instrument/source/name` + +Always written as the string `"ISIS"`. + +{#nexus_instrument_source_type} +## `raw_data_1/instrument/source/type` + +**On a neutron instrument**: the string `"Pulsed Neutron Source"` + +**On a muon instrument**: the string `"Pulsed Muon Source"` + +{#nexus_instrument_source_probe} +## `raw_data_1/instrument/source/probe` + +**On a neutron instrument**: the string `"neutrons"` + +**On a muon instrument**: the string `"positive muons"` or `"negative muons"` diff --git a/doc/nexus/nexus_measurement.md b/doc/nexus/nexus_measurement.md new file mode 100644 index 0000000000..29364ae4a6 --- /dev/null +++ b/doc/nexus/nexus_measurement.md @@ -0,0 +1,30 @@ +# `raw_data_1/measurement` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXcollection` containing measurement-related information. + +{#nexus_measurement_id} +## `raw_data_1/measurement/id` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Measurement ID + +{#nexus_measurement_subid} +## `raw_data_1/measurement/subid` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Measurement SubID + +{#nexus_measurement_type} +## `raw_data_1/measurement/type` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Measurement Type + +{#nexus_measurement_first_run} +## `raw_data_1/measurement/first_run` + +:::{danger} +**TODO** - some logic in ISISICP for this... +::: + +{#nexus_measurement_label} +## `raw_data_1/measurement/label` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Measurement Label diff --git a/doc/nexus/nexus_periods.md b/doc/nexus/nexus_periods.md new file mode 100644 index 0000000000..43f3edc756 --- /dev/null +++ b/doc/nexus/nexus_periods.md @@ -0,0 +1,149 @@ +# `raw_data_1/periods` + +This has an `NX_class` of `IXperiods`; this is not a class specified by upstream NeXus definitions. + +Periods are a way of segregating a run into discrete subcomponents during data acquisition. The run may switch between +periods using a number of hardware or software mechanisms, and may return to a previous period later. + +For example, periods may represent spin-up and spin-down measurements, or may be used to separate points during the scan +of a motion axis. + +{#nexus_periods_number} +## `raw_data_1/periods/number` + +The number of periods configured in the data-acquisition system for this measurement, as an int32. + +{#nexus_periods_highest_used} +## `raw_data_1/periods/highest_used` + +The maximum period number into which the data acquisition system collected data, as an int32. This is 1-indexed. + +For example: +- If five periods were configured and all were used during the data acquisition, `highest_used` would be `5`. +- If five periods were configured, but only the first period was used, `highest_used` would be `1`. + +{#nexus_periods_total_counts} +## `raw_data_1/periods/total_counts` + +The total number of events counted into each period. This is an array of float32, in units of `Mev` (millions of events). + +:::{caution} +The array does **NOT** necessarily have length {ref}`nexus_periods_number`; it instead has a length corresponding to +the number of configured DAQ periods. This may be smaller than or equal to {ref}`nexus_periods_number`. Dwell periods +are omitted from this array. +::: + +Attributes: +- `units`: `Mev` + +{#nexus_periods_good_frames} +## `raw_data_1/periods/good_frames` + +The total number of good frames counted into each period. This is an array of int32. + +The array has length {ref}`nexus_periods_number` + +{#nexus_periods_raw_frames} +## `raw_data_1/periods/raw_frames` + +The total number of raw frames (including both good and vetoed frames) counted into each period. This is an array of int32. + +The array has length {ref}`nexus_periods_number` + +{#nexus_periods_proton_charge} +## `raw_data_1/periods/proton_charge` + +The 'good' proton charge (corresponding to {ref}`nexus_periods_good_frames`) counted into each period. This is an array of float32. + +The array has length {ref}`nexus_periods_number` + +Attributes: +- `units`: `uAh` + +{#nexus_periods_proton_charge_raw} +## `raw_data_1/periods/proton_charge_raw` + +The 'raw' proton charge (corresponding to {ref}`nexus_periods_raw_frames`) counted into each period. This is an array of float32. + +The array has length {ref}`nexus_periods_number` + +Attributes: +- `units`: `uAh` + +{#nexus_periods_good_frames_daq} +## `raw_data_1/periods/good_frames_daq` + +The total number of good frames counted into each DAQ period. This is an array of int32. + +:::{caution} +The array does **NOT** necessarily have length {ref}`nexus_periods_number`; it instead has a length corresponding to +the number of configured DAQ periods. This may be smaller than or equal to {ref}`nexus_periods_number`. Dwell periods +are omitted from this array. +::: + +{#nexus_periods_sequences} +## `raw_data_1/periods/sequences` + +The total number of period sequences completed. This is an array of int32. + +**In software period mode**, this means the number of times each period was changed into. Beginning a run counts as changing +into the initial period. + +**In hardware period mode**, this means the number of 'period sequences' - loops of the period sequence - that have completed. +If the run was allowed to reach its maximum configured hardware period sequences, the number of sequences completed will +be that number plus one. + +The array has length {ref}`nexus_periods_number` + +Examples: +- `[17, 17, 17, 17]`, for a run with 4 hardware periods (DAQ, Dwell, DAQ, Dwell) and configured to run for 16 hardware +period sequences. +- `[1, 3, 0, 0]`, for running in software-period mode with 4 periods and the control sequence: +``` +g.begin() +g.pause() +g.change_period(2) +g.change_period(2) +g.change_period(2) +g.resume() +g.end() +``` + +{#nexus_periods_frames_requested} +## `raw_data_1/periods/frames_requested` + +The number of frames that were requested in each period, as an array of int32. **This is only meaningful in hardware period mode**; in software +period mode this may contain arbitrary, non-meaningful, data. + +The array has length {ref}`nexus_periods_number` + +{#nexus_periods_type} +## `raw_data_1/periods/type` + +The number of frames that were requested in each period, as an array of int32. **This is only meaningful in hardware period mode**; in software +period mode this may contain arbitrary, non-meaningful, data. + +The period types are: +- `0`: Unused +- `1`: DAQ +- `2`: Dwell + +The array has length {ref}`nexus_periods_number` + +{#nexus_periods_output} +## `raw_data_1/periods/output` + +The binary output configured for each hardware period, as an array of int32. **This is only meaningful in hardware period mode**; in software +period mode this may contain arbitrary, non-meaningful, data. + +The array has length {ref}`nexus_periods_number` + +{#nexus_periods_labels} +## `raw_data_1/periods/labels` + +This is the string 'label' assigned to each hardware period, concatenated together with a `;` character into a single string. +**This is only meaningful in hardware period mode**; in software period mode this may contain arbitrary, non-meaningful, data. + +Examples: +- `";;;"` for a run with 4 periods, each having the empty string as a label. +- `"label1;label2;label3;label4"`, for a run with 4 periods, each with a non-empty label. diff --git a/doc/nexus/nexus_sample.md b/doc/nexus/nexus_sample.md new file mode 100644 index 0000000000..5662170aae --- /dev/null +++ b/doc/nexus/nexus_sample.md @@ -0,0 +1,49 @@ +# `raw_data_1/sample` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXsample`. + +{#nexus_sample_name} +## `raw_data_1/sample/name` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Name + +{#nexus_sample_distance} +## `raw_data_1/sample/distance` + +This is unconditionally written as `0.0`, as a float64. + +{#nexus_sample_id} +## `raw_data_1/sample/id` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample ID + +{#nexus_sample_thickness} +## `raw_data_1/sample/thickness` + +This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Thickness + +It is in units of `mm`. + +{#nexus_sample_width} +## `raw_data_1/sample/width` + +This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Width + +It is in units of `mm`. + +{#nexus_sample_height} +## `raw_data_1/sample/height` + +This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Height + +It is in units of `mm`. + +{#nexus_sample_type} +## `raw_data_1/sample/type` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Type + +{#nexus_sample_shape} +## `raw_data_1/sample/shape` + +This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Geometry diff --git a/doc/nexus/nexus_user.md b/doc/nexus/nexus_user.md new file mode 100644 index 0000000000..51ad4f55c5 --- /dev/null +++ b/doc/nexus/nexus_user.md @@ -0,0 +1,14 @@ +# `raw_data_1/user_1` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXuser` group, containing the following data: + +{#nexus_user_affiliation} +## `raw_data_1/user_1/affiliation` + +In principle, this represents the affiliations (institutes) of the beamline users specified in {ref}`nexus_user_name`. +However, this is not currently populated by IBEX and the ICP, and so is written as an empty string. + +{#nexus_user_name} +## `raw_data_1/user_1/name` + +Users of the beamline. This is a comma-separated list of surnames, as a single string. For example `"Bloggs,Smith,Jones,Williams"`. From 90f3fd81b6bdd1b0d0de1331eb3d610881393ad2 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Sat, 5 Sep 2026 09:34:39 +0100 Subject: [PATCH 02/13] Add warnings about where we write invalid NeXus files --- doc/Nexus.md | 4 ++-- .../nexus_instrument_dae_time_channels.md | 11 +++++++++++ doc/nexus/nexus_instrument/nexus_instrument_source.md | 8 ++++++++ doc/nexus/nexus_periods.md | 2 +- doc/nexus/nexus_sample.md | 5 +++++ 5 files changed, 27 insertions(+), 3 deletions(-) create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md diff --git a/doc/Nexus.md b/doc/Nexus.md index 89f8bde0ba..bfb54fcad3 100644 --- a/doc/Nexus.md +++ b/doc/Nexus.md @@ -1,4 +1,4 @@ -# NeXus files +# NeXus Files IBEX writes out scientific data as {external+nexus:doc}`NeXus files `. The NeXus format documents the *general* layout of a NeXus file; this documentation documents the *specific* datasets written in practice by IBEX, @@ -10,7 +10,7 @@ The NeXus files conform, where possible, to the {external+nexus:doc}`TOFRaw` def Except where otherwise noted: - Strings are ASCII-encoded, null terminated. -- Little-endian datatypes are used. +- Little-endian data types are used. - 32-bit signed integers or 32-bit floats are used {#nexus_raw_data_1} diff --git a/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md new file mode 100644 index 0000000000..1c21851715 --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md @@ -0,0 +1,11 @@ +# `raw_data_1/instrument/dae/time_channels_` + +This is an `IXtime_channels` class, which is a non-standard NeXus class. + +### `raw_data_1/instrument/dae/time_channels_/time_of_flight` + +Bin-edges used by histogramming for time channel set N, as a float32. This histogramming may have been performed either in hardware or +in software. + +Attributes: +- `units`: `"microsecond"` for a neutron instrument, diff --git a/doc/nexus/nexus_instrument/nexus_instrument_source.md b/doc/nexus/nexus_instrument/nexus_instrument_source.md index 0849a1baaf..b61b4a45b0 100644 --- a/doc/nexus/nexus_instrument/nexus_instrument_source.md +++ b/doc/nexus/nexus_instrument/nexus_instrument_source.md @@ -14,9 +14,17 @@ Always written as the string `"ISIS"`. **On a muon instrument**: the string `"Pulsed Muon Source"` +:::{note} +The string "Pulsed Neutron Source" differs from the allowed names in {external+nexus_manual:doc}`classes/base_classes/NXsource`. +::: + {#nexus_instrument_source_probe} ## `raw_data_1/instrument/source/probe` **On a neutron instrument**: the string `"neutrons"` **On a muon instrument**: the string `"positive muons"` or `"negative muons"` + +:::{note} +These strings differ from the allowed names in {external+nexus_manual:doc}`classes/base_classes/NXsource`. +::: diff --git a/doc/nexus/nexus_periods.md b/doc/nexus/nexus_periods.md index 43f3edc756..8ad2b89b09 100644 --- a/doc/nexus/nexus_periods.md +++ b/doc/nexus/nexus_periods.md @@ -2,7 +2,7 @@ This has an `NX_class` of `IXperiods`; this is not a class specified by upstream NeXus definitions. -Periods are a way of segregating a run into discrete subcomponents during data acquisition. The run may switch between +Periods are a way of segregating a run into discrete subsections during data acquisition. The run may switch between periods using a number of hardware or software mechanisms, and may return to a previous period later. For example, periods may represent spin-up and spin-down measurements, or may be used to separate points during the scan diff --git a/doc/nexus/nexus_sample.md b/doc/nexus/nexus_sample.md index 5662170aae..95bea0760d 100644 --- a/doc/nexus/nexus_sample.md +++ b/doc/nexus/nexus_sample.md @@ -43,6 +43,11 @@ It is in units of `mm`. This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Type +:::{note} +{external+nexus_manual:doc}`classes/base_classes/NXsample` defines that this can only be one of a defined set of values. +Our files impose no such restriction and will write the arbitrary user-specified string. +::: + {#nexus_sample_shape} ## `raw_data_1/sample/shape` From f1b38065c279adede63396244cc790da13ce6be3 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Mon, 7 Sep 2026 09:22:19 +0100 Subject: [PATCH 03/13] Add more notes about where we write invalid NeXus files --- doc/Nexus.md | 7 +++++-- doc/nexus/nexus_sample.md | 14 ++++++++++---- 2 files changed, 15 insertions(+), 6 deletions(-) diff --git a/doc/Nexus.md b/doc/Nexus.md index bfb54fcad3..c2923d662d 100644 --- a/doc/Nexus.md +++ b/doc/Nexus.md @@ -38,8 +38,7 @@ Attributes: {#nexus_definition_local} ### `raw_data_1/definition_local` -`"ISISTOFRAW"`. See {external+nexus_manual:doc}`classes/applications/NXtofraw` for the application -definition this references. +`"ISISTOFRAW"`. Attributes: - `version`: `"1.0"` @@ -112,6 +111,10 @@ Attributes: The duration of the measurement, as a float32. Identical to {ref}`nexus_collection_time`. This is not necessarily equal to the difference between {ref}`nexus_start_time` and {ref}`nexus_end_time`, because time spent paused or under run control is not included. +:::{note} +The datatype used by the ISISICP (float32) is incompatible with the type specified by the NeXus standard (`NX_INT`). +::: + Attributes: - `units`: `"second"` diff --git a/doc/nexus/nexus_sample.md b/doc/nexus/nexus_sample.md index 95bea0760d..690e7db436 100644 --- a/doc/nexus/nexus_sample.md +++ b/doc/nexus/nexus_sample.md @@ -10,7 +10,11 @@ This is user-settable string metadata. It is set in IBEX under Experiment Detail {#nexus_sample_distance} ## `raw_data_1/sample/distance` -This is unconditionally written as `0.0`, as a float64. +This is unconditionally written as `0.0`, **as a float64**. + +:::{note} +No units attribute is written by the ISISICP, which is invalid as per the NeXus standard. +::: {#nexus_sample_id} ## `raw_data_1/sample/id` @@ -22,21 +26,23 @@ This is user-settable string metadata. It is set in IBEX under Experiment Detail This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Thickness -It is in units of `mm`. +:::{note} +It is in units of `mm`, but no units attribute is written by the ISISICP, which is invalid as per the NeXus standard. +::: {#nexus_sample_width} ## `raw_data_1/sample/width` This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Width -It is in units of `mm`. +It is in units of `mm`, but no units attribute is written. {#nexus_sample_height} ## `raw_data_1/sample/height` This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Height -It is in units of `mm`. +It is in units of `mm`, but no units attribute is written. {#nexus_sample_type} ## `raw_data_1/sample/type` From fab84c968d034e7d7fc95132519161643c0a3ad5 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Mon, 7 Sep 2026 18:39:37 +0100 Subject: [PATCH 04/13] Add more nxs --- doc/nexus/nexus_detector.md | 19 ++++ .../nexus_instrument/nexus_instrument_dae.md | 8 ++ .../nexus_instrument_dae_time_channels.md | 29 ++++- .../nexus_instrument_detector.md | 103 ++++++++++++++++++ 4 files changed, 157 insertions(+), 2 deletions(-) create mode 100644 doc/nexus/nexus_detector.md create mode 100644 doc/nexus/nexus_instrument/nexus_instrument_detector.md diff --git a/doc/nexus/nexus_detector.md b/doc/nexus/nexus_detector.md new file mode 100644 index 0000000000..9439d13076 --- /dev/null +++ b/doc/nexus/nexus_detector.md @@ -0,0 +1,19 @@ +# `raw_data_1/detector_` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXdata`. + +### `raw_data_1/detector_/counts` + +This is a NeXus link to {ref}`nexus_instrument_detector_counts`. It is only written if histogramming is enabled. + +### `raw_data_1/detector_/period_index` + +This is a NeXus link to {ref}`nexus_instrument_dae_period_index`. + +### `raw_data_1/detector_/spectrum_index` + +This is a NeXus link to {ref}`nexus_instrument_detector_spectrum_index`. + +### `raw_data_1/detector_/time_of_flight` + +This is a NeXus link to {ref}`nexus_instrument_dae_time_channels_tof`. \ No newline at end of file diff --git a/doc/nexus/nexus_instrument/nexus_instrument_dae.md b/doc/nexus/nexus_instrument/nexus_instrument_dae.md index 583f150581..b34d75fd73 100644 --- a/doc/nexus/nexus_instrument/nexus_instrument_dae.md +++ b/doc/nexus/nexus_instrument/nexus_instrument_dae.md @@ -42,6 +42,14 @@ A string identifying the type of data acquisition hardware in use; for example: Future acquisition systems may use different type identifiers; this list should not be treated as exhaustive. +{#nexus_instrument_dae_period_index} +### `raw_data_1/instrument/dae/period_index` + +This is always an array with length equal to the number of DAQ periods, and values equal to `idx + 1`, as an int32. + +Examples: +- For a run in software period mode with 5 periods: `[1, 2, 3, 4, 5]` + --- The `raw_data_1/instrument/dae` group also contains the following groups: diff --git a/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md index 1c21851715..8cf1303a26 100644 --- a/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md +++ b/doc/nexus/nexus_instrument/nexus_instrument_dae/nexus_instrument_dae_time_channels.md @@ -2,10 +2,35 @@ This is an `IXtime_channels` class, which is a non-standard NeXus class. +{#nexus_instrument_dae_time_channels_tof} ### `raw_data_1/instrument/dae/time_channels_/time_of_flight` Bin-edges used by histogramming for time channel set N, as a float32. This histogramming may have been performed either in hardware or -in software. +in software. This is only written on a Neutron instrument. Attributes: -- `units`: `"microsecond"` for a neutron instrument, +- `units`: `"microsecond"` +- `axis`: `1` +- `primary`: `1` + +{#nexus_instrument_dae_time_channels_tofraw} +### `raw_data_1/instrument/dae/time_channels_/time_of_flight_raw` + +Bin-edges used by histogramming for time channel set N, as an int32. For ICP-written files, this represents a multiple of the DAE clock frequency, +and the underlying binning as passed to hardware. +This dataset is **not** necessarily in cycles of a 32 MHz clock as the `frequency` attribute would indicate. + +This is only written on a Neutron instrument. + +Attributes: +- `units`: `"pulses"` +- `frequency`: `"32 MHz"` + +{#nexus_instrument_dae_time_channels_rawtime} +### `raw_data_1/instrument/dae/time_channels_/raw_time` + +Bin-edges used by histogramming for time channel set N, as a float32. This histogramming may have been performed either in hardware or +in software. This is only written on a Muon instrument. + +Attributes: +- `units`: `"microsecond"` diff --git a/doc/nexus/nexus_instrument/nexus_instrument_detector.md b/doc/nexus/nexus_instrument/nexus_instrument_detector.md new file mode 100644 index 0000000000..c6e5985c59 --- /dev/null +++ b/doc/nexus/nexus_instrument/nexus_instrument_detector.md @@ -0,0 +1,103 @@ +# `raw_data_1/instrument/detector_` + +This is a {external+nexus_manual:doc}`classes/base_classes/NXdetector`. + +Spectra are grouped into different `detector_` groups based on the time-regime set they share. In other words, +`detector_N` contains data from all spectra configured to use time regime `N`. + +{#nexus_instrument_detector_counts} +### `raw_data_1/instrument/detector_/counts` + +This is the histogram data for detector N, as int32 counts. It has dimensions `periods,spectra,time_channels`, where periods is the number +of DAQ periods (which may be different from the *total* number of periods). + +This is written only if histograms were requested. + +Attributes: +- `units`: `"counts"` +- `signal`: `1` +- `axes`: + - On a muon instrument: `"period_index,spectrum_index,raw_time"` + - On a neutron instrument: `"period_index,spectrum_index,time_of_flight"` +- `long_name`: `"positron_counts"` (written on Muon instruments only) + +{#nexus_instrument_detector_spectrum_index} +### `raw_data_1/instrument/detector_/spectrum_index` + +Mapping of array position in the {ref}`histogram array ` to spectrum IDs, as an int32. + +### Muon-specific datasets + +:::{caution} +TODO +::: + +{#nexus_instrument_detector_period_index} +### `raw_data_1/instrument/detector_/period_index` + +This is a NeXus link to {ref}`nexus_instrument_dae_period_index`. + +{#nexus_instrument_detector_tof} +### `raw_data_1/instrument/detector_/time_of_flight` + +This is a NeXus link to {ref}`nexus_instrument_dae_time_channels_tof`. + +{#nexus_instrument_detector_tofraw} +### `raw_data_1/instrument/detector_/time_of_flight_raw` + +This is a NeXus link to {ref}`nexus_instrument_dae_time_channels_tofraw`. + +It is only written on Neutron instruments. + +{#nexus_instrument_detector_sourcedetdist} +### `raw_data_1/instrument/detector_/source_detector_distance` + +This is user-settable float32 metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Detector Distance + +:::{caution} +The parameter name in the NeXus file is the *source*-detector distance, whereas in IBEX it is called the *sample*-detector distance. +This is only consistent it one interprets the "source" of the scattered radiation as being the sample. +::: + +{#nexus_instrument_detector_polarangle} +### `raw_data_1/instrument/detector_/polar_angle` + +This is the average polar angle (two-theta) of a spectrum, as a float32. It is an average because a spectrum may be +made from multiple detector pixels. + +Attributes: +- `units`: `"degree"` + +{#nexus_instrument_detector_distance} +### `raw_data_1/instrument/detector_/distance` + +This is the L2 flight path of a spectrum, as a float32. It is an average because a spectrum may be +made from multiple detector pixels. + +Attributes: +- `units`: `"metre"` + +{#nexus_instrument_detector_delt} +### `raw_data_1/instrument/detector_/delt` + +This is the "hold-off time" of a spectrum, as a float32. It is an average because a spectrum may be +made from multiple detector pixels. It is in `us`, but a units attribute is not written. + +{#nexus_instrument_detector_azangle} +### `raw_data_1/instrument/detector_/azimuthal_angle` + +This is the azimuthal angle of a spectrum, as a float32. It is an average because a spectrum may be +made from multiple detector pixels. + +Attributes: +- `units`: `"degree"` + +{#nexus_instrument_detector_usertable} +### `raw_data_1/instrument/detector_/user_table` + +These are arbitrary user-specified tables for each detector, as a float32. It is an average because a spectrum may be +made from multiple detector pixels. + +User tables start from `user_table01`. + +`user_table01` is always equivalent to {ref}`nexus_instrument_detector_azangle`. From 40adbb5ebe70105fc1fc274b5c0a83157f05f69a Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Tue, 8 Sep 2026 11:20:52 +0100 Subject: [PATCH 05/13] NeXus monitor --- doc/nexus/nexus_monitor.md | 39 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) create mode 100644 doc/nexus/nexus_monitor.md diff --git a/doc/nexus/nexus_monitor.md b/doc/nexus/nexus_monitor.md new file mode 100644 index 0000000000..297c2132b5 --- /dev/null +++ b/doc/nexus/nexus_monitor.md @@ -0,0 +1,39 @@ +# `raw_data_1/monitor_` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXmonitor`. + +Monitors are numbered from 1; the groups are called `monitor_1`, `monitor_2`, and so on. + +{#nexus_monitor_data} +### `raw_data_1/monitor_/data` + +An int32 array with dimensions (number of DAQ periods, number of time channels) containing the histogrammed counts +for this monitor. + +Attributes: +- `axes`: + - `period_index,spectrum_index,raw_time` on a neutron instrument + - `period_index,spectrum_index,time_of_flight` on a muon instrument +- `long_name`: `"positron counts"` (only written on a muon instrument) + +{#nexus_monitor_monnum} +### `raw_data_1/monitor_/monitor_number` + +An int32 monitor number. + +The `raw_data_1/monitor_/monitor_number` dataset always has value `N`. + +{#nexus_monitor_period_index} +### `raw_data_1/monitor_/period_index` + +This is a NeXus link to {ref}`nexus_instrument_dae_period_index`. + +{#nexus_monitor_spectrum_index} +### `raw_data_1/monitor_/spectrum_index` + +This is a scalar int32 dataset describing which spectrum this monitor corresponds to. + +{#nexus_monitor_tof} +### `raw_data_1/monitor_/time_of_flight` + +This is a NeXus link to {ref}`nexus_instrument_dae_time_channels_tof`. From da451c8f729056d581722318a1ae4d7e7e667b64 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Wed, 9 Sep 2026 19:53:30 +0100 Subject: [PATCH 06/13] selog --- doc/nexus/nexus_detector_events.md | 7 ++ doc/nexus/nexus_runlog.md | 5 ++ doc/nexus/nexus_sample.md | 34 ++++++++ doc/nexus/nexus_selog.md | 86 +++++++++++++++++++ doc/nexus/nexus_selog/nexus_selog_valuelog.md | 63 ++++++++++++++ doc/spelling_wordlist.txt | 2 + 6 files changed, 197 insertions(+) create mode 100644 doc/nexus/nexus_detector_events.md create mode 100644 doc/nexus/nexus_runlog.md create mode 100644 doc/nexus/nexus_selog.md create mode 100644 doc/nexus/nexus_selog/nexus_selog_valuelog.md diff --git a/doc/nexus/nexus_detector_events.md b/doc/nexus/nexus_detector_events.md new file mode 100644 index 0000000000..9c7cfc8139 --- /dev/null +++ b/doc/nexus/nexus_detector_events.md @@ -0,0 +1,7 @@ +# `raw_data_1/detector__events` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +:::{caution} +TODO +::: diff --git a/doc/nexus/nexus_runlog.md b/doc/nexus/nexus_runlog.md new file mode 100644 index 0000000000..3f45b69c1b --- /dev/null +++ b/doc/nexus/nexus_runlog.md @@ -0,0 +1,5 @@ +# `raw_data_1/runlog` + +This has an `NX_class` of `IXrunlog`; this is not a class specified by upstream NeXus definitions. + + diff --git a/doc/nexus/nexus_sample.md b/doc/nexus/nexus_sample.md index 690e7db436..6bf8703b9f 100644 --- a/doc/nexus/nexus_sample.md +++ b/doc/nexus/nexus_sample.md @@ -58,3 +58,37 @@ Our files impose no such restriction and will write the arbitrary user-specified ## `raw_data_1/sample/shape` This is user-settable string metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Sample Geometry + +{#nexus_sample_temperature} +## `raw_data_1/sample/temperature` + +This is user-settable metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Temperature Label as a string, +but written to the NeXus file as a float32. + +This dataset is only written on a muon instrument. + +Attributes: +- `units`: `"Kelvin"` + +{#nexus_sample_magneticfield} +## `raw_data_1/sample/magnetic_field` + +This is user-settable metadata. It is set in IBEX under Experiment Details -> Sample Parameters -> Magnetic Field Label as a string, +but written to the NeXus file as a float32. + +This dataset is only written on a muon instrument. + +Attributes: +- `units`: `"Gauss"` + +{#nexus_sample_temperaturelog} +## `raw_data_1/sample/temperature_log` + +If this is a muon instrument **and** a block named exactly `temp_sample` exists, this is a NeXus link to the +{ref}`raw_data_1/selog/temp_sample ` group. + +{#nexus_sample_fieldlog} +## `raw_data_1/sample/magnetic_field_log` + +If this is a muon instrument **and** a block named exactly `field_sample` exists, this is a NeXus link to the +{ref}`raw_data_1/selog/field_sample ` group. diff --git a/doc/nexus/nexus_selog.md b/doc/nexus/nexus_selog.md new file mode 100644 index 0000000000..c30f44b3ce --- /dev/null +++ b/doc/nexus/nexus_selog.md @@ -0,0 +1,86 @@ +{#nexus_selog} +# `raw_data_1/selog` + +This has an `NX_class` of `IXselog`; this is not a class specified by upstream NeXus definitions. Each block is +a group with an `NX_class` of `IXseblock`, which is also not a standard NeXus class. + +The list of blocks to be written is defined by the IBEX configuration, and is user-editable in IBEX. + +In addition to the list of blocks from IBEX, the following logs are written: +- `EPICS_PUTLOG`: a string log containing the EPICS channel access put log, with each put operation as a string. +- `ICP_DAE_TD`: TODO +- `IDP_SYS_TD`: TODO + +Each `IXseblock` contains the following datasets: + +{#nexus_selog_viname} +### `raw_data_1/selog//vi_name` + +Under IBEX, this is always written as an empty string. It was previously used to refer to the +filepath of a LabVIEW VI responsible for serving this data. + +{#nexus_selog_readcontrol} +### `raw_data_1/selog//read_control` + +Under IBEX, this is always written as an empty string. It was previously used to refer to the +path to this data within a LabVIEW VI. + +{#nexus_selog_setcontrol} +### `raw_data_1/selog//set_control` + +Under IBEX, this is always written as an empty string. It was previously used to refer to the +path to this data within a LabVIEW VI. + +{#nexus_selog_value} +### `raw_data_1/selog//value` + +The latest value of the block **in each period**, as an array. +This is written with a datatype corresponding to the underlying EPICS type. + +For example, the array `[12.34, 56.78]` means that the most recent value while the instrument was in period 1 was `12.34`, +and the most recent value while the instrument was in period 2 was `56.78` + +Attributes: +- `units`: Engineering units of the block, as per EPICS `.EGU` field. + +{#nexus_selog_valuespread} +### `raw_data_1/selog//value_spread` + +The difference between the minimum recorded value and maximum recorded value for a block **in each period**, as an array. +This dataset is only written if the underlying EPICS type is numeric. + +For example, the array `[1.23, 0]` means that: +- The difference between the min and max values recorded *while the instrument was in period 1* was `1.23` +- The difference between the min and max values recorded *while the instrument was in period 2* was zero. + +Attributes: +- `units`: Engineering units of the block, as per EPICS `.EGU` field. + +{#nexus_selog_setpoint} +### `raw_data_1/selog//setpoint` + +The latest setpoint value of a block **in each period**, as an array. +This is written with a datatype corresponding to the underlying EPICS type. + +Attributes: +- `units`: Engineering units of the block, as per EPICS `.EGU` field. + +{#nexus_selog_setpointspread} +### `raw_data_1/selog//setpoint_spread` + +The difference between the minimum recorded value and maximum setpoint values for a block **in each period**, as an array. +This dataset is only written if the underlying EPICS type is numeric. + +Attributes: +- `units`: Engineering units of the block, as per EPICS `.EGU` field. + +--- + +The selog group also contains the following groups: + +```{toctree} +:glob: +:titlesonly: + +nexus_selog/* +``` diff --git a/doc/nexus/nexus_selog/nexus_selog_valuelog.md b/doc/nexus/nexus_selog/nexus_selog_valuelog.md new file mode 100644 index 0000000000..4cf4e1f7dd --- /dev/null +++ b/doc/nexus/nexus_selog/nexus_selog_valuelog.md @@ -0,0 +1,63 @@ +# `raw_data_1/selog//value_log` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXlog`. + +{#nexus_selog_valuelog_name} +### `raw_data_1/selog//value_log/name` + +Under IBEX, this is written as an empty string. + +{#nexus_selog_valuelog_value} +### `raw_data_1/selog//value_log/value` + +This is a time-series log of block values. For scalar blocks, this will be a 1-D dataset. For array blocks, +this will be a 2D dataset where the outermost dimension is time. + +The datatype may be string or float32, depending on the type of the underlying EPICS block. + +Attributes: +- `units`: Units as per the `.EGU` field of the EPICS PV. + +{#nexus_selog_valuelog_time} +### `raw_data_1/selog//value_log/time` + +This is a float32 log of the timestamps, relative to the start of the run, when the {ref}`value ` +readings were taken. A negative time is used to indicate that the block update happened before the run started. + +Attributes: +- `units`: `"second"` +- `start`: Timestamp of the run start, for example `"2026-09-09T08:33:04"` + +{#nexus_selog_valuelog_valuevalid} +### `raw_data_1/selog//value_log/value_valid` + +This is a mask, written as an int32, which indicates whether a logged value was 'valid' according to EPICS at the +time it was logged. It is `1` if the value was valid, or `0` otherwise. Values may be invalid due to disconnected +hardware, corrupt responses from hardware, or misconfigurations - an invalid value should not be trusted by downstream +analysis programs. + +A value is invalid if it's EPICS severity (per the `.SEVR` field) was greater or equal to 3. + +{#nexus_selog_valuelog_alarmtime} +### `raw_data_1/selog//value_log/alarm_time` + +This logs the timestamps of EPICS alarm status or severity changes. + +The timestamps here are not necessarily the same as those in {ref}`time `; new entries +in this dataset are only created when the EPICS `.STAT` or `.SEVR` fields change. + +Attributes: +- `units`: `"second"` +- `start`: Timestamp of the run start, for example `"2026-09-09T08:33:04"` + +{#nexus_selog_valuelog_alarmseverity} +### `raw_data_1/selog//value_log/alarm_severity` + +This logs the string value of the EPICS severity (`.SEVR`) field. This dataset uses the timestamps from +{ref}`alarm_time `. + +{#nexus_selog_valuelog_alarmstatus} +### `raw_data_1/selog//value_log/alarm_status` + +This logs the string value of the EPICS status (`.STAT`) field. This dataset uses the timestamps from +{ref}`alarm_time `. diff --git a/doc/spelling_wordlist.txt b/doc/spelling_wordlist.txt index 76420db5b2..9cc501f9db 100644 --- a/doc/spelling_wordlist.txt +++ b/doc/spelling_wordlist.txt @@ -525,6 +525,7 @@ Mihai minidump mins Mirion +misconfigurations misconfigured mitigations mK @@ -786,6 +787,7 @@ scsi sdk searchable seci +selog sendSlackMessage Sensirion setIOCName From e262461f1d589cd73c1dd37b0f68f5d3c2b191fd Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 10 Sep 2026 16:08:48 +0100 Subject: [PATCH 07/13] runlog --- doc/nexus/nexus_runlog.md | 64 +++++++++++++++++++ .../nexus_runlog/nexus_runlog_icpevent.md | 57 +++++++++++++++++ doc/spelling_wordlist.txt | 1 + 3 files changed, 122 insertions(+) create mode 100644 doc/nexus/nexus_runlog/nexus_runlog_icpevent.md diff --git a/doc/nexus/nexus_runlog.md b/doc/nexus/nexus_runlog.md index 3f45b69c1b..be2553fe0c 100644 --- a/doc/nexus/nexus_runlog.md +++ b/doc/nexus/nexus_runlog.md @@ -2,4 +2,68 @@ This has an `NX_class` of `IXrunlog`; this is not a class specified by upstream NeXus definitions. +Each subgroup is an {external+nexus_manual:doc}`classes/base_classes/NXdata`. +The datasets written are: +- `count_rate` (float32, units="counts") + - The count rate reported by the DAE, in Millions of events per hour (despite the units attribute) +- `dae_beam_current` (float32, units="uAh") + - The beam current reported by the DAE +- `good_frames` (int32, units="frames") + - The cumulative number of good frames so far in this run +- `good_uah_log` (float32, units="uAh") + - The cumulative number of good uAh so far in this run +- `icp_event` (string, no units) + - See {doc}`nexus_runlog/nexus_runlog_icpevent` for details about this dataset +- `is_running` (int32, units="none") + - `1` if the DAE was in the `RUNNING` state, `0` otherwise +- `is_waiting` (int32, units="none") + - `1` if the DAE was in the `WAITING` state, `0` otherwise +- `monitor_sum_1` (int32, units="counts") + - Cumulative monitor sum so far in this run +- `np_ratio` (float32, units="none") + - neutron-proton ratio +- `period` (int32, units="none") + - The current period +- `raw_frames` (int32, units="frames") + - The cumulative number of raw frames so far in this run +- `raw_uah_log` (float32, units="uAh") + - The cumulative number of raw uAh so far in this run +- `run_status` (int32, units="none") + - The run status as an integer, where + - `0` is PROCESSING + - `1` is SETUP + - `2` is RUNNING + - `3` is PAUSED + - `4` is WAITING + - `5` is VETOING + - `6` is ENDING + - `7` is SAVING +- `total_counts` (int32, units="counts") + - The cumulative number of counts so far in this run + +{#nexus_runlog_value} +### `raw_data_1/runlog//value` + +This contains the data for the given parameter. + +Attributes: +- `units`: Units specified in the list above + +{#nexus_runlog_time} +### `raw_data_1/runlog//time` + +This contains int32 timestamps of the values. + +It is written approximately every 30 seconds, and is relative to the start of the run. + +Attributes: +- `start`: the start time of the run, for example `"2026-09-09T16:16:57"` +- `units`: `second` + +```{toctree} +:hidden: +:glob: + +nexus_runlog/* +``` diff --git a/doc/nexus/nexus_runlog/nexus_runlog_icpevent.md b/doc/nexus/nexus_runlog/nexus_runlog_icpevent.md new file mode 100644 index 0000000000..42155f956b --- /dev/null +++ b/doc/nexus/nexus_runlog/nexus_runlog_icpevent.md @@ -0,0 +1,57 @@ +# `raw_data_1/runlog/icp_event` + +The `icp_event` dataset describes DAE state transitions. A new entry is written on each state transition. + +The entries are written as strings, in the following formats: + +### `CHANGE_PERIOD %d` + +This is written on a transition from one period to another (including one just before run start, resetting period to 1). + +### `START_COLLECTION PERIOD %d GF %d RF %d GUAH %f` + +This is written whenever the acquisition transitions from a not-collecting state (e.g. SETUP, PAUSED, WAITING) to a +collecting state (e.g. RUNNING). + +The GF (good frames), RF (raw frames) and GUAH (good uAh-hour) counters reflect the total in the run so far. + +### `STOP_COLLECTION PERIOD %d GF %d RF %d GUAH %f DUR %d` + +This is written whenever the acquisition transitions from a not-collecting state (e.g. SETUP, PAUSED, WAITING) to a +collecting state (e.g. RUNNING). + +The GF (good frames), RF (raw frames) and GUAH (good uAh-hour) counters reflect the total in the run so far. + +The DUR (duration) parameter describes the duration, in seconds, of the acquisition period since the `START_ACQUISITION` message + +### `BEGIN` + +This is written at the beginning of a run. + +### `FAILED_BEGIN` + +This is written when a run attempted to start, but failed. + +### `END` + +This is written at the end of a run. + +### `PAUSE` + +This is written when a run pauses. + +### `RESUME` + +This is written when a run resumes from a pause. + +### `WAIT_START` + +This is written when run control transitions to out-of-range. + +### `WAIT_FINISH` + +This is written when run control transitions to in-range. + +### `ABORT` + +This is written when a run is aborted (ended without saving data). diff --git a/doc/spelling_wordlist.txt b/doc/spelling_wordlist.txt index 9cc501f9db..0c248ce9ba 100644 --- a/doc/spelling_wordlist.txt +++ b/doc/spelling_wordlist.txt @@ -899,6 +899,7 @@ TwinCAT txt ua uA +uAh uamps UC UDP From ed6ff30b36833da1282d91c37768579cbef213cc Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 10 Sep 2026 18:39:12 +0100 Subject: [PATCH 08/13] isis_vms_compat --- doc/nexus/libget.txt | 686 +++++++++++++++++++++++++++++ doc/nexus/nexus_isis_vms_compat.md | 466 ++++++++++++++++++++ 2 files changed, 1152 insertions(+) create mode 100644 doc/nexus/libget.txt create mode 100644 doc/nexus/nexus_isis_vms_compat.md diff --git a/doc/nexus/libget.txt b/doc/nexus/libget.txt new file mode 100644 index 0000000000..00976e3fc0 --- /dev/null +++ b/doc/nexus/libget.txt @@ -0,0 +1,686 @@ +CALL OPEN_FILE(RUNID, FOUND) + +-------------------------------------------------------------- +Extracting Data from a RAW file +-------------------------------------------------------------- + +In the descriptions below: + · [in] describes a parameter which must be set before a call, + but whose value is unchanged by the call. + + · [out] describes a parameter that is altered by the routine, + but need hold no particular value before the call + · [in,out] describes a parameter that contains input + information for the routine and whose value may be altered + by the routine. + +Data can be extracted from a RAW file with the GETDAT routine: + + SUBROUTINE GETDAT(RUNID, IFSN, NOS, IDATA, LENGTH, ERRCODE) + CHARACTER*(*) RUNID ! [in] Name of the RAW file to use + INTEGER IFSN ! [in] Number of first spectrum required + INTEGER NOS ! [in] Total number of spectra required + INTEGER IDATA(LENGTH) ! [out] Array to hold the returned data + INTEGER ERRCODE ! [out] Error status variable + + Error codes returned in ERRCODE are: + + 0 - All OK + 2 - Attempt to access CRPT (not supported) + 4 - Too many spectra (NOS) asked for + 5 - Error in unpacking compressed data in the file + 6 - The routine cannot understand the data section of the file + +********************** +*** Important Note *** +********************** +In collecting the data there is a 'channel 0' which contains no useful +information; This will appear in the first channel of each spectrum and should +not be used in calculations. This means that LENGTH needs to be at least +NTC1+1 in size, and the NTC1 useful values are returned in elements +(2) ... (NTC1+1) of the array. See the example programs libget_test.f +and libget_test2.f for more information + +Extracting Run Parameters from a RAW file +----------------------------------------- +Parameters can be extracted by + CALL GETPARI(RUNID,NAME,IARRAY,LENGTH_IN,LENGTH_OUT,ERRCODE) + CALL GETPARR(RUNID,NAME,RARRAY,LENGTH_IN,LENGTH_OUT,ERROCDE) + CALL GETPARC(RUNID,NAME,CARRAY,LENGTH_IN,LENGTH_OUT,ERRCODE) +where RUNID is a character string specifying the RAW file name. + NAME is a character string identifying the parameter. + IARRAY,RARRAY and CARRAY are arrays of Integer, Real of Character type + for returning the requested values. + + SUBROUTINE GETPARI(RUNID, NAME, IVALUE, LENGTH_IN, LENGTH_OUT, ERRCODE) + CHARACTER*(*) RUNID ! [in] Name of the RAW file + CHARACTER*4 NAME ! [in] The name of the item to return + INTEGER*4 LENGTH_IN ! [in] size of the input array + INTEGER*4 IVALUE(LENGTH_IN) ! [out] Array to hold the returned values + INTEGER*4 LENGTH_OUT ! [out] Amount of IVALUE used + INTEGER*4 ERRCODE ! [out] Returned status value + + Error codes returned by ERRCODE are: + + 0 = All OK + 2 = Attempt to access CRPT, which is not supported + 3 = Non existent parameter NAME specified + 4 = Some other error (see screen output) + + SUBROUTINE GETPARC(RUNID, NAME, IVALUE, LENGTH_IN, LENGTH_OUT, ERRCODE) + + CHARACTER*(*) RUNID ! [in] Name of the RAW file to use + CHARACTER*4 NAME ! [in] The name of the item to return + INTEGER*4 LENGTH_IN ! [in] Size of the input array + CHARACTER*(*) CVALUE(LENGTH_IN) ! [out] Array to hold the returned values + INTEGER*4 LENGTH_OUT ! [out] Amount of CVALUE used + INTEGER*4 ERRCODE ! [out] Error status value + + Error codes returned by ERRCODE are: + 0 = All OK + 2 = Attempt to access the CRPT (not supported) + 3 = Non existent parameter NAME specified + 4 = Some other error occurred (see screen output) + + SUBROUTINE GETPARR(RUNID, NAME, RVALUE, LENGTH_IN, LENGTH_OUT, ERRCODE) + CHARACTER*(*) RUNID ! [in] Name of the RAW file + CHARACTER*4 NAME ! [in] The name of the item to return + INTEGER*4 LENGTH_IN ! [in] Size of the data array + REAL*4 RVALUE(LENGTH_IN) ! [out] Array to hold the returned value(s) + INTEGER*4 LENGTH_OUT ! [out] Portion of IVALUE used + INTEGER*4 ERRCODE ! [out] Returned status value + + Error codes returned by ERRCODE are: + + 0 = All OK + 2 = Attempt to access the CRPT (not supported) + 3 = Non existent parameter NAME specified + 4 = Some other error occurred (see screen output) + + +The most common parameters required are + NSP1 total number of spectra + NTC1 total number of channels per spectrum (ignoring 'channel 0') + TCB1 time channel boundaries in microseconds +For example to pick these up + INTEGER IARRAY(100) + REAL RARRAY(10000) + INTERGER ERRCODE,LOUT,NUMCHAN,NUMSPECT + CALL GETPARI(RUNID,'NSP1',IARRAY,100,LOUT,ERRCODE) + NUMSPECT=IARRAY(1) + CALL GETPARI(RUNID,'NTC1',IARRAY,100,LOUT,ERRCODE) + NUMCHAN=IARRAY(1) + CALL GETPARR(RUNID,'TCB1',RARRAY,10000,LOUT,ERRCODE) +The RARRAY should now hold NUMCHAN time channel boundaries. + +If the parameter you require is not named individually you will need to use +the name of the parameter block to return a series of parameters into an array +and then extract the appropriate element of the array. For example, the total +number of good frames is element 10 of the 'Run Parameter Block' which has the +name RPB. eg. + INTEGER IARRAY(32) + INTERGER ERRCODE,LOUT,NGOOD + CALL GETPARI(RUNID,'RPB',IARRAY,32,LOUT,ERRCODE) + NGOOD=IARRAY(10) + +Parameter Names +--------------- +SECTION ITEM NAME TYPE CONTENTS + 1 1 HDR C*80 HEADER BLOCK + 1 2 VER1 I*4 FORMAT version number + 1 3 ADD I*4(10) SECTION addresses + + 2 1 VER2 I*4 RUN section version # + 2 2 RUN I*4 RUN # + 2 3 TITL C*80 RUN title + 2 4 USER C*20(8) USER information + 2 5 RPB I*4(32) RUN parameter block + + 3 1 VER3 I*4 INSTRUMENT section version # + 3 2 NAME C*8 INSTRUMENT name + 3 3 IVPB R/I*4(64) INSTRUMENT parameter block + 3 4 NDET I*4 No. of detectors + 3 5 NMON I*4 No. of monitors + 3 6 NUSE I*4 NO. of USER defined (UTn) tables ***B + 3 7 MDET I*4(nmon) detector no. for each monitor + 3 8 MONP I*4(nmon) prescale value for each monitor + 3 9 SPEC I*4(ndet) SPECTRUM # table + 3 10 DELT R*4(ndet) HOLD OFF table ***B + 3 11 LEN2 R*4(ndet) L2 table + 3 12 CODE I*4(ndet) CODE for UTn tables ***B + 3 13 TTHE R*4(ndet) 2Theta table (scattering ang) + 3 14 PHI R*4(ndet) PHI table (azimuthal angle) + 3 15 UT01 R*4(ndet) USER defined table 1 ***B + ..cont. to UTn + + 4 1 VER4 I*4 SE section version # + 4 2 SPB I*4(32) SAMPLE parameter block + 4 3 NSEP I*4 number of controlled SEPs + 4 4 SE01 I*4(24) SE parameter block #1 + ..cont. to SEnn + + 5 1 VER5 I*4 DAE section version # + 5 2 DAEP I*4(64) DAE parameter block + 5 3 CRAT I*4(ndet) crate no.for each detector + 5 4 MODN I*4(ndet) module no. for each detector + 5 5 MPOS I*4(ndet) position in module for each detector + 5 6 TIMR I*4(ndet) TIME REGIME # table + 5 7 UDET I*4(ndet) 'USER detector #' for each detector + + 6 1 VER6 I*4 TCB secton version # + 6 2 NTRG I*4 # of time regimes (normally =1) + 6 3 NFPP I*4 No. of frames per period + 6 4 NPER I*4 No. of periods + 6 5 PMAP I*4 period # for each basic period + 6 6 NSP1 I*4 No. of spectra for time reg.= 1(tr=1) + 6 7 NTC1 I*4 No. of time chan. (tr=1) + 6 8 TCM1 I*4(5) time channel mode (0,1,2, ..) (tr=1) *A + 6 9 TCP1 R*4(20) time channel parameters (tr=1) *A + 6 10 PRE1 I*4 prescale value for 32MHz clock (tr=1) + (6.6 - 6.10 repeated for each time regime : NSP2,NTC2..etc.) + 6 11 TCB1 I*4(ntc1+1) Time channel boundaries (tr=1) + (6.11 repeated for each time regime : TCB2,TCB3..etc.) + + 7 USER DEFINED (max length =400 R*4 wds.) + + 8 1 VER7 I*4 DATA version # + 8 2 DAT1 I*4(ntc1,nsp1) raw data for first time regime + ..cont. for NTRG time regimes + +Detailed Listing of Sections +---------------------------- + 1 FORMAT + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + P 1 HDR(1:8) c*8 RUN identifier (eg. LAD12345 ) + + - 3 characters for INSTRUMENT, + + - 5 characters for RUN NUMBER.) + + U HDR(9:28) c*20 User name + U HDR(29:52) c*24 Experiment short title + P HDR(53:64) c*12 start date + P HDR(65:72) c*8 start time + HDR(73:80) c*8 run duration (µA.Hr) + -------------------------------------------------------------- + F 2 VER1 I*4 format version number (ver=2) ***B + -------------------------------------------------------------- + P 3 ADD(1) I*4 start address of RUN section + P ADD(2) I*4 start address of INST section + P ADD(3) I*4 start address of SE section + P ADD(4) I*4 start address of DAE section + P ADD(5) I*4 start address of TCB section + P ADD(6) I*4 start address of USER section ***B + P ADD(7) I*4 start address of DATA section ***B + P ADD(8) I*4 start address of LOG section ***B + ADD(9) I*4 spare + FORM I*4 data format flag (0 or 1) ***C + 0 - all TC for each spectrum + 1 - the same TC from every spectrum + -------------------------------------------------------------- + + 2 RUN + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + F 1 VER2 I*4 RUN section version number + -------------------------------------------------------------- + P 2 RUN I*4 run number (starting from 1) + -------------------------------------------------------------- + U 3 TITL C*80 run title + -------------------------------------------------------------- + U 2 USER(1) C*20 user name + U USER(2) C*20 user telephone no. 1(day) + U USER(3) C*20 user telephone no. 2(day) + U USER(4) C*20 user telephone no. (night) + U USER(5) C*20 user institution + C*20(3) spare + -------------------------------------------------------------- + P 4 RPB(1) I*4 actual run duration + U RPB(2) I*4 scaler for RPB(1),(22) (1=sec,..) + U RPB(3) I*4 test interval of RPB(2) (sec) + U RPB(4) I*4 dump interval + U RPB(5) I*4 scaler for RPB(4) + U RPB(6) I*4 test interval of RPB(5)(sec) + U RPB(7) I*4 2**k (SNS frequency(Hz)=50/2**k) + P RPB(8) R*4 good proton charge (uA.hr) + P RPB(9) R*4 total proton charge (uA.hr) + P RPB(10) I*4 number of 'good' frames + P RPB(11) I*4 total number of frames + U RPB(12) I*4 required run duration(units=RPB(1))***B + P RPB(13) I*4 actual run duration (seconds) ***B + P RPB(14) I*4 monitor sum 1 + P RPB(15) I*4 monitor sum 2 + RPB(16) I*4 monitor sum 3 + P RPB(17-19) C*12 finish date (dd-mmm-yyyy_) + P RPB(20-21) C*8 finish time (hh-mm-ss) + RPB(22) I*4 RAL Proposal Number + RPB(- 32) spare + -------------------------------------------------------------- + + 3 INSTRUMENT + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + F 1 VER3 I*4 INSTRUMENT section version no.(=2) ***B + -------------------------------------------------------------- + F 2 NAME C*8 instrument name + -------------------------------------------------------------- + 3 INSTRUMENT VARIABLE PARAMETER BLOCK + U IVPB(1) R*4 frequency chopper 1 (Hz) + U IVPB(2) R*4 frequency chopper 2 (Hz) + U IVPB(3) R*4 frequency chopper 3 (Hz) + U IVPB(4) I*4 delay c1 (µs) + U IVPB(5) I*4 delay c2 (µs) + U IVPB(6) I*4 delay c3 (µs) + U IVPB(7) I*4 max error on delay c1 (µs) + U IVPB(8) I*4 max error on delay c2 (µs) + U IVPB(9) I*4 max error on delay c3 (µs) + U IVPB(10) I*4 apperture c1 + U IVPB(11) I*4 apperture c2 + U IVPB(12) I*4 apperture c3 + U IVPB(13) I*4 status c1 (run,stopped,stop open) + U IVPB(14) I*4 status c2 (run,stopped,stop open) + U IVPB(15) I*4 status c3 (run,stopped,stop open) + U IVPB(16) I*4 main shutter (open=1) + U IVPB(17) I*4 thermal shutter( " ) + U IVPB(18) R*4 beam apperture horizontal(mm) + U IVPB(19) R*4 beam apperture vertical(mm) + U IVPB(20) I*4 scattering posn.(eg 1or2 HRPD) + U IVPB(21) I*4 moderator type no. (e.g HET=3) + U IVPB(22) I*4 detector tank vacuum (1=vacuum on) + U IVPB(23) R*4 L1 + IVPB(24) I*4 Rotor Frequency HET + IVPB(25) R*4 Rotor Energy HET + IVPB(26) R*4 Rotor Phase HET + IVPB(27) I*4 Rotor Slit Package (0="unknown",1="L",2="Med",3="Hi") HET + IVPB(28) I*4 Slow Chopper (1=on,0=off) HET + IVPB(29) R*4 LOQ X centre LOQ + IVPB(30) R*4 LOQ Y centre LOQ + IVPB(31) I*4 Beam stop LOQ + IVPB(32) R*4 Radius Beam Stop LOQ + IVPB(33) R*4 Source to detector distance LOQ + IVPB(34) R*4 FOE angle LOQ + IVPB(35) R*4 Angle of Incidence CRISP + IVPB( - 64) spare + -------------------------------------------------------------- + + U 4 NDET I*4 no. of detectors + -------------------------------------------------------------- + U 5 NMON I*4 no. of monitors + -------------------------------------------------------------- + U 6 NUSE I*4 no. of UTn tables ***B + -------------------------------------------------------------- + U 7 MDET I*4(nmon) detector nos. of the monitors + -------------------------------------------------------------- + U 8 MONP I*4(nmon) prescale values for the monitors + -------------------------------------------------------------- + U 9 SPEC I*4(ndet) spectrum # table + -------------------------------------------------------------- + U 10 DELT R*4(ndet) 'HOLD OFF' in (µs) ***B + -------------------------------------------------------------- + U 11 LEN2 R*4(ndet) L2 table (m) (upstrem monitors -ve. + -------------------------------------------------------------- + U 12 CODE I*4(ndet) code to define use of UT values ***B + -------------------------------------------------------------- + U 13 TTHE R*4(ndet) 2Theta table (scattering angle) + -------------------------------------------------------------- + U 14 PHI R*4(ndet) PHI table (azimuthal angle) + -------------------------------------------------------------- + U 15 UT01 R*4(ndet) User defined table ***B + -------------------------------------------------------------- + . + U . UTn R*4(ndet) User defined table ***B + -------------------------------------------------------------- + + + 4 SAMPLE ENVIRONMENT + + ITEM NAME TYPE CONTENTS + -------------------------------------------------------------- + F 1 VER4 I*4 SE section version # (=2) + -------------------------------------------------------------- + 2 SAMPLE PARAMETER BLOCK + U SPB(1) I*4 position of sample changer + U SPB(2) I*4 sample type (1 = sample+can + 2 = empty can + 3 = vanadium + 4 = absorber + 5 = nothing + 6 = sample, no can) + U SPB(3) I*4 sample geometry (1 = cylinder + 2 = flat plate + 3 = HRPD slab) + U SPB(4) R*4 sample thickness normal to sampl.(mm) + U SPB(5) R*4 sample height (mm) + U SPB(6) R*4 sample width (mm) + U SPB(7) R*4 omega sample angle (deg) + U SPB(8) R*4 psi sample angle (deg) + U SPB(9) R*4 phi sample angle (deg) + U SPB(10) R*4 scat.geom.(1=trans.2=reflect. ***B + U SPB(11) R*4 sample sCOH (barns) ***B + U SPB(12) R*4 sample sINC (barns) ***B + U SPB(13) R*4 sample sABS (barns) ***B + U SPB(14) R*4 sample number density (atoms.A-3) + U SPB(15) R*4 can wall thickness (mm) ***B + U SPB(16) R*4 can sCOH (barns) ***B + U SPB(17) R*4 can sINC (barns) ***B + U SPB(18) R*4 can sABS (barns) ***B + U SPB(19) R*4 can number density (atoms.A-3) + U SPB(20) C*40 sample name or chemical formula + SPB( -64) spare + -------------------------------------------------------------- + F=0 3 NSEP I*4 no. of SE parameters + -------------------------------------------------------------- + 4 SE01(32) SE parameter block #1 ***B + C*4(2) name + I*4 value + I*4 value exponent + C*4(2) units of value + I*4 low trip + I*4 high trip + I*4 current value + I*4 status (in bounds ?) + I*4 controlled parameter (true/false) + I*4 run control parameter (true/false) + I*4 log parameter changes (true/false) + R*4 stability value (units per sec) + R*4 monitor repeat period + I*4 CAMAC location N + I*4 CAMAC location A + I*4 CAMAC offset (added to value) + I*4 CAMAC register group (1 or 2) + I*4 Pre process routine number + I*4(12) CAMAC values + -------------------------------------------------------------- + 3+nsep SEnsep I*4(32) SE parameter block #nsep + + 5 DATA ACQUISTION ELECTRONICS + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + F 1 VER5 I*4 DAE section version # (=2) + --------------------------------------------------------------- + 2 DAE PARAMETER BLOCK + U DAEP(1) I*4 Word length in bulk store memory + F DAEP(2) I*4 Length of bulk store memory (bytes)**A + U DAEP(3) I*4 PPP minimum value ***B + P DAEP(4) I*4 good PPP total (high 32 bits) ***B + P DAEP(5) I*4 good PPP total (low 32 bits) ***B + P DAEP(6) I*4 raw PPP total (high 32 bits) ***B + P DAEP(7) I*4 raw PPP total (low 32 bits) ***B + P DAEP(8) I*4 good ext. neut tot (high 32bits)***B + P DAEP(9) I*4 good ext. neut tot (low 32 bits)***B + P DAEP(10) I*4 raw ext. neut tot (high 32 bits)***B + P DAEP(11) I*4 raw ext. neut tot (low 32 bits)***B + P DAEP(12) I*4 ext. neutron gate (t1) (µs) ***B + P DAEP(13) I*4 ext. neutron gate (t2) (µs) ***B + U DAEP(14) I*4 detector for MON 1 (12 bits) ***B + U DAEP(15) I*4 module for MON 1 ( 4 bits) ***B + U DAEP(16) I*4 crate for MON 1 ( 4 bits) ***B + U DAEP(17) I*4 mask for MON 1 (c4:m4:d12) ***B + U DAEP(18) I*4 detector for MON 2 (12 bits) ***B + U DAEP(19) I*4 module for MON 2 ( 4 bits) ***B + U DAEP(20) I*4 crate for MON 2 ( 4 bits) ***B + U DAEP(21) I*4 mask for MON 2 (c4:m4:d12) ***B + P DAEP(22) I*4 total GOOD EVENTS (high 32 bits)***B + P DAEP(23) I*4 total GOOD EVENTS (low 32 bits)***B + P DAEP(24) I*4 frame synch delay (4µs steps) ***B + U DAEP(25) I*4 frm snch origin(0:none/1:ext/2:int)***B + U DAEP(26) I*4 Secondary Master Pulse (0:en,1:dis) + U DAEP(27-29) I*4 External vetoes 0,1,2 (0 dis,1 en) + DAEP( -64) Spare + -------------------------------------------------------------- + U 3 CRAT I*4(ndet) crate no.for each detector + -------------------------------------------------------------- + U 4 MODN I*4(ndet) module no. for each detector + -------------------------------------------------------------- + U 5 MPOS I*4(ndet) posn. in module for each detector + -------------------------------------------------------------- + U 6 TIMR I*4(ndet) TIME REGIME # table + -------------------------------------------------------------- + U 7 UDET I*4(ndet) 'USER DETECTOR #' for each det ***B + -------------------------------------------------------------- + + + 6 TIME CHANNEL BOUNDARIES + + ITEM NAME TYPE CONTENTS + -------------------------------------------------------------- + F 1 VER6 I*4 TCB secton version # + -------------------------------------------------------------- + U/P 2 NTRG I*4 # of time regimes (normally =1) + -------------------------------------------------------------- + F=1 3 NFPP I*4 # of frames per period + -------------------------------------------------------------- + F=1 4 NPER I*4 # of periods + -------------------------------------------------------------- + F=1 5 PMAP I*4(256) period # for each basic period + -------------------------------------------------------------- + repeat following block for each time regime + time regime =1 + -------------------------------------------------------------- + U/P 3 NSP1 I*4 No. of spectra (+1 for zeroth) + -------------------------------------------------------------- + U/P 4 NTC1 I*4 No. of time channels ( " ) + -------------------------------------------------------------- + U 5 TCM1 I*4(5) time chan. mode (see footnote) + -------------------------------------------------------------- + U 6 TCP1 R*4(4,5) time chan. parameters (µs) + -------------------------------------------------------------- + P 7 PRE1 I*4 prescale value for 32MHz clock (<=15) + -------------------------------------------------------------- + time regime =2 + -------------------------------------------------------------- + 11 NSP2 I*4 ) + -------------------------------------------------------------- + 12 NTC2 I*4 ) + -------------------------------------------------------------- + 13 TCM2 I*4(5) ) As for 6.3,6.4,6.5,6.6, + -------------------------------------------------------------- + 14 TCP2 R*4(20) ) 6.7,6.8 for regime=2 + -------------------------------------------------------------- + 15 PRE2 I*4 ) + -------------------------------------------------------------- + 16 TCB2 I*4(ntc2+1) ) + -------------------------------------------------------------- + ...cont for additional time regimes + U 8 TCB1 I*4(ntc1+1) time chan. bound. for TR=1(clock pulses) + -------------------------------------------------------------- + U 8 TCB2 I*4(ntc1+1) time chan. bound. for TR=2(clock pulses) + -------------------------------------------------------------- + ...cont for additional time regimes + +TIME CHANNEL MODES: + 0 Boundaries set by a table held in file TCB.DAT + + 1 TCB1(n) = (TCP1(1) + (n-1)*TCP1(2)-DAEP(24)*4)*32/PRE1 + (ie. Dt = c ) + (ie. less frm.synch delay) + + 2 temp(1) = TCP1(1) + TCB1(1) = (TCP1(1)-DAEP(24)*4)*32/PRE1 (ie. Dt = c.t + temp(n+1)= temp(n)*(1 + TCP1(2)) + TCB1(n+1)= (temp(n+1) - DAEP(24)*4)*32/PRE1 + + 7 USER DEFINED (suggested layout) + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + 1 VER7 I*4 DATA version # + -------------------------------------------------------------- + 2 ULEN I*4 USER_LEN + -------------------------------------------------------------- + 3 DAT1 R*4(user_len) USER data + -------------------------------------------------------------- + + 8 RAW DATA + + Two formats exist for the data section. The earlier Version1 is + simply an array of consecutive data channels in 32bit integers. + Version2 compresses each spectrum to reduce the file size and + also has a data section header to describe the exact format. + + 8a RAW DATA Version 1 + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + F 1 VER7 I*4 DATA version # (=1) + -------------------------------------------------------------- + P 2 DAT1 I*4 raw data for first time regime + (ntc1+1)*(nsp1+1) channels + If data format flag=0 + arranged (ntc1+1,nsp1+1) + (nsp1 spectra each containing + ntc1 data points) + If data format flag=1 + arranged (nsp1+1,ntc1+1) + (for each time channel all + spectra stored together) + -------------------------------------------------------------- + 3 DAT2 I*4 raw data for 2nd time regime + (nt2+1,nsp2+1) channels + -------------------------------------------------------------- + ...cont for additional time regimes.. + + +---Section Continued Overleaf--- + + 8b RAW DATA Version 2 ***C + + ITEM NAME TYPE CONTENTS + + -------------------------------------------------------------- + F 1 VER7 I*4 DATA version # (=2) + -------------------------------------------------------------- + 2 Data Section Header Block + DHDR(1) I*4 Compression Type + 0 no compression + 1 byte relative + DHDR(2) I*4 Reserved + DHDR(3) I*4 Offset to Spectrum Descriptor Array + DHDR(4) I*4 Equivalent version 1 filesize (blocks) + DHDR(5) R*4 Compression Ratio for data section + DHDR(6) R*4 Compression Ratio for whole file + DHDR(7-32) Unused + -------------------------------------------------------------- + 3 Spectrum Descriptor Array (spect=0..nspec) + DDES(2*spect+1) I*4 Number of words in compressed spectrum + DDES(2*spect+2) I*4 Offset to compressed spectrum + . + . nspec = total of all spectra in all time regimes + . in all periods + -------------------------------------------------------------- + 4 DATA(0) Compressed raw data for each + . spectrum from each time regime + . for each period + . (**see below for details) + . On expansion a spectrum contains +. channel numbers from 0 to ntc + DATA(nspec) + --------------------------------------------------------------- + + ***The data section now allows the use of a compressed format for the +data. At the time of writing only one compressed format is used. This is +byte relative. Each integer value in the data is stored as a relative offset +to the previous data point. This offset is stored within a byte thus allowing +offsets of + or - 127 between consecutive values. Should the offset +be outside this range then -128 is inserted in the byte as a flag and the +absolute value of the data point is stored in the next 4bytes. As the reduction +of space used for a value is from a 32bit integer to one byte, the maximum +compression ratio is 4. + + STATUS SECTION + +The status section is used during the run to store temporary values. It is +not copied to the raw data file. + +It generally is placed in the gap between the FORMAT and RUN sections. The +table below names the parameters stored in the section. + +OFFSET is I*4 array element if load all of status section into array +NAME refers to name in SYS$PUBLIC:CRPTSECT.DEF with the "ST_" prefix removed + +OFFSET ITEM NAME TYPE CONTENTS + -------------------------------------------------------------- +1 1 RUN I*4 Run state (-3=waiting, -2=waiting, + -1=paused,0=setup,1=running) + -------------------------------------------------------------- +2 2 START I*4(2) Start time in binary + -------------------------------------------------------------- +4 3 PAUSED I*4(2) Paused time in binary + -------------------------------------------------------------- +6 4 PREV I*4(2) Run time up to last pause + in binary + -------------------------------------------------------------- +8 5 ICPCODE I*4 Internal error code + 1 - OK + other odd Warning + even error + -------------------------------------------------------------- +9 6 SYSCODE I*4 VAX status code + -------------------------------------------------------------- +10 7 SPECT I*4 Monitor Spectrum Number + FIRST R*4 Start time in µsecs + LAST R*4 End time in µsecs + TCB1 I*4 Start boundary + TCB2 I*4 End boundary + -------------------------------------------------------------- +15 8 CAMACFLAGS I*4 Status display flags for CAMAC + -------------------------------------------------------------- +16 9 HEARTBEAT I*4 CAMAC heartbeat + -------------------------------------------------------------- +17 10 SEP_RUN_OK I*4 SE run OK + -------------------------------------------------------------- +18 11 SIMULATE I*4 Simulate flag + 0 for normal running + 1 for simulation mode + (no reads/writes to DAE) + -------------------------------------------------------------- +19 12 NOMSG I*4 No message flag + -------------------------------------------------------------- +20 13 HOLD I*4 Hold flag (do not return till + end of run) + -------------------------------------------------------------- +21 14 ENDWANTED I*4 End wanted? + -------------------------------------------------------------- +26 15 GOODFRM I*4(2) Good frame up to last pause + RAWFRM Raw frame + +---Section Continued Overleaf--- + + -------------------------------------------------------------- +28 16 I*4() Maximum values allowed for CRPT + MAX_CRPT + MAX_DET + MAX_NUT + MAX_SPEC + MAX_CHAN + MAX_MON + MAX_VAX + MAX_DAE + MAX_SEP + MAX_TRG + MAX_UDAT + MAX_PER + MAX_TOTSPEC + -------------------------------------------------------------- +46 17 CPERIOD I*4 Current Period Number + -------------------------------------------------------------- +47 18 PERMAX I*4 Number of channels in period + -------------------------------------------------------------- +48 19 TPOFFSET I*4(16) 16 time channel offsets + (One for each crate) + -------------------------------------------------------------- +64 20 COUNTSUM I*4 Monitor Sum of counts + -------------------------------------------------------------- +65 21 TABLES I*4 Tables? + -------------------------------------------------------------- diff --git a/doc/nexus/nexus_isis_vms_compat.md b/doc/nexus/nexus_isis_vms_compat.md new file mode 100644 index 0000000000..e1596a5bf1 --- /dev/null +++ b/doc/nexus/nexus_isis_vms_compat.md @@ -0,0 +1,466 @@ +# `raw_data_1/isis_vms_compat` + +This has an `NX_class` of `IXvms`; this is not a class specified by upstream NeXus definitions. + +Data under these groups are provided to aid backwards-compatibility with older analysis programs. + +:::{seealso} +See also {download}`libget.txt` +::: + +### `raw_data_1/isis_vms_compat/HDR` + +This is the "header block". Is is a fixed-size 80 character string. + +``` + P 1 HDR(1:8) c*8 RUN identifier (eg. LAD12345 ) + - 3 characters for INSTRUMENT, + - 5 characters for RUN NUMBER.) + U HDR(9:28) c*20 User name + U HDR(29:52) c*24 Experiment short title + P HDR(53:64) c*12 start date + P HDR(65:72) c*8 start time + HDR(73:80) c*8 run duration (µA.Hr) +``` + +### `raw_data_1/isis_vms_compat/VER1` + +``` + F 2 VER1 I*4 format version number (ver=2) ***B +``` + +### `raw_data_1/isis_vms_compat/ADD` + +``` + P 3 ADD(1) I*4 start address of RUN section + P ADD(2) I*4 start address of INST section + P ADD(3) I*4 start address of SE section + P ADD(4) I*4 start address of DAE section + P ADD(5) I*4 start address of TCB section + P ADD(6) I*4 start address of USER section ***B + P ADD(7) I*4 start address of DATA section ***B + P ADD(8) I*4 start address of LOG section ***B + ADD(9) I*4 spare +``` + +### `raw_data_1/isis_vms_compat/FORM` + +``` + FORM I*4 data format flag (0 or 1) ***C + 0 - all TC for each spectrum + 1 - the same TC from every spectrum +``` + +### `raw_data_1/isis_vms_compat/VER2` + +``` + F 1 VER2 I*4 RUN section version number +``` + +### `raw_data_1/isis_vms_compat/RUN` + +``` + P 2 RUN I*4 run number (starting from 1) +``` + +### `raw_data_1/isis_vms_compat/TITL` + +``` + U 3 TITL C*80 run title +``` + +### `raw_data_1/isis_vms_compat/USER` + +``` + U 2 USER(1) C*20 user name + U USER(2) C*20 user telephone no. 1(day) + U USER(3) C*20 user telephone no. 2(day) + U USER(4) C*20 user telephone no. (night) + U USER(5) C*20 user institution + C*20(3) spare +``` + +### `raw_data_1/isis_vms_compat/IRBP` / `RRBP` / `CRBP` + +`IRBP` and `RRBP` represent the `RBP` table, represented as integer and float types respectively. + +- `IRPB` is an integer representation of this data +- `RRBP` is a float representation +- `CRBP` is a string representation. Non-printable characters, `\0`, and any character outside the ASCII range are replaced with the `?` character. The string is then null-terminated. + +``` + P 4 RPB(1) I*4 actual run duration + U RPB(2) I*4 scaler for RPB(1),(22) (1=sec,..) + U RPB(3) I*4 test interval of RPB(2) (sec) + U RPB(4) I*4 dump interval + U RPB(5) I*4 scaler for RPB(4) + U RPB(6) I*4 test interval of RPB(5)(sec) + U RPB(7) I*4 2**k (SNS frequency(Hz)=50/2**k) + P RPB(8) R*4 good proton charge (uA.hr) + P RPB(9) R*4 total proton charge (uA.hr) + P RPB(10) I*4 number of 'good' frames + P RPB(11) I*4 total number of frames + U RPB(12) I*4 required run duration(units=RPB(1))***B + P RPB(13) I*4 actual run duration (seconds) ***B + P RPB(14) I*4 monitor sum 1 + P RPB(15) I*4 monitor sum 2 + RPB(16) I*4 monitor sum 3 + P RPB(17-19) C*12 finish date (dd-mmm-yyyy_) + P RPB(20-21) C*8 finish time (hh-mm-ss) + RPB(22) I*4 RAL Proposal Number + RPB(- 32) spare +``` + +### `raw_data_1/isis_vms_compat/VER3` + +``` + F 1 VER3 I*4 INSTRUMENT section version no.(=2) ***B +``` + +### `raw_data_1/isis_vms_compat/NAME` + +``` + F 2 NAME C*8 instrument name +``` + +### `raw_data_1/isis_vms_compat/IVBP` and `raw_data_1/isis_vms_compat/RVBP` + +`IVPB` and `RVBP` represent the `IVPB` table, represented as integer and float types respectively. + +``` + 3 INSTRUMENT VARIABLE PARAMETER BLOCK + U IVPB(1) R*4 frequency chopper 1 (Hz) + U IVPB(2) R*4 frequency chopper 2 (Hz) + U IVPB(3) R*4 frequency chopper 3 (Hz) + U IVPB(4) I*4 delay c1 (µs) + U IVPB(5) I*4 delay c2 (µs) + U IVPB(6) I*4 delay c3 (µs) + U IVPB(7) I*4 max error on delay c1 (µs) + U IVPB(8) I*4 max error on delay c2 (µs) + U IVPB(9) I*4 max error on delay c3 (µs) + U IVPB(10) I*4 apperture c1 + U IVPB(11) I*4 apperture c2 + U IVPB(12) I*4 apperture c3 + U IVPB(13) I*4 status c1 (run,stopped,stop open) + U IVPB(14) I*4 status c2 (run,stopped,stop open) + U IVPB(15) I*4 status c3 (run,stopped,stop open) + U IVPB(16) I*4 main shutter (open=1) + U IVPB(17) I*4 thermal shutter( " ) + U IVPB(18) R*4 beam apperture horizontal(mm) + U IVPB(19) R*4 beam apperture vertical(mm) + U IVPB(20) I*4 scattering posn.(eg 1or2 HRPD) + U IVPB(21) I*4 moderator type no. (e.g HET=3) + U IVPB(22) I*4 detector tank vacuum (1=vacuum on) + U IVPB(23) R*4 L1 + IVPB(24) I*4 Rotor Frequency HET + IVPB(25) R*4 Rotor Energy HET + IVPB(26) R*4 Rotor Phase HET + IVPB(27) I*4 Rotor Slit Package (0="unknown",1="L",2="Med", 3="Hi") HET + IVPB(28) I*4 Slow Chopper (1=on,0=off) HET + IVPB(29) R*4 LOQ X centre LOQ + IVPB(30) R*4 LOQ Y centre LOQ + IVPB(31) I*4 Beam stop LOQ + IVPB(32) R*4 Radius Beam Stop LOQ + IVPB(33) R*4 Source to detector distance LOQ + IVPB(34) R*4 FOE angle LOQ + IVPB(35) R*4 Angle of Incidence CRISP + IVPB( - 64) spare +``` + +### `raw_data_1/isis_vms_compat/NDET` + +``` + U 4 NDET I*4 no. of detectors +``` + +### `raw_data_1/isis_vms_compat/NMON` + +``` + U 5 NMON I*4 no. of monitors +``` + +### `raw_data_1/isis_vms_compat/NUSE` + +``` + U 6 NUSE I*4 no. of UTn tables ***B +``` + +### `raw_data_1/isis_vms_compat/MDET` + +``` + U 7 MDET I*4(nmon) detector nos. of the monitors +``` + +### `raw_data_1/isis_vms_compat/MONP` + +``` + U 8 MONP I*4(nmon) prescale values for the monitors +``` + +### `raw_data_1/isis_vms_compat/VER4` + +``` + F 1 VER4 I*4 SE section version # (=2) +``` + +### `raw_data_1/isis_vms_compat/SPB` / `ISPB` / `RSPB` and ` +CSPB` + +- `SPB` and `ISPB` are identical integer representations of this data +- `RSPB` is a float representation +- `CSPB` is a string representation. Non-printable characters, `\0`, and any character outside the ASCII range are replaced with the `?` character. The string is then null-terminated. + +``` + 2 SAMPLE PARAMETER BLOCK + U SPB(1) I*4 position of sample changer + U SPB(2) I*4 sample type (1 = sample+can + 2 = empty can + 3 = vanadium + 4 = absorber + 5 = nothing + 6 = sample, no can) + U SPB(3) I*4 sample geometry (1 = cylinder + 2 = flat plate + 3 = HRPD slab) + U SPB(4) R*4 sample thickness normal to sampl.(mm) + U SPB(5) R*4 sample height (mm) + U SPB(6) R*4 sample width (mm) + U SPB(7) R*4 omega sample angle (deg) + U SPB(8) R*4 psi sample angle (deg) + U SPB(9) R*4 phi sample angle (deg) + U SPB(10) R*4 scat.geom.(1=trans.2=reflect. ***B + U SPB(11) R*4 sample sCOH (barns) ***B + U SPB(12) R*4 sample sINC (barns) ***B + U SPB(13) R*4 sample sABS (barns) ***B + U SPB(14) R*4 sample number density (atoms.A-3) + U SPB(15) R*4 can wall thickness (mm) ***B + U SPB(16) R*4 can sCOH (barns) ***B + U SPB(17) R*4 can sINC (barns) ***B + U SPB(18) R*4 can sABS (barns) ***B + U SPB(19) R*4 can number density (atoms.A-3) + U SPB(20) C*40 sample name or chemical formula + SPB( -64) spare +``` + +### `raw_data_1/isis_vms_compat/NSEP` + +:::{note} +This is unconditionally written as zero. +::: + +``` + F=0 3 NSEP I*4 no. of SE parameters +``` + +### `raw_data_1/isis_vms_compat/VER5` + +``` + F 1 VER5 I*4 DAE section version # (=2) +``` + +### `raw_data_1/isis_vms_compat/DAEP` + +``` + 2 DAE PARAMETER BLOCK + U DAEP(1) I*4 Word length in bulk store memory + F DAEP(2) I*4 Length of bulk store memory (bytes)**A + U DAEP(3) I*4 PPP minimum value ***B + P DAEP(4) I*4 good PPP total (high 32 bits) ***B + P DAEP(5) I*4 good PPP total (low 32 bits) ***B + P DAEP(6) I*4 raw PPP total (high 32 bits) ***B + P DAEP(7) I*4 raw PPP total (low 32 bits) ***B + P DAEP(8) I*4 good ext. neut tot (high 32bits)***B + P DAEP(9) I*4 good ext. neut tot (low 32 bits)***B + P DAEP(10) I*4 raw ext. neut tot (high 32 bits)***B + P DAEP(11) I*4 raw ext. neut tot (low 32 bits)***B + P DAEP(12) I*4 ext. neutron gate (t1) (µs) ***B + P DAEP(13) I*4 ext. neutron gate (t2) (µs) ***B + U DAEP(14) I*4 detector for MON 1 (12 bits) ***B + U DAEP(15) I*4 module for MON 1 ( 4 bits) ***B + U DAEP(16) I*4 crate for MON 1 ( 4 bits) ***B + U DAEP(17) I*4 mask for MON 1 (c4:m4:d12) ***B + U DAEP(18) I*4 detector for MON 2 (12 bits) ***B + U DAEP(19) I*4 module for MON 2 ( 4 bits) ***B + U DAEP(20) I*4 crate for MON 2 ( 4 bits) ***B + U DAEP(21) I*4 mask for MON 2 (c4:m4:d12) ***B + P DAEP(22) I*4 total GOOD EVENTS (high 32 bits)***B + P DAEP(23) I*4 total GOOD EVENTS (low 32 bits)***B + P DAEP(24) I*4 frame synch delay (4µs steps) ***B + U DAEP(25) I*4 frm snch origin(0:none/1:ext/2:int)***B + U DAEP(26) I*4 Secondary Master Pulse (0:en,1:dis) + U DAEP(27-29) I*4 External vetoes 0,1,2 (0 dis,1 en) + DAEP( -64) Spare +``` + +### `raw_data_1/isis_vms_compat/VER6` + +``` + F 1 VER6 I*4 TCB secton version # +``` + +### `raw_data_1/isis_vms_compat/NTRG` + +``` + U/P 2 NTRG I*4 # of time regimes (normally =1) +``` + +### `raw_data_1/isis_vms_compat/NFPP` + +``` + F=1 3 NFPP I*4 # of frames per period +``` + +### `raw_data_1/isis_vms_compat/NPER` + +``` + F=1 4 NPER I*4 # of periods +``` + +### `raw_data_1/isis_vms_compat/PMAP` + +``` + F=1 5 PMAP I*4(256) period # for each basic period +``` + +### `raw_data_1/isis_vms_compat/NSP1` + +``` + U/P 3 NSP1 I*4 No. of spectra (+1 for zeroth) +``` + +### `raw_data_1/isis_vms_compat/NTC1` + +``` + U/P 4 NTC1 I*4 No. of time channels ( " ) +``` + +### `raw_data_1/isis_vms_compat/TCM1` + +``` + U 5 TCM1 I*4(5) time chan. mode (see footnote) + + TIME CHANNEL MODES: + 0 Boundaries set by a table held in file TCB.DAT + + 1 TCB1(n) = (TCP1(1) + (n-1)*TCP1(2)-DAEP(24)*4)*32/PRE1 + (ie. Dt = c ) + (ie. less frm.synch delay) + + 2 temp(1) = TCP1(1) + TCB1(1) = (TCP1(1)-DAEP(24)*4)*32/PRE1 (ie. Dt = c.t + temp(n+1)= temp(n)*(1 + TCP1(2)) + TCB1(n+1)= (temp(n+1) - DAEP(24)*4)*32/PRE1 +``` + +### `raw_data_1/isis_vms_compat/TCP1` + +``` + U 6 TCP1 R*4(4,5) time chan. parameters (µs) +``` + +### `raw_data_1/isis_vms_compat/PRE1` + +``` + P 7 PRE1 I*4 prescale value for 32MHz clock (<=15) +``` + +### `raw_data_1/isis_vms_compat/VER7` + +``` + 1 VER7 I*4 DATA version # +``` + +### `raw_data_1/isis_vms_compat/ULEN` + +:::{note} +This is unconditionally written as zero. +::: + +``` + 2 ULEN I*4 USER_LEN +``` + +### `raw_data_1/isis_vms_compat/VER9` + +Undocumented. Written as `2` (int32). + +### `raw_data_1/isis_vms_compat/NTNL` + +Undocumented. Written as `0` (int32). + +### `raw_data_1/isis_vms_compat/NTLL` + +Undocumented. Written as `0` (int32). + +### `raw_data_1/isis_vms_compat/NOTE` + +Unconditionally written as the string `"No notes were made"`. + +### `raw_data_1/isis_vms_compat/TTHE` + +``` + U 13 TTHE R*4(ndet) 2Theta table (scattering angle) +``` + +### `raw_data_1/isis_vms_compat/SPEC` + +``` + U 9 SPEC I*4(ndet) spectrum # table +``` + +### `raw_data_1/isis_vms_compat/DELT` + +``` + U 10 DELT R*4(ndet) 'HOLD OFF' in (µs) ***B +``` + +### `raw_data_1/isis_vms_compat/LEN2` + +``` + U 11 LEN2 R*4(ndet) L2 table (m) (upstrem monitors -ve. +``` + +### `raw_data_1/isis_vms_compat/CODE` + +``` + U 12 CODE I*4(ndet) code to define use of UT values ***B +``` + +### `raw_data_1/isis_vms_compat/CRAT` + +``` + U 3 CRAT I*4(ndet) crate no.for each detector +``` + +### `raw_data_1/isis_vms_compat/MODN` + +``` + U 4 MODN I*4(ndet) module no. for each detector +``` + +### `raw_data_1/isis_vms_compat/MPOS` + +``` + U 5 MPOS I*4(ndet) posn. in module for each detector +``` + +### `raw_data_1/isis_vms_compat/UDET` + +``` + U 7 UDET I*4(ndet) 'USER DETECTOR #' for each det ***B +``` + +### `raw_data_1/isis_vms_compat/TIMR` + +``` + U 6 TIMR I*4(ndet) TIME REGIME # table +``` + +### `raw_data_1/isis_vms_compat/UT01` - `UT99` + +There are `NUSE` user-tables defined. + +``` + 3 15 UT01 R*4(ndet) USER defined table 1 ***B + ..cont. to UTn +``` From 59af6bbbf37c5100c51e9dd9c85ee4f7faab9116 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 10 Sep 2026 19:13:45 +0100 Subject: [PATCH 09/13] icp_sys_td and icp_dae_td --- doc/nexus/nexus_selog.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/nexus/nexus_selog.md b/doc/nexus/nexus_selog.md index c30f44b3ce..1f76cf5da4 100644 --- a/doc/nexus/nexus_selog.md +++ b/doc/nexus/nexus_selog.md @@ -8,8 +8,8 @@ The list of blocks to be written is defined by the IBEX configuration, and is us In addition to the list of blocks from IBEX, the following logs are written: - `EPICS_PUTLOG`: a string log containing the EPICS channel access put log, with each put operation as a string. -- `ICP_DAE_TD`: TODO -- `IDP_SYS_TD`: TODO +- `ICP_DAE_TD`: This describes the difference between the DAE's internal clock and a *monotonic* software clock +- `IDP_SYS_TD`: This describes the difference between the *monotonic* software clock and an (NTP-corrected) system clock Each `IXseblock` contains the following datasets: From 58e9713754f162234d7766c57cbfde188af517a3 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Thu, 10 Sep 2026 19:27:44 +0100 Subject: [PATCH 10/13] measurement/first_run --- doc/nexus/nexus_measurement.md | 4 +--- doc/spelling_wordlist.txt | 1 + 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/doc/nexus/nexus_measurement.md b/doc/nexus/nexus_measurement.md index 29364ae4a6..bb5943bd06 100644 --- a/doc/nexus/nexus_measurement.md +++ b/doc/nexus/nexus_measurement.md @@ -20,9 +20,7 @@ This is user-settable string metadata. It is set in IBEX under Experiment Detail {#nexus_measurement_first_run} ## `raw_data_1/measurement/first_run` -:::{danger} -**TODO** - some logic in ISISICP for this... -::: +This gives the run number of the first run which had the same {ref}`nexus_measurement_id` as this run, as an int32. {#nexus_measurement_label} ## `raw_data_1/measurement/label` diff --git a/doc/spelling_wordlist.txt b/doc/spelling_wordlist.txt index 0c248ce9ba..1b04af9426 100644 --- a/doc/spelling_wordlist.txt +++ b/doc/spelling_wordlist.txt @@ -602,6 +602,7 @@ npcap NPort npp nt +NTP nxs oercone Oerlikon From 07d186329d15e941fd345448ed6ddc405a7d85e8 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Fri, 11 Sep 2026 08:45:14 +0100 Subject: [PATCH 11/13] framelog --- doc/nexus/nexus_framelog.md | 37 +++++++++++++++++++++++++++++++++++++ doc/nexus/nexus_runlog.md | 4 ++-- 2 files changed, 39 insertions(+), 2 deletions(-) create mode 100644 doc/nexus/nexus_framelog.md diff --git a/doc/nexus/nexus_framelog.md b/doc/nexus/nexus_framelog.md new file mode 100644 index 0000000000..d8c32a7197 --- /dev/null +++ b/doc/nexus/nexus_framelog.md @@ -0,0 +1,37 @@ +# `raw_data_1/framelog` + +This is an {external+nexus_manual:doc}`classes/base_classes/NXcollection`. It is only written if event-mode is enabled. +Each subgroup is an {external+nexus_manual:doc}`classes/base_classes/NXlog`. + +The datasets written are: +- `events_log` (int32, units="events") + - The number of events in this frame +- `frame_log` (int32, units="frame_number") + - The frame number (this will always monotonically increase by 1 for each frame) +- `good_frame_log` (int32, units="is_good") + - `1` if the frame was a good frame, `0` otherwise +- `period_log` (int32, units="period_number") + - The period number for each frame +- `proton_charge` (float32, units="uAh") + - The proton charge delivered by this frame +- `raw_events_log` (int32, units="events") + - The number of raw events (good+bad) in this frame + +{#nexus_framelog_value} +### `raw_data_1/framelog//value` + +This contains the data for the given parameter. + +Attributes: +- `units`: Units specified in the list above + +{#nexus_framelog_time} +### `raw_data_1/framelog//time` + +This contains float32 timestamps of the values, relative to the start of the run. + +It is written once per raw frame. + +Attributes: +- `start`: the start time of the run, for example `"2026-09-09T16:16:57"` +- `units`: `second` \ No newline at end of file diff --git a/doc/nexus/nexus_runlog.md b/doc/nexus/nexus_runlog.md index be2553fe0c..2123744dc6 100644 --- a/doc/nexus/nexus_runlog.md +++ b/doc/nexus/nexus_runlog.md @@ -2,7 +2,7 @@ This has an `NX_class` of `IXrunlog`; this is not a class specified by upstream NeXus definitions. -Each subgroup is an {external+nexus_manual:doc}`classes/base_classes/NXdata`. +Each subgroup is an {external+nexus_manual:doc}`classes/base_classes/NXlog`. The datasets written are: - `count_rate` (float32, units="counts") @@ -53,7 +53,7 @@ Attributes: {#nexus_runlog_time} ### `raw_data_1/runlog//time` -This contains int32 timestamps of the values. +This contains float32 timestamps of the values. It is written approximately every 30 seconds, and is relative to the start of the run. From 0d861787004e3271a089532305ef9dde67461fdb Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Fri, 11 Sep 2026 09:23:40 +0100 Subject: [PATCH 12/13] detector__events --- doc/nexus/nexus_detector_events.md | 44 +++++++++++-- .../nexus_instrument_detector.md | 63 +++++++++++++++++++ 2 files changed, 103 insertions(+), 4 deletions(-) diff --git a/doc/nexus/nexus_detector_events.md b/doc/nexus/nexus_detector_events.md index 9c7cfc8139..75b125d94b 100644 --- a/doc/nexus/nexus_detector_events.md +++ b/doc/nexus/nexus_detector_events.md @@ -1,7 +1,43 @@ # `raw_data_1/detector__events` -This is an {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. +This is an {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. It is only written in event mode. -:::{caution} -TODO -::: +{#nexus_detevents_framenumber} +### `raw_data_1/detector__events/event_frame_number` + +NeXus link to {ref}`nexus_instrument_detector_eventframenumber` + +{#nexus_detevents_eventid} +### `raw_data_1/detector__events/event_id` + +NeXus link to {ref}`nexus_instrument_detector_eventid` + +{#nexus_detevents_eventindex} +### `raw_data_1/detector__events/event_index` + +NeXus link to {ref}`nexus_instrument_detector_eventindex` + +{#nexus_detevents_eventtimezero} +### `raw_data_1/detector__events/event_time_zero` + +NeXus link to {ref}`nexus_instrument_detector_eventtimezero` + +{#nexus_detevents_eventtimebins} +### `raw_data_1/detector__events/event_time_bins` + +NeXus link to {ref}`nexus_instrument_dae_time_channels_tof` + +{#nexus_detevents_eventtimeoffset} +### `raw_data_1/detector__events/event_time_offset` + +NeXus link to {ref}`nexus_instrument_detector_eventtimeoffset` + +{#nexus_detevents_eventtimeoffsetshift} +### `raw_data_1/detector__events/event_time_offset_shift` + +NeXus link to {ref}`nexus_instrument_detector_eventtimeoffsetshift` + +{#nexus_detevents_totalcounts} +### `raw_data_1/detector__events/total_counts` + +NeXus link to {ref}`nexus_instrument_detector_totalcounts` diff --git a/doc/nexus/nexus_instrument/nexus_instrument_detector.md b/doc/nexus/nexus_instrument/nexus_instrument_detector.md index c6e5985c59..4facd6236a 100644 --- a/doc/nexus/nexus_instrument/nexus_instrument_detector.md +++ b/doc/nexus/nexus_instrument/nexus_instrument_detector.md @@ -101,3 +101,66 @@ made from multiple detector pixels. User tables start from `user_table01`. `user_table01` is always equivalent to {ref}`nexus_instrument_detector_azangle`. + +{#nexus_instrument_detector_eventframenumber} +### `raw_data_1/instrument/detector_/event_frame_number` + +This is the frame number corresponding to each entry in {ref}`nexus_detevents_eventtimezero`, as an int32. It always +increments by 1 for each frame. + +This is only written in event mode. See also {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +{#nexus_instrument_detector_eventid} +### `raw_data_1/instrument/detector_/event_id` + +This is the *spectrum* (not *detector*) that each event was detected on. + +This is only written in event mode. See also {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +{#nexus_instrument_detector_eventindex} +### `raw_data_1/instrument/detector_/event_index` + +The index into the event_time_offset, event_id pair for the pulse occurring at the matching entry in event_time_zero. + +This is only written in event mode. See also {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +{#nexus_instrument_detector_eventtimezero} +### `raw_data_1/instrument/detector_/event_time_zero` + +This is the time that each pulse started, as float32 microseconds, with respect to the offset which is the beginning +of a run. + +This is only written in event mode. See also {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +Attributes: +- `offset`: The start of run timestamp, for example `"2025-03-27T13:38:28"` +- `units`: `"second"` + +{#nexus_instrument_detector_eventtimeoffset} +### `raw_data_1/instrument/detector_/event_time_offset` + +This is the timestamp of each event, as float32 microseconds, with respect to the offset in {ref}`nexus_instrument_detector_eventtimezero`. + +This is only written in event mode. See also {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. + +Attributes: +- `units`: `"second"` + +{#nexus_instrument_detector_eventtimeoffsetshift} +### `raw_data_1/instrument/detector_/event_time_offset_shift` + +This dataset contains the string `"random"` if event time offsets have been assigned a randomised position within the +time bin given by {ref}`nexus_detevents_eventtimebins`. + +This randomisation is needed to avoid artefacts in downstream reduction software when using event-mode hardware with a +low resolution clock (for example DAE2/3). + +This is only written in event mode. + +{#nexus_instrument_detector_totalcounts} +### `raw_data_1/instrument/detector_/total_counts` + +This dataset contains the total number of event-mode events, as an int64. It is equal to the length of the +{ref}`event_time_offset ` or {ref}`event_id ` datasets. + +This is only written in event mode. From 83d7c1392c78255f6745867b56cecb4195d68f43 Mon Sep 17 00:00:00 2001 From: Tom Willemsen Date: Fri, 11 Sep 2026 16:28:13 +0100 Subject: [PATCH 13/13] Review comments --- doc/Glossary.md | 7 +++++++ doc/Nexus.md | 7 ++++++- doc/nexus/nexus_detector.md | 3 ++- doc/nexus/nexus_detector_events.md | 1 + doc/nexus/nexus_framelog.md | 3 ++- 5 files changed, 18 insertions(+), 3 deletions(-) diff --git a/doc/Glossary.md b/doc/Glossary.md index b5a0a8a477..2c8e187fd8 100644 --- a/doc/Glossary.md +++ b/doc/Glossary.md @@ -205,6 +205,13 @@ A network-based control system. Some elements of it are used in the [script ser **Operator Interface**. A file used in [CSS](#css) to enable a user to interact with an IOC graphically. +## Period + +This is a data acquisition concept, used to segregate a run into logical components. Periods can be changed: +- By software, for example to separate data points during a scan of a motion axis +- By the data acquisition hardware, which may for example alternate between two periods and provide signals to external hardware +- By external hardware, which may tell the data acquisition system which period to collect in (for example, pulsed magnet on or off) + ## Perspective ## Plugin diff --git a/doc/Nexus.md b/doc/Nexus.md index c2923d662d..40d7d940c9 100644 --- a/doc/Nexus.md +++ b/doc/Nexus.md @@ -1,9 +1,14 @@ # NeXus Files -IBEX writes out scientific data as {external+nexus:doc}`NeXus files `. The NeXus format documents the *general* +IBEX writes out scientific data as {external+nexus:doc}`NeXus files `, which are a set of definitions built +on top of the HDF5 format and used by most scattering-type experiments (including x-ray, lasers, Neutron and Muon +facilities worldwide). The NeXus format documents the *general* layout of a NeXus file; this documentation documents the *specific* datasets written in practice by IBEX, including any quirks or facility-specific details of their interpretation. +On a DAE2 or DAE3 instrument, these files are written by the ISISICP process. On a streaming instrument, these files +are written by the data streaming filewriter. + The NeXus files conform, where possible, to the {external+nexus:doc}`TOFRaw` definition. ## Data types diff --git a/doc/nexus/nexus_detector.md b/doc/nexus/nexus_detector.md index 9439d13076..f1fb78d78a 100644 --- a/doc/nexus/nexus_detector.md +++ b/doc/nexus/nexus_detector.md @@ -1,6 +1,7 @@ # `raw_data_1/detector_` -This is an {external+nexus_manual:doc}`classes/base_classes/NXdata`. +This is an {external+nexus_manual:doc}`classes/base_classes/NXdata`. This is always a histogram; for event data, see +{ref}`nexus_detevents`. ### `raw_data_1/detector_/counts` diff --git a/doc/nexus/nexus_detector_events.md b/doc/nexus/nexus_detector_events.md index 75b125d94b..58e0cca340 100644 --- a/doc/nexus/nexus_detector_events.md +++ b/doc/nexus/nexus_detector_events.md @@ -1,3 +1,4 @@ +{#nexus_detevents} # `raw_data_1/detector__events` This is an {external+nexus_manual:doc}`classes/base_classes/NXevent_data`. It is only written in event mode. diff --git a/doc/nexus/nexus_framelog.md b/doc/nexus/nexus_framelog.md index d8c32a7197..2be29c4bbb 100644 --- a/doc/nexus/nexus_framelog.md +++ b/doc/nexus/nexus_framelog.md @@ -1,6 +1,7 @@ # `raw_data_1/framelog` -This is an {external+nexus_manual:doc}`classes/base_classes/NXcollection`. It is only written if event-mode is enabled. +This is an {external+nexus_manual:doc}`classes/base_classes/NXcollection` which contains a set of time-series arrays +with equal sizes, indexed by the frame number. It is only written if event-mode is enabled. Each subgroup is an {external+nexus_manual:doc}`classes/base_classes/NXlog`. The datasets written are: