Changes to CPLHIST mode for robustness and getting the right default settings for CLM and an updated 1850 ndep file

[ekluzek]

Description of changes

Work with 1850_clim settings for ndep. As well as some work with CPLHIST options.

Specific notes

Contributors other than yourself, if any: @billsacks

CDEPS Issues Fixed (include github issue #):

• Fixes Add error checking and set defaults for DATM_CPLHIST_DIR and DATM_CPLHIST_CASE #399
• Fixes Bring in option for the new preliminary CMIP7 1850 ndep dataset #388
• Fixes Add expect statements to the DATM buildnml for the NEON/PLUMBER2/generic tower settings #432
• Fixes Update CPLHIST data for DATM for cesm3 spinup with preliminary data #397
• Fixes New DATM CMIP7 CO2 file replacing earlier one that would fail after 6 month simulation #410

Are there dependencies on other component PRs (if so list):

Are changes expected to change answers (bfb, different to roundoff, more substantial): No
This will add some new options, but won't change defaults (which primarily need to be changed in compsets anyway)

Any User Interface Changes (namelist or namelist defaults changes): Yes

Testing performed (e.g. aux_cdeps, CESM prealpha, etc): Have tested a few cases will test aux_cdeps against cesm3_0_beta08 tests

Hashes used for testing:

Definition of done:

• Finish work on all of the modes
• Add needed aux_cdeps tests
• Test against the aux_cdeps cesm3_0_beta09 baseline and make sure identical
• Test in the context of CTSM and make sure it does everything needed

[wwieder]

It's not clear to me if the intent of this PR + #405 are needed for:

1. CLM to be able to use CPL_HIST output (and therefore lower priority for an alpha tag); or
2. CESM to write out CPL_HIST files that we'll use for spinup (and therefore a higher priority)?

[billsacks]

It's not clear to me if the intent of this PR + #405 are needed for:

1. CLM to be able to use CPL_HIST output (and therefore lower priority for an alpha tag); or
2. CESM to write out CPL_HIST files that we'll use for spinup (and therefore a higher priority)?

It's (1) - to be able to use CPLHIST output.

[wwieder]

OK, if this is only needed for CTSM to read CPL_HIST data my read is that this PR isn't critical for the next alpha09 tag. What do others think?

[ekluzek]

@fischer-ncar I didn't find aux_cdeps baselines for either cesm3_0_alpha09g or cesm3_0_beta09. So I'm going to make my own baseline with the previous CDEPS tag.cd .

[ekluzek]

The aux_cdeps testlist all passed just now. But, I need to make a new baseline to compare it to the previous version.

[ekluzek]

I ran aux_cdeps and it passes. And everything is identical to the new baseline I made otehr than the cplhist tests.

 ../../CIME/Tools/compare_test_results -b cesm3_0_alpha09g.cdeps1.0.105 --baseline-root $CPL_BASELINE/ -r . | & grep FAIL
FAIL ERS_Ld5.f10_f10_mt232.1850_DATM%CRUJRA2024b_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist_create NLCOMP See ./ERS_Ld5.f10_f10_mt232.1850_DATM%CRUJRA2024b_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist_create.GC.cesm3A9gcdeps115cplhistaacd/compare.log.cesm3_0_alpha09g.cdeps1.0.105.20260821_145458
FAIL SMS_Ld5.f10_f10_mt232.1850_DATM%CPLHIST_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist NLCOMP See ./SMS_Ld5.f10_f10_mt232.1850_DATM%CPLHIST_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist.GC.cesm3A9gcdeps115cplhistaacd/compare.log.cesm3_0_alpha09g.cdeps1.0.105.20260821_145458
FAIL SMS_Ld5.f10_f10_mt232.1850_DATM%CPLHIST_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist BASELINE DIFF

The first case cplhist_create is just different in NLCOMP because it was done postrun. So it's actually identical.

The second case shows differences due to NDEP, and using CPRNC by hand I see this...

cases/SMS_Ld5.f10_f10_mt232.1850_DATM%CPLHIST_SLND_SICE_SOCN_SROF_SGLC_SWAV_SESP.derecho_intel.datm-cplhist.GC.cesm3A9gcdeps115cplhistaacd> grep RMS cprnc.out 
 RMS atmImp_Faxa_ndep1                1.4839E-12            NORMALIZED  4.2846E+00
 RMS atmImp_Faxa_ndep2                2.7585E-12            NORMALIZED  9.0720E+00
tail -13 cprnc.out
SUMMARY of cprnc:
 A total number of     51 fields were compared
          of which      2 had non-zero differences
               and      0 had differences in fill patterns
               and      0 had different dimension sizes
               and      0 had different data types
 A total number of      0 fields could not be analyzed
 A total number of      0 time-varying fields on file 1 were not found on file 2.
 A total number of      0 time-constant fields on file 1 were not found on file 2.
 A total number of      0 time-varying fields on file 2 were not found on file 1.
 A total number of      0 time-constant fields on file 2 were not found on file 1.
  diff_test: the two files seem to be DIFFERENT 

so this is as expected.

[ekluzek]

Note, the new baseline I created was identical to cesm3_0_alpha09f baselines with the exception of the scam test which I assume must be expected.

[ekluzek]

Some notes on things I figured out.

[ekluzek]

I think I have everything in place now, so opening up for review. I'm going to redo the testing. And I'll have some peeps review it as well, before asking for merge.

[slevis-lmwg]

@ekluzek and I went over this in detail, and I'm approving.
@ekluzek posted some notes.

[billsacks]

@mvertens - I'm adding you as a reviewer mainly to look at whether this will impact anything in NorESM that you want to comment on.

[ekluzek]

OK, I did the things from the code review from @slevis-lmwg. I'm going to redo testing, and verifty things are expected, and then I'd like to merge it, so I can have a tag for CTSM.

[ekluzek]

OK, I am sending testing for aux_clm and aux_cdeps and will want to merge once that testing is complete.

[billsacks]

I have a lot of specific comments here, but by and large I really appreciate all of the work you've done to both get cplhist mode working and also to do some cleanup and add some error checking to the datm buildnml! Thank you!!

[billsacks]

I want to confirm this: It looks like the previous version of this was set up for cpl7-based cplhist, so I see the need for this change. It looks like this triggers the cmip6 (as opposed to use_cmip7_ndep) block in the Fortran. It looks like this leads to the following block in datm_pres_ndep_advance:

       ! convert ndep flux to units of kgN/m2/s (input is in gN/m2/s)
       Faxa_ndep(1,:) = strm_Faxa_ndep_nhx(:) / 1000._r8
       Faxa_ndep(2,:) = strm_Faxa_ndep_noy(:) / 1000._r8

Does that look correct - in particular this unit conversion? If not, we'll have to disambiguate these cases.

[billsacks]

It's not clear to me why you have this commented-out alternative block. Is this needed for some files? Can you either clarify in the comment why you might want to switch to this or else just remove it?

Also, spelling error: Alturnative -> Alternative.

[billsacks]

Typo: wtih -> with

[billsacks]

What is the rationale for this change - i.e., only setting cplhist mode for compsets with CLM?

If this is kept, the regex should be changed to "_DATM%CPLHIST.*_CLM"

[billsacks]

This expect is helpful - thanks! - since it checks consistency of user inputs.

[billsacks]

It might feel kind of arbitrary, but I also feel like this expect is helpful. It's hard for me to explain my thinking on this one vs. the other ones in _handle_model_grid. Maybe the difference is that this is a single, simple expect that feels like it's doing a bigger job, and also less likely to need changes if you change the production code, partly because the expect is unconditional, so you don't need to replicate logic from earlier in the routine in order to do this expect? But I do acknowledge that this all feels subjective, so I'm open to disagreements with my feelings on where there should / shouldn't be expects.

However: looking at the above logic, it looks like config['single_column'] isn't set if case.get_value('PTS_DOMAINFILE') gives None or an empty string. So maybe you need to add to the logic to make this expect always pass?

[billsacks]

Minor nitpick: I would rewrite these two separate expect statements into a single expect statement based on cplhist_case is not None and cplhist_case != "UNSET": I find it clearer when reading the code to have a single expect on a variable when possible. (I read the first expect and was initially confused about why you were checking against None rather than UNSET... then realized that you had two expects for this variable.)

I do appreciate the expects ensuring that these various DATM_CPLHIST variables are set when they should be!!

[billsacks]

Tied in with my comment elsewhere about preferring a default value of UNSET for DATM_CPLHIST_DIR: that would make this error check more straightforward (you could just check to make sure that it isn't still UNSET).

I don't love the check of os.path.isdir here: I appreciate the intent, but it feels too early in the process to check this: I could imagine someone setting up a case and running preview_namelists before setting up the directory structure that actually contains the cplhist forcings. It may be unlikely, but it seems like that should be an acceptable order and we shouldn't force the directory to already exist at preview_namelists time. But I could be convinced otherwise, particularly if we already enforce directory existence at preview_namelists time in other places.

[billsacks]

As with my comment for DATM_CPLHIST_DIR, I feel like preview_namelists might be too early to do this file existence check. Do we do file existence checks elsewhere at preview_namelists time? (If so, then I stand corrected, so ignore this comment.)