close
Skip to content

Add MPAS_ESMF for external ESMF linking#1405

Merged
mgduda merged 1 commit intoMPAS-Dev:developfrom
NCAR:extesmf
Mar 10, 2026
Merged

Add MPAS_ESMF for external ESMF linking#1405
mgduda merged 1 commit intoMPAS-Dev:developfrom
NCAR:extesmf

Conversation

@danrosen25
Copy link
Copy Markdown
Collaborator

@danrosen25 danrosen25 commented Feb 4, 2026 •
edited
Loading

This commit replaces hardcoded links to external/esmf_time_f90 in Makefiles with MPAS_ESMF_INC and MPAS_ESMF_LIBS. MPAS_ESMF_INC and MPAS_ESMF_LIBS are calculated in the root Makefile depending on MPAS_ESMF=embedded (default) or MPAS_ESMF=external. MPAS_ESMF=embedded will use the external/esmf_time_f90 folder and MPAS_ESMF=external will read INC and LIBS variables in a ESMFMKFILE file. These variables are then exported for all other Makefiles.

I first tried adding include paths and libraries to FCINCLUDES and LIBS but this failed while checking PIO when executing MPAS_ESMF=embedded because external/esmf_time_f90 isn't build until after the PIO check.

As I'm new to MPAS and the MPAS build system please let me know if I'm on the right track.

Testing

Testing within a Docker container that includes mpich, netcdf-c, netcdf-fortran, parallel-netcdf, parallelio, and esmf (libraries installed using spack).
make clean gnu CORE="atmosphere"
make clean gnu CORE="atmosphere" MPAS_ESMF=embedded
make clean gnu CORE="atmosphere" MPAS_ESMF=external

  • tested with and without ESMFMKFILE environment variable

Before change
clean build ~2m24s
rebuild. ~0m18s

MPAS_ESMF=embedded
clean build ~2m25s
rebuild ~0m18s

MPAS_ESMF=external
clean build ~3m38s
rebuild ~0m35s

@danrosen25
Copy link
Copy Markdown
Collaborator Author

danrosen25 commented Feb 4, 2026 •
edited
Loading

@scrasmussen @uturuncoglu
I have opened a pull request to allow external ESMF libraries to link into MPAS using the Makefile build system. The embedded version of ESMF is static and outdated. It is only a partial set of the ESMF library does not include NUOPC.

@danrosen25
Copy link
Copy Markdown
Collaborator Author

danrosen25 commented Feb 4, 2026

@mgduda
This is the work I mentioned in the meeting. The Makefile build system can swap between linking to the internal (which I think of as embedded) ESMF code versus and external ESMF library. I called the configuration option MPAS_ESMF but I'm open to suggestions. The external ESMF option leverages and ESMFMKFILE, which is pretty standard.

@danrosen25 danrosen25 mentioned this pull request Feb 4, 2026
Closed
@danrosen25 danrosen25 linked an issue Feb 9, 2026 that may be closed by this pull request
Link external ESMF library during GNU Make build NCAR/MPAS-Model-NUOPC-dev#11
Closed
@mgduda mgduda self-requested a review March 6, 2026 20:26
@mgduda mgduda added the Build System Changes related to the build system, either `Make` or `CMake`. label Mar 6, 2026
@mgduda mgduda changed the base branch from master to develop March 6, 2026 20:27
@danrosen25 danrosen25 linked an issue Mar 6, 2026 that may be closed by this pull request
Duplicate: Modify Makefile build system to reference external ESMF library NCAR/MPAS-Model-NUOPC-dev#13
Closed
mgduda
mgduda requested changes Mar 6, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@mgduda mgduda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In addition to the comment about the error message, I'd suggest a couple of other changes:

  1. Somewhere around this point in the top-level Makefile, should we include a message in the build summary to indicate which ESMF timekeeping library was used?
  2. Around this point in the top-level Makefile, we could include a description of the new MPAS_ESMF build option that's been introduced in this PR.
  3. In the commit message (not the GitHub PR description), could you include a bit more detail regarding the changes in the commit?

Comment thread Makefile Outdated
@danrosen25
Copy link
Copy Markdown
Collaborator Author

danrosen25 commented Mar 7, 2026

Hi @mgduda
Thanks for your review. I think I've address all of your comments. Some of which I had already handled in the 'nuopc/build' follow on work after learning the MPAS build system better.

Note, I also did some build timing tests to see if linking to an external ESMF library makes a difference and it does. I think the external ESMF library is much larger and has a few downstream 3rd party libraries that get linked in too. There may be a way to speed this up using dynamic linking. But I think the increased build time is to be expected.

Before change
clean build ~2m24s
rebuild. ~0m18s

MPAS_ESMF=embedded
clean build ~2m25s
rebuild ~0m18s

MPAS_ESMF=external
clean build ~3m38s
rebuild ~0m35s

@danrosen25 danrosen25 requested a review from mgduda March 7, 2026 17:06
@danrosen25
Copy link
Copy Markdown
Collaborator Author

danrosen25 commented Mar 7, 2026 •
edited
Loading

@mgduda
I forgot that this is a request to merge into develop so I've rebased onto develop and fixed the conflict. Develop is the correct branch for this work, right? Also, the timings above are for 1 build task. And I made one more edit to clean up the new references to 'esmf_time_f90' that were added with chemistry

@danrosen25
Add MPAS_ESMF to Makefile opts for external ESMF
66517f6 
- MPAS_ESMF=embedded (default), uses internal ESMF timekeeping lib
- MPAS_ESMF=external, uses external ESMF lib defined by ESMFMKFILE
- Replaces hardcoded esmf_time_f90 include with MPAS_ESMF_INC
- Replaces hardcoded esmf_time lib with MPAS_ESMF_LIBS
@mgduda mgduda merged commit 8da696a into MPAS-Dev:develop Mar 10, 2026
@danrosen25 danrosen25 deleted the extesmf branch March 10, 2026 16:04
mgduda added a commit to mgduda/MPAS-Model that referenced this pull request Mar 19, 2026
@mgduda
Remove defc_a and defc_b from the init_atmosphere and atmosphere cores
968254e 
With the introduction of the deformation_coef_* fields in merge commit 63f2d44
(PR MPAS-Dev#1405), the defc_a and defc_b fields are no longer needed in either the
init_atmosphere or the atmosphere core. Accordingly, this commit removes the
defc_a and defc_b fields from the Registry.xml files as well as source code in
both of these cores.
@mgduda mgduda mentioned this pull request Mar 19, 2026
Merged
mgduda added a commit that referenced this pull request Mar 20, 2026
@mgduda
Merge branch 'atmosphere/remove_defc_ab' into develop (PR #1422)
662394e 
This merge removes the defc_a and defc_b fields from the init_atmosphere and
atmosphere cores.

With the introduction of the deformation_coef_* fields in merge commit 63f2d44
(PR #1405), the defc_a and defc_b fields are no longer needed in either the
init_atmosphere core or the atmosphere core. Accordingly, this merge removes the
defc_a and defc_b fields from the Registry.xml files as well as from the source
code of both of these cores.

* atmosphere/remove_defc_ab:
  Remove defc_a and defc_b from the init_atmosphere and atmosphere cores
mgduda added a commit that referenced this pull request Apr 16, 2026
@mgduda
Merge branch 'release-v8.4.0'
515e520 
MPAS Version 8.4.0

This release of MPAS introduces new capabilities and improvements in the
MPAS-Atmosphere model and its supporting software infrastructure, and it is the
first release to provide the foundation for CheMPAS-A, a chemistry capability
for MPAS-Atmosphere. Notable changes are listed below.

MPAS-Atmosphere Initialization:

 * Fix an issue in snow initialization over areas covered by seaice. (PR #1438)

 * Introduce namelist control over the vertical coordinate configuration for
   real-data case initialization. Two new namelist options may now be specified
   in the &vertical_grid namelist group in the namelist.init_atmosphere file:

    - config_hybrid_coordinate (default: true)
      Whether to employ a hybrid vertical coordinate

    - config_hybrid_top_z (default: 30000 m)
      Height at which coordinate surfaces become constant height surfaces

   Additionally, the smoothing coefficient formula for the hybrid coordinate now
   uses the transition height instead of the model top height. (PR #1382)

 * Add computation of the edgeNormalVectors, cellTangentPlane, and
   localVerticalUnitVectors fields for MPAS-A initialization case 13 (CAM-MPAS
   3-d grid) to better support alternative initialization workflows for
   CAM-MPAS. (PR #1351)

MPAS-Atmosphere Dynamics:

 * Implement an initial Large-Eddy Simulation (LES) capability. Similar to WRF,
   it contains two subgrid turbulence models -- a diagnostic Turbulent Kinetic
   Energy (TKE) formulation based on a 3-d Smagorinsky formulation, and a
   1.5-order prognostic TKE formulation. These formulations generally follow
   the implementation in WRF (see the WRF Technical Note Version 4, sections
   4.2.3 and 4.2.4 for a description of the formulations). (PR #1404)

 * Generalize the epssm parameter, allowing level-dependent values to be
   specified through four new namelist options:

    - config_epssm_minimum (default value: 0.1)
    - config_epssm_maximum (default value: 0.5)
    - config_epssm_transition_bottom_z (default value: 30000.0)
    - config_epssm_transition_top_z (default value: 50000.0)

   The epssm parameter is an off-centering coefficient for the vertically
   semi-implicit acoustic / gravity-wave integration, and it was previously
   specified as a single, constant value using the config_epssm namelist option.
   (PR #1381)

 * Add the capability to turn off microphysics tendencies above a specified
   height with a new namelist option, config_microphysics_top, in the &physics
   namelist group. Ignoring tendencies from the microphysics at upper levels has
   been found to alleviate some instabilities in deep-domain (i.e., model tops
   above the stratopause) MPAS-Atmosphere simulations. (PR #1380)

 * Optimize OpenACC data movement throughout the MPAS-A dycore, so that the
   majority of fields are transferred to the device at the start of a dynamics
   step and from the device at the end of a dynamics step. (PR #1315)

 * Remove the scaling of the gravity-wave absorbing layer coefficient (dss) by
   the local mesh size. Previously, the absorbing layer coefficient was scaled
   by dx/dx_fine, and for large values of dx/dx_fine, instabilities have been
   encountered associated with this scaling of the absorbing layer coefficient.
   (PR #1379)

MPAS-Atmosphere Physics:

 * Correct an issue with high q2 values over urban cells in Noah-MP. (PR #1383)

 * Fix crashes when using the CAM SW radiation scheme. (PR #1391)

 * Fix an indexing typo in mpas_atmphys_interface.F, where evapprod(k,k) was
   used instead of evapprod(k,i). The fix corrects diagnostic output, but should
   not affect the model state or simulation results. (PR #1396)

 * Correct the units designations of the gravity wave drag diagnostics variables
   dusfcg, dvsfcg, dtaux3d and dtauy3d in the atmosphere core's Registry.xml
   file. (PR #1378)

MPAS-Atmosphere Misc.:

 * Remove PV diagnostic fields from the default "output" stream for
   MPAS-Atmosphere, reducing the size of the default "history" netCDF files by
   ~20%. (PR #1428)

 * Modify the MPAS-Atmosphere CMake build system to use manage_externals to
   obtain specific tags from the https://github.com/NCAR/MMM-physics.git and
   https://github.com/NOAA-GSL/UGWP.git repositories, rather than simply
   obtaining the HEAD of the default branch. This modification corrects build
   failures when using CMake. (PR #1421)

CheMPAS-A:

 * Enable linking with the MUSICA-Fortran library to support development of
   chemistry capabilities based on MICM and TUV-x. (PR #1309)

 * Add a new namelist group, &musica, with a single option, config_micm_file, to
   control the MICM configuration in MPAS-Atmosphere. The new namelist group is
   only included in the namelist.atmosphere file (and therefore, only
   recognized) when MPAS is compiled with support for the MUSICA library.
   (PR #1376)

 * Introduce directory structure and stub modules for MUSICA, and more broadly,
   for chemistry options in the atmosphere core. (PR #1360)

   New directories and modules include:

    src/
      core_atmosphere/
        chemistry/              <--|
          mpas_atm_chemistry.F  <--|
          musica/               <--| new in this release
            mpas_musica.F       <--|

Infrastructure:

 * Introduce the ability to partition meshes "online" at model startup using the
   PT-Scotch library. (PR #1364)

 * Enable execution of halo exchanges directly on GPU-resident fields when MPAS
   is linked with an MPI library that is GPU-aware. To enable GPU-aware halo
   exchanges, a new namelist option, config_gpu_aware_mpi, in the &development
   namelist group must be set to true at runtime. (PR #1355)

 * Improve support for using an external ESMF library through a new build
   option, MPAS_ESMF. (PR #1405)

 * Set the default PnetCDF header alignment to 128 KiB when creating new output
   files with SMIOL. This has been found to fix I/O issues when overwriting
   existing model output files under certain conditions. (PR #1386)

 * Fix an issue with macro expansion with newer Intel oneAPI compilers by adding
   a flag for the 'intel' build target to change the order of macro expansion in
   the Fortran pre-processor. This change in macro expansion order fixes an
   issue with some nested macro use in MPAS. (PR #1392)

 * Fix a buffer overflow in the streaminfo_query function in the
   mpas_stream_inquiry module , where the local array c_attname was allocated
   without space for a C null character. (PR #1352)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgduda mgduda mgduda approved these changes

Assignees

No one assigned

Labels

Build System Changes related to the build system, either `Make` or `CMake`.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Duplicate: Modify Makefile build system to reference external ESMF library Link external ESMF library during GNU Make build

2 participants

@danrosen25 @mgduda