convert most common block to runtime allocated memory blocks by yslan · Pull Request #11 · yslan/Nek5000

yslan · 2026-07-24T15:06:31Z

WIP, this is to test new mem branch

Progress:

merge master to import upstream changes
(makenek) SIZE -> SIZE.usr logic with warnning.
(makenek) fix NEK_WARN via obj/.nekwarn
(makenek) remove stray .usr in auto-detect-usr mode
(makenek) add MODFLAG to support different compiler
(makenek) address linking deps for make lib, add libtest
(make) fix dependency. usr -> mod -> .f -> lib
(mkSIZE, nekTestCase.py) now generate SIZE.usr
(usr) fix implicit none locations
(timer) fix if (icalld.eq.0) counter=0, and move all in initialization to time00
Even original code has bugs in counter. This also fix some conflict of icalld.
Drop icalld from CTIMER as all variables are initialized in time00 now,
mpi_dummy.fis pure F77 and implicit none needs to be place before include.
fix a equivalent -> transfer bug. duplicate local memory and use chcopy.
Move memory allocation to 1nek_mem_init()1 and called in 1nek_init()1
This is for downstream app to link libnek5000.a directly.
Add libtest into makefile for passing flags, add pytest (LibLink) and CI (LibLink).
(NekTests.py) interface use bin to call any executable.
(NekTests.py) fix Popen to properly close process and remove ResourceWarning.
Check err after allocate.
- Make exitt calls pre-MPI safe
- module alloc: catch err inside init
- in-place alloc: do not use stat=ierr and let it fail

TODOs

position of implicit none, module files and include files need changes.
Should we add checks in makenek?
Not all all-cap files are modules. There are still header like PARDICT.
SDMP2 is dropped on purpose?
Do we prefer "allocate/free" or "ahead-or-time" for one time function like restart? I always have doubt whether restart in the middle of timestep will ruin the scratch memory for time advancing.
Should we also initialize some variable in module -> init()?
Should we deallocate modules in nek_end?
Follow up from the previous items. valgrind reports lots of uninitialized vars and conditional jump. Not necessary a bug, but it's a concern.
(make) similar to USR, a USR_MOD in case there are mod used in usr?

For 1 and 2, maybe have a lint tool that help identify or even auto fix F77 -> F90 transition. Especially, for usr.

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ec1ea1a24

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

PR: Nek5000 Nek5000#899 PR: Nek5000 Nek5000#901 PR: Nek5000 Nek5000#903 PR: Nek5000 Nek5000#908 (open)

Fix copying SIZE to SIZE.usr Fix NEK_WARN, populated to new file: obj/.nekwarn

touch $1.usr doesn't support auto-detect-usr mode

- make sure all timer variables are declared in CTIMER and initialized in time00 - drop icalld from CTIMER

yslan · 2026-07-25T03:30:20Z

@codex please review again

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6d2faa5a15

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yslan · 2026-07-25T05:14:03Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 72821d6960

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

for lib linker

- Also fix NekTests.py ResourceWarnings

real*8 timers

yslan · 2026-07-25T07:15:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 61aa65de01

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

clean up timestamp

yslan · 2026-07-25T16:24:44Z

Makefile: the graph has 4 forced phases (three barriers)

PHASE 0: 3rd party libs
PHASE 1: size_mod.o (0.03s)
PHASE 2: 70 modules (parallel,~1.6s)
PHASE 3: ~147 objects (parallel, ~6.4s) 
PHASE 4: archive LIB -> compile drive.f -> link (0.43s)

yslan · 2026-07-25T16:27:13Z

@codex review. check if previous issues are indeed addressed.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e5798d34c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yslan · 2026-07-25T16:49:13Z

@codex review focusing exclusively on correctness, edge cases, and regressions

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e5798d34c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yslan · 2026-07-25T17:14:59Z

@codex review focusing exclusively on memory safety, initialization, ownership, and leaks

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e5798d34c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

yslan · 2026-07-25T17:23:21Z

@codex review focusing exclusively on MPI correctness, parallel races, and multi-rank behavior

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e5798d34c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

- This avoid SEGV at 2D and low lx1 cases - Systematically identified by audit_cb_alloc.py

yslan · 2026-07-25T18:25:48Z

Here is the overall logic of the makenek to help user for SIZE.usr transition.
Triggered when there is SIZE presented in the case folder.

If there is no SIZE.usr
  mv SIZE SIZE.usr
If there is also SIZE.usr
  if they are the same
     rm SIZE
  if they are different
    ask user if it's safe to overwrite SIZE.usr with SIZE
    if the answer is no, abort and let user resolve conflict.

Either case, a warning message will be printed twice.

The first time is right before compilation
The second time is printed before "Compilation Successful"
1 is for debugging. 2 is to raise attention to users.

catch memory allocation error, which turns into harder to detect SEGV later.

yslan · 2026-07-25T19:25:36Z

Make exit calls pre-MPI safe

Routine	Signature	Prints on	Collective?	Calls	Pre-MPI safe?	Single-rank-fail safe?	MPI=0 safe?
`nek_die`	`(ierr)`	—	No	`mpi_finalize`†, `cexit`	Yes (after guard†)	mpiexec reaps‡	Yes — guard skips finalize (dummy `mpi_initialized`→0), then `cexit`
`exitt`	`()`	`nid.eq.0`	No	`nek_die(1)`	Yes (via guarded nek_die)	mpiexec reaps‡	Yes
`exitt0`	`()`	`nid.eq.0`	Yes (`print_runtime_info`→`glsum`,`dnekclock_sync`)	`print_runtime_info`, `nek_die(0)`	No — glsum needs MPI	for normal exit only	Yes — dummy glsum/allreduce copy data through
`exitti`	`(str$,idata)`	`nid.eq.0`	No	`exitt`	Yes (via exitt)	mpiexec reaps‡	Yes
`exittr`	`(str$,rdata,idata)`	`nid.eq.0`	No	`exitt`	Yes (via exitt)	mpiexec reaps‡	Yes
`err_chk`	`(ierr,str$)`	`nid.eq.0`	Yes (`iglsum(ierr,1)`)	`exitt`	No — iglsum needs MPI	Yes — syncs all ranks then all exit	Yes — dummy `mpi_allreduce` copies data1→data2, ierr preserved
`nek_end`	`()`	—	Yes (`nekgsync`, `exitt0`)	normal shutdown	No	n/a (normal path)	Yes

MPI=0 build uses core/mpi_dummy.f. Verified: dummy mpi_initialized returns
0/false (so the nek_die guard skips mpi_finalize — never even reaches the
dummy finalize), dummy mpi_allreduce copies data1→data2 preserving ierr (so
iglsum/err_chk work as a local check), and cexit is plain exit(). Every exit
routine terminates cleanly under MPI=0.

† nek_die originally called mpi_finalize unconditionally, which is illegal
before MPI_Init and HANGS if only one rank calls it mid-run (verified). F2 part
(a) guards it: call mpi_initialized(flag); if(flag) call mpi_finalize. After this
guard, nek_die/exitt/exitti/exittr are pre-MPI safe.

‡ "mpiexec reaps" = a single rank calling cexit (plain exit()) does NOT hang the
others: the launcher (mpiexec/mpirun) detects the non-zero exit and tears down
the whole job. Verified empirically. It is best-effort (brief window where
survivors advance) — not as clean as a collective abort, but it terminates.

convert most common block to runtime allocated memory blocks

4ec1ea1

chatgpt-codex-connector Bot reviewed Jul 24, 2026

View reviewed changes

Comment thread core/SIZE

Comment thread core/makenek.inc Outdated

Comment thread core/makefile.template Outdated

yslan added 13 commits July 24, 2026 10:19

Merge remote-tracking branch 'origin/master' into mem_stefan

e71752f

PR: Nek5000 Nek5000#899 PR: Nek5000 Nek5000#901 PR: Nek5000 Nek5000#903 PR: Nek5000 Nek5000#908 (open)

(makenek) fix SIZE, fix warning msg

b1931fd

Fix copying SIZE to SIZE.usr Fix NEK_WARN, populated to new file: obj/.nekwarn

rm stray .usr

7519b86

touch $1.usr doesn't support auto-detect-usr mode

Support MODFLAG for different compilers

b79327d

Use var to shorten makefile.template

5bcc3de

upd gitignore

48e82f3

mnor fixes

b5f3cfa

SIZE cannot exist

fac031c

fix mkSIZE for SIZE.usr

ecad13c

Fix module use and implicit none positions

2678dea

Fix CVode interface, MvCylCvode

caccd62

Clean up timer variables

131ba23

- make sure all timer variables are declared in CTIMER and initialized in time00 - drop icalld from CTIMER

add missing deps for LIB

6d2faa5

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/modules/ADJOINT.f

Comment thread core/modules/NEKNEK.f

yslan added 5 commits July 24, 2026 22:52

fix fshydro

b0c8c93

fix /inbc/ nfld_neknek

401182a

add missing init for ifadj

fdfa7bd

fix transfer module bug

259729d

revert implicit none loc for mpi_dummy.f

72821d6

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/drive.f Outdated

Comment thread core/CTIMER

yslan added 5 commits July 25, 2026 00:24

icalld fix

8c69147

rm dbg print

8821cb0

Move mem alloc to nek_mem_init called in nek_init

5dfdca2

for lib linker

Add LIB test intp CI (makefile, pytest, ci.yml)

0a6a7eb

- Also fix NekTests.py ResourceWarnings

clean up

61aa65d

real*8 timers

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/makefile.template

Comment thread core/math.f Outdated

yslan added 2 commits July 25, 2026 10:58

fix counter

8c3e4eb

(make) add dependency, usr and mod first

6e5798d

clean up timestamp

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/modules/SCREV.f Outdated

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/modules/SCREV.f Outdated

Comment thread core/modules/SCRUZ.f Outdated

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/postpro.f

Comment thread core/modules/SOLN.f

chatgpt-codex-connector Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread core/comm_mpi.f

Add fixed size into scratch memory allocation

c426fd1

- This avoid SEGV at 2D and low lx1 cases - Systematically identified by audit_cb_alloc.py

yslan added 3 commits July 25, 2026 13:32

Improve SIZE/SIZE.usr logic

4024549

chk MPI init to allow pre-MPI check

70ac0f0

check err in module init

57bc1f7

catch memory allocation error, which turns into harder to detect SEGV later.

fix mkSIZE in nekTestCase.py

72a9506

yslan mentioned this pull request Jul 25, 2026

Updates for mem refactor Nek5000/Nek5000#909

Merged

Conversation

yslan commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

yslan commented Jul 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

yslan commented Jul 24, 2026 •

edited

Loading