Skip to content

Simplify test strategy#35

Merged
ttl-octave merged 2 commits into
gnu-octave:mainfrom
MitchellThompkins:mitchellthompkins/make-ci-simple
Jul 20, 2026
Merged

Simplify test strategy#35
ttl-octave merged 2 commits into
gnu-octave:mainfrom
MitchellThompkins:mitchellthompkins/make-ci-simple

Conversation

@MitchellThompkins

@MitchellThompkins MitchellThompkins commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Description

#32 added a CI environment but did so in a very hacky way that I was unhappy with basically immediately. It did let me test test both locally and in CI but by doing some non-ideal searching through files to aggregate all the test files.

This was all in an effort to avoid installing the control package in the docker container because install in the Makefile depends on dist, which depends on docs and the docker image I was using from https://github.com/gnu-octave/docker lacked the tooling to actually build the docs (FWIW I think https://github.com/gnu-octave/octave-buildbot does have the tooling but I don't believe actually builds octave itself and therefore can't be used for testing).

I realized I could just copy the installation makefile recipe and strip out the docs generation stuff. That combined with the fact that I realized __run_test_suite__ existed let me drastically reduce the complexity of the CI environment. It also makes the output much more legible. See the standard octave test results vs what I was doing before (both of those produce 389 tests, but the new version is way cleaner).

The only trade-off is that installation uses git archive which means that for a file to be tested it has to be tracked by git. For files that are already tracked by git I've used git stash create to temporarily grab the hash that would be represented if you were to make the commit. However files that are completely untracked (so new files) won't be picked up at all, so you need to git add them in order to actually test. The trade-off for simplicity seems acceptable to me. I've updated the CONTRIBUTION.md to reflect this as well.

Changes

  • Minor copy/paste changes to makefile to copy existing installation recipe
  • Drastically reduces the complexity of the script used to run tests
  • Updates local test contribution guidelines

path.

force a failre

Revert "force a failre"

This reverts commit 6c35dfc.
@MitchellThompkins MitchellThompkins changed the title Simply test stragety Simplify test stragety Jul 13, 2026
@ttl-octave

Copy link
Copy Markdown
Member

Would it be possible to have the identical part of the recipes for both tarballs only once?

@MitchellThompkins

MitchellThompkins commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

Would it be possible to have the identical part of the recipes for both tarballs only once?

Yes it would be. I actually did this initially and it's somewhere in my git log but was worried about messing up the release-work flow today and figured it was better just to leave it alone. I can add a commit here that includes that change and I'll try to test that it doesn't break the release build locally, but I think you should definitely verify before merging.

Edit:
I just did this quickly in 6c50cac. Originally I introduced a common recipe that both paths depended on, but I htink define is simpler and clean. The tests run locally and I don't see why the dist wouldn't build, but I'd test it all the same.

Let me know if some thing like this isn't what you had in mind though and I can think about some other ideas.

@ttl-octave
ttl-octave merged commit a57bb55 into gnu-octave:main Jul 20, 2026
4 checks passed
@ttl-octave

Copy link
Copy Markdown
Member

make dist, install or release are worling as expected.
Thanks for the contribution!

@MitchellThompkins MitchellThompkins changed the title Simplify test stragety Simplify test strategy Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants