OCPBUGS-100315: Fix baremetal registry and pull secret after #1922 by tusharjadhav3302 · Pull Request #1937 · openshift-metal3/dev-scripts

tusharjadhav3302 · 2026-07-29T05:40:10Z

Summary

Fixes two regressions introduced by #1922 where the early exit 0 in
02_configure_host.sh for NODES_PLATFORM=baremetal skipped both local
registry setup and REGISTRY_CREDS file creation.

Problems fixed

1. Baremetal without mirroring (`MIRROR_IMAGES` unset)

write_pull_secret() in utils.sh unconditionally merged REGISTRY_CREDS
(private-mirror-ocp.json), causing:

jq: error: Could not open file /home/zuul/private-mirror-ocp.json: No such file or directory

Affects ShiftStack/RHOSO CI jobs on real baremetal hardware.

2. Baremetal with mirroring (`MIRROR_IMAGES=true`)

The local registry at port 5000 never starts, causing 04_setup_ironic.sh
to fail with "connection refused" when mirroring release images.
Affects virtual-baremetal CI using sushy-emulator (e.g. RHOSO uni04delta-ipv6 jobs).

Changes

`02_configure_host.sh` — registry setup in baremetal early-exit path

use_registry "podman": starts registry, creates REGISTRY_CREDS via podman login
use_registry "" (quay): starts registry
No registry: creates empty {} authfile for write_pull_secret() compatibility
Mirrors the existing pattern in the libvirt path (lines 471–487)

`utils.sh` — conditional REGISTRY_CREDS merge in write_pull_secret()

Guards REGISTRY_CREDS merge behind use_registry "" check
When mirroring is disabled: merges personal pull secret + CI registry token only
Applies to both OPENSHIFT_CI=true and the standard CI_TOKEN code paths

Root cause

#1922 assumed baremetal deployments never need a local registry. This is
true for "real" baremetal (standalone provisioning hosts pulling from
external registries), but not for:

Virtual-baremetal setups using MIRROR_IMAGES=true
Any baremetal path where write_pull_secret() is called without mirroring

Test plan

NODES_PLATFORM=baremetal, MIRROR_IMAGES unset: empty authfile created,
write_pull_secret() succeeds without registry
NODES_PLATFORM=baremetal, MIRROR_IMAGES=true: registry starts,
oc adm release mirror succeeds
NODES_PLATFORM=libvirt: no behavior change (early-exit block not reached)
Verified against ShiftStack RHOSO CI job failure on serval70 (2026-07-28)

Are you using "quay" here because that's what is on the libvirt path? I think this works but it might be a little clearer if we also use "" like in utils.sh (there's nothing special about quay here, it's just not podman, I think). This also catches "podman" registry, which might not matter but it might also be unintended

@fonta-rh
Updated to use_registry "" to match the guard in utils.sh.
Thanks for the review.

dtantsur · 2026-07-29T09:16:10Z

/ok-to-test

tusharjadhav3302 · 2026-07-29T10:27:21Z

/retest

tusharjadhav3302 · 2026-07-29T12:37:05Z

/retest

tusharjadhav3302 · 2026-07-30T05:07:02Z

/retest

imatza-rh · 2026-07-30T12:11:52Z

/lgtm

openshift-ci · 2026-07-30T12:11:55Z

@imatza-rh: changing LGTM is restricted to collaborators

Details

In response to this:

/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

mkowalski · 2026-07-30T21:38:42Z

/retitle OCPBUGS-100315: Fix baremetal registry and pull secret after #1922

rebtoor · 2026-07-30T21:45:33Z

/retest-required

mkowalski · 2026-07-30T21:47:18Z

One possible gap in the MIRROR_IMAGES=true baremetal case: registry DNS resolution.

In the libvirt path, an /etc/hosts entry for LOCAL_REGISTRY_DNS_NAME (default virthost.<cluster>.<domain>) is added before the podman login (02_configure_host.sh lines ~478–480), but that code sits after the baremetal exit 0, and the baremetal configure_dnsmasq() only adds api/ingress VIP records. So in the new baremetal block, podman login ${LOCAL_REGISTRY_DNS_NAME}:${LOCAL_REGISTRY_PORT} and the subsequent release-image mirroring in 04_setup_ironic.sh may fail on name resolution — unless the affected CI jobs already export a resolvable LOCAL_REGISTRY_DNS_NAME.

Suggest adding the /etc/hosts entry to the baremetal block too, e.g. before the use_registry checks:

sudo sed -i "/${LOCAL_REGISTRY_DNS_NAME}/d" /etc/hosts
echo "${PROVISIONING_HOST_EXTERNAL_IP} ${LOCAL_REGISTRY_DNS_NAME}" | sudo tee -a /etc/hosts

(and possibly a dnsmasq host record so cluster nodes can resolve the mirror as well).

This message was generated using AI. Please verify before acting on it.
Assisted-By: Claude Fable 5

PR openshift-metal3#1922 introduced NODES_PLATFORM=baremetal support for agent-based installs but left gaps in the early-exit path of 02_configure_host.sh: 1. LOCAL_REGISTRY_DNS_NAME (default virthost.<cluster>.<domain>) has no /etc/hosts entry. The libvirt path adds this at line ~480, but that code sits after the `exit 0` and never runs for baremetal deployments. Without it, podman login and release-image mirroring fail on name resolution. 2. When MIRROR_IMAGES=true the local registry is not started and no podman login creates the authfile, causing downstream scripts to fail reading REGISTRY_CREDS. 3. write_pull_secret() unconditionally merges REGISTRY_CREDS into the combined pull secret even when no local registry is configured, injecting a blank or stale authfile. Add the same sed+tee /etc/hosts pattern from the libvirt path, set up the local registry and authfile when mirroring is enabled, and gate the REGISTRY_CREDS merge in write_pull_secret() behind use_registry checks. Signed-off-by: tusharjadhav3302 <tjadhav@redhat.com> Co-authored-by: Cursor <cursoragent@cursor.com>

tusharjadhav3302 · 2026-07-31T11:04:11Z

One possible gap in the MIRROR_IMAGES=true baremetal case: registry DNS resolution.

In the libvirt path, an /etc/hosts entry for LOCAL_REGISTRY_DNS_NAME (default virthost.<cluster>.<domain>) is added before the podman login (02_configure_host.sh lines ~478–480), but that code sits after the baremetal exit 0, and the baremetal configure_dnsmasq() only adds api/ingress VIP records. So in the new baremetal block, podman login ${LOCAL_REGISTRY_DNS_NAME}:${LOCAL_REGISTRY_PORT} and the subsequent release-image mirroring in 04_setup_ironic.sh may fail on name resolution — unless the affected CI jobs already export a resolvable LOCAL_REGISTRY_DNS_NAME.

Suggest adding the /etc/hosts entry to the baremetal block too, e.g. before the use_registry checks:
sudo sed -i "/${LOCAL_REGISTRY_DNS_NAME}/d" /etc/hosts
echo "${PROVISIONING_HOST_EXTERNAL_IP} ${LOCAL_REGISTRY_DNS_NAME}" | sudo tee -a /etc/hosts
(and possibly a dnsmasq host record so cluster nodes can resolve the mirror as well).

This message was generated using AI. Please verify before acting on it. Assisted-By: Claude Fable 5

Added /etc/hosts entry to the baremetal block

mkowalski · 2026-07-31T12:58:51Z

If I understand correctly, failure of e2e-metal-ipi-ovn-bgp-virt-ipv4 is now a signal for this PR. Can we run this job with this change and see if it brings the green result?

openshift-ci · 2026-07-31T13:34:09Z

@tusharjadhav3302: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-agent-5control-ipv4	`31e0472`	link	false	`/test e2e-agent-5control-ipv4`
ci/prow/e2e-metal-ipi-virtualmedia	`31e0472`	link	false	`/test e2e-metal-ipi-virtualmedia`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci Bot requested review from cybertron and sadasu July 29, 2026 05:40

openshift-ci Bot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Jul 29, 2026

fonta-rh approved these changes Jul 29, 2026

View reviewed changes

tusharjadhav3302 force-pushed the fix-baremetal-registry-creds-without-mirror branch from 85f8453 to fe210fd Compare July 29, 2026 08:10

openshift-ci Bot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jul 29, 2026

rebtoor mentioned this pull request Jul 30, 2026

Fix missing local registry for baremetal with MIRROR_IMAGES #1938

Closed

3 tasks

tusharjadhav3302 force-pushed the fix-baremetal-registry-creds-without-mirror branch from fe210fd to a4a05ad Compare July 30, 2026 11:25

tusharjadhav3302 changed the title ~~Fix missing REGISTRY_CREDS on baremetal without local registry~~ Fix baremetal registry and pull secret after #1922 Jul 30, 2026

openshift-ci Bot changed the title ~~Fix baremetal registry and pull secret after #1922~~ OCPBUGS-100315: Fix baremetal registry and pull secret after #1922 Jul 30, 2026

tusharjadhav3302 force-pushed the fix-baremetal-registry-creds-without-mirror branch from a4a05ad to 31e0472 Compare July 31, 2026 10:57

+                  # Other scripts may read REGISTRY_CREDS even when local registry mirroring
+                  # is disabled; ensure a valid empty authfile exists before the early return.
+                  if ! use_registry "quay"; then

Uh oh!

Conversation

tusharjadhav3302 commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Problems fixed

1. Baremetal without mirroring (MIRROR_IMAGES unset)

2. Baremetal with mirroring (MIRROR_IMAGES=true)

Changes

02_configure_host.sh — registry setup in baremetal early-exit path

utils.sh — conditional REGISTRY_CREDS merge in write_pull_secret()

Root cause

Test plan

Related

Uh oh!

openshift-ci Bot commented Jul 29, 2026

Uh oh!

openshift-ci Bot commented Jul 29, 2026

Uh oh!

tusharjadhav3302 commented Jul 29, 2026

Uh oh!

tusharjadhav3302 commented Jul 29, 2026

Uh oh!

fonta-rh left a comment

Choose a reason for hiding this comment

Uh oh!

fonta-rh Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

tusharjadhav3302 Jul 29, 2026

Choose a reason for hiding this comment

Uh oh!

dtantsur commented Jul 29, 2026

Uh oh!

tusharjadhav3302 commented Jul 29, 2026

Uh oh!

tusharjadhav3302 commented Jul 29, 2026

Uh oh!

tusharjadhav3302 commented Jul 30, 2026

Uh oh!

imatza-rh commented Jul 30, 2026

Uh oh!

openshift-ci Bot commented Jul 30, 2026

Uh oh!

mkowalski commented Jul 30, 2026

Uh oh!

rebtoor commented Jul 30, 2026

Uh oh!

mkowalski commented Jul 30, 2026

Uh oh!

tusharjadhav3302 commented Jul 31, 2026

Uh oh!

mkowalski commented Jul 31, 2026

Uh oh!

openshift-ci Bot commented Jul 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

tusharjadhav3302 commented Jul 29, 2026 •

edited

Loading

1. Baremetal without mirroring (`MIRROR_IMAGES` unset)

2. Baremetal with mirroring (`MIRROR_IMAGES=true`)

`02_configure_host.sh` — registry setup in baremetal early-exit path

`utils.sh` — conditional REGISTRY_CREDS merge in write_pull_secret()