Skip to content

fix(prometheus): drop sslcacert from grafana repo config - #6

Merged
SNThrailkill merged 2 commits into
mainfrom
fix/prometheus-grafana-repo-ca
Aug 10, 2026
Merged

fix(prometheus): drop sslcacert from grafana repo config#6
SNThrailkill merged 2 commits into
mainfrom
fix/prometheus-grafana-repo-ca

Conversation

@SNThrailkill

Copy link
Copy Markdown
Owner

Summary

The prometheus image build has been failing for months at dnf install loki grafana -y in the Containerfile. This PR fixes the root cause.

Root cause

The build failed with:

Curl error (77): Problem with the SSL CA cert (path? access rights?) for https://rpm.grafana.com/repodata/repomd.xml
[error adding trust anchors from file: /etc/pki/tls/certs/ca-bundle.crt]

An earlier attempt (dnf update -y && update-ca-trust) didn't help — the log showed "Nothing to do." and the same error persisted.

The real cause: the grafana yum repo config explicitly pinned sslcacert=/etc/pki/tls/certs/ca-bundle.crt, but that classic Fedora symlink does not exist in the quay.io/fedora/fedora-bootc:44 base image. Verified empirically:

  • /etc/pki/tls/certs/ca-bundle.crtdoes not exist
  • /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem → exists (223 KB, populated by ca-certificates)
  • curl --cacert /etc/pki/tls/certs/ca-bundle.crt → fails (bad CA file)
  • curl with the system default (no --cacert) → HTTP 200
  • dnf install loki grafana with sslcacert removed → installs cleanly (grafana 13.1.1 + loki 3.7.6)

dnf update/update-ca-trust couldn't help because nothing in the base image creates that symlink (the bundle is a static layer).

Fix

Containerfile (prometheus stage): removed the sslcacert= line from the grafana repo config. sslverify=1 is kept; omitting sslcacert lets dnf/curl use the system-default CA trust store, which works correctly. This matches Grafana's own docs.

Validation

  • Reproduced the exact failing command (dnf install loki grafana -y) against the real base image with the fixed repo config — completes successfully.
  • DNM note: I could not run the full prometheus build locally (no buildah/podman), but the exact failing command was validated against the actual base image.

This PR was created by an AI agent (OpenHands) on behalf of SNThrailkill.

@SNThrailkill can click here to continue refining the PR

The fedora-bootc base image does not contain the classic Fedora symlink
/etc/pki/tls/certs/ca-bundle.crt, so grafana's repo config pinning
sslcacert to that path made dnf fail with curl error 77
("error adding trust anchors from file") when fetching repomd.xml,
blocking `dnf install loki grafana`.

The system-default CA trust store is populated by ca-certificates at
/etc/pki/ca-trust/extracted/pem/ and works fine (verified: curl HTTP 200
and a full dnf install of loki+grafana against rpm.grafana.com). Removing
the explicit sslcacert line lets dnf use that default.

Co-authored-by: openhands <openhands@all-hands.dev>
@SNThrailkill
SNThrailkill marked this pull request as ready for review August 10, 2026 04:38
- reusable-build-image.yml: guard Add SBOM Attestation with the same
  release-only condition as Install Cosign / Sign container image. On
  pull_request runs cosign is never installed and the digest is empty,
  so the step failed with 'cosign: command not found' (exit 127).
- build-artifact.yml: drop the pull_request trigger. The workflow
  requires build_type/flavor inputs that only exist on workflow_dispatch,
  so PR runs produced an empty flavor -> 'ghcr.io/.../:latest' and failed
  with invalid reference format.

Co-authored-by: openhands <openhands@all-hands.dev>
@SNThrailkill
SNThrailkill merged commit 85be8c5 into main Aug 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants