Switch all remote access from XRootD to WebDAV

[djw8605]

What

Convert every remote file operation in cvmfs-sync from the XRootD.client library to WebDAV (HTTPS), using the webdav4 library (httpx-based). This covers directory listing, file reading, and metadata.

Why

All target repositories are globally read-only and already serve WebDAV over HTTPS on the same host/port, so no authentication is required. Moving to WebDAV drops the xrootd-python dependency.

Changes

bin/cvmfs_sync

• Remove import XRootD.client; add webdav4 + xml.etree.
• propfind_dir(): one Depth: 1 PROPFIND per directory (same request count as the old dirlist), parsed by XML local-name since servers vary on namespace prefixes. Returns name / is_dir / size / executable.
• Executable bit comes from the Apache mod_dav executable property (T/F), falling back to 0644 when absent.
• process_download_file(): streams the file via an httpx GET in 1 MB chunks into the existing cvmfs_swissknife graft pipe — all deadline checks, EPIPE handling, child cleanup, and final chmod preserved.
• Remove lookup_dataserver(): HTTP redirects (redirector → data server) are followed transparently by httpx.
• Remove the unused GridFTP / uberftp checksum path.
• Multiprocessing download pool and threaded directory-listing workers are unchanged.

config/*.config — rewrite source = URLs from root://host:port/path to https://host:port/path (same host/port).

update-scripts/cvmfs-sync-driver — drop gridftp_source handling.

config/cvmfs-sync.spec — Requires: python3-webdav4 instead of xrootd-python; update user description text.

Verification

Tested against the live public origin pelican-osdf-public.tempest.uchicago.edu:8443:

• Directory listing returns correct sizes and executable flags (.sif files flagged executable).
• Recursive process_dir yields correct (relpath, size, executable) tuples.
• process_download_file streamed 640 B and 15 KB files end-to-end through a stand-in cvmfs_swissknife; byte counts matched and modes applied correctly (0644 / 0755).
• Listing a nonexistent/forbidden path raises a "permission denied"-tagged error → routed to the skip path, not a hard failure.

Reviewer notes

• Packaging: python3-webdav4 / python3-httpx may not be in base EL repos — they may need EPEL, pip, or a bundled venv.
• No-port hosts: stashredirector.fnal.gov and stash-origin.icecube.wisc.edu had no explicit port under root:// (xrootd default 1094) and now default to HTTPS 443. Worth confirming those serve WebDAV on 443, or adding an explicit port.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Convert cvmfs-sync remote filesystem access from XRootD to WebDAV/HTTPS using webdav4, updating both the sync implementation and the shipped configs/spec so deployments no longer depend on xrootd-python.

Changes:

• Replace XRootD directory listing + file reads in bin/cvmfs_sync with WebDAV PROPFIND/GET streaming via webdav4 (httpx).
• Update config/*.config source = URLs from root://… to https://….
• Remove legacy gridftp_source handling from the driver and switch RPM dependency to python3-webdav4.

Reviewed changes

Copilot reviewed 26 out of 27 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
bin/cvmfs_sync	Replaces XRootD operations with WebDAV PROPFIND/GET streaming and updates directory-walk metadata plumbing.
update-scripts/cvmfs-sync-driver	Removes gridftp_source concatenation so the driver passes a single source URL.
config/cvmfs-sync.spec	Switches runtime dependency to python3-webdav4 and updates user description text.
config/virgo.storage.igwn.org.config	Updates source to https://….
config/uhawaii.osgstorage.org.config	Updates source to https://… (multiple sync sections).
config/uboone.osgstorage.org.config	Updates source (and commented alternatives) to https://….
config/snomass21.osgstorage.org.config	Updates commented source examples to https://….
config/shared.storage.igwn.org.config	Updates source to https://….
config/sdsc-nrp-osdf-origin.osgstorage.org.config	Updates source to https://….
config/scitokens.osgstorage.org.config	Updates commented source to https://….
config/sbnd.osgstorage.org.config	Updates source (and commented alternative) to https://….
config/sbn.osgstorage.org.config	Updates source (and commented alternative) to https://….
config/public-uc.osgstorage.org.config	Updates source to https://….
config/nova.osgstorage.org.config	Updates source entries (and commented alternatives) to https://….
config/mu2e.osgstorage.org.config	Updates source (and commented alternatives) to https://….
config/minerva.osgstorage.org.config	Updates source (and commented alternatives) to https://….
config/ligo.storage.igwn.org.config	Updates source to https://….
config/ligo-test.storage.igwn.org.config	Updates source to https://….
config/kagra.storage.igwn.org.config	Updates source to https://….
config/jlab.osgstorage.org.config	Updates source entries to https://….
config/icecube.osgstorage.org.config	Updates source to https://….
config/icarus.osgstorage.org.config	Updates source (and commented alternative) to https://….
config/gwosc.osgstorage.org.config	Updates source to https://….
config/gluex.osgstorage.org.config	Updates source entries to https://….
config/et-gw.osgstorage.org.config	Updates source to https://….
config/dune.osgstorage.org.config	Updates source (and commented alternatives) to https://….
config/des.osgstorage.org.config	Updates source (and commented alternative) to https://….

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.