Skip to content

feat(verification): add XML Schema, OpenAPI, and Avro Docker verification targets - #270

Merged
ekarademir merged 4 commits into
accordproject:mainfrom
apoorv7g:CI-XML
Jul 21, 2026
Merged

feat(verification): add XML Schema, OpenAPI, and Avro Docker verification targets#270
ekarademir merged 4 commits into
accordproject:mainfrom
apoorv7g:CI-XML

Conversation

@apoorv7g

@apoorv7g apoorv7g commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

This PR extends the Docker based codegen verification harness with three new schema targets: XML Schema, OpenAPI, and Avro. Each target gets a Docker image, an entrypoint that generates from the corpus and runs an external validator, a local Mocha verification test, registration in verification/docker/targets.json and scripts/verification/docker-run.js, a verify:docker:<target> npm script, and a matrix entry in .github/workflows/verify-codegen.yml.

Where generated output is known to be rejected by the external tool because of existing visitor limitations, the affected case and target pairs are skipped with an explicit reason (same pattern as protobuf, graphql, rust, and java).

Changes

Shared wiring

  • Registered xmlschema, openapi, and avro in verification/docker/targets.json.
  • Extended scripts/verification/docker-run.js TARGETS to include all three.
  • Added npm scripts: verify:docker:xmlschema, verify:docker:openapi, verify:docker:avro.
  • Added all three targets to the verify-codegen GitHub Actions matrix.
  • Extended per target skip documentation in test/verification/cases.js and, for the Docker corpus metamodel case, in verification/corpus/manifest.json.

XML Schema (xmlschema)

  • Added verification/docker/xmlschema/Dockerfile and entrypoint.sh.
  • Image installs libxml2-utils and downloads the W3C XML Schema for Schemas (XMLSchema.xsd) at build time.
  • Entrypoint generates CTO to .xsd via CLI target XMLSchema, then validates each file with xmllint --noout --schema against the bundled meta schema.
  • Added test/verification/xmlschema.validate.test.js: generates with XmlSchemaVisitor, validates with xmllint --schema. Locally skips the suite when xmllint is not installed. Falls back to the W3C URL if the local meta schema path is missing (override with XMLSCHEMA_META_SCHEMA).

OpenAPI (openapi)

  • Added verification/docker/openapi/Dockerfile, entrypoint.sh, and redocly.yaml (extends: minimal).
  • Image installs @redocly/cli@1.34.3 globally. Entrypoint generates via CLI target OpenApi, then runs redocly lint on each openapi.json.
  • Added test/verification/openapi.validate.test.js: generates with OpenApiVisitor, lints with the local @redocly/cli package (devDependency) via require.resolve and node, so no global CLI install is required on developer machines.
  • Uses the same verification/docker/openapi/redocly.yaml config locally (override with REDOCLY_CONFIG).

Avro (avro)

  • Added verification/docker/avro/Dockerfile and entrypoint.sh.
  • Image installs a JRE and downloads Apache avro-tools-1.12.0.jar. Entrypoint generates via CLI target Avro, then runs java -jar … idl on each .avdl.
  • Added test/verification/avro.validate.test.js: mirrors the Java Jackson jar pattern. Downloads avro-tools-1.12.0.jar from Maven Central into verification/templates/ on first run when missing (override with AVRO_TOOLS_JAR). Only skips when Java is not installed.
  • Gitignored verification/templates/avro-tools-*.jar (same approach as jackson-annotations-*.jar).

Skips (what and why)

These skips document known visitor output issues. They are not Docker or harness bugs. Fixing them belongs in follow up visitor PRs (plus snapshot updates).

Avro: illegal versioned namespaces

AvroVisitor emits @namespace("concerto.decorator@1.0.0") (and similar). Avro IDL rejects that namespace because version segments such as 0 are illegal identifiers (SchemaParseException: Illegal name: 0).

Case Target Why skipped
metamodel avro Generates versioned system namespaces including concerto.decorator@1.0.0
hr_base avro Same versioned namespace problem when decorator or system models are present
hr_integration avro Same
stringlength avro Same
model-base avro Same
agreement avro Same
circular avro Same

Docker corpus: metamodel case also skips avro for the same reason in verification/corpus/manifest.json.

OpenAPI: $decorators and invalid schema nesting

OpenApiVisitor embeds JSON Schema from JSONSchemaVisitor. That payload includes $decorators (and in some graphs invalid schema nesting) which Redocly's OpenAPI struct rules reject. Component map keys with @ in FQNs also warn under Redocly naming rules; the hard failures that forced skips are the struct errors.

Case Target Why skipped
hr_integration openapi $decorators and invalid schema nesting fail OpenAPI struct validation
model-base openapi $decorators fail OpenAPI struct validation
agreement openapi $decorators fail OpenAPI struct validation

Cases that still run for OpenAPI (for example metamodel, hr_base, stringlength, circular) lint clean under the current minimal Redocly config.

Unrelated existing skips (unchanged intent)

Other skips already present for protobuf, graphql, rust, java, and csharp remain; this PR does not change their meaning. New avro / openapi entries were added alongside them where needed.

Flags

  • XML Schema local tests require xmllint (libxml2). Without it, those tests are pending rather than failing.
  • Avro local tests require a JRE. The tools jar is downloaded automatically like Jackson annotations for Java verification.
  • OpenAPI local tests use @redocly/cli from devDependencies; Docker still installs it globally in the target image.
  • Docker images download remote assets at build time (W3C XMLSchema.xsd, Maven/Apache avro-tools, npm @redocly/cli).
  • Skipped case and target pairs should be revisited when AvroVisitor sanitizes versioned namespaces and when OpenAPI generation stops emitting non OAS $decorators (or maps them to x- extensions).

Screenshots or Video

N/A. Verification is exercised via Mocha (npm run test:verify) and Docker (npm run verify:docker:xmlschema, verify:docker:openapi, verify:docker:avro).

Author Checklist

  • Ensure you provide a DCO sign-off for your commits using the --signoff option of git commit.
  • Vital features and changes captured in unit and/or integration tests
  • Commits messages follow AP format
  • Extend the documentation, if necessary
  • Merging to main from fork:branchname

apoorv7g added 3 commits July 20, 2026 16:14
- Introduced XML Schema as a new target in the Docker verification setup.
- Updated package.json, Dockerfiles, and entrypoint scripts to support XML Schema validation using xmllint.
- Added a new validation test for XML Schema to ensure proper generation and verification.
- Enhanced the verification manifest to include XML Schema as a target.

This commit expands the verification capabilities to include XML Schema, ensuring a consistent environment for testing and validation.

Signed-off-by: Apoorv <130035517+APOORV7G@users.noreply.github.com>
Signed-off-by: Apoorv <130035517+APOORV7G@users.noreply.github.com>
…ification

- Introduced OpenAPI and Avro as new targets in the Docker verification setup.
- Updated package.json and package-lock.json to include necessary dependencies for OpenAPI and Avro.
- Enhanced the verification manifest and Docker configuration to support validation for OpenAPI and Avro.
- Added skip reasons for specific test cases related to OpenAPI and Avro in the verification tests.

This commit expands the verification capabilities to include OpenAPI and Avro, ensuring a consistent environment for testing and validation.

Signed-off-by: Apoorv <130035517+APOORV7G@users.noreply.github.com>
@apoorv7g apoorv7g changed the title feat(verification): add XML Schema support for Docker-based verification feat(verification): add XML Schema, OpenAPI, and Avro Docker verification targets Jul 21, 2026
…ignment for consistency

Signed-off-by: Apoorv <130035517+APOORV7G@users.noreply.github.com>
@apoorv7g

apoorv7g commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@DianaLease @ekarademir and if possible @mttrbrts , please review and approve this, Avro and OpenAPI fixes can be done after GSoC Period.

@ekarademir
ekarademir merged commit 8291c85 into accordproject:main Jul 21, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants