docs: coordinate Python quality gates and UI package migration - #689
docs: coordinate Python quality gates and UI package migration#689Paulo Lacerda (placerda) wants to merge 19 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Record the maintainer-approved design, compatibility contracts and 46 executable tasks for #681. Keep runtime and administrative acceptance separate from planning. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
⛔ Pull requests to
|
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve all 46 task IDs and recorded progress. Make contract coverage, policy ownership, acceptance evidence and execution dependencies explicit for the requested speckit-tasks refresh. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Orchestrator source reconciliation is unchanged at c6d0ccb01a40071f82f30bd17c9fe566b3d0ad18; isolated branch feature/python-module-boundaries. Python 3.12.9 environment restored from requirements.txt. The researched Ruff 0.16.6 / Import Linter 2.15 / Grimp 3.17 pins are not published on the available package index; installed alternatives are Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16. Full AST plus Grimp/import-linter architecture analysis passes after the local OBO extraction (including flat, deferred and type-only edges); 209 focused tests pass. Important scope conflict: source contains 155 pre-existing broad handlers (154 after narrowing the JSON parser), including configuration/retrieval best-effort fallbacks beyond audit. I will not bulk-approve or silently change these runtime contracts. Building a fail-closed draft PR with exact inventory and review blockers unless these can be individually resolved with existing failure evidence. No active handler exceptions have been invented. Bootstrap policy also correctly remains non-green until independently reviewed; no settings changes. Contributor CLI is python .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json --test-results .artifacts/pytest.xml; test evidence is emitted by python -m pytest -q --junitxml=.artifacts/pytest.xml. Initial typing scope is the four agreed schema/type modules plus new connectors.obo; no global import skips. Final counts and PR/SHA will follow. |
|
UI implementation status: isolated feature/ui-package-quality-gates from c635bc6696714b543feec24b4a062a8a8f3ff6d0 (current develop unchanged). All 32 runtime inventory entries mapped; citation cycle removed; root legacy modules now explicit adapters; existing 410 unittest tests pass after owner-target migration. Package uses setuptools, requirements.txt remains runtime dependency authority, Docker retains uvicorn main:app at /app and installs with pip install --no-deps .; contributor setup adds pip install --no-deps -e . after runtime requirements. Assets remain externally staged .chainlit/public/chainlit.config.yaml/chainlit.md/VERSION, CHAINLIT_APP_ROOT or staged cwd, no sys.path/sys.modules proxies. Python 3.12. Tested available quality pins: Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16 (research candidate Ruff 0.16.6/Import Linter 2.15/Grimp 3.17 were unavailable from configured index). Quality CLI implemented: python .github/scripts/check-quality.py --check all --base-ref --report .artifacts/quality.json; individual checks lint/typing/architecture/exceptions/policy. Seed typing remains chat_backend/panel_config/hosted_continuity_config canonical config modules; adds newly introduced modules/adapters. 13 quality mutation fixtures pass, area Import Linter contract passes. Installed-wheel matrix, complete policy integrity and CI wiring are still in progress. Inherited broad-handler inventory is intentionally NOT auto-approved; current source has fallback behaviors conflicting with blanket gate activation, to be reported exactly for maintainer decision rather than silently changed. No settings or live operations performed. No send_session_message/create_pull_request tool is exposed in this session; reporting coordination here and will create PR with gh fallback. |
|
Ingestion implementation status (isolated Stabilized interface: Python 3.12 ( Typing: audit_contract + audit_sanitizer seeds and new jobs.runtime, zero in-scope findings and zero baseline entries; 354 imported diagnostics reported, not globally ignored. Full static graph includes flat roots, namespace chunkers, deferred and type-only imports; Grimp overlap checked. Removed the scheduler back edges; found an overwritten legacy SharepointPurger import and four existing private helper accesses for explicit compatibility review. Acceptance blocker: 218 remaining broad runtime handlers after removing two overwritten definitions, including success-shaped configuration/indexing/deletion fallbacks. No mass whitelist/automatic justification is being added; exceptions ledger is empty and this gate remains red. Preserving audit best-effort behavior. Preparing a DRAFT component PR with exact residual acceptance rather than claim Q4/SC-003 complete. Workflow uses real same-workflow jobs, protected-base evaluator/aggregate, bound reports, and verified CODEOWNERS (Paulo Lacerda (@placerda) has admin permission). Settings and Azure validation/recovery remain unauthorized/pending. No merging/deploying/publishing. This session does not expose send_session_message or create_pull_request; reporting here and will create the develop-targeted draft through gh. Docs remain parent-owned (#688), coordination #689. |
|
Orchestrator stabilized evidence: Python suite 838 passed; unchanged frontend npm install --no-audit --no-fund + npm run build passed. Typing is green with 5/77 runtime modules protected and zero baseline entries; imported out-of-scope diagnostics remain visible. Architecture is green (complete AST graph; Grimp overlap 65 package modules). Exceptions are intentionally not green: all 154 remaining broad handlers are inventoried, including 100 Ruff BLE001 findings and logged/re-raised handlers BLE001 misses. Source review found security-sensitive legacy fallback behavior (configuration/auth readiness and retrieval-context fallbacks) incompatible with blanket approval under the only-audit-best-effort constraint; no behavior-changing fix or approval will be invented. Local checker has executable mutation fixtures, protected-base comparison and fail-closed matrix/test/frontend aggregate wiring. Draft PR will carry exact blockers and audit-specific proposed justifications with failure evidence, not pretend enforcement is complete. Verified existing admin login placerda for proposed CODEOWNERS; latest-head/code-owner controls and independent owner-author review remain administrative prerequisites. Tool pins/CLI are in my previous coordination comment. Docs must remain gated. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Stable ingestion checkpoint is now pushed and open as draft Azure/gpt-rag-ingestion#296, targeting develop, exact head bbe52923dbaf2b8ce4f6f371e492ad32ae7ffe45. Both remote head and base were verified; worktree clean. Required Copilot trailer included. No merges/settings/deployments/releases. At that committed SHA, Implemented checkpoint: T003/T007/T010/T013/T016/T019/T022/T040, with qualifications in the PR task table. T025 incomplete; T028 partial; component T045 local evidence complete but remote/full acceptance pending. Individual primary-operation failure remediation, final mutation/surface review, administrative required checks and live integration/recovery remain follow-up scope. No bulk handler approval. CLI and exact available pins are in Session-message/create_pull_request tools are unavailable here, so this handoff uses the parent PR comment and gh-created draft. Full commands, recovery/peer SHAs, changed surfaces and residual acceptance are preserved in the component PR; no need to infer completion from a passing unit suite. |
|
Orchestrator checkpoint PR is open: Azure/gpt-rag-orchestrator#346 (draft, target develop), branch feature/python-module-boundaries, commit ef649eeab6144156b4c90c4422d62f229454dedc. Full maintained Python suite: 840 passed; quality fixture subset: 83 passed; existing frontend build and Copilot asset validator passed. Post-commit report binds exactly that head to c6d0ccb01a40071f82f30bd17c9fe566b3d0ad18. Corrected inventory count: 78 runtime modules, 5 blocking typing / 73 uncovered, zero baseline entries; graph 196 edges and zero detected violations with 65-module Grimp overlap. Deliberately incomplete: 100 BLE001 findings, 154 broad handlers, zero approved exceptions, one bootstrap-policy failure. No handler approval is fabricated; concrete ledger proposals and review of legacy non-audit failure contracts are still needed. Strict schema/adversarial review and real required-check activation/live validation remain open. Full exact commands, tools (Ruff 0.16.5/mypy 2.3.1/Import Linter 2.14/Grimp 3.16), changed scope, task accounting, compatibility and recovery are in the draft PR. Parent checkpoint requested: this turn stops here pending review; no merge/settings/publish/deploy occurred. Source checkout is clean. |
|
The follow-up instruction resumed implementation after the checkpoint. I am closing strict policy/dynamic-import and exact audit-record gaps while preserving the draft. Current GitHub Actions confirm tests, frontend, typing, architecture and assets pass; lint/exceptions/policy/aggregate fail closed exactly as reported. Concrete blocker needing maintainer direction: retaining non-audit best-effort behavior and preserving existing auth/SSE semantics conflict with the instruction to retain only intentional best-effort audit semantics. Examples include main.orchestrator_endpoint/validate_user_access setting auth_configured=False on configuration failure; FoundryIQContextProvider/SearchContextProvider returning Context() after source failure; multimodal search retry removing source authorization. I will not approve these as audit exceptions or change identity/wire semantics under a tooling-only assumption. Exact audit exception proposals and checker hardening can proceed independently. |
|
UI checkpoint is now Azure/gpt-rag-ui#110, an open draft targeting develop, at All 32 researched inventory entries have canonical package owners; this is not merely a skeleton. Contributor commands: Non-editable install: The artifact is code plus externally staged Evidence: 410 original/migrated tests passed; final complete suite passed 436 tests, including 17 quality, 3 legacy compatibility and 6 installed-package tests. Installed tests build a real wheel and run isolated subprocesses outside the checkout, assert non-editable UI module origins, exercise both import orders/once-only registration and ASGI startup/resource cases. They reuse third-party dependencies via system-site-packages: independent clean dependency resolution and the full installed security matrix remain pending. Actual Uvicorn import/ASGI TestClient evidence is not a deployed-listener test. Agent asset validation passed. Full-suite checkpoint was Quality CLI: Actual pins: Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16, setuptools 80.9.0. Research's higher Ruff/Import Linter/Grimp pins were unavailable. Blocking typing covers canonical At final head: typing and architecture pass; lint has 20 BLE001 findings, exceptions have 62 unapproved inherited handlers, policy has 1 bootstrap-review finding; combined exit 1. Do not document completed/activated quality adoption. Remaining checker work includes exhaustive schema/suppression/policy mutations, move/split debt identity, exact adapters, dynamic-import allowances and executed exception-test evidence. Real required-check/latest-head enforcement is not established. T023 and T031-T036 implementation is present; T004/T030/T038 compatibility closure, T008/T011/T014/T017/T020/T026/T029 gate/error-policy closure, T037 container evidence and UI T045 final acceptance remain incomplete. T041 ownership/install/rollback guidance is in the PR and Linux container execution is blocked by the unavailable Docker Desktop Linux daemon. Live exact-peer integration (orchestrator v4.1.1, ingestion v2.7.3), recovery rehearsal and administrator/negative-PR enforcement evidence remain unperformed. No merges, releases/tags/images, deployment, settings changes or parent file edits occurred. Dedicated cross-session/PR-creation tools were not exposed; this PR comment supplies the durable coordination handoff via GitHub CLI. |
|
Follow-up batch (still isolated, not yet pushed): strict policy/scope/debt/exception parsing and tool-pin consistency; unresolved first-party exports, exact single-use variable-import records with passing evidence, relative literal dynamic imports, reflective loader/exception aliases and conflicting-alias fail-closed cases. The stronger export resolver found a real stale connectors.AzureOpenAIClient import in the legacy RetrievalPlugin; it now reuses get_genai_client and awaits async embeddings, with security-filter and explicit failure-result tests. Five exact audit exception proposals are recorded as proposed (zero active approvals) and their source/context/test matches are tested. Full suite at the pre-final-fixture point: 873 passed; typing/full graph remain green. Published docs search of contributing/services_orchestrator/howto_retrieval_optimization/orchestrator_visual_guide found no old/new connector/plugin references requiring an operator edit; AGENTS and Unreleased changelog reflect the changes. I acknowledge your clarification that FR008 follows existing public contracts, not a blanket ban on non-audit recovery. No auth/identity/SSE behavior was changed. I will push this coherent batch and report the SHA for reviewer reconciliation; the frozen ef649ee review remains useful. |
|
Coherent follow-up pushed to Azure/gpt-rag-orchestrator#346: 4e7fdcad98af6e217d37d4d7fa6f3a314a3d91ca. Full Python suite now 874 passed; asset validator, tooling Ruff and git diff checks passed. This adds strict record/pin parsing, unresolved-export detection, dynamic-loader/evidence/alias adversarial coverage, five exact non-authorizing audit proposals, and the legacy retrieval connector/async-embedding repair with security-filter/error-envelope tests. Zero active exceptions; 154 broad handlers still require individual disposition, 100 BLE001 findings remain, bootstrap remains red. PR body now distinguishes both SHAs and incorporates your conditional FR008 clarification. No need to restart the frozen ef649ee reviews; please send findings for reconciliation against this follow-up. Parent/docs own umbrella tracking. Runtime identity/SSE/defaults remain unchanged; no settings/merge/publication/deploy. |
|
Follow-up is addressing concrete contract violations, not treating every recovery as forbidden. Before/after fixtures reproduced: Search single-delete used an invalid SDK call and swallowed failure; batch delete counted unconfirmed/missing outcomes; NL2SQL purge ignored returned failed counts and fabricated successful totals; audit treated missing results as succeeded; config apply/refresh returned success despite failure. The fixes preserve bool/count failure contracts and existing success wire shapes, and sanitize affected error diagnostics. Schema bytes/labels/credentials remain unchanged. Cron/manual wrappers now delegate failure event + propagation to the existing audit_run boundary instead of duplicate blanket handlers. A newly added lifespan regression caught the induced startup-sequencing change; the coordinator now retains ordered independent startup execution after a worker failure (not fail-fast across unrelated startup jobs). Existing audit side-effect recovery remains best-effort, with tests proving it neither fails the primary operation nor leaks exception payloads. Preparing a separate follow-up commit and four exact audit-boundary proposals, explicitly NOT active approvals. No change to the frozen bbe529 review target. Original CI at that target has now completed: unit-tests, typing, architecture and agent assets passed; lint/exceptions/policy failed for the reported findings; real quality-gate correctly failed. Will post new SHA and complete command evidence after the coherent batch. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Latest orchestrator draft head is 29df99d9b1e1393d6b775d9d5362c445a7fdd021 (Azure/gpt-rag-orchestrator#346, still develop). Additional safe work completed: all 10 broad sites in audit emitter/config/sanitizer now have exact proposed records and passing failure evidence; unreadable mappings/sequences and iteration failures cannot serialize content or fail the primary operation. All records remain proposed, 0 active. A second verified existing admin gxjorge is included with placerda in the proposed CODEOWNERS file, providing an independent eligible owner without any settings change. Private class/literal-reflective member cases and a qualified-child/facade regression bring quality fixtures to 117. Exact latest-head full suite: 884 passed (7 existing warnings); typing 5/78 with 0 baseline entries and graph 78/196 remain green. Runtime lint 100 BLE001, 154 unapproved handlers plus 10 proposed-record findings, and bootstrap policy remain red; real GitHub aggregate has demonstrated fail-closed behavior. Remaining non-audit contract/disposition work is 144 sites, plus independent activation/live acceptance. This is not completed enforcement. I await the already-running frozen architecture/tooling review to avoid guessing those failure contracts; both previous frozen SHAs remain available and all follow-up commits are documented in the PR. |
|
Scoped quality follow-up: yes, I will wire an ephemeral Ubuntu Quality progress: new fixtures reproduced the previously reported false-green paths. Current 33 focused tests pass, including real disposable-Git-repository protected-base and unittest receipt/aggregate subprocess cases. I am completing signature alias normalization, scoped guidance and final evidence/commit. |
|
Ingestion follow-up committed and pushed to Azure/gpt-rag-ingestion#296 (still draft, develop): 46d08d31e5090045e481f22da699e09f444159dc. The frozen bbe529 review target is unchanged and remains available. Concrete fixes: Search single/batch delete SDK use and matching confirmation; NL2SQL failed scan/count/delete propagation; missing/malformed/duplicate results cannot emit positive audit events; governance provider failures cannot silently disable governance; config apply/refresh/write and manual scheduling failures are explicit with payload-free diagnostics. Cron/manual exceptions propagate through audit_run, while ordered independent startup isolation remains intact. FR008 is applied per established public contract, not as a blanket ban on non-audit recovery. Exact-head maintained suite: 348 passed / 95 existing deprecation warnings / 39.61s. Same-head pytest evidence and full quality report generated. Quality exits 1 with 346 findings: 202 unapproved handlers, 131 BLE001, 4 review-pending proposals, 1 bootstrap and 8 protected-policy findings. Typing: 3 blocking modules, 0 in-scope diagnostics, 0 baseline entries, 350 visible imported diagnostics. Graph: 53 nodes / 145 edges / 41 Grimp package nodes, no reported findings. Assets and diff checks pass; contracts/dependencies/runtime requirements/frontend/VERSION remain unchanged. Four exact audit proposals remain proposed, 0 active. Added failure-first fixtures for exact active-record/Ruff integration and expired-stage rejection: only a matching protected active record can waive its BLE001 header, and the independent required exceptions job still needs same-run behavior evidence. No blanket suppression or self-approval. Pinned tool versions remain Python 3.12.9, Ruff 0.16.5, mypy 2.3.1, Import Linter 2.14, Grimp 3.16. PR body now carries current files, exact commands/results, task accounting, intentional failure-response changes and recovery. T022/T040 complete; T025 and the rest of T028 remain incomplete. Strict/adversarial checker/surface review, latest-head approval/administrative activation and live integration/recovery evidence remain open; I will not invent handler approvals or speculate about unrelated failure contracts. Current-head CI: https://github.com/Azure/gpt-rag-ingestion/actions/runs/34041647661 (unit-tests, typing, architecture and assets already passed; lint/policy failed for declared findings when last observed). No merge/settings/publication/deploy or parent edits. |
|
Final current-head ingestion CI for 46d08d31e5090045e481f22da699e09f444159dc has completed: unit-tests, typing, architecture and agent assets passed; lint (131), exceptions (206) and policy (9) returned explicit violations; actual quality-gate failed. No tool-execution error or false-green aggregate. Azure/gpt-rag-ingestion#296 has the complete follow-up body and current-head CI comment. Worktree clean, draft/develop preserved; individual approvals/remaining acceptance are still open. |
|
Acknowledged: FR008 remains conditional on the established contract, and all exception proposals stay inactive. Current bounded follow-up is evaluator-only: 18 new failing fixtures reproduced malformed-record acceptance (including boolean/float schema versions and missing structured error reports), first-party broad-exception reexports, conflicting lexical aliases and assignment/reflective import-loader bypasses. Strict parsing and conservative binding resolution are being implemented with the existing pytest runner; exact site/reflection/relative-import fixtures are next. No audit wire/schema changes or runtime changes in this batch. I will reconcile the frozen quality review against the follow-up rather than changing reviewed commits. |
|
Ownership confirmation: I authored/committed tests/container_smoke.py and its read-only-test-mount, I preserved your installed-test/.dockerignore files and consumed e9620fce81daa879c0d945113911e58ae4b574e3 as the new parent head. I will now add targeted fixtures for the additional cross-cutting concerns you listed; existing receipt validation already compares root-computed source/head/base/run/pinned-tool inputs, but I will test coordinated artifact forgery rather than assume it. No runtime changes or fabricated exception approvals. |
|
Next immutable head2be12230003c140ed3fcd3204ea73e6afbb25661 pushed. Actual signed RSA/JWT matrix covers v1/v2, expiry/audience/issuer/tenant/signature/algorithm rejection, role shape, key rotation+alternate JWKS, required-setting500, cancellation and tenant-scoped cache clearing. Two parser/verifier catches narrowed; generic fail-closed401 retained as one exact inactive proposal with bounded diagnostic. Regression2fail/15pass; expanded auth/consumer106pass; proposal/auth33pass. Exact all lint81/typing0/architecture0/exceptions145/policy1;122sites/23proposed/0active/99unproposed. No identity/audience/cache policy change. aca Linux34061635704 tests/frontend/typing/architecture/assets passed, expected adoption/bootstrap jobs red. |
|
ACK parent release/decision ownership: blob_sas two paths remain untouched by me; NL2SQL/multimodal thrown-failure terminal methods are held pending your grounded disposition. Latest coherent b5b04ea5bebf96ff3afda4d12cd3fc0805705a06 is pushed and clean, draft346 body updated. ONLY separate profile helpers changed in MafLite/MAF Service/multimodal: bounded diagnostics and not-confirmed warning instead of false Saved when Cosmos returns None; empty/load/save/cancellation/doc-shape behavior preserved. Six helper-specific inactive proposals, no approval of memory eligibility/default-user or primary outcomes. Regression24fail/21pass; profile+actualMAF+Cosmos144pass; exact-record/profile63pass; exact Linux34067719459 full1290pass/8warnings, frontend/typing/architecture/assets pass. Local first typing timeout exit2 was explicitly retried unchanged: typing0 then fullall exit1 lint81/typing0/architecture0/exceptions151/policy1.122broad/29proposed/0active/93unproposed,246qualitycases. No pin/timeout/isolation weakening. Continuing already-known secondary/backend boundaries, not another exploratory audit. |
|
Frontend ownership released to parent: frontend/package.json, frontend/package-lock.json, and frontend validation/test scope. I have made no frontend edits. Existing ERESOLVE: @types/react@19.2.18 conflicts with @types/react-dom@18.3.7 requiring @types/react ^18.0.0; runtime manifest also pairs react ^19.2.8 with react-dom ^18.3.1. npm view confirms react-dom 19.2.8 requires react ^19.2.8, and current @types/react-dom 19.2.5 requires @types/react ^19.2.0. Additional build compatibility to inspect: Vite ^8.0.16 on local Node 20.14.0, and Tailwind ^4.3.3 with existing v3 PostCSS plugin/directives. Docker uses Node20 and plain npm ci. No force/legacy-peer bypass or Python pin changes. Backend now has 65 inventoried broad sites, all individually proposed with zero active approvals; ordinary Python lint findings are zero, remaining 60 BLE001 findings are proposed-boundary adoption findings. Final full-suite/receipt validation is next; these are not yet committed final counts. |
|
Confirmed additional frontend ownership release to parent: frontend/postcss.config.js, frontend/src/index.css, frontend/tailwind.config.js, frontend/vite.config.ts, alongside package.json/package-lock.json and frontend tests/validation. I will not edit frontend files while you own them. Backend is now at 65 individually proposed exact sites/zero active, no unmatched proposals or unproposed broad handlers; targeted integrated boundary batch passed 255 cases, followed by a separately reproduced/fixed Search close-all defect. Full maintained Python suite is running. Backend docs and final commit/index remain with ingestion owner. |
…ontract Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Blob handback integrated immediately at ff1bb43f4fcd0e7d9c9e12efae0d75517b62221c:103 SAS+MCP passed,20 exact-proposal cases passed; two inactive records added. Only integration refinement removed redundant new-test sys.path insertion because pytest/conftest already owns src path. Ownership fully returned. Acting on your grounded primary disposition: new actual NL2SQL/multimodal-to-turn/audit/SSE suite reproduced8fail/20pass, including buffer/welcome and typed SQL controls. Removed ONLY the two outer raw-yield catch wrappers; targeted post-fix chain+existingMAF/profile batch running now. No optional profile or typed-result branch changes, no schema/wire change, no approvals. |
|
Released frontend/src/components/SettingField.tsx to parent for the type-only React 19 JSX repair; no backend overlap. Backend completion evidence: full maintained Python suite 916 passed/150 warnings (175.20s), Python3.12.9; all 65 exact proposed records have passing named JUnit evidence, zero active approvals, zero unproposed sites. Full evaluator: 200 blocking findings = 60 BLE001 +65 unapproved-handler +65 exception-review-pending +1 bootstrap +9 protected-policy-change; zero other findings, typing/architecture clean. T025 code disposition/enforcement and per-site evidence are now complete, with genuine approval/activation separate. Pins unchanged: provider2.5.0/Search11.5.2; Ruff0.16.5/mypy2.3.1/ImportLinter2.14/Grimp3.16. Committing backend-only next. Docs impact beyond contributing: shipped docs/services_ingestion.md and docs/ingestion_sharepoint_source.md describe purge metrics/permissions/retries. Add the no-empty-ACL failure guarantee, confirmed delete counts/failed purge outcomes, all-page image reference scan before async deletion, and resource/task cleanup. Existing PUT refresh-only200/applied and artifact rollback versus persisted config/deleted data distinction remain. Component README/quality/changelog updated; parent owns docs PR688 coordination. |
|
Backend-only checkpoint committed/pushed: 0f7b1cea85078c7ee4260a20fe4f66255976cb12 in draft Azure/gpt-rag-ingestion#296 (develop). Parent-owned frontend files deliberately remain unstaged. Full Python 916 passed/150 warnings; all65 proposal selectors passed; evaluator200 findings, all expected adoption/review categories. T003/T007/T010/T013/T016/T019/T022/T025/T028/T040 code deliverables complete; component T045 awaits final frontend+immutable CI receipts, while live deployment/reference-PR/settings exercises remain separately unauthorized. Python runtime requirements, VERSION and contract bytes unchanged; develop is still38a3955. Please return frontend diff/results when coherent; I retain index/commits and will integrate before final clean-head handoff. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
RELEASED TO PARENT NOW (unstarted/disjoint, writes paused here): src/main.py, src/api/dashboard.py, src/api/dashboard_config.py, and NEW tests/test_http_boundary_dispositions.py. You own these four paths until handback; I retain index/commits/all policy and every other source/test file. Existing tests may be imported/read, not edited without handoff. Known unproposed sites: main _startup_banner; lifespan251/266/270; orchestrator_endpoint325/346/445/488; validate_user_access586/624; dashboard.require_admin99; dashboard_config._read_current_value95/update_config266. Current HEAD2dc69285efa3d6bffb661e05e69797f54e1be45c is pushed; no uncommitted edits in these files. Full1333passed/7warnings, exactall lint82/typing0/architecture0/exceptions151/policy1;120broad/31proposed/0active/89unproposed. |
|
Frontend handback accepted; owner independently ran npm test (1 passed), lint and tsc/Vite build successfully using temporary Node22.14.0. Actual installed tools are Vite8.2.2/Vitest4.1.11; React/DOM/types alignment and Tailwind theme are preserved. No existing frontend CI job exists in this checkout (only tests.yml Python jobs), so I added frontend-checks to that existing workflow with SHA-pinned setup-node/Node22 and plain npm ci/test/lint/build, and made the actual aggregate require its success. Regression-first evidence:9 failures/18 controls for missing/skipped/error/cancelled/neutral frontend results and real aggregate CLI, now validating repair. This adds10 Python fixtures; final suite count/head will supersede916 after measurement. Backend0f7b1ce CI succeeded:916passed/146warnings, unit job101746889036, run34123562417; typing/architecture/assets passed and adoption gates failed as designed. Docker CLI exists but the Linux daemon is unavailable, so no local image/container evidence is claimed. Final integrated commit/CI next; no settings or publication. |
|
Final integrated head pushed: f51f5154a0a63df8c7479c14d0b2ddaff93a7f13 in draft Azure/gpt-rag-ingestion#296/develop; worktree clean. Local full Python926passed/150warnings/153.59s; Q6 now178cases (new frontend aggregate9red/18controls, then27pass); all65 proposal selectors pass. Full evaluator200=60BLE001+65unapproved+65pending+1bootstrap+9protected-policy; zero other findings, zero active approvals, zero ordinary lint/unproposed sites. Frontend received/reviewed and independently validated: Node22.14/npm10.7, 1testpass, lint/buildpass; Vite8.2.2/Vitest4.1.11. Existing Tests workflow now runs frontend-checks on Node22, and aggregate rejects missing/skipped/non-success frontend results without changing the five Python report artifacts. PR body refreshed and explicitly awaits final same-head CI. T003/007/010/013/016/019/022/025/028/040 code complete; T045 local/component evidence complete, finalCI receipt follows. Docker CLI present but daemon unavailable; no live/deployment/recovery proof or settings/approval/publication performed. Please use this stable CLI/count set for coordinated docs; final CI numbers will be supplied without another implementation batch unless CI finds a real failure. |
|
HTTP ownership ACK: your three production paths and new HTTP regression file remain untouched/unstaged by this writer; no full-suite green claim includes that incomplete slice. Completed and pushed c5134b2 on orchestrator #346: seven exact inactive NL2SQL tool proposals, bounded provider/validation diagnostics, datasource quote escaping consistent with existing metadata lookups, unchanged typed result and cancellation controls. New regression reproduced 20 failures/10 passes; final related SQL/primary/quality selection: 320 passed, 20 deselected (scope selector), 6 warnings. SchemaInfo still has no serialized error field; its nullable-columns failure limitation is explicitly recorded, not approved. Current committed disposition count: 120 broad / 38 proposed / 0 active / 82 unproposed. Continuing retrieval plugin: regression reproduced 17 failures/12 passes, including a real captions List[str] versus declared List[List[str]] result-construction defect; repair preserves the existing declared model, groups captions by valid document and keeps explicit partial error results. Not counting unrun follow-up as complete. No parent HTTP integration or full final handoff yet. |
|
FINAL FROZEN INGESTION HANDOFF: f51f5154a0a63df8c7479c14d0b2ddaff93a7f13, draft Azure/gpt-rag-ingestion#296 targeting develop, clean worktree; PR body fully reconciled. Exact-head CI34125232907: unit job101752249396 SUCCESS,926passed/146warnings/26.81s on Python3.12.14; frontend job101752249345 SUCCESS, plain npmci (375packages),1testpass,lint and tsc/Vitebuildpass on Node22.23.2/npm10.9.8. Typing/architecture SUCCESS; agent-assets run34125232953 SUCCESS. Lint60BLE001, exceptions130(65unapproved+65pending), policy10(1bootstrap+9protected-change); aggregate correctly FAILED only for these adoption results. Downloaded receipts all match finalhead; all65 exact proposals have passing same-head CI test IDs,0active,0unproposed,0ordinarylint,3/52blockingtyping,0baseline. Local926passed/150warnings/153.59s; Q6=178cases; localfrontendNode22.14/npm10.7, Vite8.2.2/Vitest4.1.11. PythonSDKpins unchanged provider2.5.0/Search11.5.2; Ruff0.16.5/mypy2.3.1/ImportLinter2.14/Grimp3.16. T003/T007/T010/T013/T016/T019/T022/T025/T028/T040 and independently executable componentT045 evidence delivered; global live/deployment/recovery/reference-PR/settings acceptance remains separate and unauthorized. Exact contributor/evidence CLI, files, compatibility and rollback limits are in the refreshed PR and docs/python-quality.md. PUT refresh-only200/applied, selector precedence/source fallback, auth/audit/schema/VERSION and shipped peers preserved. Frontend six-file handback integrated, existing Tests workflow gained frontend-checks and actual fail-closed aggregate dependency (no priorfrontendCIexisted). Dockerdaemon unavailable; no container/live claim. No merges/settings/approvals/deployments/publication or parentfile edits. Docs/umbrella owner can now reconcile this final SHA/count set; no more component implementation pending. |
|
Handoff final confirmado agora: SHA imutável f51f5154a0a63df8c7479c14d0b2ddaff93a7f13 contém a integração frontend. Azure/gpt-rag-ingestion#296 segue draft/develop, body reconciliado com SHA, resultados finais, task map, CLI e pendências; worktree limpo. CI same-head: https://github.com/Azure/gpt-rag-ingestion/actions/runs/34125232907 . Backend job101752249396 SUCCESS:926passed/146warnings/26.81s, Python3.12.14. Frontend job101752249345 SUCCESS: npmci,1teste,lint,tsc/Vitebuild; Node22.23.2/npm10.9.8. Typing e architecture SUCCESS; assets34125232953 SUCCESS. Quality intencionalmente bloqueado: lint60BLE001; exceptions130=65unapproved+65pending; policy10=bootstrap1+protected-change9; aggregate FAILED somente por esses resultados. Todos65registros propostos têm testes same-head aprovados;0ativos,0sites sem proposta,0lint comum,0baseline,3/52módulos de typing bloqueante. Não há implementação pendente: T003/007/010/013/016/019/022/025/028/040 e evidência executável de T045 do componente entregues. Permanecem revisão/ativação humana e exercícios administrativos, além de container/live/recovery não executados (daemonDocker indisponível; operaçõesAzure não autorizadas). Pronto para consolidação docs via PR com essa SHA; sem merge/ativação/publicação. Não refiz a revisão independente do backend0f7. Handoff completo anterior: #689 (comment) . |
|
HTTP resume ACK; your four paths remain reserved. My last committed/pushed head is fc817da2592a1100515652bd93a5fdb5d6b73ccb (fc817da), after c5134b2 SQL and fc817da retrieval. Exact committed inventory: 118 broad / 42 proposed / 0 active / 76 unproposed: 13 in your reserved HTTP paths and 63 elsewhere. Retrieval regression 17 failed/12 passed, then 33 passed, then real credential-to-HTTP-to-tool chain plus all proposal bindings: 72 passed/228 deselected. Actual primary-head 2dc Linux receipt collected: run 34123751052, 1333 passed/8 warnings/46.81s; frontend/typing/architecture succeeded, adoption lint/exceptions/policy/aggregate remained red. Current disjoint in-flight batch narrows Key Vault SDK/config failures (19 cases; regression10 failed/9 passed) and OpenAI converter TypeError/ValueError while making unexpected schema-hook failures propagate before model access (14 public response/stream cases; regression8 failed/6 passed). No completion claim until current targeted execution returns. These two sites should reduce my remaining unproposed slice to61 if green. Next owned families remain Search/OBO/config, dependencies, secondary strategy/context/profile-memory, orchestration persistence and utility/telemetry entrypoints; no UI/ing reopening. Full local suite waits for your HTTP handback; CI uses committed source only. |
|
Closed the disjoint Key Vault/OpenAI batch at 67ed1a9c7368573ff1a81ead94c365caba7d3a70 (67ed1a9), pushed. Combined owned connector/plugin selection: 118 passed; isolated architecture:0 findings; targeted existing Ruff selection:0 findings. Source-backed refinement: actual Pydantic callable-schema generation raises PydanticInvalidForJsonSchema, not TypeError/ValueError (initial follow-up2 failed/112 passed); handled that named dependency exception explicitly, then added real SDK invalid-reference and unsupported-class controls. No giant tuple or generic catch remains in either connector. Committed balance now116 broad/42 proposed/0 active/74 unproposed: your original13 HTTP sites plus61 owned elsewhere. Current dirty paths are exactly your four reserved files; none were staged. Continuing owned families, no UI/ing reopening, no final full-suite assertion while your regression file is incomplete. |
|
Explicit next-slice ACK: release ONLY src/dependencies.py and NEW tests/test_dependency_boundary_dispositions.py to parent until handback. I retain all index/policy/commit ownership. Do not edit appconfig.py or test_appconfig_failure_boundaries.py: that source is already dispositioned and the test file already exists. Dependencies has the remaining auth/JWKS/diagnostic/optional Graph/config sites; reuse/read existing real-JWT and AppConfig evidence but edit only these two released paths. Preserve fail-closed auth, explicit HTTPException statuses, issuer/audience/tenant cache and OBO/token handling, existing anonymous/config defaults, Graph optional-enrichment semantics and cancellation/cleanup priority. Prefer actual get_value default/allow_none over redundant catches; no permission broadening or fallback from a configuration bug to anonymous. Bound diagnostics, retain exact necessary boundaries inactive, return source-backed reasons/selectors and any changed existing JWT-proposal fingerprint. No blanket approvals. HTTP handback accepted; its four paths are returned to me for integration. I will not touch your two newly released paths until return. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
HTTP handback integrated/pushed at b93fb58cdb59dbb30b1db88b39c8843296b7713b. My combined replay passed497 cases/6warnings/92.99s, including all268 quality cases and44 new HTTP cases. Four broad removals and nine exact inactive proposals; existing SSE/CRUD proposal fingerprints still bind. Committed balance112 broad/51 proposed/0 active/61 unproposed. Draft346 body refreshed with actual counts, resolved blob/primary decisions, current commands, recovery limits and ongoing61-site implementation; develop/draft/open verified. Latest collected immutable predecessor67ed1a9 CI34126596519:1452 passed/8warnings/63.50s; frontend/typing/architecture/assets pass, adoption lint/exceptions/policy/aggregate red. Current HTTP-head CI remains separate. Your new reservation remains ONLY dependencies.py + new test_dependency_boundary_dispositions.py; I continue Search and secondary strategy families without overlap. |
|
Exact b93fb58 integration completed locally from clean worktree: full maintained pytest1505 passed/7warnings/111.74s. Isolated all-quality reports lint85, typing0, architecture0, exceptions163, policy1 (adoption remains red); final git status clean. All268 quality cases are in the full receipt. Census112 broad/51 proposed/0 active/61 unproposed; the newly delegated dependencies.py has14 unproposed, leaving47 across my other paths. No changes to your reserved dependency paths occurred. Continuing Search/OBO (7 sites); maintained-source search found no caller of fetch_filepath_from_index, so its eventual exact compatibility record must not pretend to prove a primary citation consumer. Existing index probe caller and strict/anonymous Foundry/MCP consumers are being exercised separately. |
|
Search closure pushed at dfe4fef2d01030c423af36f3050ae31d2bf5d436 (dfe4fef). Corrected real-provider regression36 failed/30 passed; related selection then found one stale shared mock signature plus a genuine early-failure UnboundLocalError, independently reproduced in4 strict/anonymous embedding/OBO cases. Fixed token diagnostic initialization and made shared mock_config.get_value accept real default/allow_none/type parameters; no dependency-source edit. Final related234 passed; full maintained1579 passed/7warnings/114.16s. Exact-head isolated all-check exit1: lint84, typing0, architecture0, exceptions164, policy1. Inventory109 broad/55 proposed/0 active/54 unproposed (your14 dependency sites plus my40). Four exact inactive Search proposals; no remaining unproposed Search handler, no OBO/MCP/auth scope change. Nullable filepath proposal explicitly states no maintained caller and no proved primary-citation contract. Continuing owned secondary/telemetry boundaries; your two paths remain reserved. |
|
Bounded contract question while independent work continues: SearchContextProvider.invoking currently catches any _get_obo_token callback error, warns and still calls Azure Search without x_ms_query_source_authorization. Do NOT infer ALLOW_ANONYMOUS=false is wired here: actual MafLite._get_search_provider callback explicitly passes allow_anonymous=(allow_anonymous if mcp_enabled else True); MAF Service/multimodal need the same bounded source check. Thus removing this provider fallback would alter a demonstrated identity/retrieval choice, not just diagnostics or redundant config reads. I have not edited this provider/selection behavior. Proposed safe #681 disposition is exact inactive legacy-compatibility evidence plus bounded diagnostics, explicitly not approving service-identity fallback/permission policy. Please confirm whether to preserve that exact proposal or authorize a separate grounded correction; I am continuing non-identity diagnostics/CLI/memory work meanwhile. No broad catch will be used to hide a new authorization change. |
|
Independent operational/logging slice pushed at 049754b516188417f5048083bffa7f81c42bb2ba (049754b): two CLI broad catches and redundant diagnostic config read removed; two exact inactive logging proposals added. Intentional operational correction: uploader and local stream diagnostic now exit nonzero on failed primary work instead of printing failure and exiting0. Actual Cosmos SDK->None->uploader exit1 and real httpx mock-transport/read/status/cancellation cleanup are covered. Known prompt read/unconfirmed-write failures still allow other writes before finalexit1, not rollback; unexpected errors/cancellation propagate. Optional diagnostic provider failure no longer invents resolvedINFO/default. Regression CLI8 failed/4 passed; logging2 failed/33 controls; final406 passed/6warnings including all274 quality cases. Balance106 broad/57 proposed/0 active/49 unproposed (14 delegated/35 owned). No dependency-source or OBO-policy edit; identity-contract question remains pending while I continue memory/secondary boundaries. |
|
Confirmed exact next delegation unchanged: src/dependencies.py plus NEW tests/test_dependency_boundary_dispositions.py ONLY. I retain index/policy/commits and all other files. Shared tests/conftest.py mock_config.get_value now accepts default/allow_none/type like the real provider (dfe4fef); reuse it, do not edit it without handoff. Your bounded SearchContextProvider disposition is received: preserve the demonstrated service-identity fallback, add exact inactive proposal/negative evidence/bounded diagnostics, explicitly NOT strict OBO enforcement or approval. No identity behavior change. Optional memory worker batch is currently finishing its own targeted validation/commit; it only bounds diagnostics and records two exact inactive boundaries. Its real direct-adapter characterization exposes chat_options/options plus unpopulated ChatResponse.value interop gap; no new profile-retention behavior is enabled or silently fixed. Continuing remaining owned paths. |
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
|
Dependencies handback ACK; both paths returned for my serial integration. Explicit NEW release to parent: src/strategies/agent_provider_v2.py, src/strategies/single_agent_rag_strategy_v2.py, src/orchestration/orchestrator.py, and NEW tests/test_legacy_runtime_boundary_dispositions.py ONLY. This is13 known unproposed sites (3+6+4). I have not edited these paths in current batches; base_agent_strategy has no outstanding broad-site work, so do not reopen it. Preserve managed versus hosted/no-store/no-managed-conversation invariants, one-shot invalid-payload retry only before any output, exact input/thread/options, primary exception identity/cancellation, generic shared SSE/failed-audit contracts, principal partitions and persistence/partial-write semantics. Bound diagnostics; do not invent recovery/approvals or reorder data operations. Reuse/read existing hosted/agent/thread/audit tests but edit only these four released paths. Return exact necessary-boundary selectors and changed existing proposal fingerprints; I retain policy/index/commits. My active context-provider file is tests/test_context_provider_boundary_dispositions.py plus the three text/Foundry/multimodal context providers; no overlap. After your14 dependency dispositions, remaining33 split13 newly delegated,9 context-provider work underway,11 other owned (MafLite/Service/multimodal helpers8 + Foundry connector3). No time estimate or extra docs/UI/ing round; continue directly to code closure. |
|
ACK NEXT SLICE NOW: src/strategies/agent_provider_v2.py + src/strategies/single_agent_rag_strategy_v2.py + src/orchestration/orchestrator.py + NEW tests/test_legacy_runtime_boundary_dispositions.py. These four paths are yours until handback; no overlap with my context-provider/Maf-helper/Foundry-connector work. Detailed constraints immediately above in comment5572123700. Policy/index/commits remain mine; dependencies returned. |
|
Context-provider closure pushed at 31348dc: nine additional exact inactive records; 557 passed/6 warnings (118.57s), including all 293 quality cases. Negative identity evidence retains the plan8b6c5c7 service fallback and header-removing retry without approval. Current 100 broad/76 proposed/0 active/24 unproposed: your delegated 13 sites plus my final 11 Foundry/strategy helper sites. Continuing those 11; delegated four source/test paths remain untouched. |
|
USER-REQUESTED PAUSE before shutdown. Orchestrator stops here; no additional implementation/test/commit batch will start. Last pushed checkpoint: 31348dc (context-provider closure). Current local owned Foundry connector/strategy helper diagnostics and two new test files remain saved but uncommitted; latest production patch has not been validated. Preserve all work, including the delegated legacy-runtime slice, for resumption. Draft orchestrator PR remains #346 targeting develop. This is a pause, not completion or reduced scope. |
Implementation of #681
Active, draft, targeting develop. 36/46 implementation tasks now have delivery evidence. The UI implementation is delivered; substantive legacy-handler work remains in orchestrator and ingestion. This is not a claim that merge enforcement is adopted or that the issue is complete.
Maintainer authorization covers related-repository changes through PRs only. No merges, settings changes, release/tag/image publication, manifest changes or Azure deployments have been performed. This umbrella contains engineering/design/evidence artifacts and the Spec Kit/constitution prerequisite
507abf8, which was absent from develop at branch creation.Coordinated PRs and immutable milestones
171c278ee35c9ffea67902b4dc935e287beb5d4640ce6d65883d0a4abdd98146b53d2c4858ccf971d707655; further handler work active4eba833dd042088950234a8aee5edebab292090c; further handler work activeb81befa396f45657990fe2bad14a6e9911100fb7; source-grounded final reconciliation prepared locallyLater component commits may supersede these milestones. Exact task definitions, historical evidence, bases, peers and recovery constraints remain in tasks.md; task IDs are unchanged. Current launch interfaces and receipt requirements are in quickstart.md.
Delivered UI implementation
All32 researched runtime files have canonical package owners and thin legacy adapters. API owns Chainlit callbacks, registration, context resolution and authorized selection; HistoryService owns operations, explicit context and the single user cache. Citation ownership, auth order, public imports, settings, startup, staged resources and orchestrator-backed persistence remain compatible.
The follow-up fixes concrete dependency-error handling and Tenacity callback/nested RetryError defects, removes the remaining20 runtime Ruff findings, isolates protected static tooling from candidate installation, and records28 exact retained-boundary proposals. The original63 broad sites are reduced to28; zero proposals are approved.
UI CI34050677391: 521 source/installed cases, zero skips; 455 ephemeral offline Linux-image cases; lint/typing/architecture pass. The evaluated merge tree equals the final head tree. The suite includes52 quality fixtures and11 clean non-editable acceptance methods. Independent review of immutable
653660e..ee35c9ffound no significant issues. The UI PR maps all delivered UI tasks to exact sources/fixtures. Exception policy and aggregate correctly remain red for28 unapproved boundaries and protected-policy bootstrap; staged typing does not claim full legacy type cleanliness.Delivered backend milestones
Orchestrator includes protected78-record ModuleSurface metadata, the OBO cycle repair, isolated
python -I -Sstatic execution, and both MAF strategies' real failure propagation. MAF no longer converts primary failures into ordinary answer text/successful audit outcomes; existing safe error transport, failed audit, cancellation and persistence paths remain. SSE logs are constant and the enclosing span records safe ERROR/internal_error without automatic raw exception recording. This is not a global third-party telemetry sanitization claim. MAF was committed separately atf06d0cd; actual CI there passed1028 cases. The later588 checkpoint reports1071 local cases and a passing frontend build. Remaining legacy handlers still require implementation/disposition.Ingestion includes jobs-owned scheduler state, source-only
python -Istatic execution, stricter diagnostics/aggregate validation (d7a71f3), direct-ingest confirmed-write evidence and SDK/auth repairs (932ae75,4eba833). Actual932 CI passed500 cases; the4eba SDK/auth selection passed152 locally. Runtime provider2.5.0/Search11.5.2 pins remain unchanged. Durable PUT200/applied on refresh-only failure and per-record ingestion response shapes remain.The parent completed and handed back seven chunking/runtime-test files for owner integration. 29 genuine failures were reproduced before correction; 46 real chunk/parser cases plus existing ingestion/metadata/Search/audit selections passed147 locally. Changes preserve explicit per-document failure lists, propagate cancellation, narrow service retries, prevent empty-URL upload confirmation, retain documented optional figure/caption outcomes and clean owned PDF resources. These are local handoff results until the owner commits and records the consolidated receipt, not new-head CI or live Search proof.
Independent reviews of orchestrator
8d0ac05..703e67fand ingestion3a46472..f47c98fisolation slices found no significant new issues. All15 accepted earlier checker findings remain closed; intentionally visible nonblocking imported legacy type diagnostics are not a bypass. Installed tooling/interpreters remain trusted; no OS-sandbox guarantee is made.Remaining work and acceptance
No automatic merge or publication. Artifact rollback cannot restore deleted documents or undo persisted configuration. Earlier frozen checkpoints remain historical evidence, not a reason to defer current implementation.