Skip to content

Add standalone JSON logging (cre_logging) - #1101

Merged
northdpole merged 1 commit into
mainfrom
feat/json-structured-logging
Sep 13, 2026
Merged

northdpole merged 1 commit into
mainfrom
feat/json-structured-logging

Conversation

@northdpole

Copy link
Copy Markdown
Collaborator

Summary

  • Add top-level cre_logging so every module can from cre_logging import get_logger without importing Flask/application.__init__.
  • Always emit one JSON object per stderr line, including module, file, method, and level.
  • Replace per-module logging.basicConfig() / unused pprint, and send relevant print progress/errors through the logger. CLI machine output (print(summary.to_json())) stays on stdout.

Supersedes #856 (optional JSON + application.utils.logging_config, which still sits under the heavy application package).

Test plan

  • python -m unittest application.tests.cre_logging_test (includes subprocess check that import cre_logging does not load application or flask)
  • mapping fixture / AISVS / cheatsheets parser tests
  • CI make lint / make mypy / make test

Made with Cursor

Keep cre_logging off the Flask application package so importing a logger never loads create_app. Emit one JSON object per line with module, file, and calling method.
@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Too many files!

This PR contains 305 files, which is 205 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

Usage-priced reviews support at most 300 files.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Advanced

Run ID: eecf01bb-0a12-4fe0-ad51-820c671bb3b6

📥 Commits

Reviewing files that changed from the base of the PR and between 68ac01d and 333a727.

📒 Files selected for processing (305)
  • Makefile
  • application/__init__.py
  • application/cmd/__init__.py
  • application/cmd/cre_main.py
  • application/config.py
  • application/database/__init__.py
  • application/database/db.py
  • application/database/inmemory_graph.py
  • application/database/pgvector_utils.py
  • application/defs/__init__.py
  • application/defs/cheatsheet_defs.py
  • application/defs/cre_defs.py
  • application/defs/cre_exceptions.py
  • application/defs/osib_defs.py
  • application/feature_flags.py
  • application/mcp/__init__.py
  • application/mcp/__main__.py
  • application/mcp/catalog.py
  • application/mcp/openapi_loader.py
  • application/mcp/rest_client.py
  • application/mcp/server.py
  • application/prompt_client/embed_alignment.py
  • application/prompt_client/llm_error_utils.py
  • application/prompt_client/prompt_client.py
  • application/tests/__init__.py
  • application/tests/admin_imports_api_test.py
  • application/tests/ai_exchange_csv_test.py
  • application/tests/auth_routes_test.py
  • application/tests/capec_parser_test.py
  • application/tests/ccmv4_parser_test.py
  • application/tests/chat_completion_test.py
  • application/tests/cheatsheet_extractor_test.py
  • application/tests/cheatsheet_record_adapter_test.py
  • application/tests/cheatsheets_parser_test.py
  • application/tests/cheatsheets_workstream_f_test.py
  • application/tests/cloud_native_security_controls_parser_test.py
  • application/tests/cre_logging_test.py
  • application/tests/cre_main_test.py
  • application/tests/cwe_parser_test.py
  • application/tests/db_test.py
  • application/tests/defs_test.py
  • application/tests/dsomm_parser_test.py
  • application/tests/embed_alignment_test.py
  • application/tests/embeddings_sync_scripts_test.py
  • application/tests/export_cli_test.py
  • application/tests/ga_parity_test.py
  • application/tests/gap_analysis_db_test.py
  • application/tests/gap_analysis_pair_job_test.py
  • application/tests/gap_analysis_test.py
  • application/tests/harvester_test/__init__.py
  • application/tests/harvester_test/artifact_registry_test.py
  • application/tests/harvester_test/change_detector_test.py
  • application/tests/harvester_test/checkpoint_manager_test.py
  • application/tests/harvester_test/checkpoint_store_test.py
  • application/tests/harvester_test/chunk_pipeline_test.py
  • application/tests/harvester_test/chunk_record_builder_test.py
  • application/tests/harvester_test/chunk_record_validator_test.py
  • application/tests/harvester_test/chunker_test.py
  • application/tests/harvester_test/chunking_benchmark_test.py
  • application/tests/harvester_test/config_loader_test.py
  • application/tests/harvester_test/content_hash_test.py
  • application/tests/harvester_test/deduplication_metrics_test.py
  • application/tests/harvester_test/diff_normalizer_test.py
  • application/tests/harvester_test/diff_parser_test.py
  • application/tests/harvester_test/diff_pipeline_test.py
  • application/tests/harvester_test/diff_retriever_test.py
  • application/tests/harvester_test/document_builder_test.py
  • application/tests/harvester_test/document_deduplicator_test.py
  • application/tests/harvester_test/document_validator_test.py
  • application/tests/harvester_test/file_filter_test.py
  • application/tests/harvester_test/filtering_benchmark_test.py
  • application/tests/harvester_test/filtering_metrics_test.py
  • application/tests/harvester_test/git_repository_client_integration_test.py
  • application/tests/harvester_test/git_repository_client_test.py
  • application/tests/harvester_test/harvest_pipeline_test.py
  • application/tests/harvester_test/heading_extractor_test.py
  • application/tests/harvester_test/incremental_pipeline_test.py
  • application/tests/harvester_test/repos_validator_test.py
  • application/tests/harvester_test/repository_cache_test.py
  • application/tests/import_dashboard_test.py
  • application/tests/import_diff_test.py
  • application/tests/import_pipeline_pair_scheduler_test.py
  • application/tests/import_post_apply_test.py
  • application/tests/import_run_test.py
  • application/tests/import_staging_test.py
  • application/tests/inmemory_graph_test.py
  • application/tests/juiceshop_test.py
  • application/tests/librarian/__init__.py
  • application/tests/librarian/candidate_retriever_test.py
  • application/tests/librarian/config_loader_test.py
  • application/tests/librarian/cross_encoder_test.py
  • application/tests/librarian/dataset_test.py
  • application/tests/librarian/decision_engine_test.py
  • application/tests/librarian/decision_queue_test.py
  • application/tests/librarian/emitter_test.py
  • application/tests/librarian/envelope_sink_test.py
  • application/tests/librarian/evaluate_harness_test.py
  • application/tests/librarian/explicit_link_resolver_test.py
  • application/tests/librarian/factory_test.py
  • application/tests/librarian/hub_firewall_test.py
  • application/tests/librarian/knowledge_source_test.py
  • application/tests/librarian/pipeline_test.py
  • application/tests/librarian/queue_consumer_test.py
  • application/tests/librarian/queue_runner_test.py
  • application/tests/librarian/safety_guard_test.py
  • application/tests/librarian/schemas_test.py
  • application/tests/librarian/scoring_test.py
  • application/tests/librarian/section_validator_test.py
  • application/tests/librarian/temperature_test.py
  • application/tests/litellm_router_test.py
  • application/tests/master_spreadsheet_parser_test.py
  • application/tests/mcp_public_tools_test.py
  • application/tests/mdutils_test.py
  • application/tests/misc_tools_parser_test.py
  • application/tests/monitor_chatbot_health_test.py
  • application/tests/monitor_ga_health_test.py
  • application/tests/myopencre_parser_test.py
  • application/tests/neo_node_lookup_test.py
  • application/tests/noise_filter/__init__.py
  • application/tests/noise_filter/config_loader_test.py
  • application/tests/noise_filter/llm_classifier_test.py
  • application/tests/noise_filter/pipeline_test.py
  • application/tests/noise_filter/queue_writer_test.py
  • application/tests/noise_filter/regex_filter_test.py
  • application/tests/noise_filter/sanitize_test.py
  • application/tests/noise_filter/schemas_test.py
  • application/tests/openapi_guardrail_test.py
  • application/tests/opencre_gap_analysis_test.py
  • application/tests/oscal_utils_test.py
  • application/tests/osib_defs_test.py
  • application/tests/owasp_aisvs_parser_test.py
  • application/tests/owasp_api_top10_2023_parser_test.py
  • application/tests/owasp_kubernetes_top10_2022_parser_test.py
  • application/tests/owasp_kubernetes_top10_2025_parser_test.py
  • application/tests/owasp_llm_top10_2025_parser_test.py
  • application/tests/owasp_mapping_fixtures_test.py
  • application/tests/owasp_top10_2025_parser_test.py
  • application/tests/pci_dss_parser_test.py
  • application/tests/pgvector_utils_test.py
  • application/tests/prompt_client_incremental_embeddings_test.py
  • application/tests/prompt_client_pdf_embeddings_test.py
  • application/tests/prompt_client_pgvector_similarity_test.py
  • application/tests/prompt_client_smart_embed_test.py
  • application/tests/register_standard_incremental_embeddings_test.py
  • application/tests/resource_filter_test.py
  • application/tests/secure_headers_parser_test.py
  • application/tests/spreadsheet_parsers_test.py
  • application/tests/spreadsheet_test.py
  • application/tests/test_smart_embeddings_e2e_llm.py
  • application/tests/user_model_test.py
  • application/tests/user_resources_api_test.py
  • application/tests/utils/data_gen.py
  • application/tests/web_main_test.py
  • application/tests/worker_test.py
  • application/tests/zap_alerts_parser_test.py
  • application/utils/cres_csv_export.py
  • application/utils/db_backend.py
  • application/utils/external_project_parsers/__init__.py
  • application/utils/external_project_parsers/base_parser.py
  • application/utils/external_project_parsers/base_parser_defs.py
  • application/utils/external_project_parsers/parsers/__init__.py
  • application/utils/external_project_parsers/parsers/ai_exchange/__init__.py
  • application/utils/external_project_parsers/parsers/ai_exchange/csv_source.py
  • application/utils/external_project_parsers/parsers/ai_exchange/exchange_source.py
  • application/utils/external_project_parsers/parsers/capec_parser.py
  • application/utils/external_project_parsers/parsers/ccmv4.py
  • application/utils/external_project_parsers/parsers/cheatsheet_extractor.py
  • application/utils/external_project_parsers/parsers/cheatsheet_record_adapter.py
  • application/utils/external_project_parsers/parsers/cheatsheets_parser.py
  • application/utils/external_project_parsers/parsers/cheatsheets_workstream_f.py
  • application/utils/external_project_parsers/parsers/cloud_native_security_controls.py
  • application/utils/external_project_parsers/parsers/cwe.py
  • application/utils/external_project_parsers/parsers/dsomm.py
  • application/utils/external_project_parsers/parsers/export_format_parser.py
  • application/utils/external_project_parsers/parsers/iso27001.py
  • application/utils/external_project_parsers/parsers/juiceshop.py
  • application/utils/external_project_parsers/parsers/master_spreadsheet_parser.py
  • application/utils/external_project_parsers/parsers/misc_tools_parser.py
  • application/utils/external_project_parsers/parsers/myopencre_parser.py
  • application/utils/external_project_parsers/parsers/owasp_aisvs.py
  • application/utils/external_project_parsers/parsers/owasp_api_top10_2023.py
  • application/utils/external_project_parsers/parsers/owasp_kubernetes_top10_2022.py
  • application/utils/external_project_parsers/parsers/owasp_kubernetes_top10_2025.py
  • application/utils/external_project_parsers/parsers/owasp_llm_top10_2025.py
  • application/utils/external_project_parsers/parsers/owasp_mapping_fixture_parser.py
  • application/utils/external_project_parsers/parsers/owasp_top10_2025.py
  • application/utils/external_project_parsers/parsers/pci_dss.py
  • application/utils/external_project_parsers/parsers/secure_headers.py
  • application/utils/external_project_parsers/parsers/spreadsheet_asvs.py
  • application/utils/external_project_parsers/parsers/spreadsheet_biml.py
  • application/utils/external_project_parsers/parsers/spreadsheet_cloud_controls_matrix.py
  • application/utils/external_project_parsers/parsers/spreadsheet_cwe.py
  • application/utils/external_project_parsers/parsers/spreadsheet_enisa.py
  • application/utils/external_project_parsers/parsers/spreadsheet_etsi.py
  • application/utils/external_project_parsers/parsers/spreadsheet_iso27001.py
  • application/utils/external_project_parsers/parsers/spreadsheet_mitre_atlas.py
  • application/utils/external_project_parsers/parsers/spreadsheet_nist_800_53_v5.py
  • application/utils/external_project_parsers/parsers/spreadsheet_nist_800_63.py
  • application/utils/external_project_parsers/parsers/spreadsheet_nist_ai_100_2.py
  • application/utils/external_project_parsers/parsers/spreadsheet_nist_ssdf.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_ai_exchange.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_cheat_sheets.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_proactive_controls.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_top10_2017.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_top10_2021.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_top10_llm.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_top10_ml.py
  • application/utils/external_project_parsers/parsers/spreadsheet_owasp_wstg.py
  • application/utils/external_project_parsers/parsers/spreadsheet_samm.py
  • application/utils/external_project_parsers/parsers/spreadsheet_standard_family.py
  • application/utils/external_project_parsers/parsers/spreadsheet_standards_dispatch.py
  • application/utils/external_project_parsers/parsers/zap_alerts_parser.py
  • application/utils/file.py
  • application/utils/ga_parity.py
  • application/utils/gap_analysis.py
  • application/utils/git.py
  • application/utils/harvester/__init__.py
  • application/utils/harvester/artifact_id.py
  • application/utils/harvester/artifact_registry.py
  • application/utils/harvester/change_detector.py
  • application/utils/harvester/checkpoint_manager.py
  • application/utils/harvester/checkpoint_store.py
  • application/utils/harvester/chunk_pipeline.py
  • application/utils/harvester/chunk_record_builder.py
  • application/utils/harvester/chunk_record_validator.py
  • application/utils/harvester/chunker.py
  • application/utils/harvester/config_loader.py
  • application/utils/harvester/content_hash.py
  • application/utils/harvester/deduplication_metrics.py
  • application/utils/harvester/diff_normalizer.py
  • application/utils/harvester/diff_parser.py
  • application/utils/harvester/diff_retriever.py
  • application/utils/harvester/document_builder.py
  • application/utils/harvester/document_deduplicator.py
  • application/utils/harvester/document_validator.py
  • application/utils/harvester/file_filter.py
  • application/utils/harvester/filtering_benchmark.py
  • application/utils/harvester/filtering_metrics.py
  • application/utils/harvester/git_repository_client.py
  • application/utils/harvester/harvest_writer.py
  • application/utils/harvester/heading_extractor.py
  • application/utils/harvester/incremental_pipeline.py
  • application/utils/harvester/models.py
  • application/utils/harvester/pipeline.py
  • application/utils/harvester/repos_validator.py
  • application/utils/harvester/repository_cache.py
  • application/utils/harvester/repository_client.py
  • application/utils/harvester/repository_lock.py
  • application/utils/harvester/schemas.py
  • application/utils/hash.py
  • application/utils/import_apply.py
  • application/utils/import_diff.py
  • application/utils/import_graph.py
  • application/utils/import_impact.py
  • application/utils/import_pipeline.py
  • application/utils/import_post_apply.py
  • application/utils/librarian/__init__.py
  • application/utils/librarian/calibration/__init__.py
  • application/utils/librarian/calibration/temperature.py
  • application/utils/librarian/candidate_retriever.py
  • application/utils/librarian/config_loader.py
  • application/utils/librarian/cross_encoder.py
  • application/utils/librarian/decision_engine.py
  • application/utils/librarian/emitter.py
  • application/utils/librarian/envelope_sink.py
  • application/utils/librarian/explicit_link_resolver.py
  • application/utils/librarian/factory.py
  • application/utils/librarian/hub_firewall.py
  • application/utils/librarian/knowledge_source.py
  • application/utils/librarian/pipeline.py
  • application/utils/librarian/queue_consumer.py
  • application/utils/librarian/queue_runner.py
  • application/utils/librarian/safety_guard.py
  • application/utils/librarian/schemas.py
  • application/utils/librarian/scoring.py
  • application/utils/librarian/section_validator.py
  • application/utils/mapping_fixtures.py
  • application/utils/mdutils.py
  • application/utils/noise_filter/__init__.py
  • application/utils/noise_filter/config_loader.py
  • application/utils/noise_filter/hashing.py
  • application/utils/noise_filter/llm_classifier.py
  • application/utils/noise_filter/pipeline.py
  • application/utils/noise_filter/prompts.py
  • application/utils/noise_filter/queue_writer.py
  • application/utils/noise_filter/regex_filter.py
  • application/utils/noise_filter/sanitize.py
  • application/utils/noise_filter/schemas.py
  • application/utils/oie_orchestrator/__init__.py
  • application/utils/oie_orchestrator/pipeline.py
  • application/utils/oscal_utils.py
  • application/utils/redirectors.py
  • application/utils/redis.py
  • application/utils/spreadsheet.py
  • application/utils/spreadsheet_parsers.py
  • application/utils/telemetry.py
  • application/web/__init__.py
  • application/web/openapi_registry.py
  • application/web/openapi_schemas.py
  • application/web/visualise.py
  • application/web/web_main.py
  • application/worker.py
  • cre.py
  • cre_logging/__init__.py
  • cre_logging/py.typed

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@northdpole

Copy link
Copy Markdown
Collaborator Author

Merging with admin bypass: Test, Lint, Frontend Lint, Librarian, Analyze (python), and both e2e jobs are green. CodeQL failed on pre-existing web_main.py alerts (path expression, clear-text secret, URL redirect, exception leak) — GitHub notes these can surface on large diffs even when not introduced by the PR. Logging change does not add those sinks.

@northdpole
northdpole merged commit 7368352 into main Sep 13, 2026
12 of 13 checks passed
@northdpole
northdpole deleted the feat/json-structured-logging branch September 13, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant