Skip to content

Add return_not_null_data_as_optional parameter to read_table - #880

Open
krasnovdm wants to merge 1 commit into
ydb-platform:mainfrom
krasnovdm:feature/read-table-return-not-null-as-optional
Open

Add return_not_null_data_as_optional parameter to read_table#880
krasnovdm wants to merge 1 commit into
ydb-platform:mainfrom
krasnovdm:feature/read-table-return-not-null-as-optional

Conversation

@krasnovdm

Copy link
Copy Markdown

Problem

read_table always returns NOT NULL columns wrapped in Optional, and there is no way to opt out in the Python SDK.

The gRPC ReadTableRequest already has a return_not_null_data_as_optional field, and the C++ SDK exposes it via TReadTableSettings().ReturnNotNullAsOptional(false). The Python SDK simply never set this field, so users reading a table with NOT NULL columns always get Optional values back.

Change

Add a return_not_null_data_as_optional parameter to read_table (sync ydb/table.py and async ydb/aio/table.py) and plumb it through read_table_request_factory. It accepts a bool (mapped to FeatureFlag.ENABLED/DISABLED) or a raw FeatureFlag.Status.

Default behavior is unchanged: when the parameter is not provided, the field is left unset and NOT NULL columns are still returned as Optional (backward compatible).

# opt out of the Optional wrapping for NOT NULL columns:
session.read_table(path, return_not_null_data_as_optional=False)

@vgvoleg vgvoleg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hi! please update your branch to latest main and add some tests

@krasnovdm
krasnovdm force-pushed the feature/read-table-return-not-null-as-optional branch from 7d89bdd to ea13389 Compare August 31, 2026 13:04
@krasnovdm

Copy link
Copy Markdown
Author

Thanks for the review! Done:

  • Rebased the branch onto the latest main (resolved the CHANGELOG conflict; mergeable should be clean now).
  • Added unit tests in tests/table/test_read_table_request.py covering read_table_request_factory for return_not_null_data_as_optional=True/False and the default (unset) case.

PTAL 🙏

@codecov

codecov Bot commented Aug 31, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.36%. Comparing base (3688a5f) to head (56c508d).

Files with missing lines Patch % Lines
ydb/_session_impl.py 66.66% 1 Missing and 1 partial ⚠️
Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #880   +/-   ##
=======================================
  Coverage   82.36%   82.36%           
=======================================
  Files          99       99           
  Lines       12753    12759    +6     
  Branches     1242     1245    +3     
=======================================
+ Hits        10504    10509    +5     
- Misses       1795     1796    +1     
  Partials      454      454           
Flag Coverage Δ
integration 80.15% <0.00%> (-0.04%) ⬇️
unit 48.72% <66.66%> (+0.14%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
ydb/aio/table.py 80.32% <ø> (ø)
ydb/table.py 66.69% <ø> (ø)
ydb/_session_impl.py 61.53% <66.66%> (+0.42%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@vgvoleg

vgvoleg commented Aug 31, 2026

Copy link
Copy Markdown
Member

@krasnovdm please fix ci checks.

@krasnovdm

Copy link
Copy Markdown
Author

@vgvoleg pushed a fix for the failing lint jobs — reformatted tests/table/test_read_table_request.py with black==25.11.0 (the test file I added had multi-line calls that black collapses). black --check and flake8 are clean locally now. PTAL.

@krasnovdm
krasnovdm force-pushed the feature/read-table-return-not-null-as-optional branch from f7aaed5 to 397be14 Compare September 7, 2026 15:50
@krasnovdm

Copy link
Copy Markdown
Author

Rebased onto latest main (resolved the CHANGELOG.md conflict — both unreleased bullets kept) and fixed the failing lint. All checks are green now: lint (black/style/mypy), unit pass. The PR is MERGEABLE. Only your earlier change request is left blocking it — could you re-review / dismiss the stale CHANGES_REQUESTED? PTAL 🙏 cc @Ovcharuk

Adds the return_not_null_data_as_optional flag to read_table (sync and
async), mirroring the existing gRPC field and the C++ SDK's
ReturnNotNullAsOptional. Default behavior is unchanged (NOT NULL columns
stay wrapped in Optional for backward compatibility). Unit tests added to
ydb/table_test.py.
@krasnovdm
krasnovdm force-pushed the feature/read-table-return-not-null-as-optional branch from 397be14 to 56c508d Compare September 7, 2026 16:02
@krasnovdm

Copy link
Copy Markdown
Author

Small follow-up: moved the unit tests into the existing ydb/table_test.py (and dropped the separate tests/table/ file) to align with AGENTS.md — unit tests live in ydb/*_test.py and should extend existing files. Squashed to a single commit. CI is green (lint/unit/mypy), still MERGEABLE.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants