Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
81 commits
Select commit Hold shift + click to select a range
3b24a7f
feat(thoughtspot): package skeleton, CI workflow, and ASF header gate
djwaldo Sep 1, 2026
bb613fd
fix(thoughtspot): add ASF header to pyproject.toml, pin CI actions to…
djwaldo Sep 1, 2026
3e3ab67
feat(thoughtspot): vocabulary constants with dialect registration flag
djwaldo Sep 1, 2026
b323f00
feat(thoughtspot): YAML 1.2 codec so on/off/yes/no survive a round trip
djwaldo Sep 1, 2026
279236d
test(thoughtspot): pin loader/dumper fixes independently with plain-P…
djwaldo Sep 1, 2026
2e4b359
feat(thoughtspot): structured never-silent issue reporting
djwaldo Sep 1, 2026
533088b
feat(thoughtspot): custom_extensions stash with staleness-guarded res…
djwaldo Sep 1, 2026
08df474
feat(thoughtspot): identifier normalisation with case-folded collisio…
djwaldo Sep 1, 2026
66039e6
fix(thoughtspot): reject ambiguous column refs; document normalise AS…
djwaldo Sep 1, 2026
a458dc2
feat(thoughtspot): key derivation that refuses to manufacture false keys
djwaldo Sep 1, 2026
c52b4e6
fix(thoughtspot): report empty-to_columns relationships, document KD3…
djwaldo Sep 1, 2026
8e80e9b
docs(thoughtspot): README with direction statement and coverage matrix
djwaldo Sep 1, 2026
53b0f66
docs(thoughtspot): add identifier-derivation limitations, tighten L2 …
djwaldo Sep 1, 2026
7fa1952
build(thoughtspot): match sibling converters' packaging and CI shape
djwaldo Sep 1, 2026
8de1d8f
fix(thoughtspot): correct KD2's upstream-warning claim; harden _yaml …
djwaldo Sep 1, 2026
dc1c7ca
fix(thoughtspot): fold diacritics in normalise(); close a column-ref …
djwaldo Sep 1, 2026
04af0e5
docs(thoughtspot): future-tense unimplemented claims; name the extern…
djwaldo Sep 1, 2026
498b198
docs(thoughtspot): fix README dev commands for the hatchling/uv packa…
djwaldo Sep 2, 2026
5b9e924
fix(thoughtspot): flip DIALECT_IS_REGISTERED now that ossie#351 merged
djwaldo Sep 2, 2026
20ba363
feat(thoughtspot): expression catalog vocabulary and spec-coverage gate
djwaldo Sep 2, 2026
8ced116
fix(thoughtspot): reconcile 137 vs 146 with an explicit divergence set
djwaldo Sep 2, 2026
2c4033e
fix(thoughtspot): correct Spelling audit, drop dead code, gate empty …
djwaldo Sep 2, 2026
e29a948
feat(thoughtspot): expression emitters for direct, passthrough and un…
djwaldo Sep 2, 2026
fd0c3e8
fix(thoughtspot): enforce the E8 wrapper instead of relying on caller…
djwaldo Sep 2, 2026
07c42d7
fix(thoughtspot): make the E8 PARTITION BY guard whitespace-tolerant
djwaldo Sep 2, 2026
52c7929
feat(thoughtspot): catalog entries for aggregate functions and type c…
djwaldo Sep 2, 2026
e9ca82e
feat(thoughtspot): catalog entries for date/time functions
djwaldo Sep 2, 2026
4a2d185
feat(thoughtspot): catalog entries for string functions
djwaldo Sep 2, 2026
41ee910
fix(thoughtspot): reject a passthrough template that wraps itself
djwaldo Sep 2, 2026
4a31cdb
feat(thoughtspot): catalog entries for mathematical and conditional f…
djwaldo Sep 2, 2026
6fb5bc1
feat(thoughtspot): catalog entries for operators and constructs
djwaldo Sep 2, 2026
2459923
fix(thoughtspot): escape IN/NOT IN braces; force external dispatch fo…
djwaldo Sep 2, 2026
550a8a1
feat(thoughtspot): catalog entries for window functions
djwaldo Sep 2, 2026
cdb341b
feat(thoughtspot): reverse-direction inventory (ThoughtSpot -> Ossie)
djwaldo Sep 2, 2026
35b025a
fix(ci): trigger thoughtspot CI when the upstream spec file changes
djwaldo Sep 2, 2026
892d605
fix(thoughtspot): custom_extensions_fragment must be keyed by column,…
djwaldo Sep 2, 2026
5d68166
fix(thoughtspot): correct the template contract and two catalog defec…
djwaldo Sep 2, 2026
b8a4723
docs(thoughtspot): remove citations to a private, deleted workspace f…
djwaldo Sep 2, 2026
b6b89fd
fix(thoughtspot): remove remaining internal-process leakage; add pass…
djwaldo Sep 2, 2026
30c1b2b
feat(thoughtspot): shallow formula tokenizer for reference rewriting …
djwaldo Sep 2, 2026
b851d91
fix(thoughtspot): stop quote state leaking into bracket bodies, deleg…
djwaldo Sep 2, 2026
2593d96
fix(thoughtspot): loosen column-ref table grammar to match its own am…
djwaldo Sep 2, 2026
af4ba1f
docs(thoughtspot): drop review-process wording from a test comment
djwaldo Sep 2, 2026
7f5a1a6
feat(thoughtspot): bidirectional datatype map with declared losses named
djwaldo Sep 2, 2026
edceb22
docs(thoughtspot): remove citations to an internal revision id
djwaldo Sep 2, 2026
0c26763
feat(thoughtspot): TML document set with the guid/brace/ordering/YAML…
djwaldo Sep 2, 2026
8d96f86
fix(thoughtspot): sanitise dump_document_set filenames and strip nest…
djwaldo Sep 2, 2026
21e53c7
fix(thoughtspot): reserve minted filenames by their final form, and c…
djwaldo Sep 2, 2026
8c46738
test(thoughtspot): add a self-policing guard against unresolvable ref…
djwaldo Sep 2, 2026
449bf99
feat(thoughtspot): convert TML columns to Ossie fields with verbatim …
djwaldo Sep 2, 2026
b7660f9
fix(thoughtspot): give convert_field a formulas map instead of an inl…
djwaldo Sep 3, 2026
8ff27be
fix(thoughtspot): log the genuine silent losses review round 2 found
djwaldo Sep 3, 2026
b4b991c
feat(thoughtspot): convert TML measures to Ossie metrics, composing c…
djwaldo Sep 3, 2026
8ca1a14
fix(thoughtspot): stash the source metric shape for round-trip fidelity
djwaldo Sep 3, 2026
a1f1524
fix(thoughtspot): stop silent double aggregation on group_aggregate a…
djwaldo Sep 3, 2026
517c8d7
fix(thoughtspot): only warn when an aggregate is nested, not when it …
djwaldo Sep 3, 2026
ee42a01
feat(thoughtspot): complete the TML -> Ossie direction with keys, sta…
djwaldo Sep 3, 2026
ab648d2
fix(thoughtspot): stash unconsumed column properties instead of dropp…
djwaldo Sep 3, 2026
e611cdd
fix(thoughtspot): validate relationship targets, centralize the ident…
djwaldo Sep 3, 2026
cb6e791
fix(thoughtspot): read SQL View columns from sql_view_columns[], not …
djwaldo Sep 3, 2026
6855083
feat(thoughtspot): build Table TML documents from Ossie datasets
djwaldo Sep 3, 2026
aefc647
fix(thoughtspot): a bare field's portable expression names the wareho…
djwaldo Sep 3, 2026
9624196
fix(thoughtspot): read the stashed warehouse column name back, and th…
djwaldo Sep 3, 2026
d117f13
refactor(thoughtspot): centralize every custom_extensions[THOUGHTSPOT…
djwaldo Sep 3, 2026
d1f42e6
feat(thoughtspot): build the Model TML document under the R3-R9 impor…
djwaldo Sep 3, 2026
1c01727
fix(thoughtspot): rewrite formula cross-references to match normalise…
djwaldo Sep 3, 2026
1035756
fix(thoughtspot): rename FULL_OUTER to OUTER in every emitted join type
djwaldo Sep 3, 2026
71c3d24
fix(thoughtspot): drop is_hidden/was_auto_generated from emitted columns
djwaldo Sep 3, 2026
fec2ba2
feat(thoughtspot): complete the Ossie -> TML direction with joins and…
djwaldo Sep 3, 2026
d220224
fix(thoughtspot): a formula cross-reference is not a runtime parameter
djwaldo Sep 3, 2026
eeaf5ff
fix(thoughtspot): audit and enforce X5's stash-currency rule across e…
djwaldo Sep 3, 2026
cec8dd5
fix(thoughtspot): unsurfaced-column membership is a second, orthogona…
djwaldo Sep 3, 2026
0dc6840
test(thoughtspot): TPC-DS and minimal fixture pairs
djwaldo Sep 3, 2026
f445fac
feat(thoughtspot): CLI entry point for both directions
djwaldo Sep 3, 2026
539d46f
test(thoughtspot): round-trip both directions, asserting preservation…
djwaldo Sep 3, 2026
6d16a76
fix(thoughtspot): restore a referencing join's own name and stop dupl…
djwaldo Sep 3, 2026
880ecbd
test(thoughtspot): property-based round-trip over adversarial identif…
djwaldo Sep 3, 2026
5e0e510
docs(thoughtspot): README, packaging and CI for the bidirectional con…
djwaldo Sep 3, 2026
49e5b3e
docs(thoughtspot): generate expression, reverse, datatype and payload…
djwaldo Sep 7, 2026
0f2f2b6
fix(thoughtspot): state the substance of internal rule citations, dro…
djwaldo Sep 7, 2026
e767092
fix(thoughtspot): close the last mapping-doc rule-id family (I)
djwaldo Sep 7, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/converter-thoughtspot-ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
#

name: Converters ThoughtSpot CI
on:
push:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
# catalog.py's spec_construct_names() parses this file directly (it is the
# upstream-spec coverage oracle, not vendored into converters/thoughtspot/),
# so a change here must trigger this workflow even though it touches no
# converter file. No sibling workflow parses the spec, hence this addition
# is not mirrored anywhere else.
- 'core-spec/expression_language.md'
pull_request:
branches: [ "main" ]
paths:
- 'converters/thoughtspot/**'
- '.github/workflows/converter-thoughtspot-ci.yml'
- 'core-spec/expression_language.md'

jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]

steps:
- name: Checkout project
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

- name: Install uv
run: |
curl -LsSf https://astral.sh/uv/install.sh | sh
echo "${HOME}/.local/bin" >> "${GITHUB_PATH}"

- name: Sync dependencies
working-directory: converters/thoughtspot
run: |
uv sync

- name: Unit Tests
working-directory: converters/thoughtspot
run: |
uv run pytest
1 change: 1 addition & 0 deletions converters/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ The Ossie specification currently defines extensions for the following vendors:
| `OMNI` | Omni semantic model |
| `WISDOM` | WisdomAI domain |
| `NVIDIA_GSF` | NVIDIA Generative Semantic Fabric standalone YAML |
| `THOUGHTSPOT` | ThoughtSpot TML (Model + Table/SQL View) |

Each vendor may define custom extensions (via the `custom_extensions` field in the Ossie spec) to carry vendor-specific metadata that does not have an equivalent in the core specification.

Expand Down
25 changes: 25 additions & 0 deletions converters/thoughtspot/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

__pycache__/
*.py[cod]
.pytest_cache/
*.egg-info/
dist/
build/
.venv/
venv/
311 changes: 311 additions & 0 deletions converters/thoughtspot/README.md

Large diffs are not rendered by default.

76 changes: 76 additions & 0 deletions converters/thoughtspot/docs/datatype-map.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

<!-- GENERATED FILE -- do not edit by hand.
Produced by `tools/generate_reference_docs.py` from:
- `src/ossie_thoughtspot/datatypes.py`
Regenerate with:
uv run --python 3.13 python tools/generate_reference_docs.py
tests/test_reference_docs_current.py fails the suite if this file
drifts from what the generator currently produces. -->

# Ossie <-> ThoughtSpot Datatype Map

The bidirectional Ossie <-> ThoughtSpot TML datatype map. The map is **not injective** — several Ossie types collapse onto one TML spelling and cannot be told apart on the way back; see "Not injective" below.

## The closed Ossie datatype enum

`Boolean`, `Date`, `DateTime`, `DateTimeTz`, `Decimal`, `Float`, `Integer`, `Opaque`, `String`, `Time`

## Ossie -> TML

| Ossie datatype | TML `data_type` (default) | Notes |
|---|---|---|
| `Boolean` | `BOOLEAN` | connection-dependent spelling — `BOOLEAN` by default, `BOOL` when the connection's own TML spells it that way |
| `Date` | `DATE` | exact, single spelling |
| `DateTime` | `DATE_TIME` | exact, single spelling |
| `DateTimeTz` | `DATE_TIME` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Decimal` | `DOUBLE` | exact, single spelling |
| `Float` | `DOUBLE` | connection-dependent spelling — `DOUBLE` by default, `FLOAT` when the connection's own TML spells it that way; ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Integer` | `INT64` | exact, single spelling |
| `Opaque` | `VARCHAR` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `String` | `VARCHAR` | exact, single spelling |
| `Time` | `VARCHAR` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

A column with no declared `datatype` at all infers `INT64` rather than raising — `datatype` is optional in Ossie, but TML rejects a column with no `db_column_properties` block at all.

## TML -> Ossie

| TML `data_type` | Ossie datatype |
|---|---|
| `BOOL` | `Boolean` |
| `BOOLEAN` | `Boolean` |
| `DATE` | `Date` |
| `DATE_TIME` | `DateTime` |
| `DOUBLE` | `Decimal` |
| `FLOAT` | `Float` |
| `INT64` | `Integer` |
| `VARCHAR` | `String` |

A TML `data_type` outside this map returns no Ossie datatype at all — `datatype` is optional in Ossie, so omitting it is preferred over inventing one.

## Not injective — declared losses

| Ossie datatype | Why the round trip is lossy |
|---|---|
| `DateTimeTz` | ThoughtSpot has no offset-aware column type; the value becomes DATE_TIME and returns as DateTime. |
| `Float` | ThoughtSpot has one approximate numeric type, so Float and Decimal both become DOUBLE and return as Decimal. |
| `Opaque` | Opaque is Ossie's marker for a type outside the portable vocabulary; it becomes VARCHAR and returns as String. |
| `Time` | ThoughtSpot has no time-of-day column type; the value becomes VARCHAR. |

Loading