Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/source/components/analyse.rst
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ Limitations

**Current Limitations:**

- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``) and JSONC (``//``, ``/* */``) comment styles are supported
- **Language Support**: C/C++ (``//``, ``/* */``), C# (``//``, ``/* */``, ``///``), Python (``#``), YAML (``#``), Rust (``//``, ``/* */``, ``///``), Go (``//``, ``/* */``), JSONC (``//``, ``/* */``) and Bash (``#``) comment styles are supported
- **Single Comment Style**: Each analysis run processes only one comment style at a time

Extraction Examples
Expand Down
6 changes: 5 additions & 1 deletion docs/source/components/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ Specifies the comment syntax style used in the source code files. This determine

**Type:** ``str``
**Default:** ``"cpp"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``
**Supported values:** ``"cpp"``, ``"python"``, ``"cs"``, ``"yaml"``, ``"rust"``, ``"go"``, ``"jsonc"``, ``"bash"``

.. code-block:: toml

Expand Down Expand Up @@ -326,6 +326,10 @@ Specifies the comment syntax style used in the source code files. This determine
``/* */`` (multi-line)
- ``.jsonc`` (always); ``.json`` only when the file opens with a comment
(e.g. the mode line ``// -*- mode: jsonc -*-``)
* - Bash / POSIX shell
- ``"bash"``
- ``#`` (single-line)
- ``.sh``, ``.bash``, ``.zsh``, ``.ksh``

.. note:: Future versions may support additional programming languages.

Expand Down
32 changes: 32 additions & 0 deletions docs/source/components/features.rst
Original file line number Diff line number Diff line change
Expand Up @@ -234,6 +234,38 @@ Features
.. fault:: Sphinx-codelinks hallucinates traceability objects in JSONC
:id: FAULT_JSONC_2

.. feature:: Bash Language Support
:id: FE_BASH

Support for defining traceability objects in Bash and POSIX-shell scripts.

The Bash language parser leverages tree-sitter to identify and extract
single-line (``#``) comments from shell scripts, associating each marker with
the surrounding function definition it annotates.

``.sh``, ``.bash``, ``.zsh``, and ``.ksh`` files are auto-discovered when
``comment_type = "bash"``. zsh and ksh share Bash's ``#`` comment syntax and
are parsed with the same grammar.

Key capabilities:

* Hash-style comment (``#``) detection
* Association of comments with function definitions
* Support for standard shell comment conventions
* Shebang lines (``#!/bin/bash``) never produce spurious markers

.. note::

Fish shell is not supported. Fish is not POSIX-compatible and no
``tree-sitter-fish`` grammar is published on PyPI, so it cannot be wired
into the Python package.

.. fault:: Traceability objects are not detected in Bash language
:id: FAULT_BASH_1

.. fault:: Sphinx-codelinks hallucinates traceability objects in Bash
:id: FAULT_BASH_2

.. feature:: Customized comment styles
:id: FE_CMT

Expand Down
13 changes: 13 additions & 0 deletions docs/source/development/change_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,19 @@
Changelog
=========

Unreleased
----------

New and Improved
................

- ✨ Added Bash language support for the ``analyse`` module.

Comments in shell scripts are now parsed for need ID references and one-line need
definitions. ``.sh``, ``.bash``, ``.zsh``, and ``.ksh`` files are discovered when
``comment_type = "bash"``. The supported comment style is ``#``. Fish shell is not
supported (no ``tree-sitter-fish`` grammar is published for the Python package).

.. _`release:1.3.0`:

1.3.0
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ dependencies = [
"tree-sitter-rust>=0.23.0",
"tree-sitter-go>=0.23.0",
"tree-sitter-json>=0.24.8",
"tree-sitter-bash>=0.25.1",
]

[project.optional-dependencies]
Expand Down
11 changes: 10 additions & 1 deletion src/sphinx_codelinks/analyse/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,8 @@
"type_declaration",
"type_spec",
},
# @Bash Scope Node Types, IMPL_BASH_2, impl, [FE_BASH]
CommentType.bash: {"function_definition"},
}

logger = get_logger(__name__)
Expand Down Expand Up @@ -74,6 +76,8 @@
(comment) @comment
"""
JSONC_QUERY = """(comment) @comment"""
# @Bash comment query for tree-sitter, IMPL_BASH_3, impl, [FE_BASH]
BASH_QUERY = """(comment) @comment"""

# JSON value node types that can be associated with a comment.
JSON_STRUCTURE_TYPES = {
Expand Down Expand Up @@ -103,7 +107,7 @@ def is_text_file(filepath: Path, sample_size: int = 2048) -> bool:
return False


# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC]
# @Tree-sitter parser initialization for multiple languages, IMPL_LANG_1, impl, [FE_C_SUPPORT, FE_CPP, FE_PY, FE_YAML, FE_RUST, FE_GO, FE_JSONC, FE_BASH]
def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:
if comment_type == CommentType.cpp:
import tree_sitter_cpp # noqa: PLC0415
Expand Down Expand Up @@ -140,6 +144,11 @@ def init_tree_sitter(comment_type: CommentType) -> tuple[Parser, Query]:

parsed_language = Language(tree_sitter_json.language())
query = Query(parsed_language, JSONC_QUERY)
elif comment_type == CommentType.bash:
import tree_sitter_bash # noqa: PLC0415

parsed_language = Language(tree_sitter_bash.language())
query = Query(parsed_language, BASH_QUERY)
else:
raise ValueError(f"Unsupported comment style: {comment_type}")
parser = Parser(parsed_language)
Expand Down
8 changes: 8 additions & 0 deletions src/sphinx_codelinks/source_discover/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,12 @@
"rust": ["rs"],
"go": ["go"],
"jsonc": ["jsonc", "json"],
# Bash uses `#` line comments; zsh and ksh share the same comment syntax and
# are scanned with the bash grammar. Fish is intentionally excluded: it is
# not POSIX-compatible and no tree-sitter-fish distribution is published on
# PyPI, so it cannot be wired in here. Track fish separately if a PyPI
# grammar becomes available.
"bash": ["sh", "bash", "zsh", "ksh"],
}


Expand All @@ -27,6 +33,8 @@ class CommentType(str, Enum):
go = "go"
# @Support JSONC style comments, IMPL_JSONC_1, impl, [FE_JSONC];
jsonc = "jsonc"
# @Support Bash style comments, IMPL_BASH_1, impl, [FE_BASH];
bash = "bash"


class SourceDiscoverSectionConfigType(TypedDict, total=False):
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"needs": [
{
"id": "IMPL_BASH",
"title": "Bash Title",
"type": "impl",
"links": {
"links": [
"REQ_BASH"
]
},
"metadata": {},
"line": 1
}
],
"need_refs": [],
"marked_rst": [],
"warnings": []
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"needs": [
{
"id": "IMPL_BASH_SHEBANG",
"title": "Bash Title",
"type": "impl",
"links": {
"links": [
"REQ_BASH"
]
},
"metadata": {},
"line": 2
}
],
"need_refs": [],
"marked_rst": [],
"warnings": []
}
2 changes: 1 addition & 1 deletion tests/data/extraction/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Each `*.yaml` file in this directory is a map of `case_name → case`:

```yaml
default_oneliner_cpp:
lang: cpp # cpp | c | python | csharp | rust | yaml | go | jsonc
lang: cpp # cpp | c | python | csharp | rust | yaml | go | jsonc | bash
config: default # "default", or an inline config block (see below)
source: |
// @My Title, IMPL_1, impl, [REQ_1]
Expand Down
17 changes: 17 additions & 0 deletions tests/data/extraction/oneline.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,20 @@ default_oneliner_yaml:
source: |
# @Yaml Title, IMPL_YAML, impl, [REQ_YAML]
key: value

default_oneliner_bash:
lang: bash
config: default
source: |
# @Bash Title, IMPL_BASH, impl, [REQ_BASH]
greet() { echo hi; }

# the shebang is a comment node but must not yield a marker or warning;
# the marker anchors to line 2 and the `function` keyword form also parses
shebang_oneliner_bash:
lang: bash
config: default
source: |
#!/bin/bash
# @Bash Title, IMPL_BASH_SHEBANG, impl, [REQ_BASH]
function greet { echo hi; }
56 changes: 56 additions & 0 deletions tests/test_analyse_utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import pytest
from tree_sitter import Language, Parser, Query
from tree_sitter import Node as TreeSitterNode
import tree_sitter_bash
import tree_sitter_c_sharp
import tree_sitter_cpp
import tree_sitter_go
Expand Down Expand Up @@ -75,6 +76,14 @@ def init_jsonc_tree_sitter() -> tuple[Parser, Query]:
return parser, query


@pytest.fixture(scope="session")
def init_bash_tree_sitter() -> tuple[Parser, Query]:
parsed_language = Language(tree_sitter_bash.language())
query = Query(parsed_language, utils.BASH_QUERY)
parser = Parser(parsed_language)
return parser, query


@pytest.mark.parametrize(
("code", "result"),
[
Expand Down Expand Up @@ -425,6 +434,53 @@ def test_find_associated_scope_jsonc(code, result, init_jsonc_tree_sitter):
assert result in jsonc_structure


@pytest.mark.parametrize(
("code", "result"),
[
# comment above a POSIX-style function definition
(
b"""
# @req-id: need_001
greet() {
echo hi
}
""",
"greet()",
),
# comment above the `function` keyword form
(
b"""
# @req-id: need_002
function greet {
echo hi
}
""",
"function greet",
),
# comment inside a function body falls back to the enclosing function
(
b"""
greet() {
# @req-id: need_003
echo hi
}
""",
"greet()",
),
],
)
def test_find_associated_scope_bash(code, result, init_bash_tree_sitter):
parser, query = init_bash_tree_sitter
comments = utils.extract_comments(code, parser, query)
node: TreeSitterNode | None = utils.find_associated_scope(
comments[0], CommentType.bash
)
assert node
assert node.text
func_def = node.text.decode("utf-8")
assert func_def.startswith(result)


@pytest.mark.parametrize(
("code", "result"),
[
Expand Down
1 change: 1 addition & 0 deletions tests/test_extraction_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@
"yaml": (CommentType.yaml, "yaml"),
"go": (CommentType.go, "go"),
"jsonc": (CommentType.jsonc, "jsonc"),
"bash": (CommentType.bash, "sh"),
}


Expand Down
3 changes: 2 additions & 1 deletion tests/test_source_discover.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@
"comment_type": "java",
},
[
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']"
"Schema validation error in field 'comment_type': 'java' is not one of ['bash', 'cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']"
],
),
(
Expand Down Expand Up @@ -182,6 +182,7 @@ def create_source_files(tmp_path: Path) -> Path:
[
("cpp", len(COMMENT_FILETYPE["cpp"])),
("python", len(COMMENT_FILETYPE["python"])),
("bash", len(COMMENT_FILETYPE["bash"])),
],
)
def test_comment_filetype(
Expand Down
2 changes: 1 addition & 1 deletion tests/test_src_trace.py
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@
[
"Project 'dcdc' has the following errors:",
"Schema validation error in field 'exclude': 123 is not of type 'string'",
"Schema validation error in field 'comment_type': 'java' is not one of ['cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']",
"Schema validation error in field 'comment_type': 'java' is not one of ['bash', 'cpp', 'cs', 'go', 'jsonc', 'python', 'rust', 'yaml']",
"Schema validation error in field 'gitignore': '_true' is not of type 'boolean'",
"Schema validation error in field 'include': 345 is not of type 'string'",
"Schema validation error in field 'src_dir': ['../dcdc'] is not of type 'string'",
Expand Down
Loading