From 86fc89b6a85b7309bcd1fb0431030e9614885505 Mon Sep 17 00:00:00 2001 From: Antti Haapala Date: Tue, 16 Jun 2026 02:04:43 +0300 Subject: [PATCH 1/2] docs: comprehensive narrative documentation for the framework Add narrative guides for the previously-undocumented subsystems: services (dependency injection), request/response, views & controllers, viewlets, Tonnikala templating, i18n, static assets, and decorators. Each page is grounded in the actual source APIs. Audit and correct the existing narrative pages (configuration, json, sqlalchemy, security, utilities, testing) against the real code -- utilities and testing were largely rewritten as the originals documented many nonexistent APIs. Refresh the introduction (0.5.0 / Python 3.8+) and reorganize the narrative index into logical sections. Normalize RST heading underlines and fix an i18n table / XSS-demo code-block so the Sphinx build is warning-free for the narrative and tutorial docs. --- docs/narr/configuration.rst | 112 +++-- docs/narr/decorators.rst | 190 +++++++ docs/narr/i18n.rst | 309 ++++++++++++ docs/narr/index.rst | 41 +- docs/narr/introduction.rst | 166 ++++--- docs/narr/json.rst | 22 +- docs/narr/request_response.rst | 227 +++++++++ docs/narr/security.rst | 26 +- docs/narr/services.rst | 369 ++++++++++++++ docs/narr/sqlalchemy.rst | 457 +++++++++-------- docs/narr/static.rst | 198 ++++++++ docs/narr/templating.rst | 226 +++++++++ docs/narr/testing.rst | 715 ++++++++++++--------------- docs/narr/utilities.rst | 517 +++++++------------ docs/narr/viewlets.rst | 297 +++++++++++ docs/narr/views.rst | 312 ++++++++++++ docs/tutorials/database_tutorial.rst | 46 +- docs/tutorials/index.rst | 4 +- docs/tutorials/json_tutorial.rst | 42 +- docs/tutorials/quickstart.rst | 2 +- docs/tutorials/security_tutorial.rst | 30 +- 21 files changed, 3188 insertions(+), 1120 deletions(-) create mode 100644 docs/narr/decorators.rst create mode 100644 docs/narr/i18n.rst create mode 100644 docs/narr/request_response.rst create mode 100644 docs/narr/services.rst create mode 100644 docs/narr/static.rst create mode 100644 docs/narr/templating.rst create mode 100644 docs/narr/viewlets.rst create mode 100644 docs/narr/views.rst diff --git a/docs/narr/configuration.rst b/docs/narr/configuration.rst index 2a0254c..81c243b 100644 --- a/docs/narr/configuration.rst +++ b/docs/narr/configuration.rst @@ -5,14 +5,18 @@ Configuration Tet provides enhanced configuration capabilities that extend Pyramid's configuration system with additional directives and conveniences. Basic Configuration -================== +=================== Tet modules are configured through Pyramid's ``Configurator`` using the ``include`` directive. Application Factory Pattern --------------------------- -Tet uses an application factory decorator that automatically configures features: +Tet provides the :func:`tet.config.application_factory` decorator, which wraps +a configuration function so it becomes a standard Pyramid/Paster application +entry point accepting ``(global_config, **settings)``. The wrapped function +receives a single argument -- the :class:`~pyramid.config.Configurator` -- and +by default the wrapper returns a WSGI application built from it: .. code-block:: python @@ -25,35 +29,62 @@ Tet uses an application factory decorator that automatically configures features config.add_route('home', '/') config.scan() - # Or with minimal features - @application_factory(included_features=MINIMAL_FEATURES) + # The decorator can also be applied without arguments. In that case the + # default is MINIMAL_FEATURES (no features are auto-included), so you add + # what you need manually. + @application_factory def minimal_main(config): """Minimal Tet application.""" config.include('tet.renderers.json') # Add features manually config.add_route('api', '/api') +The decorator accepts the following keyword arguments: + +* ``included_features`` -- iterable of feature names to include automatically + (defaults to :data:`~tet.config.MINIMAL_FEATURES`, i.e. an empty list). + Nested iterables are flattened. +* ``excluded_features`` -- iterable of feature names to skip even if present in + ``included_features``. +* ``configure_only`` -- if ``True``, the wrapper returns whatever the wrapped + function returns (typically the configurator) instead of building a WSGI + application. Defaults to ``False``. +* ``package`` -- the package passed to the ``Configurator``; defaults to the + caller's package. + +Any extra keyword arguments are passed through to +:func:`~tet.config.create_configurator`. + +If the wrapped function returns a ``Configurator``, that returned configurator +is used to build the WSGI application; otherwise the configurator created by +the decorator is used. + Available Features ------------------ -Tet provides predefined feature sets: +Tet provides two predefined feature sets in :mod:`tet.config`: -* ``ALL_FEATURES``: All Tet features enabled -* ``MINIMAL_FEATURES``: No features (empty list) +* :data:`~tet.config.ALL_FEATURES` -- all standard Tet features (see the list + below) +* :data:`~tet.config.MINIMAL_FEATURES` -- no features (an empty list) -Individual features can be included: +Individual features are named with the part of the dotted module path that +follows ``tet.``; ``create_configurator`` includes each as ``tet.``. +The available feature names are: -* ``"services"`` - Service configuration +* ``"services"`` - Dependency injection via pyramid_di * ``"i18n"`` - Internationalization support -* ``"renderers.json"`` - Enhanced JSON renderer -* ``"renderers.tonnikala"`` - Tonnikala template renderer -* ``"renderers.tonnikala.i18n"`` - Tonnikala with i18n -* ``"security.authorization"`` - Enhanced authorization -* ``"security.csrf"`` - CSRF protection +* ``"renderers.json"`` - JSON rendering with custom type adapters +* ``"renderers.tonnikala"`` - Tonnikala template engine integration +* ``"renderers.tonnikala.i18n"`` - Tonnikala with i18n support +* ``"security.authorization"`` - Custom authorization policy +* ``"security.csrf"`` - CSRF token protection Manual Configuration -------------------- -For fine-grained control, create the configurator manually: +For fine-grained control, create the configurator manually with +:func:`tet.config.create_configurator`. All of its parameters are +keyword-only: .. code-block:: python @@ -64,7 +95,7 @@ For fine-grained control, create the configurator manually: global_config=global_config, settings=settings, included_features=['renderers.json', 'security.csrf'], - excluded_features=['i18n'] + excluded_features=['i18n'], ) # Your configuration @@ -73,6 +104,25 @@ For fine-grained control, create the configurator manually: return config.make_wsgi_app() +The most commonly used keyword arguments are: + +* ``global_config`` -- the global configuration mapping (from PasteDeploy). +* ``settings`` -- the application settings mapping. +* ``merge_global_config`` -- when ``True`` (the default) and ``global_config`` + is a mapping, it is merged into ``settings`` via a ``ChainMap``. +* ``included_features`` / ``excluded_features`` -- iterables of feature names; + both are flattened, and the effective set is + ``included_features - excluded_features``. The resulting set is stored on + ``config.registry.tet_features``. +* ``configurator_class`` -- the ``Configurator`` subclass to instantiate + (defaults to :class:`pyramid.config.Configurator`). +* ``package`` -- the package for the configurator; defaults to the caller's + package. The package name is also used as the default i18n domain. + +Any remaining keyword arguments are forwarded to the ``Configurator`` +constructor (the ``default_i18n_domain`` setting, if given, is extracted and +applied via ``add_settings``). + Configuration Directives ======================== @@ -114,7 +164,7 @@ JSON Renderer Directives ) Authorization Directive ----------------------- +----------------------- **set_authorization_policy** Enhanced authorization policy registration that supports Tet's ``INewAuthorizationPolicy``: @@ -130,12 +180,12 @@ Authorization Directive config.set_authorization_policy(policy) Module Configuration -=================== +==================== Individual Tet modules can be configured with specific options. CSRF Configuration ------------------ +------------------ The CSRF module sets secure defaults but can be customized: @@ -185,12 +235,12 @@ Customize the JSON renderer behavior: return config.make_wsgi_app() Settings Integration -=================== +==================== Tet modules respect Pyramid's settings system for configuration. Database Settings ----------------- +----------------- Configure SQLAlchemy integration through settings: @@ -211,7 +261,7 @@ Configure SQLAlchemy integration through settings: session.cookie_secure = true Security Settings ----------------- +----------------- Configure security-related settings: @@ -228,7 +278,7 @@ Configure security-related settings: auth.secret = auth-signing-secret Application Settings -------------------- +-------------------- Access settings in your application code: @@ -244,7 +294,7 @@ Access settings in your application code: return {'debug': debug_mode} Environment Configuration -======================== +========================= Tet applications can be configured for different environments. @@ -273,7 +323,7 @@ Development Configuration return config.make_wsgi_app() Production Configuration ------------------------ +------------------------ .. code-block:: python @@ -298,7 +348,7 @@ Production Configuration return config.make_wsgi_app() Testing Configuration --------------------- +--------------------- .. code-block:: python @@ -321,12 +371,12 @@ Testing Configuration return config.make_wsgi_app() Advanced Configuration -===================== +====================== Complex configuration scenarios and patterns. Factory Configuration --------------------- +--------------------- Configure root factories and other components: @@ -353,7 +403,7 @@ Configure root factories and other components: return config.make_wsgi_app() Service Configuration --------------------- +--------------------- Configure services with pyramid_di: @@ -442,7 +492,7 @@ Settings Helper return SettingsHelper(settings) Configuration Profiles -===================== +====================== Managing different configuration profiles. @@ -482,7 +532,7 @@ Profile System return config.make_wsgi_app() Best Practices -============= +============== **Validate Early** Validate configuration at application startup to catch errors early. diff --git a/docs/narr/decorators.rst b/docs/narr/decorators.rst new file mode 100644 index 0000000..d4462ad --- /dev/null +++ b/docs/narr/decorators.rst @@ -0,0 +1,190 @@ +========================== +Decorators and Descriptors +========================== + +The :mod:`tet.decorators` module collects a small set of general-purpose +helpers that are useful throughout a Tet application: a decorator for marking +APIs as deprecated and a reify-style cached-property descriptor that is aware +of the attribute name it is bound to. + +Both helpers are importable directly from the package: + +.. code-block:: python + + from tet.decorators import deprecated, reify_attr + +The module is deliberately tiny and dependency-free, so it is safe to use in +library code, models, services, and views alike. + + +``deprecated`` +-------------- + +:func:`deprecated` is a function decorator that marks a callable as +deprecated. The wrapped function continues to work exactly as before, but +every call now emits a :class:`DeprecationWarning` before delegating to the +original implementation. + +Use it when you want to retire a function or method but cannot remove it yet +because callers still depend on it. The warning gives downstream code a clear +signal (and, in test suites that turn warnings into errors, a hard failure) +without breaking runtime behaviour. + +Signature +~~~~~~~~~ + +.. code-block:: python + + deprecated(func) + +It takes a single callable and returns a wrapper with the same name, +docstring, and ``__dict__`` as the original. The warning message is built from +the function's ``__qualname__``, so it correctly identifies methods nested in +classes (e.g. ``MyService.old_method``). + +Behaviour +~~~~~~~~~ + +When the wrapped function is called, it issues: + +.. code-block:: text + + Call to deprecated function . + +The warning is raised with ``stacklevel=2``, which means the warning is +attributed to the *caller* of the deprecated function rather than to the +wrapper inside :mod:`tet.decorators`. That makes the warning point at the line +of code that actually needs to change. + +.. note:: + + Python silences :class:`DeprecationWarning` by default outside of + ``__main__`` and test runners. To see the warnings during development, run + Python with ``-W default::DeprecationWarning`` or configure the + :mod:`warnings` filter explicitly. Most test runners (including pytest) + surface these warnings out of the box. + +Example +~~~~~~~ + +.. code-block:: python + + from tet.decorators import deprecated + + + @deprecated + def render_legacy_template(name): + """Old rendering path; use render_template() instead.""" + return _legacy_render(name) + + + # Calling it still works, but emits: + # DeprecationWarning: Call to deprecated function render_legacy_template. + html = render_legacy_template("home") + +It works equally well on methods, where ``__qualname__`` produces a fully +qualified name in the warning: + +.. code-block:: python + + class ReportService: + @deprecated + def export_csv(self, report): + # Warning text: "Call to deprecated function ReportService.export_csv." + return self._export(report, fmt="csv") + + +``reify_attr`` +-------------- + +:class:`reify_attr` is a cached-property descriptor. The first time the +attribute is accessed on an instance, the wrapped function is called and its +return value is computed; that value is then written back onto the instance so +that subsequent accesses read a plain attribute and never call the function +again. + +It is similar in spirit to Pyramid's ``pyramid.decorator.reify``, but with one +key difference: ``reify_attr`` caches under the *name the descriptor is bound +to on the class*, not the name of the decorated method. This matters when the +descriptor is assigned to an attribute whose name differs from the wrapped +function, or assigned dynamically. It is intended as a building block for +descriptors such as ``autowired`` in ``pyramid_di``, which need to know their +own attribute name in order to cache the resolved value on the instance. + +Signature +~~~~~~~~~ + +.. code-block:: python + + class reify_attr: + def __init__(self, wrapped): ... + +It is used as a decorator on a method that takes ``self`` and returns the +value to cache. The descriptor copies the wrapped function's metadata via +:func:`functools.update_wrapper`. + +How caching and name resolution work +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +- **Name discovery via ``__set_name__``:** when the descriptor is created as + part of a class body, Python calls ``__set_name__`` and ``reify_attr`` + records the attribute name (or names, if the same descriptor object is bound + to several attributes). +- **Fallback discovery:** if ``__set_name__`` was never called (for example, + the descriptor was attached to the class dynamically after definition), the + name is discovered lazily on first access by scanning the owner class's + ``__dict__`` for the attributes that point at this descriptor. +- **Write-back on access:** on first ``__get__`` for an instance, the wrapped + function is invoked and the result is stored on the instance under every + resolved name via :func:`setattr`. Because instance attributes shadow class + descriptors for non-data descriptors, later accesses return the cached value + directly without invoking the function again. +- **Class access:** accessing the attribute on the class itself (``inst`` is + ``None``) returns the descriptor object rather than computing a value. + +.. note:: + + ``reify_attr`` is a *non-data* descriptor (it defines ``__get__`` but not + ``__set__``), which is exactly what allows the instance attribute written on + first access to take precedence on subsequent reads. If you need to force + recomputation, delete the cached instance attribute (``del inst.name``). + +Example +~~~~~~~ + +.. code-block:: python + + from tet.decorators import reify_attr + + + class Report: + def __init__(self, rows): + self.rows = rows + + @reify_attr + def summary(self): + # Computed once, then cached on the instance as `summary`. + print("computing summary...") + return { + "count": len(self.rows), + "total": sum(r.amount for r in self.rows), + } + + + report = Report(load_rows()) + report.summary # prints "computing summary..." and computes the dict + report.summary # returns the cached dict; no print, no recompute + +Because caching uses the bound attribute name, you can rely on the cached value +living under the attribute you actually access, which is what makes it suitable +for descriptor-composition patterns like dependency-injection ``autowired`` +fields. + + +See also +-------- + +- :doc:`utilities` -- the broader set of helper utilities in ``tet.util`` + (cryptography, base64, collections, paths, and JSON helpers). +- :doc:`configuration` -- configuring a Tet application and including Tet + components via ``config.include(...)``. diff --git a/docs/narr/i18n.rst b/docs/narr/i18n.rst new file mode 100644 index 0000000..9514217 --- /dev/null +++ b/docs/narr/i18n.rst @@ -0,0 +1,309 @@ +=========================== +Internationalization (i18n) +=========================== + +Tet builds on Pyramid's translation machinery and wires it up so that +translation is convenient from both view code and templates. Once i18n is +enabled, every request gains ``request.translate`` and ``request.pluralize`` +helpers that know the application's default translation domain, and every +template -- including viewlet fragments -- automatically receives the familiar +``_``, ``gettext``, ``ngettext`` and ``localizer`` globals. + +This page explains how to enable i18n, how the request helpers behave, how the +template globals are injected, and how to lay out your translation catalogs. + +Enabling i18n +------------- + +The i18n support lives in :mod:`tet.i18n`. There are two ways to turn it on. + +The recommended way is to list ``"i18n"`` among the features of your +application factory. The application factory derives a sensible default +translation domain (your package name) automatically. + +.. code-block:: python + + from tet.config import application_factory + + @application_factory(included_features=["i18n"]) + def main(config): + config.add_translation_dirs("myapp:locale") + config.scan() + +You can also include the module directly the way you would include any Pyramid +add-on: + +.. code-block:: python + + config.include("tet.i18n") + +When included this way, the default translation domain is taken from the +``default_i18n_domain`` setting if present, otherwise it falls back to the +configured package name: + +.. code-block:: ini + + [app:main] + use = egg:myapp + default_i18n_domain = myapp + +Choosing the default domain explicitly +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you want full control, call :func:`tet.i18n.configure_i18n` yourself and +pass the domain you want to use: + +.. code-block:: python + + from tet.i18n import configure_i18n + + def main(global_config, **settings): + config = Configurator(settings=settings) + configure_i18n(config, default_domain="myapp") + config.add_translation_dirs("myapp:locale") + # ... + +``configure_i18n`` is what actually does the work. It: + +- subscribes ``add_renderer_globals`` to both ``pyramid.events.BeforeRender`` + and ``tet.viewlet.IBeforeViewletRender``; +- creates a :class:`pyramid.i18n.TranslationStringFactory` for the default + domain and stores it on the registry as ``config.registry.tsf``; +- adds the ``request.translate`` and ``request.pluralize`` request methods + (both reified properties); +- adds a ``request.localize`` reified property bound to + :func:`pyramid.i18n.get_localizer`. + +Translation string factories +----------------------------- + +A *translation string factory* turns a plain string into a Pyramid +:class:`~pyramid.i18n.TranslationString` bound to a particular domain. Tet +creates one for your default domain and keeps it on the registry: + +.. code-block:: python + + tsf = config.registry.tsf + msg = tsf("Hello, World!") + +You rarely need to touch ``registry.tsf`` directly, because +``request.translate`` and ``request.pluralize`` apply it for you (see below). +You can still create additional factories for other domains in the usual +Pyramid way when you need them: + +.. code-block:: python + + from pyramid.i18n import TranslationStringFactory + + _ = TranslationStringFactory("myapp") + label = _("Save") + +Translating in view code +------------------------- + +``request.translate`` is a reified property that returns a callable. Calling +it with a string wraps that string in the registry's translation string +factory (so it picks up the default domain) and then translates it with the +request's localizer. + +.. code-block:: python + + from pyramid.view import view_config + + @view_config(route_name="hello", renderer="json") + def hello(request): + return {"message": request.translate("Hello, World!")} + +The callable accepts keyword-only arguments for fine control: + +.. code-block:: python + + # Override the domain for a single call + request.translate("Save", domain="otherapp") + + # Disambiguate with a message context + request.translate("Open", context="verb") + + # Interpolate placeholders with a mapping + request.translate( + "Welcome, %(name)s", + mapping={"name": user.name}, + ) + +The full signature of the returned callable is:: + + auto_translate(string, *, domain=, mapping=None, context=None) + +If you pass a value that is already a ``TranslationString`` (rather than a +plain ``str``), it is translated as-is and the ``context`` argument is ignored +-- the factory is only applied to bare strings. + +Pluralization +~~~~~~~~~~~~~~ + +``request.pluralize`` works the same way for plural forms. It returns a +callable that selects the singular or plural message based on ``n`` and +translates the result: + +.. code-block:: python + + @view_config(route_name="cart", renderer="json") + def cart(request): + count = len(request.cart) + return { + "summary": request.pluralize( + "%(num)d item in cart", + "%(num)d items in cart", + count, + mapping={"num": count}, + ), + } + +The full signature is:: + + auto_pluralize(singular, plural, n, *, domain=, + mapping=None, context=None) + +As with ``translate``, the ``context`` argument is applied only when the +``singular`` value is a plain string. + +Template globals +---------------- + +When i18n is configured, Tet injects four globals into every template render +context via the ``add_renderer_globals`` subscriber: + +=============== ===================== +Global Bound to +=============== ===================== +``_`` ``request.translate`` +``gettext`` ``request.translate`` +``ngettext`` ``request.pluralize`` +``localizer`` ``request.localizer`` +=============== ===================== + +``_`` and ``gettext`` are aliases for the same translate callable, so you can +use whichever reads better in a given template. + +Tonnikala templates use ``$`` for interpolation, so calling these globals is +natural: + +.. code-block:: html + +

$_("Welcome to our site!")

+ + + +When you need to pass a mapping or a context, use braces to delimit the full +expression: + +.. code-block:: html + +

${_('Welcome, %(name)s', mapping={'name': user.name})}

+ + ${_('Open', context='verb')} + +Pluralization in templates uses ``ngettext`` exactly like the request helper: + +.. code-block:: html + +

${ngettext('%(num)d message', + '%(num)d messages', + count, + mapping={'num': count})}

+ +The ``localizer`` global is the request's Pyramid +:class:`~pyramid.i18n.Localizer`, which is handy for locale-aware formatting +or for inspecting the negotiated locale: + +.. code-block:: html + + + +Globals in viewlets +~~~~~~~~~~~~~~~~~~~~ + +Viewlet fragments are rendered through a separate event, +``tet.viewlet.IBeforeViewletRender``, rather than the normal Pyramid +``BeforeRender``. Because ``configure_i18n`` subscribes ``add_renderer_globals`` +to *both* events, the same ``_``, ``gettext``, ``ngettext`` and ``localizer`` +globals are available inside viewlet templates with no extra work: + +.. code-block:: python + + from tet.viewlet import viewlet + + @viewlet("myapp:templates/sidebar.tk") + def sidebar(request): + return {"user": request.user} + +.. code-block:: html + + + + +The subscriber resolves the current request from the event payload, falling +back to :func:`pyramid.threadlocal.get_current_request` when the event does +not carry one, so the globals are populated reliably in both rendering paths. + +Setting up translation directories and locales +---------------------------------------------- + +Translations are loaded from translation directories registered with +:meth:`~pyramid.config.Configurator.add_translation_dirs`. A typical layout +places GNU ``gettext`` catalogs under a ``locale`` directory in your package: + +.. code-block:: text + + myapp/ + locale/ + myapp.pot + fi/ + LC_MESSAGES/ + myapp.mo + myapp.po + sv/ + LC_MESSAGES/ + myapp.mo + myapp.po + +Register the directory during configuration. The package name used here should +match your default translation domain so that ``.mo`` files are found: + +.. code-block:: python + + config.add_translation_dirs("myapp:locale") + +Pyramid negotiates the active locale per request (for example via a locale +negotiator or the ``_LOCALE_`` request parameter); the localizer used by +``request.translate`` and ``request.pluralize`` reflects that negotiated +locale automatically. + +Extracting and compiling catalogs follows the standard ``gettext`` workflow. +Extract messages into a ``.pot`` template, create per-locale ``.po`` files, and +compile them to ``.mo``. For example, with ``Babel``: + +.. code-block:: console + + $ pybabel extract -o myapp/locale/myapp.pot myapp + $ pybabel init -D myapp -i myapp/locale/myapp.pot \ + -d myapp/locale -l fi + $ pybabel compile -D myapp -d myapp/locale + +Because ``request.translate`` wraps bare strings with the default-domain +factory, the message ids you write in code and templates (``"Hello, World!"``, +``"Save"``, ...) are exactly the strings the extractor should pick up. Make +sure your extraction configuration scans both your Python modules and your +Tonnikala templates. + +See also +-------- + +- :doc:`configuration` -- the application factory, features, and settings + such as ``default_i18n_domain``. +- :doc:`templating` -- the Tonnikala renderer and ``$`` interpolation syntax. +- :doc:`viewlets` -- reusable template fragments and the + ``IBeforeViewletRender`` event. diff --git a/docs/narr/index.rst b/docs/narr/index.rst index 1d753ed..d8f5283 100644 --- a/docs/narr/index.rst +++ b/docs/narr/index.rst @@ -1,16 +1,47 @@ -================== +======================= Narrative Documentation -================== +======================= -This section contains comprehensive guides and explanations of Tet's features and concepts. +This section contains comprehensive guides and explanations of Tet's features +and concepts. Start with the :doc:`introduction`, then read the topics most +relevant to your application — each guide is self-contained. .. toctree:: :maxdepth: 2 + :caption: Getting started introduction - security + configuration + +.. toctree:: + :maxdepth: 2 + :caption: Application building blocks + + services + request_response + views + viewlets + templating + i18n + static + +.. toctree:: + :maxdepth: 2 + :caption: Data and rendering + json sqlalchemy + +.. toctree:: + :maxdepth: 2 + :caption: Security + + security + +.. toctree:: + :maxdepth: 2 + :caption: Utilities and workflow + utilities - configuration + decorators testing diff --git a/docs/narr/introduction.rst b/docs/narr/introduction.rst index 1efbd41..0eabe8b 100644 --- a/docs/narr/introduction.rst +++ b/docs/narr/introduction.rst @@ -5,7 +5,14 @@ Introduction What is Tet? ============ -Tet is an "unearthly intelligent batteries-included application framework built on Pyramid." It extends the robust Pyramid web framework with additional utilities, security features, and developer conveniences that make building web applications more productive and secure. +Tet is an "unearthly intelligent batteries-included application framework built +on Pyramid." It extends the robust Pyramid web framework with additional +utilities, security features, and developer conveniences that make building web +applications more productive and secure. + +Tet does not replace Pyramid — it *extends* it. Every Tet feature is an +ordinary Pyramid include or directive, so you can adopt as much or as little of +Tet as you like and remain fully compatible with the Pyramid ecosystem. Core Philosophy =============== @@ -13,114 +20,117 @@ Core Philosophy Tet follows these core principles: **Batteries Included** - Tet provides commonly needed functionality out of the box, reducing the need to find and integrate multiple third-party packages. + Tet provides commonly needed functionality out of the box, reducing the need + to find and integrate multiple third-party packages. **Security First** - Security features like CSRF protection and safe JSON serialization are enabled by default and designed to prevent common vulnerabilities. + Security features like CSRF protection and safe JSON serialization are enabled + by default and designed to prevent common vulnerabilities. **Pyramid Compatible** - Tet extends rather than replaces Pyramid, maintaining full compatibility with existing Pyramid applications and ecosystem. + Tet extends rather than replaces Pyramid, maintaining full compatibility with + existing Pyramid applications and ecosystem. **Developer Friendly** - Enhanced development experience with better error handling, type hints, and comprehensive documentation. + Enhanced development experience with better error handling, type hints, and + comprehensive documentation. Key Features Overview -==================== - -Enhanced Security ------------------ - -Tet provides several security enhancements: - -* **CSRF Protection**: Automatically enabled CSRF protection for forms -* **Authorization Policies**: Enhanced authorization with request-aware policies -* **Safe JSON Serialization**: Prevents XSS attacks when embedding JSON in HTML -* **SQL Injection Prevention**: Proper exception handling in SQLAlchemy factories - -JSON Handling ------------- - -Tet includes advanced JSON handling capabilities: - -* **XSS Prevention**: Automatic escaping of dangerous characters for inline JavaScript -* **Custom Type Adapters**: Built-in support for SQLAlchemy and datetime objects -* **Safe Serialization**: Unicode and HTML-safe JSON output - -SQLAlchemy Integration ---------------------- - -Enhanced database support: - -* **Root Factories**: Custom traversal root factories with proper exception handling -* **Session Management**: Enhanced session handling patterns -* **Type Safety**: Proper conversion of SQL exceptions to appropriate HTTP errors - -Utility Modules --------------- - -Comprehensive utility modules: - -* **Cryptography**: Password hashing and security utilities -* **Collections**: Enhanced collection types and utilities -* **Path Handling**: File and path manipulation utilities -* **Export Functions**: Data export and serialization helpers +===================== + +Application Assembly +-------------------- + +* **App factory**: the ``application_factory`` decorator and + ``create_configurator`` helper wire up a configured application with sensible + defaults. See :doc:`configuration`. +* **Dependency injection**: request-scoped services via ``pyramid_di``, exposed + through ``tet.services``. See :doc:`services`. +* **Views and viewlets**: enhanced ``view_config``, class-based controllers, and + a composable viewlet system for reusable template fragments. See :doc:`views` + and :doc:`viewlets`. + +Rendering and Templating +------------------------ + +* **Tonnikala templates**: a fast templating renderer with ``$`` interpolation. + See :doc:`templating`. +* **Safe JSON**: XSS-safe JSON serialization with custom type adapters for + SQLAlchemy and datetime objects. See :doc:`json`. +* **Internationalization**: translation and pluralization helpers wired into + requests and templates. See :doc:`i18n`. +* **Static assets**: cache-breaking static views so browsers always pick up new + asset versions. See :doc:`static`. + +Security +-------- + +* **CSRF Protection**: automatically enabled CSRF protection. +* **Authorization Policies**: request-aware authorization policies. +* **Safe serialization**: prevents XSS when embedding JSON in HTML. + +See :doc:`security`. + +Data and Utilities +------------------ + +* **SQLAlchemy integration**: root factories that convert SQL lookup errors into + ``KeyError`` for clean traversal, plus session helpers. See :doc:`sqlalchemy`. +* **Utility modules**: cryptography, Base64/Crockford Base32, collections, path + handling, and more. See :doc:`utilities`. +* **Decorators**: small helpers such as ``deprecated`` and ``reify_attr``. See + :doc:`decorators`. Framework Integration -==================== +===================== Tet integrates with the broader Python web ecosystem: -* **Pyramid**: Built on top of Pyramid's solid foundation -* **SQLAlchemy**: Enhanced ORM integration -* **pyramid_di**: Dependency injection with request-scoped services -* **Passlib**: Secure password handling +* **Pyramid**: built on top of Pyramid's solid foundation. +* **SQLAlchemy**: enhanced ORM integration. +* **pyramid_di**: dependency injection with request-scoped services. +* **Passlib**: secure password handling. Architecture -=========== +============ -Tet uses a modular architecture where each component can be included independently: +Tet uses a modular architecture where each component can be included +independently: .. code-block:: python from pyramid.config import Configurator - def main(): - with Configurator() as config: + def main(global_config, **settings): + with Configurator(settings=settings) as config: # Include only the Tet features you need - config.include('tet.security.csrf') - config.include('tet.renderers.json') - config.include('tet.security.authorization') + config.include("tet.security.csrf") + config.include("tet.renderers.json") + config.include("tet.security.authorization") # Your application configuration # ... return config.make_wsgi_app() -This modular approach allows you to adopt Tet features gradually and only include what your application needs. - -Version History -============== +This modular approach lets you adopt Tet features gradually and include only +what your application needs. For a higher-level entry point that wires the +common features together, see the ``application_factory`` decorator in +:doc:`configuration`. -**Version 0.4.1** (Current) - * Request-scoped services with pyramid_di integration - * Enhanced SQLAlchemy root factory - * Improved namespace package support - * Python 3.6+ compatibility +Requirements +============ -**Version 0.4.0** - * Replace zodb integration - * pyramid_di integration improvements - * Various bug fixes +* **Python**: 3.8 or newer. +* **Pyramid**: 1.9 or newer. +* Core dependencies: ``pyramid``, ``passlib``, ``sqlalchemy``, ``pyramid_di``. -**Earlier Versions** - * Initial namespace package conversion - * SQLAlchemy factory improvements - * Package renamed to 'tet' +Tet is currently at version **0.5.0** and is tested on Python 3.8 through 3.14. Getting Help -=========== +============ -* **Documentation**: This comprehensive documentation covers all aspects of Tet -* **Source Code**: Available on GitHub (if public repository exists) -* **Issues**: Report bugs and feature requests through the issue tracker -* **Community**: Connect with other Tet users and contributors +* **Documentation**: this documentation covers the framework in depth — see the + :doc:`tutorials <../tutorials/index>` for step-by-step walkthroughs. +* **Source Code**: https://github.com/tetframework/tet +* **Issues**: report bugs and feature requests through the issue tracker. diff --git a/docs/narr/json.rst b/docs/narr/json.rst index 2228007..e6f5399 100644 --- a/docs/narr/json.rst +++ b/docs/narr/json.rst @@ -27,7 +27,7 @@ Standard JSON serialization can be unsafe when embedded in HTML: When embedded in HTML, this could execute malicious JavaScript: -.. code-block:: html +.. code-block:: text -The ``|n`` filter prevents double-escaping in template engines like Chameleon. +The ``|n`` filter outputs the value without HTML-escaping in the Tonnikala template engine, which is what you want since ``js_safe_dumps`` has already produced a string that is safe to embed in a `` + Logo + +At render time these expand to URLs such as:: + + /static/001718539201234/style.css + /static/001718539201234/app.js + /static/001718539201234/img/logo.png + +After the next deploy the token changes, every emitted URL changes with it, +and browsers fetch the new assets -- while any page still pointing at the old +token gets a ``301`` redirect to the current one. + +You can of course use the same call from Python view code or any other +renderer: + +.. code-block:: python + + @view_config(renderer="myapp:templates/index.html") + def index(request): + return { + "stylesheet_url": request.static_url("myapp:static/style.css"), + } + +.. note:: + + Always go through :meth:`request.static_url` (or its template + equivalent) rather than hard-coding the ``/static/...`` path yourself. + Hard-coded paths will not contain the cachebreaker token, defeating the + whole mechanism, and will break if the token-bearing prefix ever changes. + + +See also +-------- + +- :doc:`templating` -- writing Tonnikala templates and using ``$`` + interpolation, including expressions like ``${request.static_url(...)}``. +- :doc:`configuration` -- the application factory, ``config.include`` and + registering directives during configuration. diff --git a/docs/narr/templating.rst b/docs/narr/templating.rst new file mode 100644 index 0000000..1b63b30 --- /dev/null +++ b/docs/narr/templating.rst @@ -0,0 +1,226 @@ +========================= +Templating with Tonnikala +========================= + +Tet ships first-class support for the `Tonnikala +`_ template engine. Tonnikala is a +fast, XML-based template language that compiles your templates straight to +Python bytecode, so rendering is quick and template errors surface as ordinary +Python tracebacks. + +This guide shows how to enable the Tonnikala renderer in a Tet application, how +to return template renderings from views, and how the Tonnikala interpolation +syntax works. + +The integration lives in :mod:`tet.renderers.tonnikala`. The public surface is +small and deliberate: + +- ``includeme(config)`` -- the Pyramid includeme that registers the renderer + and the ``.tk`` template extension. +- ``i18n(config)`` -- an includeme that pulls in the base renderer and turns on + Tonnikala's localization support. + + +Enabling the renderer +---------------------- + +The renderer is wired up with a standard Pyramid include. You can include it +directly on the configurator: + +.. code-block:: python + + from pyramid.config import Configurator + + def main(global_config, **settings): + config = Configurator(settings=settings) + config.include("tet.renderers.tonnikala") + config.add_route("home", "/") + config.scan() + return config.make_wsgi_app() + +Calling ``config.include("tet.renderers.tonnikala")`` runs +:func:`tet.renderers.tonnikala.includeme`, which does two things: + +#. Includes ``tonnikala.pyramid``, registering Tonnikala as a Pyramid renderer. +#. Calls ``config.add_tonnikala_extensions(".tk")`` so that any renderer whose + name ends in ``.tk`` is rendered with Tonnikala. + +If your application is assembled with Tet's feature-based +:func:`tet.config.application_factory`, request the renderer as a feature +instead of including it by hand: + +.. code-block:: python + + from tet.config import application_factory + + @application_factory(included_features=["renderers.tonnikala"]) + def main(config): + config.add_route("home", "/") + config.scan() + +Both approaches end up calling the same ``includeme``; pick whichever matches +how the rest of your application is configured. + + +Rendering templates from views +------------------------------ + +Once the renderer is enabled, point a view's ``renderer`` argument at a +template whose name ends in ``.tk``. The view returns a plain dictionary; the +keys of that dictionary become the top-level variables available inside the +template. + +.. code-block:: python + + from pyramid.view import view_config + + @view_config(route_name="home", renderer="templates/home.tk") + def home(request): + return {"title": "Welcome", "name": "World"} + +The ``renderer`` value is resolved as a Pyramid asset specification. A bare +path such as ``"templates/home.tk"`` is interpreted relative to the package the +view is defined in. To reference a template in another package, use the fully +qualified ``package:path`` form: + +.. code-block:: python + + @view_config(route_name="home", renderer="myapp:templates/home.tk") + def home(request): + return {"title": "Welcome", "name": "World"} + +Inside the template, in addition to the keys you returned, the current +``request`` is always available, so you can reach request attributes like +``request.application_url`` directly. + + +Template syntax +--------------- + +Tonnikala templates are well-formed XML/HTML documents. Dynamic content is +produced with ``$`` interpolation and a handful of control attributes. The +example below shows a complete template that consumes the dictionary returned +by the ``home`` view above: + +.. code-block:: html + + + $title + +

Hello $name.

+

Welcome to $request.application_url

+ + + +Save this as ``templates/home.tk`` next to your views and the ``home`` view +will render it. + + +Interpolating expressions with ``$`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The ``$`` sign interpolates an expression. What makes Tonnikala convenient is +that ``$`` does **not** always require braces: it greedily resolves a +*continuous chain* of attribute access, method calls, and indexing as a single +expression. + +.. code-block:: html + + $name + $user.name + $user.get_full_name() + $items[0].title + $viewlets.sidebar() + +The chain stops at the first character that cannot continue a Python +expression of this kind. This is why ``Hello $name.`` renders correctly: the +``.`` is immediately followed by the end of the line (a non-letter), so it does +not continue the attribute chain and is treated as literal punctuation. + + +When to use braces +~~~~~~~~~~~~~~~~~~~ + +Braces ``${...}`` are only needed in two situations. + +First, when the expression is immediately followed by characters that could be +read as part of the identifier. Without braces, Tonnikala would try to resolve +the whole run as one name: + +.. code-block:: html + + + $namesuffix + + + ${name}suffix + +Second, for any expression more complex than a chain of access and calls, such +as arithmetic or operators: + +.. code-block:: html + +

Total: ${x + y}

+ +If in doubt, braces are always safe; they simply make the expression +boundaries explicit. + + +Outputting raw HTML with ``$literal`` +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +By default every interpolated value is HTML-escaped, which is what you want for +user-supplied data. When a value is already trusted markup -- for example the +HTML produced by a viewlet -- wrap it in ``$literal(...)`` to emit it verbatim, +without escaping: + +.. code-block:: html + + + +Only use ``$literal`` on content you control. Passing unescaped user input +through it reintroduces the cross-site scripting risk that escaping exists to +prevent. + + +Internationalized templates +--------------------------- + +To translate template text, enable the i18n-aware variant of the renderer. +Instead of including ``tet.renderers.tonnikala`` directly, include the +:func:`tet.renderers.tonnikala.i18n` includeme: + +.. code-block:: python + + config.include("tet.renderers.tonnikala.i18n") + +This includeme first includes the base renderer and then calls +``config.set_tonnikala_l10n(True)``, which switches on Tonnikala's localization +machinery so that translatable strings in your templates are passed through the +active translation domain. + +With the feature-based factory, request the i18n feature instead: + +.. code-block:: python + + from tet.config import application_factory + + @application_factory(included_features=["renderers.tonnikala.i18n"]) + def main(config): + config.add_route("home", "/") + config.scan() + +Enabling i18n does not change the syntax shown above; it only adds translation +of marked-up text. See the internationalization guide for how to mark and +extract translatable strings. + + +See also +-------- + +- :doc:`views` -- defining views and choosing renderers. +- :doc:`viewlets` -- composing reusable fragments such as ``viewlets.sidebar()`` + for use with ``$literal``. +- :doc:`i18n` -- setting up translation domains and extracting messages. diff --git a/docs/narr/testing.rst b/docs/narr/testing.rst index d19049f..a61cd3c 100644 --- a/docs/narr/testing.rst +++ b/docs/narr/testing.rst @@ -1,493 +1,444 @@ -========= +======= Testing -========= +======= -Tet applications can be thoroughly tested using pytest and various testing utilities. This chapter covers testing patterns, fixtures, and best practices for Tet applications. +Tet applications and the Tet framework itself are tested with `pytest +`_. This chapter documents the test layout actually +used in this project (Tet 0.5.0, Python 3.8+), the fixtures that ship in +``tests/conftest.py``, and practical patterns for testing your own Tet +applications. -Testing Framework -================ +Running the Tests +================= -Tet applications use pytest as the primary testing framework with additional utilities for web application testing. +Install Tet together with its test dependencies in editable mode and run +``pytest``:: -Basic Test Setup ---------------- + # Test dependencies only (pytest, pytest-cov) + pip install -e '.[test]' -.. code-block:: python + # Full development toolchain (pytest, pytest-cov, black, ruff, mypy) + pip install -e '.[dev]' - # conftest.py - import pytest - from pyramid.config import Configurator - from pyramid.testing import setUp, tearDown + # Run the whole suite + pytest - @pytest.fixture(scope='function') - def config(): - """Pyramid configurator for testing.""" - config = setUp() - config.include('tet.renderers.json') - yield config - tearDown() - - @pytest.fixture(scope='function') - def request(config): - """Mock request object for testing.""" - from pyramid.testing import DummyRequest - request = DummyRequest() - request.registry = config.registry - return request +The project configures pytest in ``pyproject.toml`` under +``[tool.pytest.ini_options]``. The relevant settings are:: -Testing Views -============= + [tool.pytest.ini_options] + testpaths = ["tests"] + python_files = ["test_*.py", "*_test.py"] + addopts = "-ra -q --strict-markers" + markers = [ + "slow: marks tests as slow (deselect with '-m \"not slow\"')", + "integration: marks tests as integration tests", + ] + +Because ``testpaths`` is set to ``tests``, a bare ``pytest`` invocation +collects only the ``tests/`` directory. ``--strict-markers`` means every +marker must be declared in ``markers`` (above) or collection fails, so use the +declared markers rather than inventing new ones. -Testing Pyramid views with Tet enhancements. +Two markers are available: -Basic View Testing ------------------ +``slow`` + Mark long-running tests. Deselect them with ``pytest -m "not slow"``. + +``integration`` + Mark integration tests. Run only these with ``pytest -m integration``. .. code-block:: python - # test_views.py import pytest - from pyramid.testing import DummyRequest - from myapp.views import home_view - def test_home_view(): - request = DummyRequest() - response = home_view(request) + @pytest.mark.slow + def test_expensive_operation(): + ... - assert response['message'] == 'Hello, World!' + @pytest.mark.integration + def test_full_request_cycle(app): + ... -JSON View Testing ----------------- +Coverage is available through ``pytest-cov`` (installed by both the ``test`` +and ``dev`` extras):: -Test views that use Tet's JSON renderer: + pytest --cov=tet --cov-report=term-missing -.. code-block:: python +Built-in Fixtures +================= - def test_api_view(config, request): - from myapp.views import api_view +The shared ``tests/conftest.py`` provides a small set of fixtures used +throughout the suite. They are all function-scoped. - # Configure JSON renderer - config.include('tet.renderers.json') +``pyramid_config`` + A real :class:`pyramid.config.Configurator` with ``config.begin()`` already + called; ``config.end()`` runs automatically on teardown. Use it to test + ``includeme`` functions and configuration directives. - # Test the view - result = api_view(request) +``pyramid_request`` + A :class:`pyramid.testing.DummyRequest` whose ``registry`` attribute is a + :class:`unittest.mock.Mock`. - assert 'data' in result - assert isinstance(result['data'], list) +``pyramid_request_with_json`` + Like ``pyramid_request`` but with ``request.json_body`` set to an empty + ``dict``. -Integration Testing -================== +``mock_db_session`` + A :class:`unittest.mock.Mock` with ``query``, ``add``, ``commit``, + ``rollback`` and ``flush`` attributes pre-created as mocks. -Testing complete request/response cycles. +``mock_model`` + A :class:`unittest.mock.Mock` with ``__tablename__`` set to + ``"test_model"``. -WebTest Integration ------------------- +The actual definitions look like this: .. code-block:: python - # conftest.py + # tests/conftest.py + from unittest.mock import Mock + import pytest - from webtest import TestApp - from myapp import main + from pyramid import testing + from pyramid.config import Configurator - @pytest.fixture(scope='session') - def app(): - """Create test application.""" - settings = { - 'sqlalchemy.url': 'sqlite:///:memory:', - 'debug': True, - } - app = main({}, **settings) - return TestApp(app) - # test_integration.py - def test_home_page(app): - response = app.get('/') - assert response.status_code == 200 - assert b'Hello' in response.body + @pytest.fixture + def pyramid_config(): + config = Configurator() + config.begin() + yield config + config.end() - def test_api_endpoint(app): - response = app.get('/api/users') - assert response.status_code == 200 - assert response.content_type == 'application/json' -Database Testing -=============== + @pytest.fixture + def pyramid_request(): + request = testing.DummyRequest() + request.registry = Mock() + return request -Testing with SQLAlchemy and database operations. -Database Fixtures ----------------- + @pytest.fixture + def mock_db_session(): + session = Mock() + session.query = Mock() + session.add = Mock() + session.commit = Mock() + session.rollback = Mock() + session.flush = Mock() + return session + +Testing ``includeme`` Functions +=============================== + +Most Tet modules expose an ``includeme(config)`` entry point. The +``pyramid_config`` fixture makes these easy to exercise. For example, the CSRF +module sets ``require_csrf=True``: .. code-block:: python - # conftest.py - import pytest - from sqlalchemy import create_engine - from sqlalchemy.orm import sessionmaker - from myapp.models import Base + # tests/test_security_csrf.py + from unittest.mock import Mock - @pytest.fixture(scope='session') - def engine(): - """Create test database engine.""" - return create_engine('sqlite:///:memory:', echo=False) + from tet.security.csrf import includeme - @pytest.fixture(scope='session') - def tables(engine): - """Create all tables.""" - Base.metadata.create_all(engine) - yield - Base.metadata.drop_all(engine) - @pytest.fixture(scope='function') - def dbsession(engine, tables): - """Create database session for each test.""" - Session = sessionmaker(bind=engine) - session = Session() - yield session - session.rollback() - session.close() + def test_includeme_sets_csrf_defaults(pyramid_config): + pyramid_config.set_default_csrf_options = Mock() -Testing Root Factories ----------------------- + includeme(pyramid_config) + + pyramid_config.set_default_csrf_options.assert_called_once_with( + require_csrf=True + ) + +Testing Views +============= -Test Tet's SQLAlchemy root factories: +Pyramid views can be called directly with a dummy request. Use the +``pyramid_request`` fixture rather than constructing a request by hand: .. code-block:: python - def test_root_factory_success(dbsession): - from myapp.models import User - from myapp.root import UserRootFactory - from pyramid.testing import DummyRequest + # tests/test_views.py + from myapp.views import home_view + - # Create test data - user = User(name='Test User', email='test@example.com') - dbsession.add(user) - dbsession.commit() + def test_home_view(pyramid_request): + response = home_view(pyramid_request) + assert response["message"] == "Hello, World!" - # Test root factory - request = DummyRequest() - request.dbsession = dbsession +When a view reads JSON from the request body, use +``pyramid_request_with_json`` and set the body content you need: + +.. code-block:: python - root = UserRootFactory(request) - found_user = root[str(user.id)] + def test_api_view(pyramid_request_with_json): + pyramid_request_with_json.json_body = {"name": "example"} - assert found_user == user + from myapp.views import create_view - def test_root_factory_not_found(dbsession): - from myapp.root import UserRootFactory - from pyramid.testing import DummyRequest + result = create_view(pyramid_request_with_json) + assert result["created"] is True - request = DummyRequest() - request.dbsession = dbsession +Testing the JSON Renderer +========================= - root = UserRootFactory(request) +Tet's JSON renderer lives in :mod:`tet.renderers.json`. The public surface is: - with pytest.raises(KeyError): - root['nonexistent'] +``construct_default_renderer(renderer_factory=JSON, **renderer_args)`` + Builds a Pyramid :class:`pyramid.renderers.JSON` renderer pre-loaded with + adapters for :class:`datetime.datetime`, :class:`datetime.date`, and (when + SQLAlchemy is installed) SQLAlchemy keyed tuples. -Security Testing -=============== +``hook_json_renderer(config, *, renderer, name="json")`` + Registers a renderer under a name and records it in the per-registry + renderer registry. -Testing Tet's security features. +``add_json_adapter(config, *, for_, adapter, renderer="json")`` + Adds a type adapter to a named, already-registered renderer. -CSRF Testing ------------ +``includeme(config)`` + Registers the default renderer and adds the ``add_json_renderer`` and + ``add_json_adapter`` directives. + +Note that ``construct_default_renderer`` returns a Pyramid ``JSON`` renderer +*factory* instance. It is not a plain callable that turns data into a string; +to actually render, Pyramid calls it with renderer ``info`` to obtain the +render function. The simplest way to test serialization is therefore to test +the adapters and helpers directly, or to register the renderer on a +configurator. To check that the default adapters are present: .. code-block:: python - def test_csrf_protection(app): - # GET request should work - response = app.get('/form') - assert response.status_code == 200 + # tests/test_renderers_json.py + from tet.renderers.json import construct_default_renderer - # POST without CSRF token should fail - with pytest.raises(Exception): # CSRF error - app.post('/form', {'data': 'test'}) - # POST with CSRF token should work - # (Implementation depends on your CSRF setup) + def test_default_renderer_constructs(): + renderer = construct_default_renderer() + # It is a Pyramid JSON renderer factory instance with adapters added. + assert renderer is not None -Authorization Testing --------------------- +To test the configuration directives, use ``pyramid_config`` and inspect the +per-registry renderer registry that ``hook_json_renderer`` maintains: .. code-block:: python - def test_authorization_policy(): - from myapp.security import MyAuthorizationPolicy - from pyramid.testing import DummyRequest + from unittest.mock import Mock - policy = MyAuthorizationPolicy() - request = DummyRequest() + from tet.renderers.json import hook_json_renderer - # Test permission checking - result = policy.permits( - request=request, - context=None, - principals=['user:123'], - permission='edit' - ) - assert result is True # or False, depending on logic + def test_hook_json_renderer_default_name(pyramid_config): + renderer = Mock() + pyramid_config.add_renderer = Mock() + pyramid_config.registry.tet_json_renderers = {} + + hook_json_renderer(pyramid_config, renderer=renderer) -JSON Testing -=========== + pyramid_config.add_renderer.assert_called_once_with("json", renderer) + assert pyramid_config.registry.tet_json_renderers["json"] is renderer -Testing Tet's JSON functionality. +Testing Safe JSON Serialization +=============================== -JSON Serialization Testing --------------------------- +:func:`tet.util.json.js_safe_dumps` escapes characters that are dangerous +inside inline ``' - } + # tests/test_sqlalchemy_factory.py + from unittest.mock import Mock - safe_json = js_safe_dumps(dangerous_data) - - # Should escape dangerous characters - assert '<' not in safe_json - assert '\\u003c' in safe_json + import pytest + from sqlalchemy.orm.exc import NoResultFound -Mock Testing -=========== + from tet.sqlalchemy.factory import SQLARootFactory -Using mocks for isolated testing. -Service Mocking --------------- + def test_getitem_success(pyramid_request): + factory = SQLARootFactory(pyramid_request) + expected = Mock() + factory.supplier = Mock(return_value=expected) -.. code-block:: python + assert factory["test_id"] is expected + factory.supplier.assert_called_once_with("test_id") - from unittest.mock import Mock, patch - def test_view_with_service(): - from myapp.views import user_list_view - from pyramid.testing import DummyRequest + def test_getitem_raises_keyerror_on_noresult(pyramid_request): + factory = SQLARootFactory(pyramid_request) + factory.supplier = Mock(side_effect=NoResultFound("No result found")) - # Mock the database service - mock_session = Mock() - mock_session.query.return_value.all.return_value = [ - Mock(id=1, name='User 1'), - Mock(id=2, name='User 2'), - ] + with pytest.raises(KeyError) as exc_info: + _ = factory["missing_id"] - request = DummyRequest() - request.find_service = Mock(return_value=mock_session) + # NoResultFound is preserved as the cause of the KeyError. + assert isinstance(exc_info.value.__cause__, NoResultFound) - result = user_list_view(request) +The factory also converts :class:`sqlalchemy.orm.exc.MultipleResultsFound` and +:class:`sqlalchemy.exc.DataError` into ``KeyError`` in the same way. - assert len(result['users']) == 2 +Testing with a Real Database +============================ -External Service Mocking ------------------------ +The built-in ``mock_db_session`` fixture is enough for unit tests that only +need to assert how a session is used. When you need real persistence, define +your own SQLAlchemy fixtures in your application's ``conftest.py``: .. code-block:: python - @patch('myapp.services.external_api_call') - def test_external_service(mock_api_call): - mock_api_call.return_value = {'status': 'success'} - - from myapp.services import process_external_data + # conftest.py (in your application) + import pytest + from sqlalchemy import create_engine + from sqlalchemy.orm import sessionmaker - result = process_external_data('test_data') + from myapp.models import Base - assert result['status'] == 'success' - mock_api_call.assert_called_once_with('test_data') -Fixture Patterns -================ + @pytest.fixture(scope="session") + def engine(): + return create_engine("sqlite:///:memory:") -Common fixture patterns for Tet applications. -User Authentication Fixtures ----------------------------- + @pytest.fixture(scope="session") + def tables(engine): + Base.metadata.create_all(engine) + yield + Base.metadata.drop_all(engine) -.. code-block:: python @pytest.fixture - def authenticated_user(dbsession): - """Create an authenticated user for testing.""" - from myapp.models import User - - user = User( - username='testuser', - email='test@example.com', - is_active=True - ) - dbsession.add(user) - dbsession.commit() - return user + def dbsession(engine, tables): + Session = sessionmaker(bind=engine) + session = Session() + yield session + session.rollback() + session.close() - @pytest.fixture - def authenticated_request(request, authenticated_user): - """Create request with authenticated user.""" - request.user = authenticated_user - return request +Integration Testing with WebTest +================================ -Application State Fixtures --------------------------- +For full request/response cycles, build your WSGI application and wrap it in a +`WebTest `_ ``TestApp``. +WebTest is not a dependency of Tet, so add it to your own test requirements. .. code-block:: python - @pytest.fixture - def sample_data(dbsession): - """Create sample data for testing.""" - from myapp.models import User, Post - - users = [ - User(username=f'user{i}', email=f'user{i}@example.com') - for i in range(3) - ] + # conftest.py (in your application) + import pytest + from webtest import TestApp - for user in users: - dbsession.add(user) + from myapp import main - dbsession.commit() - posts = [ - Post(title=f'Post {i}', content=f'Content {i}', author=users[0]) - for i in range(5) - ] + @pytest.fixture(scope="session") + def app(): + settings = {"sqlalchemy.url": "sqlite:///:memory:"} + return TestApp(main({}, **settings)) - for post in posts: - dbsession.add(post) - dbsession.commit() + # test_integration.py + import pytest - return {'users': users, 'posts': posts} -Performance Testing -================== + @pytest.mark.integration + def test_home_page(app): + response = app.get("/") + assert response.status_code == 200 -Testing performance characteristics of your application. +Mock Testing +============ -Response Time Testing --------------------- +Use :mod:`unittest.mock` to isolate views and services from their +dependencies. The ``mock_db_session`` fixture provides a ready-made mocked +session: .. code-block:: python - import time + from unittest.mock import Mock - def test_api_response_time(app): - start_time = time.time() - response = app.get('/api/users') - end_time = time.time() - assert response.status_code == 200 - assert end_time - start_time < 1.0 # Should respond within 1 second + def test_view_with_service(pyramid_request, mock_db_session): + mock_db_session.query.return_value.all.return_value = [ + Mock(id=1, name="User 1"), + Mock(id=2, name="User 2"), + ] + pyramid_request.find_service = Mock(return_value=mock_db_session) -Load Testing with Locust ------------------------- + from myapp.views import user_list_view -.. code-block:: python + result = user_list_view(pyramid_request) + assert len(result["users"]) == 2 - # locustfile.py - from locust import HttpUser, task, between +Patch external calls at the point where they are used: - class WebsiteUser(HttpUser): - wait_time = between(1, 3) +.. code-block:: python - @task - def index_page(self): - self.client.get("/") + from unittest.mock import patch - @task(3) - def api_users(self): - self.client.get("/api/users") -Test Organization -================ + @patch("myapp.services.external_api_call") + def test_external_service(mock_api_call): + mock_api_call.return_value = {"status": "success"} -Organizing tests for maintainability. + from myapp.services import process_external_data + + result = process_external_data("test_data") + assert result["status"] == "success" + mock_api_call.assert_called_once_with("test_data") -Directory Structure ------------------- +Test Organization +================= -.. code-block:: +The Tet test suite keeps a flat ``tests/`` directory whose module names mirror +the package layout, for example:: tests/ - ├── conftest.py # Shared fixtures - ├── unit/ # Unit tests - │ ├── test_models.py - │ ├── test_views.py - │ └── test_utilities.py - ├── integration/ # Integration tests - │ ├── test_api.py - │ └── test_web.py - ├── functional/ # Functional tests - │ └── test_workflows.py - └── performance/ # Performance tests - └── test_load.py - -Test Categories --------------- - -**Unit Tests** - Test individual functions and classes in isolation. - -**Integration Tests** - Test how components work together. - -**Functional Tests** - Test complete user workflows. - -**Performance Tests** - Test response times and resource usage. + ├── conftest.py # Shared fixtures + ├── test_renderers_json.py + ├── test_security_authorization.py + ├── test_security_csrf.py + ├── test_sqlalchemy_factory.py + ├── test_util_base64.py + ├── test_util_collections.py + ├── test_util_crypt.py + └── test_util_json.py + +Tests are grouped into classes (``class TestSomething:``) with descriptive +method names. Because ``python_files`` is ``["test_*.py", "*_test.py"]``, both +``test_foo.py`` and ``foo_test.py`` are collected. Continuous Integration -===================== - -Running tests in CI environments. - -pytest Configuration -------------------- - -.. code-block:: ini - - # pytest.ini - [tool:pytest] - testpaths = tests - python_files = test_*.py - python_classes = Test* - python_functions = test_* - addopts = - --strict-markers - --disable-warnings - --cov=myapp - --cov-report=html - --cov-report=term-missing +====================== -GitHub Actions Example ---------------------- +A minimal GitHub Actions workflow that installs the test extra and runs the +suite across the supported Python versions: .. code-block:: yaml @@ -501,52 +452,42 @@ GitHub Actions Example runs-on: ubuntu-latest strategy: matrix: - python-version: [3.8, 3.9, '3.10', 3.11] + python-version: ['3.8', '3.9', '3.10', '3.11', '3.12', '3.13'] steps: - - uses: actions/checkout@v4 - - name: Set up Python ${{ matrix.python-version }} - uses: actions/setup-python@v4 - with: - python-version: ${{ matrix.python-version }} - - - name: Install dependencies - run: | - python -m pip install --upgrade pip - pip install -e .[dev] - - - name: Run tests - run: pytest + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: ${{ matrix.python-version }} + - run: | + python -m pip install --upgrade pip + pip install -e '.[test]' + - run: pytest Best Practices -============= - -**Use Fixtures Liberally** - Create reusable fixtures for common test data and setup. - -**Test Edge Cases** - Test not just the happy path, but error conditions and edge cases. - -**Mock External Dependencies** - Mock external APIs and services to make tests reliable and fast. - -**Use Meaningful Test Names** - Test function names should clearly describe what is being tested. +============== -**Keep Tests Independent** - Each test should be able to run independently of others. +**Reuse the built-in fixtures** + Prefer ``pyramid_config``, ``pyramid_request`` and ``mock_db_session`` over + re-creating equivalents in every test module. -**Test Database Interactions** - Use transactions and rollbacks to keep database tests isolated. +**Respect strict markers** + Only ``slow`` and ``integration`` are declared. With ``--strict-markers`` an + undeclared marker fails collection; declare new markers in ``pyproject.toml`` + before using them. -**Use Parametrized Tests** - Use pytest's parametrize decorator to test multiple inputs efficiently. +**Test the public API** + Import from documented entry points such as + :func:`tet.util.json.js_safe_dumps` and + :class:`tet.sqlalchemy.factory.SQLARootFactory`. -**Measure Coverage** - Use coverage tools to ensure adequate test coverage. +**Test edge cases** + Cover error conditions explicitly, for example the ``KeyError`` conversion in + ``SQLARootFactory``. -**Test Security Features** - Specifically test security-related functionality like CSRF and authorization. +**Keep tests independent** + Each test should run on its own; the function-scoped fixtures help enforce + this. -**Performance Benchmarks** - Include basic performance tests to catch regressions early. +**Measure coverage** + Use ``pytest --cov=tet`` (via ``pytest-cov``) to find untested code paths. diff --git a/docs/narr/utilities.rst b/docs/narr/utilities.rst index 3de8046..1183334 100644 --- a/docs/narr/utilities.rst +++ b/docs/narr/utilities.rst @@ -2,454 +2,285 @@ Utilities ========= -Tet provides a comprehensive set of utility modules to handle common tasks in web applications. +Tet provides a small set of focused utility modules under ``tet.util`` to +handle common tasks in web applications. Cryptographic Utilities -======================= +======================== -The ``tet.util.crypt`` module provides secure password hashing and cryptographic utilities. +The ``tet.util.crypt`` module provides password hashing built on top of +passlib's SHA-256 crypt scheme. Password Hashing ---------------- -For secure password storage, Tet integrates with passlib: +Two functions are exposed: ``crypt`` to hash a password and ``verify`` to +check a plaintext password against an existing hash. Both accept either +``str`` or ``bytes`` for the password. .. code-block:: python - from tet.util.crypt import hash_password, verify_password + from tet.util.crypt import crypt, verify # Hash a password - hashed = hash_password('user_password') + hashed = crypt("my_secret_password") # Verify a password - is_valid = verify_password('user_password', hashed) + if verify("my_secret_password", hashed): + print("Password is correct!") -The password utilities use industry-standard algorithms and automatically handle salting and timing-attack prevention. +Hashing uses ``passlib.hash.sha256_crypt`` (exposed as the module-level +``password_hash``), which handles salting automatically. For SQLAlchemy +models, consider ``tet.sqlalchemy.password.UserPasswordMixin``, which +integrates this functionality directly into your model. -Secure Random Generation ------------------------- +Base64 and Crockford Base32 Utilities +====================================== -Generate cryptographically secure random values: +The ``tet.util.base64`` module provides two codec classes, ``Base64`` and +``CrockfordBase32``, both deriving from ``BaseCodec``. Each codec exposes +``encode``, ``decode`` and ``normalize`` classmethods, plus a +``generate_characters`` classmethod inherited from ``BaseCodec``. -.. code-block:: python +Standard Base64 +--------------- - from tet.util.crypt import generate_token, generate_key +.. code-block:: python - # Generate a secure token (for CSRF, API keys, etc.) - token = generate_token(32) # 32 bytes = 256 bits + from tet.util.base64 import Base64 - # Generate a secure key for encryption - key = generate_key(algorithm='AES256') + encoded = Base64.encode(b"hello") # returns bytes + decoded = Base64.decode(encoded) # returns b"hello" -Base64 Utilities -================ +``Base64.encode`` wraps :func:`base64.b64encode` and returns ``bytes``; +``Base64.decode`` wraps :func:`base64.b64decode`. ``Base64.normalize`` is a +no-op that returns its argument unchanged. The class attributes are +``Base64.chars`` (a ``str`` of the 64-character alphabet, +``string.ascii_letters + string.digits + "+/"``), ``bits_per_char = 6`` and +``padding = True``. -The ``tet.util.base64`` module provides enhanced base64 encoding/decoding with additional safety features. +Crockford Base32 +---------------- -URL-Safe Encoding ------------------ +Crockford's Base32 is a human-friendly encoding that avoids ambiguous +characters (``0``/``O`` and ``1``/``I``/``L``). It is case-insensitive on +decode and tolerates common transcription mistakes. .. code-block:: python - from tet.util.base64 import url_safe_encode, url_safe_decode + from tet.util.base64 import CrockfordBase32 - data = b"Hello, World!" + encoded = CrockfordBase32.encode(b"hello") # returns str + decoded = CrockfordBase32.decode(encoded) # returns bytes - # Encode for safe use in URLs - encoded = url_safe_encode(data) + # Ambiguous characters are normalized: O -> 0, I/L -> 1 + CrockfordBase32.normalize("O1L") # "011" - # Decode back to original - decoded = url_safe_decode(encoded) +``CrockfordBase32.encode`` accepts ``str`` or ``bytes`` and returns a ``str`` +with any ``=`` padding stripped. ``CrockfordBase32.decode`` normalizes its +input by default (pass ``normalize=False`` to skip that), re-adds the +mandatory padding, and returns ``bytes``. ``CrockfordBase32.normalize`` +translates the ambiguous characters and upper-cases the input. The class +attributes are ``CrockfordBase32.chars`` +(``"0123456789ABCDEFGHJKMNPQRSTVWXYZ"``, a ``str``), ``bits_per_char = 5`` +and ``padding = False``. -The URL-safe encoding uses base64url format (RFC 4648) that replaces ``+`` and ``/`` with ``-`` and ``_`` respectively, making it safe for use in URLs without encoding. +Generating Random Characters +---------------------------- -Padding Handling ----------------- +``BaseCodec.generate_characters`` produces a random string of the requested +length using the codec's own alphabet, drawn from a cryptographically secure +source: .. code-block:: python - from tet.util.base64 import encode_no_padding, decode_with_padding + from tet.util.base64 import Base64, CrockfordBase32 + + # 16 random Base64 characters + token = Base64.generate_characters(16) - # Encode without padding characters - encoded = encode_no_padding(data) + # 26 random Crockford Base32 characters (good for IDs / tokens) + ident = CrockfordBase32.generate_characters(26) - # Decode with automatic padding restoration - decoded = decode_with_padding(encoded) +Internally it generates ``ceil(length * bits_per_char / 8)`` random bytes +with :func:`secrets.token_bytes`, runs them through the codec's ``encode``, +and truncates the result to ``length`` characters. Because any padding only +trails the data, the truncated slice never contains padding. A non-positive +``length`` returns an empty string. Collection Utilities ==================== -The ``tet.util.collections`` module provides enhanced collection types and utilities. +The ``tet.util.collections`` module provides a single helper, ``flatten``. -Enhanced Dictionaries ---------------------- +Flattening Nested Iterables +--------------------------- -.. code-block:: python - - from tet.util.collections import AttrDict, DefaultAttrDict - - # Dictionary with attribute access - config = AttrDict({ - 'database': { - 'host': 'localhost', - 'port': 5432 - } - }) - - # Access via attributes - host = config.database.host - - # Or traditional dictionary access - port = config['database']['port'] - -Nested Operations ----------------- +``flatten`` is a generator that recursively flattens an arbitrarily nested +iterable. ``str`` and ``bytes`` are treated as atomic values and are never +exploded into their characters. .. code-block:: python - from tet.util.collections import deep_merge, safe_get + from tet.util.collections import flatten - # Deep merge dictionaries - dict1 = {'a': {'b': 1}} - dict2 = {'a': {'c': 2}} - merged = deep_merge(dict1, dict2) - # Result: {'a': {'b': 1, 'c': 2}} + nested = [1, [2, 3, [4, 5]], 6] + list(flatten(nested)) # [1, 2, 3, 4, 5, 6] - # Safe nested access - value = safe_get(config, 'database.host', default='localhost') + with_strings = ["hello", ["world", ["!"]]] + list(flatten(with_strings)) # ["hello", "world", "!"] Path Utilities ============== -The ``tet.util.path`` module provides file and path manipulation utilities. +The ``tet.util.path`` module provides ``caller_package``, used internally by +Tet's configuration system to determine which package called into the +framework. -Path Operations ---------------- - -.. code-block:: python - - from tet.util.path import safe_join, ensure_dir, normalize_path - - # Safely join paths (prevents directory traversal) - safe_path = safe_join('/var/uploads', user_filename) - - # Ensure directory exists - ensure_dir('/var/logs/app') - - # Normalize path for consistent handling - normalized = normalize_path(user_input_path) - -File Operations ---------------- +Determining the Calling Package +------------------------------- .. code-block:: python - from tet.util.path import atomic_write, backup_file - - # Atomic file writing (prevents corruption) - with atomic_write('/important/file.txt') as f: - f.write(data) - - # Create backup before modifying - backup_path = backup_file('/important/file.txt') - # Returns path to backup file + from tet.util.path import caller_package -Temporary File Handling ------------------------ - -.. code-block:: python + # The package module of the code that called the current function + pkg = caller_package() - from tet.util.path import temp_file, temp_dir + # Skip additional modules when walking the stack + pkg = caller_package(ignored_modules=("myframework.helpers",)) - # Secure temporary file - with temp_file(suffix='.json') as tmp: - tmp.write(json_data) - process_file(tmp.name) - - # Temporary directory - with temp_dir() as tmpdir: - work_in_directory(tmpdir) +``caller_package`` walks up the call stack (starting a few frames up, and +always ignoring ``tet.util.path`` itself), skipping any module whose name is +in ``ignored_modules``. When it reaches the first non-ignored module it +returns that module if it is itself a package (its ``__file__`` ends in +``__init__.py``), otherwise it returns the package that contains the module. +It builds on Pyramid's ``pyramid.path.caller_module``, which can also be +overridden via the ``caller_module`` keyword argument for testing. Export Utilities ================ -The ``tet.util.export`` module provides data export and serialization functionality. - -Data Export ------------ - -.. code-block:: python - - from tet.util.export import export_csv, export_json, export_xml - - data = [ - {'name': 'Alice', 'age': 30}, - {'name': 'Bob', 'age': 25} - ] - - # Export to CSV - csv_content = export_csv(data) - - # Export to JSON with custom formatting - json_content = export_json(data, indent=2, sort_keys=True) - - # Export to XML - xml_content = export_xml(data, root_element='users', item_element='user') - -Format Conversion ----------------- - -.. code-block:: python - - from tet.util.export import convert_format - - # Convert between formats - xml_data = convert_format(json_data, from_format='json', to_format='xml') - -Shell Integration -================ - -The ``tet.util.pshell`` module provides Python shell integration utilities. - -Interactive Shell ------------------ - -.. code-block:: python - - from tet.util.pshell import make_shell_env - - # Create shell environment with application context - env = make_shell_env(request) - - # Available variables in shell: - # - request: Current request object - # - root: Application root - # - registry: Application registry - -Development Utilities --------------------- - -.. code-block:: python - - from tet.util.pshell import debug_request, inspect_object - - # Debug request information - debug_info = debug_request(request) - - # Inspect object properties - object_info = inspect_object(some_object, include_private=False) - -JSON Utilities -============== +The ``tet.util.export`` module provides ``exporter``, a small helper for +maintaining a module's ``__all__`` via a decorator. -Beyond the safe serialization covered in the JSON chapter, ``tet.util.json`` provides additional utilities. - -Pretty Printing ---------------- - -.. code-block:: python - - from tet.util.json import pretty_print, colorized_print - - # Pretty print JSON data - pretty_print(complex_data) - - # Colorized output for debugging - colorized_print(data, style='dark') +Maintaining ``__all__`` +----------------------- -JSON Schema Validation ---------------------- +``exporter()`` returns a ``(decorator, list)`` tuple. Bind the list to your +module's ``__all__`` and apply the decorator to anything you want exported; +each decorated object's ``__name__`` is appended to ``__all__`` and the object +is returned unchanged. .. code-block:: python - from tet.util.json import validate_json, create_schema - - schema = { - 'type': 'object', - 'properties': { - 'name': {'type': 'string'}, - 'age': {'type': 'integer', 'minimum': 0} - }, - 'required': ['name'] - } + from tet.util.export import exporter - # Validate data against schema - is_valid, errors = validate_json(user_data, schema) + export, __all__ = exporter() -Configuration Utilities -======================= - -Working with application configuration. + @export + def my_public_function(): + pass -Configuration Loading --------------------- + @export + class MyPublicClass: + pass -.. code-block:: python + def _private_function(): + pass - from tet.util.config import load_config, merge_configs + # __all__ == ["my_public_function", "MyPublicClass"] - # Load configuration from multiple sources - base_config = load_config('config/base.ini') - env_config = load_config('config/production.ini') +Shell (pshell) Utilities +======================== - # Merge configurations with precedence - final_config = merge_configs(base_config, env_config) +The ``tet.util.pshell`` module provides snippet support for the Pyramid +``pshell`` interactive environment. A *snippet* is a ``.py`` file that defines +a ``run()`` function, which can then be invoked interactively. -Environment Variables +Configuring Snippets -------------------- -.. code-block:: python - - from tet.util.config import get_env_config - - # Load configuration from environment variables - config = get_env_config( - prefix='MYAPP_', - mapping={ - 'DATABASE_URL': 'sqlalchemy.url', - 'SECRET_KEY': 'session.secret', - 'DEBUG': ('debug', bool) # Type conversion - } - ) +Point the ``tet.snippets`` setting at a directory of snippet files in your INI +file: -Validation Utilities -=================== +.. code-block:: ini -Input validation and sanitization helpers. + [app:main] + tet.snippets = %(here)s/snippets -Data Validation ---------------- +A snippet file ``snippets/create_user.py`` looks like: .. code-block:: python - from tet.util.validation import validate_email, validate_url, sanitize_filename - - # Validate email address - is_valid_email = validate_email('user@example.com') - - # Validate URL - is_valid_url = validate_url('https://example.com') - - # Sanitize filename for safe storage - safe_filename = sanitize_filename(user_uploaded_filename) - -Form Data Processing -------------------- + def run(username, email): + from myapp.models import User + session = env["request"].dbsession + user = User(username=username, email=email) + session.add(user) + return user -.. code-block:: python - - from tet.util.validation import clean_form_data, validate_form - - # Clean and validate form data - cleaned_data = clean_form_data(request.POST, { - 'name': str.strip, - 'email': str.lower, - 'age': int - }) - - # Comprehensive form validation - is_valid, errors, cleaned = validate_form(form_data, validation_rules) - -Testing Utilities -================ - -Utilities to help with testing Tet applications. +Using Snippets in pshell +------------------------ -Test Helpers ------------- +The ``Snippets`` factory builds a snippets-access object from an environment +mapping (the same ``env`` exposed in ``pshell``). Each ``.py`` file in the +configured directory becomes an attribute that, when called, executes that +file's ``run()`` function in the caller's globals: .. code-block:: python - from tet.util.testing import make_test_request, create_test_app - - # Create test request with mock data - request = make_test_request( - method='POST', - post_data={'name': 'test'}, - user_id=123 - ) + from tet.util.pshell import Snippets - # Create test application - app = create_test_app(settings={ - 'sqlalchemy.url': 'sqlite:///:memory:' - }) + snippets = Snippets(env) -Mock Utilities --------------- - -.. code-block:: python + # List available snippets + snippets() - from tet.util.testing import mock_service, patch_setting + # Invoke snippets/create_user.py's run() function + snippets.create_user("john", "john@example.com") - # Mock application service - with mock_service('dbsession', mock_db): - result = my_view(request) - - # Temporarily patch application setting - with patch_setting('feature.enabled', True): - test_feature_behavior() +JSON Utilities +============== -Performance Utilities -==================== +The ``tet.util.json`` module provides ``js_safe_dumps`` for serializing data +to JSON that is safe to embed directly inside an HTML ``"} + js_safe_dumps(data) + # '{"name": "\\u003cscript\\u003ealert(\'xss\')\\u003c\\u002fscript\\u003e"}' - from tet.util.performance import cached, cache_key +In a Tonnikala template, use ``$literal()`` so the already-escaped JSON is not +double-escaped: - # Simple function caching - @cached(timeout=300) # 5 minute cache - def expensive_calculation(param): - return complex_computation(param) +.. code-block:: html - # Generate cache keys - key = cache_key('user_data', user_id=123, version=2) + Best Practices -============= - -**Security First** - Always use the cryptographic utilities for sensitive operations like password hashing. - -**Validate Inputs** - Use validation utilities to sanitize and validate all user inputs. - -**Handle Paths Safely** - Use path utilities to prevent directory traversal and other path-related security issues. - -**Test Thoroughly** - Use the testing utilities to create comprehensive tests for your utilities usage. - -**Performance Monitoring** - Use timing and profiling utilities to identify performance bottlenecks. +============== -**Configuration Management** - Use configuration utilities to manage application settings across environments. +**Use the secure codecs for tokens** + Prefer ``CrockfordBase32.generate_characters`` / ``Base64.generate_characters`` + for identifiers and tokens; they draw from :mod:`secrets`. -**Error Handling** - All utility functions include proper error handling and meaningful error messages. +**Hash passwords, never store them** + Use ``crypt`` and ``verify`` (or ``UserPasswordMixin``) for password storage. -**Documentation** - Each utility module includes comprehensive docstrings and examples. +**Escape JSON destined for HTML** + Use ``js_safe_dumps`` whenever JSON is embedded inside a ``