[17.0][ADD] fs_attachment_azure

fs_attachment_azure/README.rst

@@ -0,0 +1,209 @@
+===================
+Fs Attachment Azure
+===================
+
+.. 
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! This file is generated by oca-gen-addon-readme !!
+   !! changes will be overwritten.                   !!
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+   !! source digest: sha256:c01d32f225802fc30d7d79a6130c073016c0d98c69ba20dd6d6c0d1213e9f92d
+   !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
+
+.. |badge1| image:: https://img.shields.io/badge/maturity-Beta-yellow.png
+    :target: https://odoo-community.org/page/development-status
+    :alt: Beta
+.. |badge2| image:: https://img.shields.io/badge/licence-AGPL--3-blue.png
+    :target: http://www.gnu.org/licenses/agpl-3.0-standalone.html
+    :alt: License: AGPL-3
+.. |badge3| image:: https://img.shields.io/badge/github-OCA%2Fstorage-lightgray.png?logo=github
+    :target: https://github.com/OCA/storage/tree/17.0/fs_attachment_azure
+    :alt: OCA/storage
+.. |badge4| image:: https://img.shields.io/badge/weblate-Translate%20me-F47D42.png
+    :target: https://translation.odoo-community.org/projects/storage-17-0/storage-17-0-fs_attachment_azure
+    :alt: Translate me on Weblate
+.. |badge5| image:: https://img.shields.io/badge/runboat-Try%20me-875A7B.png
+    :target: https://runboat.odoo-community.org/builds?repo=OCA/storage&target_branch=17.0
+    :alt: Try me on Runboat
+
+|badge1| |badge2| |badge3| |badge4| |badge5|
+
+This module extends the functionality of
+`fs_attachment <https://github.com/OCA/storage/tree/16.0/fs_attachment>`__
+to better support Azure storage. It includes features such as:
+
+- Special handling of X-Accel-Redirect headers for Azure storages.
+- Options for using signed URLs in X-Accel-Redirect. (This is required
+  to be able to serve files from a private Azure Blob Storage using
+  X-Accel-Redirect without exposing the files publicly.)
+- Bulk deletion of the orphaned files during the garbage collection,
+  using Azure blob batch requests instead of one request per file.
+
+**Table of contents**
+
+.. contents::
+   :local:
+
+Configuration
+=============
+
+On the Odoo instance, go to *Settings* > *Technical* > *Storage* > *File
+Storage*.
+
+When you create a new storage for Azure or modify an existing one, when
+you activate the option "Use X-Sendfile To Serve Internal Url", 3
+additional fields will appear:
+
+- **Azure Uses Signed URL For X-Accel-Redirect**: If checked, the
+  X-Accel-Redirect path will be a signed URL, which is useful for Azure
+  storages that require signed URLs for access.
+- **Azure Signed URL Expiration**: The expiration time for the signed
+  URL in seconds. This field is only relevant if the previous option is
+  checked. By default, it is set to 30 seconds but it could be less
+  since the url generated into the X-Accel-Redirect process is directly
+  used by the web server to serve the file.
+- **Azure Delegation Key Expiration**: The lifetime of the user
+  delegation key, in seconds. This field is only relevant when the
+  storage authenticates with an identity (see below). By default it is
+  set to 1 hour, and Azure does not allow more than 7 days.
+
+The value of these fields can also be set in the server environment, by
+installing the *fs_attachment_azure_environment* glue module and using
+the keys:
+
+- *azure_uses_signed_url_for_x_sendfile*
+- *azure_signed_url_expiration*
+- *azure_delegation_key_expiration*
+
+When the option "Use X-Sendfile To Serve Internal Url" is enabled, the
+system will generate an X-Accel-Redirect header in the response to a
+request to get a file. In the case of Azure storages, it will follow the
+format:
+
+.. code:: text
+
+   X-Accel-Redirect: /fs_x_sendfile/{scheme}/{host}/{path with query if any}
+
+Where:
+
+- ``{scheme}``: The URL scheme (http or https).
+- ``{host}``: The host of the Azure storage.
+- ``{path with query if any}``: The path to the file in the Azure
+  storage, including any query parameters. (Query parameters are set
+  when the ``azure_uses_signed_url_for_x_sendfile`` option is enabled.)
+
+In order to serve files using X-Accel-Redirect, you must ensure that
+your web server is configured to handle these headers correctly. This
+typically involves setting up a location block in your web server
+configuration that matches the X-Accel-Redirect path and proxies the
+request to the Azure storage.
+
+For example, if you are using Nginx, you would add a location block like
+this:
+
+.. code:: nginx
+
+
+       location ~ ^/fs_x_sendfile/(.*?)/(.*?)/(.*) {
+           internal;
+           set $url_scheme $1;
+           set $url_host $2;
+           set $url_path $3;
+           set $url $url_scheme://$url_host/$url_path;
+
+           proxy_pass $url$is_args$args;
+           proxy_set_header Host $url_host;
+           proxy_ssl_server_name on;
+
+       }
+
+Unlike the standard implementation of X-Accel-Redirect on non Azure
+storages, the Azure implementation does not require a base URL to be set
+in the storage configuration. The X-Accel-Redirect path is constructed
+directly from the Azure storage's URL defined for the connection, the
+directory name as bucket name, and the file path.
+
+Signing with an identity
+------------------------
+
+Signed URLs are generated with the account shared key when the storage
+is configured with a connection string or an account name/key pair.
+Otherwise (managed identity, workload identity, service principal, ...)
+they are signed with a *user delegation key*, which requires the
+identity to have the **Storage Blob Delegator** role on the storage
+account, on top of a data plane role such as *Storage Blob Data Reader*.
+
+Delegation keys are obtained from Azure with an extra request, so they
+are kept in the cache of the Odoo registry for **Azure Delegation Key
+Expiration** seconds. They are requested from Azure for slightly longer
+than that, so that a key served from the cache still covers the URLs
+signed with it.
+
+A higher value means fewer requests to Azure, but also a longer window
+during which the key of a revoked identity remains usable. Note that a
+key already issued by Azure stays valid until it expires anyway,
+whatever Odoo does with its copy.
+
+The cache is only kept in memory, so it is never shared between
+processes nor persisted in the database, and it is dropped whenever Odoo
+clears the cache of the registry, which modifying a storage does.
+Reconfiguring a storage is therefore applied right away.
+
+Changelog
+=========
+
+17.0.1.0.0 (2026-07-13)
+-----------------------
+
+- This module was "forked" from fs_attachment_s3 v17.0.1.2.1
+
+Bug Tracker
+===========
+
+Bugs are tracked on `GitHub Issues <https://github.com/OCA/storage/issues>`_.
+In case of trouble, please check there if your issue has already been reported.
+If you spotted it first, help us to smash it by providing a detailed and welcomed
+`feedback <https://github.com/OCA/storage/issues/new?body=module:%20fs_attachment_azure%0Aversion:%2017.0%0A%0A**Steps%20to%20reproduce**%0A-%20...%0A%0A**Current%20behavior**%0A%0A**Expected%20behavior**>`_.
+
+Do not contact contributors directly about support or help with technical issues.
+
+Credits
+=======
+
+Authors
+-------
+
+* ACSONE SA/NV
+* Camptocamp
+
+Contributors
+------------
+
+- Laurent Mignon laurent.mignon@acsone.eu (https://www.acsone.eu)
+- Stéphane Bidoul stephane.bidoul@acsone.eu (https://www.acsone.eu)
+- Akim Juillerat akim.juillerat@camptocamp.com
+
+Maintainers
+-----------
+
+This module is maintained by the OCA.
+
+.. image:: https://odoo-community.org/logo.png
+   :alt: Odoo Community Association
+   :target: https://odoo-community.org
+
+OCA, or the Odoo Community Association, is a nonprofit organization whose
+mission is to support the collaborative development of Odoo features and
+promote its widespread use.
+
+.. |maintainer-grindtildeath| image:: https://github.com/grindtildeath.png?size=40px
+    :target: https://github.com/grindtildeath
+    :alt: grindtildeath
+
+Current `maintainer <https://odoo-community.org/page/maintainer-role>`__:
+
+|maintainer-grindtildeath| 
+
+This module is part of the `OCA/storage <https://github.com/OCA/storage/tree/17.0/fs_attachment_azure>`_ project on GitHub.
+
+You are welcome to contribute. To learn how please visit https://odoo-community.org/page/Contribute.

fs_attachment_azure/__init__.py

@@ -0,0 +1 @@
+from . import models

fs_attachment_azure/__manifest__.py

@@ -0,0 +1,22 @@
+# Copyright 2025 ACSONE SA/NV
+# Copyright 2026 Camptocamp SA
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
+
+{
+    "name": "Fs Attachment Azure",
+    "summary": """Store attachments into Azure Blob storage""",
+    "version": "17.0.1.0.0",
+    "license": "AGPL-3",
+    "author": "ACSONE SA/NV,Camptocamp,Odoo Community Association (OCA)",
+    "website": "https://github.com/OCA/storage",
+    "depends": ["fs_attachment"],
+    "external_dependencies": {
+        "python": [
+            "adlfs",
+        ],
+    },
+    "data": [
+        "views/fs_storage.xml",
+    ],
+    "maintainers": ["grindtildeath"],
+}

fs_attachment_azure/models/__init__.py

@@ -0,0 +1,3 @@
+from . import fs_storage
+from . import fs_file_gc
+from . import ir_attachment

fs_attachment_azure/models/fs_file_gc.py

@@ -0,0 +1,106 @@
+# Copyright 2026 Camptocamp SA
+# License AGPL-3.0 or later (https://www.gnu.org/licenses/agpl).
+import logging
+
+from odoo import models
+
+from .fs_storage import AZURE_MAX_BLOBS_PER_BATCH
+
+_logger = logging.getLogger(__name__)
+
+
+class FsFileGc(models.Model):
+    _inherit = "fs.file.gc"
+
+    # One batch request is sent per iteration, so Azure's own limit is the
+    # most files we can handle at once.
+    _GC_BATCH_SIZE = AZURE_MAX_BLOBS_PER_BATCH
+
+    def _gc_files_unsafe(self) -> None:
+        """Collect the Azure storages by batch before the file by file cleanup.
+
+        Deleting blobs one by one is one request per file, which does not
+        scale to a large backlog. Azure deletes up to 256 blobs per request,
+        so the Azure storages are collected that way first and ``super()`` is
+        left with the rest: the other storages, and the blobs the batch could
+        not delete, which it retries one by one.
+        """
+        self._gc_azure_bulk_delete()
+        return super()._gc_files_unsafe()
+
+    def _gc_azure_bulk_delete(self) -> None:
+        """Delete the orphaned blobs of every Azure storage, by batch."""
+        # autovacuum_gc is not a stored field, so the storages are filtered
+        # in memory.
+        storages = (
+            self.env["fs.storage"]
+            .search([])
+            .filtered(
+                lambda storage: storage.autovacuum_gc and storage.is_azure_storage
+            )
+        )
+        for storage in storages:
+            try:
+                self._gc_azure_bulk_delete_storage(storage)
+            except Exception:
+                _logger.exception(
+                    "GC: could not batch delete the blobs of the storage %s",
+                    storage.code,
+                )
+
+    def _gc_azure_bulk_delete_storage(self, storage) -> None:
+        """Delete the orphaned blobs of one Azure storage, one batch at a time."""
+        while True:
+            self._cr.execute(
+                """
+                SELECT
+                    store_fname
+                FROM
+                    fs_file_gc
+                WHERE
+                    fs_storage_code = %s
+                    AND NOT EXISTS (
+                        SELECT 1
+                        FROM ir_attachment
+                        WHERE store_fname = fs_file_gc.store_fname
+                    )
+                LIMIT %s
+                """,
+                (storage.code, self._GC_BATCH_SIZE),
+            )
+            store_fnames = [row[0] for row in self._cr.fetchall()]
+            if not store_fnames:
+                return
+            blob_names = [
+                store_fname.partition("://")[2] for store_fname in store_fnames
+            ]
+            _logger.info(
+                "GC: batch deleting %s blobs of the storage %s",
+                len(blob_names),
+                storage.code,
+            )
+            collected = set(storage._azure_delete_blobs(blob_names))
+            deleted = [
+                store_fname
+                for store_fname, blob_name in zip(store_fnames, blob_names, strict=True)
+                if blob_name in collected
+            ]
+            if deleted:
+                self._cr.execute(
+                    """
+                    DELETE FROM
+                        fs_file_gc
+                    WHERE
+                        store_fname = ANY(%s)
+                    """,
+                    (deleted,),
+                )
+            if not self._is_test_mode():
+                # Commit each batch, so that the progress is kept even if a
+                # later batch fails, and the locks taken by _gc_files are not
+                # held for the whole backlog.
+                self._cr.commit()  # pylint: disable=invalid-commit
+            if len(deleted) < len(store_fnames):
+                # The blobs that could not be deleted would be selected again
+                # by the query above: leave them to the file by file cleanup.
+                return