DON'T REVIEW [26.04_linux-nvidia] Backport ATS always-on (pci_ats_required) from upstream v6 - #547
Closed
nirmoy wants to merge 3 commits into
Closed
Conversation
Controlled by IOMMU drivers, ATS can be enabled "on demand", when a given PASID on a device is attached to an I/O page table. This is working, even when a device has no translation on its RID (i.e., RID is IOMMU bypassed). However, certain PCIe devices require non-PASID ATS on their RID even when the RID is IOMMU bypassed. Call this "ATS always on" in IOMMU term. For example, CXL spec r4.0 notes in sec 3.2.5.13 Memory Type on CXL.cache: "To source requests on CXL.cache, devices need to get the Host Physical Address (HPA) from the Host by means of an ATS request on CXL.io." In other words, the CXL.cache capability requires ATS; otherwise, it can't access host physical memory. Introduce a new pci_ats_required() helper for the IOMMU driver to scan a PCI device and shift ATS policies between "on demand" and "always on". Add the support for CXL.cache devices first. Pre-CXL devices will be added in quirks.c file. Note that pci_ats_required() validates against pci_ats_supported(), so we ensure that untrusted devices (e.g. external ports) will not be always on. This maintains the existing ATS security policy regarding potential side- channel attacks via ATS. Cc: linux-cxl@vger.kernel.org Suggested-by: Vikram Sethi <vsethi@nvidia.com> Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Tested-by: Nirmoy Das <nirmoyd@nvidia.com> Acked-by: Nirmoy Das <nirmoyd@nvidia.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com> Reviewed-by: Yi Liu <yi.l.liu@intel.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> (cherry picked from commit 91561e1) Signed-off-by: Nirmoy Das <nirmoyd@nvidia.com>
Some NVIDIA GPU/NIC devices, though they don't implement CXL config space, have many CXL-like properties. Call this kind "pre-CXL". Similar to CXL.cache capability, these pre-CXL devices also require the ATS function even when their RIDs are IOMMU bypassed, i.e. keep ATS "always on" v.s. "on demand" when a non-zero PASID line gets enabled in SVA use cases. Introduce pci_dev_specific_ats_required() quirk function to scan a list of IDs for these devices. Then, include it in pci_ats_required(). Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Nirmoy Das <nirmoyd@nvidia.com> Tested-by: Nirmoy Das <nirmoyd@nvidia.com> Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Acked-by: Bjorn Helgaas <bhelgaas@google.com> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com> Reviewed-by: Yi Liu <yi.l.liu@intel.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> (cherry picked from commit 073e638) Signed-off-by: Nirmoy Das <nirmoyd@nvidia.com>
When a device's default substream attaches to an identity domain, the SMMU driver currently sets the device's STE between two modes: Mode 1: Cfg=Translate, S1DSS=Bypass, EATS=1 Mode 2: Cfg=bypass (EATS is ignored by HW) When there is an active PASID (non-default substream), mode 1 is used. And when there is no PASID support or no active PASID, mode 2 is used. The driver will also downgrade an STE from mode 1 to mode 2, when the last active substream becomes inactive. However, there are PCIe devices that demand ATS to be always on. For these devices, their STEs have to use the mode 1 as HW ignores EATS with mode 2. Change the driver accordingly: - always use the mode 1 - never downgrade to mode 2 - allocate and retain a CD table (see note below) Note that these devices might not support PASID, i.e. doing non-PASID ATS. In such a case, the ssid_bits is set to 0. However, s1cdmax must be set to a !0 value in order to keep the S1DSS field effective. Thus, when a master requires ats_always_on, set its s1cdmax to at least 1, meaning that the CD table will have a dummy entry (SSID=1) that will never be used. Now for these devices, arm_smmu_cdtab_allocated() will always return true, v.s. false prior to this change. When its default substream is attached to an IDENTITY domain, its first CD is NULL in the table, which is a totally valid case. Thus, add "!master->ats_always_on" to the condition. Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com> Tested-by: Nirmoy Das <nirmoyd@nvidia.com> Acked-by: Nirmoy Das <nirmoyd@nvidia.com> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Reviewed-by: Kevin Tian <kevin.tian@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> Signed-off-by: Nicolin Chen <nicolinc@nvidia.com> Signed-off-by: Joerg Roedel <joerg.roedel@amd.com> (cherry picked from commit 5ff98a0) Signed-off-by: Nirmoy Das <nirmoyd@nvidia.com>
nirmoy
changed the title
Contributor
PR Validation ReportPatchscan ✅ No Missing FixesAll cherry-picked commits checked — no missing upstream fixes found. PR Lint
|
Collaborator
Author
BaseOS Kernel ReviewWarning
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
26.04_linux-nvidiadid not haveNVIDIA: VR: SAUCEATS commits, so this is a directgit cherry-pick -x -sof the three upstream commits.nirmoy/ats-always-on-v6-26.04-bos-xs(revert SAUCE on 2017.17, then the same cherry-picks). Current26.04_linux-nvidia-bos(Ubuntu-nvidia-bos-7.0.0-2018.18) already contains that replacement.Source
91561e1dc94bPCI: Add pci_ats_required() for CXL.cache capable devices073e63812525PCI: Allow ATS to be always on for pre-CXL devices5ff98a0da6b4iommu/arm-smmu-v3: Allow ATS to be always onLP: https://bugs.launchpad.net/bugs/2150727
Test plan
lspci -nnvvvshowsATS Enable+with RID identity/bypassATC invalidate timeout/CMDQ errorin dmesg