From 865b4fce961612235d7176838b28987e4a67e07e Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 13:39:37 -0400 Subject: [PATCH 01/18] Expand supplementary reading materials for AI security Added sections for securing data sets and useful books related to security topics in AI. --- ...15.03 - Supplementary Reading Materials.md | 24 ++++++++++++------- 1 file changed, 16 insertions(+), 8 deletions(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index c4f3a8c..d30b6f4 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -1,19 +1,27 @@ # Supplementary Reading Materials - Security of and for AI + These materials further examine the topics that a developer needs to know about security of and for AI. -In each section of this page, we'll also list books and other non-OWASP publications that which have been found useful -in improving understanding. While not necessary to cover the material in the examination Body of Knowledge, you may -find them helpful as you build your understanding. By happy coincidence, many of the authors listed are also active in -the OWASP community. +## Securing Models -Keep an eye out for cost-saving ways to obtain these titles, such as e-book bundling sites like -[Humble Bundle](https://www.humblebundle.com/), or at your local public library. -## Securing Models + +## Securing Data Sets + + +## Useful Books Related to Topics in This Module + +The following books and publications are useful for building a deeper understanding of the topics in this module. +They are not required reading for the examination Body of Knowledge, but many learners find them helpful. + + +### Securing Models * Steve Wilson, [*The Developer's Playbook for Large Language Model Security*](https://www.oreilly.com/library/view/the-developers-playbook/9781098162191) (O'Reilly Media, Inc., 2024) * Harriet Farlow, [*Practical AI Security*](https://nostarch.com/practical-ai-security) (No Starch Press, 2026) -## Securing Data Sets +### Securing Data Sets * Anand Singh, [*Data Security in the Age of AI: A Guide to Protecting Data and Reducing Risk in an AI-Driven World*](https://www.amazon.com/Data-Security-Age-Protecting-AI-Driven-ebook/dp/B0FLVWDYJH) (Responsible Artificial Intelligence Institute, 2025) + +If you want lower-cost access to these titles, check library availability or e-book bundle sites such as [Humble Bundle](https://www.humblebundle.com/). From f2c7346951da09026d4eee982ea9eb48bd36af16 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 14:45:09 -0400 Subject: [PATCH 02/18] Enhance section on securing datasets for AI models Added guidelines for securing datasets used by AI models, including verifying provenance, integrity, and encryption. Provided references to relevant resources for further guidance. --- .../15.03 - Supplementary Reading Materials.md | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index d30b6f4..e1d379a 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -6,15 +6,31 @@ These materials further examine the topics that a developer needs to know about + ## Securing Data Sets +Securing datasets used by AI models requires treating them as critical assets throughout their lifecycle, from collection and storage to training, deployment and retirement. A compromised dataset can introduce data poisoning, privacy violations, intellectual property risks, and unreliable model behavior. +A secure developer should do the following to help achieve this. + +### Verify the provenance of training data. + +### Verify the integrity of datasets. + +### Encrypt data at rest and in transit. + +### Remove sensitive information from datasets. + +### Minimize data collection, processing, and retention. + +### Maintain version control for datasets. + +You can find further guidance on this subject in [Chapter 1 (Training Data Integrity & Traceability) of the OWASP Artificial Intelligence Security Verification Standard (AISVS) project](https://github.com/OWASP/AISVS/blob/main/1.0/en/0x10-C01-Training-Data-Integrity-and-Traceability.md), which is a catalogue of testable security requirements for AI-enabled systems that may be useful for developers, as well as architects, security engineers and auditors. ## Useful Books Related to Topics in This Module The following books and publications are useful for building a deeper understanding of the topics in this module. They are not required reading for the examination Body of Knowledge, but many learners find them helpful. - ### Securing Models * Steve Wilson, [*The Developer's Playbook for Large Language Model Security*](https://www.oreilly.com/library/view/the-developers-playbook/9781098162191) (O'Reilly Media, Inc., 2024) From 1cce02df7d1183b8b296d369a9b5efc0f10e8119 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 14:47:01 -0400 Subject: [PATCH 03/18] Enhance security section for AI datasets Added a section on securing datasets used by AI models, emphasizing their importance and the risks of compromised data. --- .../15.03 - Supplementary Reading Materials.md | 1 + 1 file changed, 1 insertion(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index e1d379a..16040cd 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -10,6 +10,7 @@ These materials further examine the topics that a developer needs to know about ## Securing Data Sets Securing datasets used by AI models requires treating them as critical assets throughout their lifecycle, from collection and storage to training, deployment and retirement. A compromised dataset can introduce data poisoning, privacy violations, intellectual property risks, and unreliable model behavior. + A secure developer should do the following to help achieve this. ### Verify the provenance of training data. From 6c68fa59dee417b8f982ac6d1cc4050c7b7cb6cc Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 14:59:30 -0400 Subject: [PATCH 04/18] Enhance security guidelines for AI dataset management Added guidelines for securing datasets in AI development, including verification, encryption, and access auditing. --- ...15.03 - Supplementary Reading Materials.md | 25 +++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 16040cd..1152313 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -15,16 +15,41 @@ A secure developer should do the following to help achieve this. ### Verify the provenance of training data. +Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines. + ### Verify the integrity of datasets. +Perform checksum/signature verification before processing or training. + ### Encrypt data at rest and in transit. +Enable encryption in storage services, databases, object stores, and APIs using enterprise standards. + ### Remove sensitive information from datasets. +Develop data sanitization, PII masking, secret scanning, and preprocessing workflows. + ### Minimize data collection, processing, and retention. +Design applications and pipelines to collect and retain only necessary data. + ### Maintain version control for datasets. +Integrate tools such as Git-LFS, DVC, LakeFS, or MLflow into development workflows. + +### Secure third-party datasets before use. + +Verify licenses, integrity, and approved sources before integrating external datasets. + +### Audit dataset access and usage. + +Build logging and audit trails into applications and services. + +### Retain and securely delete datasets according to organizational retention policies. + +Implement lifecycle management, deletion APIs, and retention logic in software. + + You can find further guidance on this subject in [Chapter 1 (Training Data Integrity & Traceability) of the OWASP Artificial Intelligence Security Verification Standard (AISVS) project](https://github.com/OWASP/AISVS/blob/main/1.0/en/0x10-C01-Training-Data-Integrity-and-Traceability.md), which is a catalogue of testable security requirements for AI-enabled systems that may be useful for developers, as well as architects, security engineers and auditors. ## Useful Books Related to Topics in This Module From e69e7855915ff933f7a07d58e8b69b5cb94a7bb6 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:06:36 -0400 Subject: [PATCH 05/18] Enhance dataset version control section with links Added links to external tools for better reference. --- .../15.03 - Supplementary Reading Materials.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 1152313..2668e2f 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -35,7 +35,7 @@ Design applications and pipelines to collect and retain only necessary data. ### Maintain version control for datasets. -Integrate tools such as Git-LFS, DVC, LakeFS, or MLflow into development workflows. +Integrate tools such as [Git-LFS](https://git-lfs.com), [DVC](https://dvc.org), [LakeFS](https://lakefs.io), or [MLflow](https://mlflow.org) into development workflows. ### Secure third-party datasets before use. From fe9c4041ee5c06fd57fbe25c52265d298ffa68ad Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:12:27 -0400 Subject: [PATCH 06/18] Enhance PII masking section with a reference link Added a link to the definition of Personally Identifiable Information (PII) masking for clarity. --- .../15.03 - Supplementary Reading Materials.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 2668e2f..3f557a1 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -27,7 +27,7 @@ Enable encryption in storage services, databases, object stores, and APIs using ### Remove sensitive information from datasets. -Develop data sanitization, PII masking, secret scanning, and preprocessing workflows. +Develop data sanitization, [Personally Identifiable Information (PII) masking](https://www.iso.org/information-security/data-masking), secret scanning, and preprocessing workflows. ### Minimize data collection, processing, and retention. From 54e860e4d312f8f9aa7432df46e5b27a98dd08f2 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:24:36 -0400 Subject: [PATCH 07/18] Enhance training data provenance verification guidelines Expanded guidelines on verifying training data provenance, including maintaining metadata and restrictions on dataset sources. --- .../15.03 - Supplementary Reading Materials.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 3f557a1..e7e6810 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -15,7 +15,10 @@ A secure developer should do the following to help achieve this. ### Verify the provenance of training data. -Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines. +* Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines. +* Maintain metadata describing the origin of the data. Capture the dataset name and version, original source URL or repository, data owner or provider, collection date, license and usage restrictions, intended purpose and data classification level. +* Only allow datasets from approved locations, such as Internal enterprise data platforms, such as approved cloud storage buckets, trusted open-source repositories and vendor-provided datasets with contractual agreements. +* Avoid unknown file-sharing sites, unverified public datasets, datasets without ownership information and data scraped without legal review. ### Verify the integrity of datasets. From 225edee66b58329481cd6d5999741b786aa1dc75 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:30:58 -0400 Subject: [PATCH 08/18] Enhance security guidelines for dataset verification Added details on verifying dataset integrity and actions for developers. --- .../15.03 - Supplementary Reading Materials.md | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index e7e6810..2ea89cf 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -15,7 +15,8 @@ A secure developer should do the following to help achieve this. ### Verify the provenance of training data. -* Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines. +Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines. + * Maintain metadata describing the origin of the data. Capture the dataset name and version, original source URL or repository, data owner or provider, collection date, license and usage restrictions, intended purpose and data classification level. * Only allow datasets from approved locations, such as Internal enterprise data platforms, such as approved cloud storage buckets, trusted open-source repositories and vendor-provided datasets with contractual agreements. * Avoid unknown file-sharing sites, unverified public datasets, datasets without ownership information and data scraped without legal review. @@ -24,6 +25,12 @@ A secure developer should do the following to help achieve this. Perform checksum/signature verification before processing or training. +A developer can verify the integrity of datasets by ensuring that the data has not been altered, corrupted, tampered with, or unintentionally modified from an approved state. Dataset integrity verification applies software supply chain security practices to data artifacts. +Key developer actions include the following. + +* Generate a cryptographic hash for a dataset and compare it against a trusted reference value. Common algorithms include SHA-256 and SHA-512. +* Validate that the dataset was signed by an authorized source. Implement signature verification, public key validation, certificate trust checks and key expiration checks. + ### Encrypt data at rest and in transit. Enable encryption in storage services, databases, object stores, and APIs using enterprise standards. From a99a9769e52d6b0863bbda9756a0211ead82c055 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:36:23 -0400 Subject: [PATCH 09/18] Enhance encryption guidelines for data security Added detailed guidelines for encrypting data at rest and in transit, including encryption algorithms, key management, and compliance measures. --- ...15.03 - Supplementary Reading Materials.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 2ea89cf..00539f8 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -35,6 +35,30 @@ Key developer actions include the following. Enable encryption in storage services, databases, object stores, and APIs using enterprise standards. +* Encrypt datasets at rest using strong encryption algorithms such as AES-256 to protect stored training data in databases, object storage, file systems, and data lakes. +* Enable native encryption features provided by cloud platforms and storage systems, such as encrypted buckets, managed database encryption, and encrypted volumes. +* Manage encryption keys securely using enterprise key management systems (KMS), hardware security modules (HSMs), or secrets management platforms. +* Rotate encryption keys regularly according to organizational security policies and compliance requirements. +* Restrict access to encryption keys using least-privilege permissions and role-based access control (RBAC). +* Separate encryption keys from encrypted data to prevent unauthorized users from accessing both the dataset and its decryption capability. +* Encrypt data backups, snapshots, replicas, and temporary copies to prevent exposure through secondary storage locations. +* Use TLS 1.2 or higher (preferably TLS 1.3) to encrypt data transmitted between applications, APIs, databases, storage services, and AI training infrastructure. +* Validate TLS certificates to ensure secure communication only occurs with trusted endpoints. +* Disable insecure communication protocols and deprecated cryptographic algorithms such as TLS 1.0, TLS 1.1, weak ciphers, and unencrypted HTTP connections. +* Protect API communications by requiring encrypted HTTPS connections for dataset access and AI service interactions. +* Implement mutual TLS (mTLS) for high-trust environments where both clients and servers must authenticate each other. +* Encrypt data transfers between cloud services, data pipelines, machine learning platforms, and storage systems. +* Secure message queues, data streaming platforms, and ETL/ELT pipelines with encryption and authenticated connections. +* Avoid transmitting sensitive datasets through unencrypted channels such as email, unsecured file transfers, or public storage links. +* Apply field-level or column-level encryption for highly sensitive data elements such as personally identifiable information (PII), financial data, healthcare data, or proprietary business information. +* Tokenize or pseudonymize sensitive data before storing or transmitting it when full data encryption is not required. +* Prevent unauthorized decryption by enforcing authentication and authorization checks before granting access to encrypted datasets. +* Log encryption-related events, including key usage, access attempts, and administrative changes, for auditing and incident response. +* Monitor encryption configurations continuously to detect disabled encryption, exposed keys, or unauthorized changes. +* Automate encryption checks in CI/CD and infrastructure-as-code pipelines to ensure new datasets and services meet enterprise security requirements. +* Verify encryption compliance against organizational standards and frameworks such as NIST cryptographic guidance, CIS Benchmarks, and enterprise security policies. +* Document encryption methods, key ownership, rotation schedules, and access controls as part of dataset governance and AI system documentation. + ### Remove sensitive information from datasets. Develop data sanitization, [Personally Identifiable Information (PII) masking](https://www.iso.org/information-security/data-masking), secret scanning, and preprocessing workflows. From 25db467c908fe3c9617d0b48e0af1ac9043590a8 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:42:39 -0400 Subject: [PATCH 10/18] Enhance data sanitization section with new guidelines Added guidelines for identifying and handling sensitive information in datasets, including PII, PHI, and other confidential data. --- .../15.03 - Supplementary Reading Materials.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 00539f8..f30b92e 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -63,6 +63,13 @@ Enable encryption in storage services, databases, object stores, and APIs using Develop data sanitization, [Personally Identifiable Information (PII) masking](https://www.iso.org/information-security/data-masking), secret scanning, and preprocessing workflows. +* Identify sensitive information within datasets before training or processing AI models, including personally identifiable information (PII), protected health information (PHI), payment card data, authentication credentials, API keys and secrets, confidential business information and intellectual property. +* Classify dataset fields according to their sensitivity level to determine appropriate protection and removal requirements. +* Scan datasets automatically using data discovery tools to detect sensitive content such as names, email addresses, phone numbers, addresses, government identifiers, financial account numbers, passwords and tokens. +* Implement automated PII detection using techniques such as regular expressions, pattern matching, named entity recognition (NER) and machine learning-based classifiers. +* Remove unnecessary sensitive data fields before using datasets for AI training. +* Redact sensitive values by replacing them with placeholders. + ### Minimize data collection, processing, and retention. Design applications and pipelines to collect and retain only necessary data. From e5275c93c626a299a953fa5f90890d54989e77b9 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:48:53 -0400 Subject: [PATCH 11/18] Enhance data minimization guidelines for AI models Added guidelines for minimizing data collection, processing, and retention in AI applications. --- ...15.03 - Supplementary Reading Materials.md | 32 +++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index f30b92e..2fbaf41 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -74,6 +74,38 @@ Develop data sanitization, [Personally Identifiable Information (PII) masking](h Design applications and pipelines to collect and retain only necessary data. +* Define the minimum data required to support the AI model’s intended purpose before collecting any information. +* Collect only data elements that directly contribute to the model’s functionality, accuracy, or business requirements. +* Avoid collecting unnecessary personal, confidential, or sensitive information that does not improve model performance. +* Limit data ingestion from external sources to approved and relevant datasets. +* Establish data collection requirements based on business objectives, privacy requirements, and security policies. +* Remove unnecessary fields from datasets before processing or training AI models. +* Filter irrelevant, duplicate, outdated, or low-quality records before they enter AI pipelines. +* Use data sampling techniques when full datasets are unnecessary for training, testing, or analytics. +* Aggregate data where possible to reduce exposure of individual records while preserving analytical value. +* Anonymize or pseudonymize data when individual-level information is not required. +* Separate identifying information from operational or training data whenever possible. +* Avoid using production data directly in development or testing environments unless explicitly approved. +* Create synthetic test data when realistic data is unnecessary for software development or validation. +* Restrict access to raw datasets and provide applications with only the minimum required data. +* Implement API responses that return only required fields rather than exposing entire records. +* Apply field-level filtering to prevent unnecessary data from flowing through applications and AI pipelines. +* Limit the amount of historical data retained for model training unless older data provides measurable value. +* Define data retention periods based on business needs, regulatory requirements, and security considerations. +* Automatically delete expired datasets, temporary files, cached data, and intermediate processing artifacts. +* Remove unused datasets from storage locations, backups, and AI development environments. +* Configure lifecycle policies for cloud storage systems to automatically archive or delete data according to retention rules. +* Document the purpose, ownership, retention period, and approved usage of each dataset. +* Review datasets periodically to identify unnecessary data that can be removed. +* Monitor data usage to detect excessive collection, unauthorized processing, or unexpected data growth. +* Prevent AI systems from collecting additional data through unnecessary logging, telemetry, or user tracking. +* Minimize the information included in prompts, context windows, and retrieval-augmented generation (RAG) pipelines. +* Restrict model training pipelines from accessing datasets beyond their authorized scope. +* Implement data governance controls that enforce collection, processing, and retention policies automatically. +* Audit data flows to verify that only approved information is collected, processed, stored, and retained. +* Integrate data minimization checks into CI/CD and ML lifecycle workflows to prevent excessive data usage before deployment. + + ### Maintain version control for datasets. Integrate tools such as [Git-LFS](https://git-lfs.com), [DVC](https://dvc.org), [LakeFS](https://lakefs.io), or [MLflow](https://mlflow.org) into development workflows. From 65871d07f6510f34c65ecd88c9b61f486c4adf2c Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:53:01 -0400 Subject: [PATCH 12/18] Enhance section on dataset version control and security Added recommendations for integrating dataset version control tools and managing large files with Git-LFS. --- .../15.03 - Supplementary Reading Materials.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 2fbaf41..56a9ce2 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -110,6 +110,11 @@ Design applications and pipelines to collect and retain only necessary data. Integrate tools such as [Git-LFS](https://git-lfs.com), [DVC](https://dvc.org), [LakeFS](https://lakefs.io), or [MLflow](https://mlflow.org) into development workflows. +* Integrate dataset version control tools such as Git-LFS, DVC, LakeFS, or MLflow into AI development workflows to track, manage, and reproduce dataset changes. +* Store dataset metadata alongside source code repositories, including dataset name, version identifier, creation date, data owner, source location, hash values and processing history +* Use Git-LFS to manage large dataset files that exceed traditional Git storage limits by storing file references in Git while keeping large objects in dedicated storage. +* Configure Git-LFS tracking rules for large datasets, models, and binary artifacts. + ### Secure third-party datasets before use. Verify licenses, integrity, and approved sources before integrating external datasets. From 20914ab08bcd5d1e73d0ae0d0c98770d89c4cc90 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 15:58:32 -0400 Subject: [PATCH 13/18] Enhance security guidelines for third-party datasets Added detailed guidelines for verifying and securing third-party datasets before integration into AI workflows. --- ...15.03 - Supplementary Reading Materials.md | 21 +++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 56a9ce2..c8399fc 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -119,6 +119,27 @@ Integrate tools such as [Git-LFS](https://git-lfs.com), [DVC](https://dvc.org), Verify licenses, integrity, and approved sources before integrating external datasets. +* Verify the source and reputation of third-party datasets before integrating them into AI development workflows. +* Confirm dataset ownership, licensing terms, and permitted usage rights before downloading or using external data. +* Download third-party datasets only from trusted repositories, approved vendors, or verified publishers. +* Verify dataset integrity by comparing cryptographic hashes, checksums, or digital signatures provided by the dataset publisher. +* Validate digital signatures when available to confirm that the dataset originated from the claimed provider and has not been modified. +* Scan third-party datasets for malicious content before importing them into enterprise environments. +* Confirm that third-party datasets comply with organizational security, privacy, and regulatory requirements. +* Remove unnecessary sensitive information from third-party datasets before allowing them into AI pipelines. +* Apply data classification rules to determine whether third-party datasets are appropriate for enterprise use. +* Isolate third-party datasets in controlled environments before integrating them into production workflows. +* Record the exact third-party dataset version used for every AI model training run. +* Track dataset changes over time to identify unexpected modifications between releases. +* Test third-party datasets in development and validation environments before production use. +* Review vendor or community security practices before relying on externally maintained datasets. +* Avoid automatically pulling updated third-party datasets into production pipelines without validation. +* Implement approval workflows requiring security or data governance review before introducing new external datasets. +* Automate third-party dataset security checks within CI/CD and MLOps pipelines. +* Maintain an inventory of approved third-party datasets and their associated risks. +* Retire third-party datasets that become unsupported, insecure, outdated, or non-compliant. +* Treat third-party datasets as external software supply chain components by applying the same verification, tracking, and governance practices used for open-source dependencies. + ### Audit dataset access and usage. Build logging and audit trails into applications and services. From 2c2c30aca250d7160832fcd4614b386ec52f3f6f Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Fri, 24 Jul 2026 16:08:39 -0400 Subject: [PATCH 14/18] Enhance security guidelines for dataset management Added detailed guidelines for auditing dataset access and securely managing dataset retention and deletion. --- ...15.03 - Supplementary Reading Materials.md | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index c8399fc..ac5a08e 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -144,10 +144,36 @@ Verify licenses, integrity, and approved sources before integrating external dat Build logging and audit trails into applications and services. +* Retain dataset access logs according to organizational, regulatory, and compliance requirements. +* Automate alerts when unauthorized or high-risk dataset activities occur. +* Integrate audit controls into data pipelines, MLOps platforms, and AI development workflows. +* Include dataset access monitoring in CI/CD and model deployment processes to ensure security controls remain active throughout the AI lifecycle. +* Review third-party dataset usage logs to confirm that external data sources are accessed only as authorized. +* Document dataset access policies, ownership, approved use cases, and review procedures. +* Perform periodic access reviews to remove unnecessary permissions and outdated accounts. +* Validate that deployed AI applications access only the datasets required for their intended functions. +* Maintain a complete audit trail that supports incident response, compliance reviews, and forensic investigations. + ### Retain and securely delete datasets according to organizational retention policies. Implement lifecycle management, deletion APIs, and retention logic in software. +* Define dataset retention requirements based on organizational policies, regulatory obligations, contractual requirements, and business needs. +* Assign ownership responsibilities for managing dataset lifecycle activities, including retention reviews and secure deletion approvals. +* Store datasets only in approved enterprise repositories that support access controls, encryption, auditing, and lifecycle management. +* Encrypt retained datasets at rest to protect stored information throughout the retention period. +* Apply automated lifecycle management policies to archive or delete datasets when retention periods expire. +* Separate active training datasets from archived or retired datasets to reduce unnecessary exposure. +* Revoke access permissions and credentials associated with datasets that are retired or deleted. +* Remove dataset references from AI pipelines, training configurations, and development environments after retirement. +* Validate that downstream AI models and applications no longer depend on datasets scheduled for deletion. +* Maintain audit logs for dataset retention and deletion activities to support compliance reviews and incident investigations. +* Automate retention policy enforcement within data platforms, MLOps workflows, and CI/CD pipelines. +* Implement deletion safeguards that require approval before removing critical datasets. +* Minimize retention of sensitive information by removing data that no longer provides operational or model-training value. +* Securely dispose of third-party datasets according to provider requirements, contractual terms, and licensing obligations. +* Test data deletion procedures regularly to confirm that organizational retention and privacy requirements are being enforced. +* Integrate dataset retention and deletion controls into the AI lifecycle to ensure data remains governed from collection through retirement. You can find further guidance on this subject in [Chapter 1 (Training Data Integrity & Traceability) of the OWASP Artificial Intelligence Security Verification Standard (AISVS) project](https://github.com/OWASP/AISVS/blob/main/1.0/en/0x10-C01-Training-Data-Integrity-and-Traceability.md), which is a catalogue of testable security requirements for AI-enabled systems that may be useful for developers, as well as architects, security engineers and auditors. From 653ab8237348174109faaa9848870b1106e08058 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Mon, 27 Jul 2026 13:37:36 -0400 Subject: [PATCH 15/18] Enhance supplementary reading with input validation section Added section on input validation for AI models, including resources for LLM input sanitization and traditional security guidance. --- .../15.03 - Supplementary Reading Materials.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index ac5a08e..40f564d 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -4,7 +4,12 @@ These materials further examine the topics that a developer needs to know about ## Securing Models +### Input Validation +[LLM Input Sanitizer](https://pypi.org/project/llm-input-sanitizer), a Python library that offers LLM-specific features like PII detection and masking, profanity filtering, input truncation, unicode normalization, prompt injection defense and jailbreak prevention. + +More traditional guidance that is not specific to AI can also be applied to large language models. For example, OWASP Annotated Application Security Verification Standard +includes chapter on Sanitization and Sandboxing Requirements](https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding), which covers apply to AI models, in addition to other types of applications. ## Securing Data Sets From bb64e3669f28501922c0abbef260c448f75a7074 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Mon, 27 Jul 2026 13:38:19 -0400 Subject: [PATCH 16/18] Fix link formatting and improve text clarity Corrected the formatting of a link and improved clarity in the text regarding OWASP standards. --- .../15.03 - Supplementary Reading Materials.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 40f564d..80c1697 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -8,8 +8,7 @@ These materials further examine the topics that a developer needs to know about [LLM Input Sanitizer](https://pypi.org/project/llm-input-sanitizer), a Python library that offers LLM-specific features like PII detection and masking, profanity filtering, input truncation, unicode normalization, prompt injection defense and jailbreak prevention. -More traditional guidance that is not specific to AI can also be applied to large language models. For example, OWASP Annotated Application Security Verification Standard -includes chapter on Sanitization and Sandboxing Requirements](https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding), which covers apply to AI models, in addition to other types of applications. +More traditional guidance that is not specific to AI can also be applied to large language models. For example, OWASP Annotated Application Security Verification Standard includes a [chapter on Sanitization and Sandboxing Requirements](https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding), which covers apply to AI models, in addition to other types of applications. ## Securing Data Sets From 494aaee82e4239a03cc8e7977b890f07ce0e10d0 Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Mon, 27 Jul 2026 13:46:29 -0400 Subject: [PATCH 17/18] Enhance supplementary reading with prompt injection section Added section on prompt injection and related resources. --- .../15.03 - Supplementary Reading Materials.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index 80c1697..f55236f 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -4,6 +4,10 @@ These materials further examine the topics that a developer needs to know about ## Securing Models +### Prompt Injection + +The [OWASP LLM Prompt Injection Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html) provides guidance on prevention techniques against prompt injection attacks. + ### Input Validation [LLM Input Sanitizer](https://pypi.org/project/llm-input-sanitizer), a Python library that offers LLM-specific features like PII detection and masking, profanity filtering, input truncation, unicode normalization, prompt injection defense and jailbreak prevention. From 2cbcc39cf262f7bfb06765a7f0d7dc4eb7ba69cb Mon Sep 17 00:00:00 2001 From: Zoe Braiterman Date: Thu, 30 Jul 2026 09:39:15 -0400 Subject: [PATCH 18/18] Add Output Data Control section to supplementary materials Added a section on Output Data Control, highlighting its importance in AI security and referencing OWASP LLM Top 10. --- .../15.03 - Supplementary Reading Materials.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md index f55236f..ef54431 100644 --- a/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md +++ b/2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md @@ -14,6 +14,9 @@ The [OWASP LLM Prompt Injection Prevention Cheat Sheet](https://cheatsheetseries More traditional guidance that is not specific to AI can also be applied to large language models. For example, OWASP Annotated Application Security Verification Standard includes a [chapter on Sanitization and Sandboxing Requirements](https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding), which covers apply to AI models, in addition to other types of applications. +### Output Data Control + +Improper Output Handling is [number 05 of the OWASP LLM Top 10](https://genai.owasp.org/llmrisk/llm052025-improper-output-handling). This section of the OWASP GenAI Security Project includes common examples of the vulnerability, prevention and mitigation strategies, example attack scenarios and references to other relevant resources. ## Securing Data Sets