Supplemental readings for Securing AI Data Sets

2.Curriculum/15.Security of and for AI/15.03 - Supplementary Reading Materials.md

@@ -1,19 +1,205 @@
 # Supplementary Reading Materials - Security of and for AI
+
 These materials further examine the topics that a developer needs to know about security of and for AI.
 
-In each section of this page, we'll also list books and other non-OWASP publications that  which have been found useful 
-in improving understanding. While not necessary to cover the material in the examination Body of Knowledge, you may 
-find them helpful as you build your understanding. By happy coincidence, many of the authors listed are also active in 
-the OWASP community. 
+## Securing Models
 
-Keep an eye out for cost-saving ways to obtain these titles, such as e-book bundling sites like 
-[Humble Bundle](https://www.humblebundle.com/), or at your local public library.
+### Prompt Injection
 
-## Securing Models
+The [OWASP LLM Prompt Injection Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/LLM_Prompt_Injection_Prevention_Cheat_Sheet.html) provides guidance on prevention techniques against prompt injection attacks. 
+
+### Input Validation
+
+[LLM Input Sanitizer](https://pypi.org/project/llm-input-sanitizer), a Python library that offers LLM-specific features like PII detection and masking, profanity filtering, input truncation, unicode normalization, prompt injection defense and jailbreak prevention.
+
+More traditional guidance that is not specific to AI can also be applied to large language models. For example, OWASP Annotated Application Security Verification Standard includes a [chapter on Sanitization and Sandboxing Requirements](https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding), which covers apply to AI models, in addition to other types of applications. 
+
+### Output Data Control
+
+Improper Output Handling is [number 05 of the OWASP LLM Top 10](https://genai.owasp.org/llmrisk/llm052025-improper-output-handling). This section of the OWASP GenAI Security Project includes common examples of the vulnerability, prevention and mitigation strategies, example attack scenarios and references to other relevant resources. 
+
+## Securing Data Sets
+
+Securing datasets used by AI models requires treating them as critical assets throughout their lifecycle, from collection and storage to training, deployment and retirement. A compromised dataset can introduce data poisoning, privacy violations, intellectual property risks, and unreliable model behavior.
+
+A secure developer should do the following to help achieve this. 
+
+### Verify the provenance of training data.
+
+Verify dataset sources, signatures, hashes, and approved repositories in code or CI/CD pipelines.
+
+* Maintain metadata describing the origin of the data.  Capture the dataset name and version, original source URL or repository, data owner or provider, collection date, license and usage restrictions, intended purpose and data classification level.
+* Only allow datasets from approved locations, such as Internal enterprise data platforms, such as approved cloud storage buckets, trusted open-source repositories and vendor-provided datasets with contractual agreements.
+* Avoid unknown file-sharing sites, unverified public datasets, datasets without ownership information and data scraped without legal review.
+
+### Verify the integrity of datasets.
+
+Perform checksum/signature verification before processing or training.
+
+A developer can verify the integrity of datasets by ensuring that the data has not been altered, corrupted, tampered with, or unintentionally modified from an approved state. Dataset integrity verification applies software supply chain security practices to data artifacts.
+Key developer actions include the following. 
+
+* Generate a cryptographic hash for a dataset and compare it against a trusted reference value. Common algorithms include SHA-256 and SHA-512.
+* Validate that the dataset was signed by an authorized source. Implement signature verification, public key validation, certificate trust checks and key expiration checks.
+
+### Encrypt data at rest and in transit.
+
+Enable encryption in storage services, databases, object stores, and APIs using enterprise standards.
+
+* Encrypt datasets at rest using strong encryption algorithms such as AES-256 to protect stored training data in databases, object storage, file systems, and data lakes.
+* Enable native encryption features provided by cloud platforms and storage systems, such as encrypted buckets, managed database encryption, and encrypted volumes.
+* Manage encryption keys securely using enterprise key management systems (KMS), hardware security modules (HSMs), or secrets management platforms.
+* Rotate encryption keys regularly according to organizational security policies and compliance requirements.
+* Restrict access to encryption keys using least-privilege permissions and role-based access control (RBAC).
+* Separate encryption keys from encrypted data to prevent unauthorized users from accessing both the dataset and its decryption capability.
+* Encrypt data backups, snapshots, replicas, and temporary copies to prevent exposure through secondary storage locations.
+* Use TLS 1.2 or higher (preferably TLS 1.3) to encrypt data transmitted between applications, APIs, databases, storage services, and AI training infrastructure.
+* Validate TLS certificates to ensure secure communication only occurs with trusted endpoints.
+* Disable insecure communication protocols and deprecated cryptographic algorithms such as TLS 1.0, TLS 1.1, weak ciphers, and unencrypted HTTP connections.
+* Protect API communications by requiring encrypted HTTPS connections for dataset access and AI service interactions.
+* Implement mutual TLS (mTLS) for high-trust environments where both clients and servers must authenticate each other.
+* Encrypt data transfers between cloud services, data pipelines, machine learning platforms, and storage systems.
+* Secure message queues, data streaming platforms, and ETL/ELT pipelines with encryption and authenticated connections.
+* Avoid transmitting sensitive datasets through unencrypted channels such as email, unsecured file transfers, or public storage links.
+* Apply field-level or column-level encryption for highly sensitive data elements such as personally identifiable information (PII), financial data, healthcare data, or proprietary business information.
+* Tokenize or pseudonymize sensitive data before storing or transmitting it when full data encryption is not required.
+* Prevent unauthorized decryption by enforcing authentication and authorization checks before granting access to encrypted datasets.
+* Log encryption-related events, including key usage, access attempts, and administrative changes, for auditing and incident response.
+* Monitor encryption configurations continuously to detect disabled encryption, exposed keys, or unauthorized changes.
+* Automate encryption checks in CI/CD and infrastructure-as-code pipelines to ensure new datasets and services meet enterprise security requirements.
+* Verify encryption compliance against organizational standards and frameworks such as NIST cryptographic guidance, CIS Benchmarks, and enterprise security policies.
+* Document encryption methods, key ownership, rotation schedules, and access controls as part of dataset governance and AI system documentation.
+
+### Remove sensitive information from datasets.
+
+Develop data sanitization, [Personally Identifiable Information (PII) masking](https://www.iso.org/information-security/data-masking), secret scanning, and preprocessing workflows.
+
+* Identify sensitive information within datasets before training or processing AI models, including personally identifiable information (PII), protected health information (PHI), payment card data, authentication credentials, API keys and secrets, confidential business information and intellectual property.
+* Classify dataset fields according to their sensitivity level to determine appropriate protection and removal requirements.
+* Scan datasets automatically using data discovery tools to detect sensitive content such as names, email addresses, phone numbers, addresses, government identifiers, financial account numbers, passwords and tokens.
+* Implement automated PII detection using techniques such as regular expressions, pattern matching, named entity recognition (NER) and machine learning-based classifiers.
+* Remove unnecessary sensitive data fields before using datasets for AI training.
+* Redact sensitive values by replacing them with placeholders.
+
+### Minimize data collection, processing, and retention.
+
+Design applications and pipelines to collect and retain only necessary data.
+
+* Define the minimum data required to support the AI model’s intended purpose before collecting any information.
+* Collect only data elements that directly contribute to the model’s functionality, accuracy, or business requirements.
+* Avoid collecting unnecessary personal, confidential, or sensitive information that does not improve model performance.
+* Limit data ingestion from external sources to approved and relevant datasets.
+* Establish data collection requirements based on business objectives, privacy requirements, and security policies.
+* Remove unnecessary fields from datasets before processing or training AI models.
+* Filter irrelevant, duplicate, outdated, or low-quality records before they enter AI pipelines.
+* Use data sampling techniques when full datasets are unnecessary for training, testing, or analytics.
+* Aggregate data where possible to reduce exposure of individual records while preserving analytical value.
+* Anonymize or pseudonymize data when individual-level information is not required.
+* Separate identifying information from operational or training data whenever possible.
+* Avoid using production data directly in development or testing environments unless explicitly approved.
+* Create synthetic test data when realistic data is unnecessary for software development or validation.
+* Restrict access to raw datasets and provide applications with only the minimum required data.
+* Implement API responses that return only required fields rather than exposing entire records.
+* Apply field-level filtering to prevent unnecessary data from flowing through applications and AI pipelines.
+* Limit the amount of historical data retained for model training unless older data provides measurable value.
+* Define data retention periods based on business needs, regulatory requirements, and security considerations.
+* Automatically delete expired datasets, temporary files, cached data, and intermediate processing artifacts.
+* Remove unused datasets from storage locations, backups, and AI development environments.
+* Configure lifecycle policies for cloud storage systems to automatically archive or delete data according to retention rules.
+* Document the purpose, ownership, retention period, and approved usage of each dataset.
+* Review datasets periodically to identify unnecessary data that can be removed.
+* Monitor data usage to detect excessive collection, unauthorized processing, or unexpected data growth.
+* Prevent AI systems from collecting additional data through unnecessary logging, telemetry, or user tracking.
+* Minimize the information included in prompts, context windows, and retrieval-augmented generation (RAG) pipelines.
+* Restrict model training pipelines from accessing datasets beyond their authorized scope.
+* Implement data governance controls that enforce collection, processing, and retention policies automatically.
+* Audit data flows to verify that only approved information is collected, processed, stored, and retained.
+* Integrate data minimization checks into CI/CD and ML lifecycle workflows to prevent excessive data usage before deployment.
+
+
+### Maintain version control for datasets.
+
+Integrate tools such as [Git-LFS](https://git-lfs.com), [DVC](https://dvc.org), [LakeFS](https://lakefs.io), or [MLflow](https://mlflow.org) into development workflows.
+
+* Integrate dataset version control tools such as Git-LFS, DVC, LakeFS, or MLflow into AI development workflows to track, manage, and reproduce dataset changes.
+* Store dataset metadata alongside source code repositories, including dataset name, version identifier, creation date, data owner, source location, hash values and processing history
+* Use Git-LFS to manage large dataset files that exceed traditional Git storage limits by storing file references in Git while keeping large objects in dedicated storage.
+* Configure Git-LFS tracking rules for large datasets, models, and binary artifacts.
+
+### Secure third-party datasets before use.
+
+Verify licenses, integrity, and approved sources before integrating external datasets.
+
+* Verify the source and reputation of third-party datasets before integrating them into AI development workflows.
+* Confirm dataset ownership, licensing terms, and permitted usage rights before downloading or using external data.
+* Download third-party datasets only from trusted repositories, approved vendors, or verified publishers.
+* Verify dataset integrity by comparing cryptographic hashes, checksums, or digital signatures provided by the dataset publisher.
+* Validate digital signatures when available to confirm that the dataset originated from the claimed provider and has not been modified.
+* Scan third-party datasets for malicious content before importing them into enterprise environments.
+* Confirm that third-party datasets comply with organizational security, privacy, and regulatory requirements.
+* Remove unnecessary sensitive information from third-party datasets before allowing them into AI pipelines.
+* Apply data classification rules to determine whether third-party datasets are appropriate for enterprise use.
+* Isolate third-party datasets in controlled environments before integrating them into production workflows.
+* Record the exact third-party dataset version used for every AI model training run.
+* Track dataset changes over time to identify unexpected modifications between releases.
+* Test third-party datasets in development and validation environments before production use.
+* Review vendor or community security practices before relying on externally maintained datasets.
+* Avoid automatically pulling updated third-party datasets into production pipelines without validation.
+* Implement approval workflows requiring security or data governance review before introducing new external datasets.
+* Automate third-party dataset security checks within CI/CD and MLOps pipelines.
+* Maintain an inventory of approved third-party datasets and their associated risks.
+* Retire third-party datasets that become unsupported, insecure, outdated, or non-compliant.
+* Treat third-party datasets as external software supply chain components by applying the same verification, tracking, and governance practices used for open-source dependencies.
+
+### Audit dataset access and usage.
+
+Build logging and audit trails into applications and services.
+
+* Retain dataset access logs according to organizational, regulatory, and compliance requirements.
+* Automate alerts when unauthorized or high-risk dataset activities occur.
+* Integrate audit controls into data pipelines, MLOps platforms, and AI development workflows.
+* Include dataset access monitoring in CI/CD and model deployment processes to ensure security controls remain active throughout the AI lifecycle.
+* Review third-party dataset usage logs to confirm that external data sources are accessed only as authorized.
+* Document dataset access policies, ownership, approved use cases, and review procedures.
+* Perform periodic access reviews to remove unnecessary permissions and outdated accounts.
+* Validate that deployed AI applications access only the datasets required for their intended functions.
+* Maintain a complete audit trail that supports incident response, compliance reviews, and forensic investigations.
+
+### Retain and securely delete datasets according to organizational retention policies.
+
+Implement lifecycle management, deletion APIs, and retention logic in software.
+
+* Define dataset retention requirements based on organizational policies, regulatory obligations, contractual requirements, and business needs.
+* Assign ownership responsibilities for managing dataset lifecycle activities, including retention reviews and secure deletion approvals.
+* Store datasets only in approved enterprise repositories that support access controls, encryption, auditing, and lifecycle management.
+* Encrypt retained datasets at rest to protect stored information throughout the retention period.
+* Apply automated lifecycle management policies to archive or delete datasets when retention periods expire.
+* Separate active training datasets from archived or retired datasets to reduce unnecessary exposure.
+* Revoke access permissions and credentials associated with datasets that are retired or deleted.
+* Remove dataset references from AI pipelines, training configurations, and development environments after retirement.
+* Validate that downstream AI models and applications no longer depend on datasets scheduled for deletion.
+* Maintain audit logs for dataset retention and deletion activities to support compliance reviews and incident investigations.
+* Automate retention policy enforcement within data platforms, MLOps workflows, and CI/CD pipelines.
+* Implement deletion safeguards that require approval before removing critical datasets.
+* Minimize retention of sensitive information by removing data that no longer provides operational or model-training value.
+* Securely dispose of third-party datasets according to provider requirements, contractual terms, and licensing obligations.
+* Test data deletion procedures regularly to confirm that organizational retention and privacy requirements are being enforced.
+* Integrate dataset retention and deletion controls into the AI lifecycle to ensure data remains governed from collection through retirement.
+
+You can find further guidance on this subject in [Chapter 1 (Training Data Integrity & Traceability) of the OWASP Artificial Intelligence Security Verification Standard (AISVS) project](https://github.com/OWASP/AISVS/blob/main/1.0/en/0x10-C01-Training-Data-Integrity-and-Traceability.md), which is a catalogue of testable security requirements for AI-enabled systems that may be useful for developers, as well as architects, security engineers and auditors.
+
+## Useful Books Related to Topics in This Module
+
+The following books and publications are useful for building a deeper understanding of the topics in this module.
+They are not required reading for the examination Body of Knowledge, but many learners find them helpful. 
+
+### Securing Models
 
 * Steve Wilson, [*The Developer's Playbook for Large Language Model Security*](https://www.oreilly.com/library/view/the-developers-playbook/9781098162191) (O'Reilly Media, Inc., 2024)
 * Harriet Farlow, [*Practical AI Security*](https://nostarch.com/practical-ai-security) (No Starch Press, 2026)
 
-## Securing Data Sets
+### Securing Data Sets
 
 * Anand Singh, [*Data Security in the Age of AI: A Guide to Protecting Data and Reducing Risk in an AI-Driven World*](https://www.amazon.com/Data-Security-Age-Protecting-AI-Driven-ebook/dp/B0FLVWDYJH) (Responsible Artificial Intelligence Institute, 2025)
+
+If you want lower-cost access to these titles, check library availability or e-book bundle sites such as [Humble Bundle](https://www.humblebundle.com/).