Dotentv & scripting

Architecture.md

@@ -0,0 +1,211 @@
+# BookGen Architecture
+
+## 1. Introduction and Goals
+
+BookGen is a command-line toolchain for generating books and documentation from Markdown sources.
+
+Primary goals:
+- Provide reproducible builds for multiple output formats (web, epub, feed, print, WordPress export).
+- Keep command handling extensible and testable.
+- Support plugin-based custom builds through a stable API contract (`BookGen.Api`, netstandard2.1).
+- Offer shell and script automation support for author workflows.
+
+## 2. Architecture Constraints
+
+- Core runtime targets `.NET 11` (`net11.0`) for executables and main libraries.
+- Plugin API targets `.NET Standard 2.1` for compatibility (`BookGen.Api`).
+- CLI-first architecture with dependency injection (`Microsoft.Extensions.DependencyInjection`).
+- Rendering and pipeline logic is in reusable libraries, not in program entry points.
+- Plugins are loaded dynamically and isolated via `AssemblyLoadContext`.
+
+## 3. System Scope and Context
+
+### 3.1 Business Context
+Book authors and documentation teams use BookGen to transform Markdown + configuration into publishable outputs.
+
+```nomnoml
+[Author/CI/CD] -> [BookGen CLI]
+[BookGen CLI] -> [Book Sources\n(config.json, toc.json, md files)]
+[BookGen CLI] -> [Generated Outputs\n(web, epub, feed, print, wordpress)]
+[BookGen CLI] -> [Plugin Package (.plugin/.dll)]
+```
+
+### 3.2 Technical Context
+
+```nomnoml
+[BookGen (Exe)] -> [BookGen.Cli]
+[BookGen (Exe)] -> [BookGen.Lib]
+[BookGen (Exe)] -> [BookGen.Vfs]
+[BookGen (Exe)] -> [BookGen.Api]
+[BookGen (Exe)] -> [BookGen.Shell.Shared]
+[BookGen.Shellprog (Exe)] -> [BookGen.Cli]
+[BookGen.Shellprog (Exe)] -> [BookGen.Shell.Shared]
+[BookGen.Lib] -> [BookGen.Vfs]
+[BookGen.SamplePlugin] -> [BookGen.Api]
+[Test\nBookgen.Tests] -> [BookGen + Lib + Cli + Shell.Shared]
+```
+
+## 4. Solution Strategy
+
+- **Command orchestration layer**: `BookGen` + `BookGen.Cli` provide command discovery, parsing, validation, and execution.
+- **Domain and build pipelines**: `BookGen.Lib` encapsulates environment initialization, config validation/upgrades, markdown rendering, and multi-step output pipelines.
+- **I/O abstraction**: `BookGen.Vfs` provides scoped filesystem interfaces (`IReadOnlyFileSystem`, `IWritableFileSystem`) used across commands and pipelines.
+- **Extensibility**: `BookGen.Api` defines plugin contracts; `BookGen.Infrastructure.Plugins` loads and executes plugin implementations.
+- **Shell integration**: `BookGen.Shellprog` and `BookGen.Shell.Shared` support interactive and shell-oriented workflows.
+
+## 5. Building Block View
+
+### 5.1 Level 1
+```nomnoml
+[BookGen Runtime|
+- CLI command execution
+- Build command entry points
+- DI wiring]
+[Library Core|
+- BookEnvironment
+- Pipeline steps
+- Rendering]
+[Plugin Surface|
+- BookGen.Api contracts
+- Plugin loader]
+[Infrastructure Support|
+- VFS
+- Shell helpers
+- Packaged assets]
+
+[BookGen Runtime] -> [Library Core]
+[BookGen Runtime] -> [Plugin Surface]
+[BookGen Runtime] -> [Infrastructure Support]
+```
+
+### 5.2 Level 2 (Main executable internals)
+```nomnoml
+[Program.cs]
+[CommandRunner]
+[BuildCommandBase + Commands]
+[BookEnvironment]
+[Pipeline]
+[PluginRunner]
+
+[Program.cs] -> [CommandRunner]
+[CommandRunner] -> [BuildCommandBase + Commands]
+[BuildCommandBase + Commands] -> [BookEnvironment]
+[BuildCommandBase + Commands] -> [Pipeline]
+[BuildCommandBase + Commands] -> [PluginRunner]
+```
+
+### 5.3 Key Project Responsibilities
+
+- `Source/BookGen`: main executable, command implementations, DI composition, plugin invocation.
+- `Source/BookGen.Cli`: command framework (`CommandRunner`, command tree, global options, validation).
+- `Source/BookGen.Lib`: core domain model, config migration/validation, rendering, pipelines, preview HTTP support.
+- `Source/BookGen.Vfs`: file system abstraction with scoped access.
+- `Source/BookGen.Api`: stable plugin contracts for external builders.
+- `Source/BookGen.Shellprog`: shell helper executable.
+- `Source/BookGen.Shell.Shared`: shared shell/logging/browser/process helpers.
+- `Source/BookGen.Contents`: distributable bundled content and tool assets.
+- `Test/Bookgen.Tests`: NUnit test suite across CLI and command behavior.
+
+## 6. Runtime View
+
+### 6.1 Scenario: `build web`
+
+```nomnoml
+[User] -> [BookGen Program]
+[BookGen Program] -> [CommandRunner]
+[CommandRunner] -> [BuildWebCommand]
+[BuildWebCommand] -> [BookEnvironment.Initialize]
+[BuildWebCommand] -> [Pipeline.CreateWebPipeLine]
+[Pipeline] -> [Steps: CopyAssets -> Render -> Index -> Pager]
+[Pipeline] -> [Output Folder]
+```
+
+Flow summary:
+1. `Program.cs` configures logging and services, then runs `CommandRunner`.
+2. `BuildWebCommand` (via `BuildCommandBase`) sets source/output scopes.
+3. `BookEnvironment.Initialize` validates config and TOC, applies optional overlay, acquires lock.
+4. Web pipeline executes ordered steps from `BookGen.Lib.Pipeline.StaticWebsite`.
+5. Generated files are written through `IWritableFileSystem`.
+
+### 6.2 Scenario: `build plugin`
+
+```nomnoml
+[User] -> [BuildPlugin Command]
+[BuildPlugin Command] -> [PluginPathResolver]
+[BuildPlugin Command] -> [BookEnvironment.Initialize]
+[BuildPlugin Command] -> [PluginRunner]
+[PluginRunner] -> [Extract .plugin + manifest]
+[PluginRunner] -> [AssemblyLoadContext]
+[AssemblyLoadContext] -> [IBuildPluginV1.Build]
+[IBuildPluginV1.Build] -> [Output Folder]
+```
+
+Flow summary:
+1. Plugin package path is resolved (`.plugin` or `.dll` in dev mode).
+2. Environment is initialized like regular builds.
+3. Plugin runner validates package manifest and extracts payload.
+4. Plugin assembly is loaded in collectible `AssemblyLoadContext`.
+5. Exactly one `IBuildPluginV1` implementation is instantiated and executed.
+6. Context is unloaded and GC-assisted cleanup is performed.
+
+## 7. Deployment View
+
+BookGen is primarily a local/CI process architecture.
+
+```nomnoml
+[Developer Workstation / CI Agent|
+- BookGen.exe
+- BookGen.Shellprog.exe
+- assets.zip / dictionaries.zip
+- optional plugins/*.plugin]
+
+[Developer Workstation / CI Agent] -> [File System Workspace|
+book sources + config + output]
+```
+
+Deployment characteristics:
+- No mandatory long-running backend service.
+- Outputs are static artifacts suitable for hosting elsewhere.
+- Optional preview/server capabilities are hosted in-process when used.
+
+## 8. Cross-cutting Concepts
+
+- **Dependency Injection**: service wiring in executable entry points.
+- **Logging**: configurable console/json/file logging with shared providers.
+- **Validation**: argument validation in command args + configuration schema/domain validation.
+- **Pipeline pattern**: deterministic ordered steps per output target.
+- **Filesystem abstraction**: commands and pipelines depend on VFS interfaces, not raw `System.IO` calls.
+- **Plugin isolation**: dynamic loading with unloadable contexts.
+- **Asset packaging**: content and dictionaries shipped as zipped artifacts.
+
+## 9. Architecture Decisions
+
+- Use separate projects to isolate concerns (CLI framework, domain/rendering, plugin API, VFS, shell helpers).
+- Keep plugin API in `netstandard2.1` to lower compatibility friction for plugin authors.
+- Use a step-based pipeline model for output generation to keep build stages explicit and composable.
+- Use command auto-discovery (`AddCommandsFrom`) plus explicit default command registration.
+- Use scoped filesystem wrappers to centralize path safety and testability.
+
+## 10. Quality Requirements
+
+Top quality goals:
+1. **Reliability**: deterministic command execution and clear exit codes.
+2. **Extensibility**: plugin contracts and build-command architecture.
+3. **Maintainability**: modular projects with focused responsibilities.
+4. **Usability**: strong CLI help, shell completion support, script command.
+5. **Portability**: cross-platform runtime support for core tooling and assets.
+
+## 11. Risks and Technical Debt
+
+- Dynamic plugin loading can fail due to missing dependencies or malformed manifests.
+- Pipeline failures abort generation; diagnostics quality depends on step-level logging.
+- External native/tool dependencies (rendering helpers, conversion tools) can vary by OS/runtime packaging.
+- Version drift between plugin implementations and API contracts must be managed carefully.
+
+## 12. Glossary
+
+- **BookEnvironment**: runtime context containing configuration, TOC, source/output scopes, and assets.
+- **Pipeline**: ordered set of build steps producing one output format.
+- **IBuildPluginV1**: plugin entry contract for custom build behavior.
+- **VFS**: virtual/scoped file system abstraction used by commands and pipelines.
+- **Shellprog**: helper executable for shell-oriented workflows and command interaction.

BookGen.slnx

@@ -1,9 +1,11 @@
 <Solution>
   <Folder Name="/Documents/">
+    <File Path="Architecture.md" />
     <File Path="Docs/Changelog.md" />
     <File Path="Docs/manual.md" />
     <File Path="Docs/plugin.md" />
     <File Path="Docs/readme.md" />
+    <File Path="Docs/scripts.md" />
   </Folder>
   <Folder Name="/Misc/" Id="209479ab-f6dc-4e64-8458-43e805966b28">
     <File Path=".editorconfig" />

Directory.Packages.props

@@ -4,32 +4,32 @@
     <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
   </PropertyGroup>
   <ItemGroup>
-    <PackageVersion Include="AngleSharp" Version="1.5.2" />
+    <PackageVersion Include="AngleSharp" Version="1.8.2" />
     <PackageVersion Include="BenchmarkDotNet" Version="0.15.8" />
     <PackageVersion Include="coverlet.collector" Version="10.0.1" />
     <PackageVersion Include="HtmlToOpenXml.dll" Version="3.5.0" />
-    <PackageVersion Include="Markdig" Version="1.3.2" />
-    <PackageVersion Include="Microsoft.ClearScript" Version="7.5.1" />
-    <PackageVersion Include="Microsoft.ClearScript.V8.Native.linux-x64" Version="7.5.1" />
-    <PackageVersion Include="Microsoft.ClearScript.V8.Native.win-x64" Version="7.5.1" />
-    <PackageVersion Include="Microsoft.Extensions.Caching.Memory" Version="11.0.0-preview.6.26359.118" />
-    <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="11.0.0-preview.6.26359.118" />
-    <PackageVersion Include="Microsoft.Extensions.Logging" Version="11.0.0-preview.6.26359.118" />
-    <PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="4.0.0-preview" />
-    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.8.1" />
-    <PackageVersion Include="Moq" Version="4.20.72" />
+    <PackageVersion Include="Markdig" Version="1.4.0" />
+    <PackageVersion Include="Microsoft.ClearScript" Version="7.5.1.1" />
+    <PackageVersion Include="Microsoft.ClearScript.V8.Native.linux-x64" Version="7.5.1.1" />
+    <PackageVersion Include="Microsoft.ClearScript.V8.Native.win-x64" Version="7.5.1.1" />
+    <PackageVersion Include="Microsoft.Extensions.Caching.Memory" Version="11.0.0-rc.1.26425.128" />
+    <PackageVersion Include="Microsoft.Extensions.DependencyInjection" Version="11.0.0-rc.1.26425.128" />
+    <PackageVersion Include="Microsoft.Extensions.Logging" Version="11.0.0-rc.1.26425.128" />
+    <PackageVersion Include="Microsoft.IO.RecyclableMemoryStream" Version="4.0.1-preview" />
+    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.10.1" />
+    <PackageVersion Include="Moq" Version="4.21.0" />
     <PackageVersion Include="NUnit" Version="4.6.1" />
-    <PackageVersion Include="NUnit.Analyzers" Version="4.14.0" />
-    <PackageVersion Include="NUnit3TestAdapter" Version="6.2.0" />
-    <PackageVersion Include="PreMailer.Net" Version="2.7.3" />
-    <PackageVersion Include="Roslynator.Analyzers" Version="4.15.0" />
-    <PackageVersion Include="SkiaSharp" Version="4.150.1" />
-    <PackageVersion Include="SkiaSharp.NativeAssets.Linux" Version="4.150.1" />
-    <PackageVersion Include="SkiaSharp.NativeAssets.Win32" Version="4.150.1" />
+    <PackageVersion Include="NUnit.Analyzers" Version="4.15.0" />
+    <PackageVersion Include="NUnit3TestAdapter" Version="6.3.0" />
+    <PackageVersion Include="PreMailer.Net" Version="2.7.4" />
+    <PackageVersion Include="Roslynator.Analyzers" Version="5.0.0" />
+    <PackageVersion Include="SkiaSharp" Version="4.152.1" />
+    <PackageVersion Include="SkiaSharp.NativeAssets.Linux" Version="4.152.1" />
+    <PackageVersion Include="SkiaSharp.NativeAssets.Win32" Version="4.152.1" />
     <PackageVersion Include="Spectre.Console" Version="0.57.2" />
     <PackageVersion Include="Spectre.Console.Analyzer" Version="1.0.0" />
-    <PackageVersion Include="Svg.Skia" Version="5.1.1" />
-    <PackageVersion Include="System.ServiceModel.Syndication" Version="11.0.0-preview.6.26359.118" />
+    <PackageVersion Include="Svg.Skia" Version="5.2.3" />
+    <PackageVersion Include="System.ServiceModel.Syndication" Version="11.0.0-rc.1.26425.128" />
     <PackageVersion Include="Webmaster442.WindowsTerminal" Version="4.1.1" />
     <PackageVersion Include="WeCantSpell.Hunspell" Version="7.0.1" />
     <PackageVersion Include="XmlDocMarkdown.Core" Version="2.9.0" />

Docs/bookgen.toc.json

@@ -7,7 +7,8 @@
       "Files": [
         "manual.md",
         "changelog.md",
-        "plugin.md"
+        "plugin.md",
+		"scripts.md"
       ]
     }
   ]

Docs/changelog.md

@@ -5,6 +5,9 @@ tags: ''
 
 # 2026. 08.
 
+* Breaking: Removed json args support for commands
+* Breaking: Removed config command. Configuration is now loaded from the `BookGen.env` file or from a configuration specified by the `-env` option
+* Breaking: Removed jsonargs command.
 * New: Md2html command output file now can be a folder. If a folder is specified, the output file name is generated from the input file name. 
 * Fix: Shell atuocomplete now correctly handles command tree
 * Fix: Fixed md2html external template file validation

Docs/scripts.md

@@ -0,0 +1,86 @@
+# Scripting
+
+Scripting in BookGen lets you automate common workflows for your book - building outputs, generating or transforming content.
+You can run scripts from your shell (PowerShell, Bash, etc.) that invoke BookGen commands, or you can define and run scripts
+directly inside BookGen.
+
+The advantage of scripting inside BookGen is performance: BookGen starts once and executes multiple script steps in the
+same process, avoiding repeated startup overhead. Use shell scripts when you need OS integration or CI ties;
+use BookGen scripts when you want faster, BookGen-focused automation.
+
+## Execution model
+
+BookGen scripts follow a bash `pipefail` style execution model. Commands run sequentially, and the script continues only
+as long as each command succeeds. If any single command fails, execution stops immediately and the whole script is
+considered failed - the remaining commands are not run. This fail-fast behavior ensures that a broken step never lets the
+rest of the script proceed on invalid state, making script results reliable and easy to reason about.
+
+When a script fails, its exit code is the exit code of the last command that was executed - that is, the command that
+failed and caused execution to stop. This lets you inspect the script's exit code from your shell or CI pipeline to
+determine exactly which step went wrong and to react accordingly.
+
+## Syntax
+
+Beside the execution model described above, BookGen scripts follow the usual bash conventions.
+Long arguments that contain spaces must be escaped by wrapping them in quotes. For example:
+
+```
+build --output "./my output folder"
+```
+
+Without the quotes the value would be split into multiple arguments and the command would not receive the intended input.
+
+Each line should contain only a single command together with its arguments. For readability, however, a command can be
+split across multiple lines by ending a line with the `\` character. When a line ends with `\`, the following line is
+treated as a continuation of the same command. For example:
+
+```
+build \
+    --output "./my output folder" \
+    --verbose
+```
+
+The example above is equivalent to writing the whole `build` command on one line.
+
+## Comments
+
+BookGen scripts support two comment styles, and only these two:
+
+- Shell-style comments that begin with a `#` mark.
+- C++-style line comments that begin with `//`.
+
+Everything from the comment marker to the end of the line is ignored during execution. Both styles are line comments only -
+there is no block or multi-line comment syntax. You can use either style (or mix them) to document your scripts.
+
+```
+# This is a comment
+build --output ./out   # trailing comment after a command
+
+// This is also a comment
+clean                  // trailing comment using C++ style
+```
+
+### Adding extra logging
+
+In addition to regular comments, BookGen scripts let you inject your own messages into the log output while a script is
+running. To emit a custom log message, start a line with the special `#log` or `//log` instruction, followed by the text
+you want to display:
+
+```
+#log Starting the build step
+build --output ./out
+
+//log Build finished, cleaning up
+clean
+```
+
+These log instructions must appear at the very start of the line. If `#log` or `//log` is not at the beginning of the line
+(for example when used as a trailing comment after a command), it is treated as an ordinary comment and its text is ignored
+during execution instead of being written to the log:
+
+```
+build --output ./out   #log this is NOT logged, it's just a regular trailing comment
+```
+
+Use these log instructions to make your scripts easier to follow, marking progress and highlighting important steps in the
+runtime output.