Allow saved applications to disable runtime option processing

[luciusmagn]

Hi, I would like to propose a solution to the issue that the lisp kernel invariably eats some arguments before -- which may be used by other applications.

To describe it better, CCL’s kernel removes recognized runtime options before a saved application’s Lisp toplevel runs, which means that application flags such as --batch, --debug, -R, and -S can disappear or be misinterpreted. Saved applications with an explicit toplevel also do not receive arguments following -- through *UNPROCESSED-COMMAND-LINE-ARGUMENTS*.

I recognize that there is a lot of prior art and discussion to this topic (it is possible I missed some, so this list is probably not exhaustive and there may have been some things outside of github also):

• the major discussion is CCL:*UNPROCESSED-COMMAND-LINE-ARGUMENTS* is NIL when running an image #177, @sjl identified that the issue is with the kernel, and I don't think the other hacks are good solutions
• UIOP and an application class were suggested, but then people explain why these can't recover arguments already removed by the kernel (CCL:*UNPROCESSED-COMMAND-LINE-ARGUMENTS* is NIL when running an image #177 (comment)).
• wrapper using -- was proposed, but a saved toplevel image still sees UNPROCESSED-COMMAND-LINE-ARGUMENTS as NIL (CCL:*UNPROCESSED-COMMAND-LINE-ARGUMENTS* is NIL when running an image #177 (comment)).
• unix-opts issue project visual identity #14 (unix-opts seems to be losing a few options libre-man/unix-opts#14) has application options disappearing (--debug is particularly problematic)
• pgloader issue #719 (CCL build: -S means stack-size dimitri/pgloader#719) shows an -S collision between an aplication and CCL
• I looked at SBCL’s :save-runtime-options (https://www.sbcl.org/manual/#Function-sb_002dext-save_002dlisp_002dand_002ddie) for precedent

This issue is particularly important for me, because I am developing CCLSH, which should support an open-ended argument vocab since it can be used to call arbitrary scripts, and these conflicts could bite me, or somebody else, in the ass, hypothetical scenario:

cclsh script.sh.lisp --opt1 -x -R report -S serve

Especially since hypothetically, the options that concern the kernel could change in the future and scripts that work before may not work anymore.

Solution

My solution adds an optional :PROCESS-RUNTIME-OPTIONS option to CCL:SAVE-APPLICATION, which defaults to the current behavior when omitted, so that we don't affect prior software.

When set to NIL, this policy is recorded in the saved image. For a embedded or default marked image, the kernel detects it before option scanning and leaves runtime-looking arguments untouched. When a marked image is selected through -I, runtime options before it can still configure the kernel, while subsequent args belong to the application

I also initialize *UNPROCESSED-COMMAND-LINE-ARGUMENTS* for explicit toplevel functions, giving standalone programs a reliable&complete path to args without affecting existing images.

---

Unlike wrappers or UIOP or app-level parsing, this solution happens before the kernel irreversibly removes args. Recording the policy in the image makes it generic and opt-in, not dependent on executable names, and compatible with CCL images that already exist