GDB doesn't include 'dap' (debug adapter protocol) interpreter

[JamesC01]

I'm trying to set up a debug adapter in neovim with gdb from w64devkit, and it requires GDB to have the 'dap' interpreter. I searched and it seems that for gdb to support it, it needs to be built with python support:

dap
When GDB has been built with Python support, it also supports the Debugger Adapter Protocol. This protocol can be used by a debugger GUI or an IDE to communicate with GDB. This protocol is documented at https://microsoft.github.io/debug-adapter-protocol/. See Debugger Adapter Protocol, for information about GDB extensions to the protocol.

Having it there allows you to debug code within different editors as a frontend to gdb.

Would you consider building gdb with python support by default to fix this? It would be useful for people wanting to use frontend's for GDB on windows. I understand if you don't want to, though, since mingw is much more focused on C/C++ rather than python, and maybe it bloats the installation?

[skeeto]

CPython doesn't support Mingw-w64, so it doesn't support cross-compilation and cannot be built with w64dk. At best I could download binaries built on Windows using MSVC, then somehow rig it up to the GDB cross build. If only the GDB devs had chosen a more suitable extension language like Lua, which wouldn't have this issue. You're also right: Even if CPython could be cross-compiled, I wouldn't be thrilled about shipping an entire Python environment in w64dk. It wouldn't just be some static library I could link into GDB (e.g. Lua, which would have been far more sensible). A minimal CPython installation on Windows is currently ~5,200 loose files. That is, it would about double the number of files currently in w64dk. Just for a trifling extension language. (CPython is such a mediocre programming language implementation. Ugh.) A good middle ground would be if I could enable Python support in GDB, and if it can't find a Python environment on start then it runs without just as it currently does. Then users like you could supply a compatible Python environment to get the extra features. As far as I'm aware, GDB does not support anything like this right now.

[Peter0x44]

I got around this by replacing w64devkit's GDB with:
https://github.com/ssbssa/gdb/releases/

Which does ship an entire CPython environment. Unfortunately I don't think much can be done on w64devkit's end.

[JamesC01]

CPython doesn't support Mingw-w64, so it doesn't support cross-compilation
and cannot be built with w64dk. At best I could download binaries built on
Windows using MSVC, then somehow rig it up to the GDB cross build. If only
the GDB devs had chosen a more suitable extension language like Lua, which
wouldn't have this issue.

You're also right: Even if CPython could be cross-compiled, I wouldn't be
thrilled about shipping an entire Python environment in w64dk. It wouldn't
just be some static library I could link into GDB (e.g. Lua, which would
have been far more sensible). A minimal CPython installation on Windows is
currently ~5,200 loose files. That is, it would about double the number of
files currently in w64dk. Just for a trifling extension language.

(CPython is such a mediocre programming language implementation. Ugh.)

A good middle ground would be if I could enable Python support in GDB, and
if it can't find a Python environment on start then it runs without just
as it currently does. Then users like you could supply a compatible Python
environment to get the extra features. As far as I'm aware, GDB does not
support anything like this right now.

I wasn't actually intending on using gdb with python, but with C and C++. It just seems like you can't have the dap support without compiling it with python enabled. edit: never mind, I just realised that gdb has a python API, not debugging for python itself. I understand not wanting to include python. I've found a separate debug adapter (cppdgb) which works, so I'm using that for now. I can compile with the tools from w64devkit, and use cppdbg for the debugging. I still think it would be nice to have the option to be able to use gdb as the debug adapter, though, if you can figure out any workarounds.

[Peter0x44]

Python is also required for libstdc++ pretty printers, needed to print something useful from an std::vector or std::string

[skeeto]

It just seems like you can't have the dap support without compiling it with python enabled.

That's because DAP is implemented in Python. Since CPython can't compile Python, you need a full Python language implementation on hand in order to run it. Same goes for some other features like, as @Peter0x44 pointed out, C++ pretty printing.

https://github.com/ssbssa/gdb

Interesting. Do you know anything about how this is built? We have the forked GDB source, but I don't see information on how releases are constructed.

Digging into it, I see the Python-enabled version has a full Python 3.10 standard library. Aside from .pyd modules, no new EXEs nor DLLs. GDB goes from 13MB to 36MB, strongly suggesting that Python is statically linked. I can see new import table entries for CPython, but also new entries unaccounted for from either GDB nor Python (ex. SHLWAPI.dll!PathSkipRootW), and a bunch of unusually-decorated new msvcrt.dll imports(edit: double-checking, these are all the normal decorations). 23MB also seems too large to be just CPython. Is that a static CRT linked into it? I suspect it's miscompiled, with CPython compiled against UCRT (or some VS CRT), then mistakenly linked to msvcrt.dll when linked with GDB.

[Peter0x44]

@skeeto I don't know anything about how it's built, but I have python on my system regardless.

It does seem to work.

For the libstdc++ pretty printers, I did the following:
https://www.gonwan.com/2014/07/02/setting-up-pretty-printers-in-gdb/

Except obviously w64devkit doesn't ship these scripts, I took them manually from the gcc repo.
https://github.com/gcc-mirror/gcc/tree/master/libstdc%2B%2B-v3/python/libstdcxx

And now I can pretty-print libstdc++ containers fine.

dap also works.

There are CPython forks for MinGW-w64:

https://github.com/msys2-contrib/cpython-mingw

https://github.com/redpanda-cpp/cpython-mingw-lite

(mingw-lite is said to be inspired by w64devkit: https://github.com/redpanda-cpp/mingw-lite)

I used to run ldd on binaries installed in MSYS2 UCRT64. I have seen binaries that are linked with both msvcrt and ucrt. Most if not all binaries on MSYS2 UCRT64 are this way.

Here is the result of ldd on /ucrt64/bin/python:

$ ldd /ucrt64/bin/python
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffb1afd0000)
        KERNEL32.DLL => /c/Windows/system32/KERNEL32.DLL (0x7ffb18ba0000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7ffb18300000)
        libpython3.12.dll => /ucrt64/bin/libpython3.12.dll (0x7ffaf22c0000)
        api-ms-win-crt-environment-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-environment-l1-1-0.dll (0x7ffb13780000)
        api-ms-win-crt-heap-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-heap-l1-1-0.dll (0x7ffb145d0000)
        api-ms-win-crt-math-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-math-l1-1-0.dll (0x7ffb15580000)
        api-ms-win-crt-private-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-private-l1-1-0.dll (0x7ffb11740000)
        api-ms-win-crt-runtime-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-runtime-l1-1-0.dll (0x7ffb145f0000)
        api-ms-win-crt-stdio-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-stdio-l1-1-0.dll (0x7ffb145c0000)
        api-ms-win-crt-string-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-string-l1-1-0.dll (0x7ffb145e0000)
        ADVAPI32.dll => /c/Windows/system32/ADVAPI32.dll (0x7ffb187d0000)
        bcrypt.dll => /c/Windows/SYSTEM32/bcrypt.dll (0x7ffb17c40000)
        libgcc_s_seh-1.dll => /ucrt64/bin/libgcc_s_seh-1.dll (0x7ffb02b30000)
        api-ms-win-crt-conio-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-conio-l1-1-0.dll (0x7ffb11520000)
        api-ms-win-crt-convert-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-convert-l1-1-0.dll (0x7ffb139d0000)
        api-ms-win-crt-filesystem-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-filesystem-l1-1-0.dll (0x7ffb13990000)
        api-ms-win-crt-locale-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-locale-l1-1-0.dll (0x7ffb139c0000)
        api-ms-win-crt-process-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-process-l1-1-0.dll (0x7ffb11510000)
        api-ms-win-crt-time-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-time-l1-1-0.dll (0x7ffb139a0000)
        api-ms-win-crt-utility-l1-1-0.dll => /c/Windows/SYSTEM32/api-ms-win-crt-utility-l1-1-0.dll (0x7ffb139b0000)
        VERSION.dll => /c/Windows/SYSTEM32/VERSION.dll (0x7ffb10b50000)
        WS2_32.dll => /c/Windows/system32/WS2_32.dll (0x7ffb1a960000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7ffb18720000)
        sechost.dll => /c/Windows/SYSTEM32/sechost.dll (0x7ffb1a9c0000)
        RPCRT4.dll => /c/Windows/system32/RPCRT4.dll (0x7ffb1ab40000)
        libwinpthread-1.dll => /ucrt64/bin/libwinpthread-1.dll (0x7ffb0ca70000)
        NSI.dll => /c/Windows/system32/NSI.dll (0x7ffb1aaf0000)
        ucrtbase.DLL => /c/Windows/SYSTEM32/ucrtbase.DLL (0x7ffb13580000)

p/s: I'm on Linux now. This result of ldd was from the past. I found it to be interesting so I saved it into a note.

More interesting ldd output from other notes:

$ ldd /c/Windows/SYSTEM32/ucrtbase.DLL
        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffd5e620000)
        KERNEL32.DLL => /c/Windows/system32/KERNEL32.DLL (0x7ffd5ceb0000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7ffd5bb80000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7ffd5c650000)

$ ldd /c/Windows/SYSTEM32/api-ms-*.dll

The result of this command is always:

        ntdll.dll => /c/Windows/SYSTEM32/ntdll.dll (0x7ffd5e620000)
        KERNEL32.DLL => /c/Windows/system32/KERNEL32.DLL (0x7ffd5ceb0000)
        KERNELBASE.dll => /c/Windows/system32/KERNELBASE.dll (0x7ffd5bb80000)
        msvcrt.dll => /c/Windows/system32/msvcrt.dll (0x7ffd5c650000)

[skeeto]

On ELF systems (Linux, BSD), linking multiple libc into a single process cannot work correctly. You quietly get a broken process. On PE systems (Windows), linking multiple, different CRTs into one process is supported, sound, and happens all the time during normal operations. So it's not unusual to see both ucrt and msvcrt both linked into a process. However, a particular PE image can only reliably reference exactly one CRT. Theoretically referencing multiple CRTs can be made to work, but the tooling isn't designed for it and makes it challenging. This rule applies regardless of whether CRTs are statically or dynamically linked. When multiple CRTs are linked into a process, it's because one image references one CRT, another image reference a difference CRT, so they both get linked into the process. That ldd listing suggests that this python.exe image references both ucrt and msvcrt at once, but that's just due to misleading output. I checked it myself, because I'd be surprised if MSYS2 was miscompiling something so important. The listing is flattened: ADVAPI32.dll pulls in msvcrt.dll, so its indirect, and so everything is fine.

[N-R-K]

e.g. Lua, which would have been far more sensible

One big issue with lua is that the official language/interpreter does not value backwards compatibility. mpv for example is running into this issue where they cannot upgrade to lua 5.3 or newer since it can break many existing user scripts. But v5.2/v5.1 is no longer officially supported and so on many distros luajit is now the only viable interpreter to build mpv against.

I wonder if javascript (ugh) might be better in this regard or not. There are a couple minimal, embeddable, written in C js interpreters such as duktape, mujs and Bellard's quickjs.

There are also niche ones like Tcl but they suffer from being niche and thus people not wanting to write plugins as they'd need to learn a new language. I suspect that may have been a reason why gdb went with python (but js would've also avoided the niche issue as well).