[ANE-3098] Read Go buildinfo from binaries during fossa analyze

Changelog.md

@@ -2,6 +2,7 @@
 
 ## 3.18.0
 
+- Go: `fossa analyze` can now report Go module dependencies read from the buildinfo embedded in compiled Go binaries (built with Go >= 1.18), so Go code shipped as a binary with no `go.mod` alongside it is no longer invisible to analysis. Opt in with `--enable-go-binary-analysis`; it is off by default because `fossa analyze` otherwise reports only what package managers declare. To reach binaries nested inside an archive (for example a `.so` inside an AAR or JAR), combine it with `--unpack-archives`.
 - Container scanning: `fossa container analyze` now reports Go module dependencies embedded in Go binaries (built with Go >= 1.18) found in container image layers as regular Go dependencies, supporting images without package manager metadata such as `scratch` and distroless images ([#1740](https://github.com/fossas/fossa-cli/pull/1740))
 - Bun: Dependencies reachable only through a `devDependencies` entry are now reported as development dependencies instead of production dependencies.
 - Analysis: JSON manifest files with a leading UTF-8 byte order mark (commonly written by Windows tooling, e.g. in NuGet `project.json`) no longer fail to parse.

docs/references/strategies/README.md

@@ -46,6 +46,7 @@ See the linked documentation above for details.
 ### go
 
 - [gomodules (`go mod`)](languages/golang/gomodules.md)
+- [gobinary (compiled Go binaries)](languages/golang/gobinary.md)
 - [dep](languages/golang/godep.md)
 - [glide](languages/golang/glide.md)
 
@@ -176,6 +177,7 @@ Invoke strict analysis with the `--strict` flag when running `fossa analyze`.
 | [Erlang (rebar3)](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/erlang/erlang.md)                        | Dynamic                      | ❌                    |
 | [Fortran](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/fortran/fortran.md)                              | Static                       | ❌                    |
 | [Go (dep)](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/golang/godep.md)                                | Static                       | ❌                    |
+| [Go (gobinary)](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/golang/gobinary.md)                        | Static                       | ❌                    |
 | [Go (glide)](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/golang/glide.md)                              | Static                       | ❌                    |
 | [Go (gomodules)](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/golang/gomodules.md)                      | Dynamic with static fallback | ❌                    |
 | [Gradle](https://github.com/fossas/fossa-cli/blob/master/docs/references/strategies/languages/gradle/gradle.md)                                 | Dynamic                      | ❌                    |

docs/references/strategies/languages/golang/gobinary.md

@@ -0,0 +1,98 @@
+# Go Binaries (buildinfo)
+
+Go binaries built with module support (Go >= 1.18) embed the list of every
+module linked into them. This is the same data `go version -m <binary>` prints.
+
+FOSSA CLI reads that list, so Go code shipped as a compiled binary is reported
+even when no `go.mod`, `go.sum`, or Go source is present next to it.
+
+This matters for artifacts such as:
+
+- a gomobile SDK shipping `jni/<abi>/lib<name>.so` inside an AAR
+- a Go binary vendored into a repository that is otherwise not a Go project
+- a Go binary packaged inside a JAR or other archive
+
+The embedded module list is generally more accurate than a hand-maintained
+third-party notice file, because the linker writes it from what was actually
+built into the binary.
+
+## Enabling
+
+This strategy is opt-in. `fossa analyze` otherwise reports only what package
+managers declare, and reading binaries would add dependencies to existing
+projects without the user asking for them.
+
+```bash
+fossa analyze --enable-go-binary-analysis
+```
+
+To reach a binary nested inside an archive, pass `--unpack-archives` as well.
+The two flags are independent - neither implies the other:
+
+```bash
+fossa analyze --enable-go-binary-analysis --unpack-archives
+```
+
+## Project Discovery
+
+Walk the scan directory and sniff each file for an embedded buildinfo section.
+A file is reported only if buildinfo is found and it yields at least one
+usable module version.
+
+Binaries nested inside archives are found when `--unpack-archives` is passed:
+discovery runs again over the extracted contents, so a binary inside an AAR or
+JAR is reached the same way a manifest inside one would be.
+
+Only ELF, Mach-O, and PE files at least 4 KiB in size are examined, so the
+walk is cheap on repositories that contain unrelated binary files.
+
+All Go binaries found in one directory are reported as a single project, because
+a source unit is named after its directory. Each contributing binary appears as
+an origin path, and their module lists are combined.
+
+Default path filters still apply: a binary under `vendor/` is skipped unless you
+pass `--include-path vendor`.
+
+## Analysis
+
+The module list is read directly out of the binary; no Go toolchain is invoked
+and nothing is executed. Every module found is reported as a direct `go`
+dependency.
+
+Versions are normalized the same way `go.mod` analysis normalizes them:
+pseudo-versions are reduced to their commit hash, and semantic versions keep
+their `v` prefix.
+
+The main module is skipped when it is unversioned (the linker records `(devel)`
+for a locally built binary), and reported when it carries a real version, which
+happens for binaries built via `go install <module>@<version>`.
+
+## Limitations
+
+- Binaries built by Go < 1.18 use an older pointer-based buildinfo encoding and
+  are skipped.
+- Binaries built without module support (`GOPATH` mode, or `CGO`-only objects)
+  carry no module list.
+- Buildinfo records modules, not the dependency edges between them, so the
+  resulting graph is flat. The set of modules is complete.
+- Stripping a binary does not remove buildinfo, but rewriting or packing it
+  (for example with UPX) can.
+
+## FAQ
+
+### How do I only perform analysis for Go binaries?
+
+Pass `--only-target gobinary` alongside the enabling flag:
+
+```bash
+fossa analyze --enable-go-binary-analysis --only-target gobinary
+```
+
+`--only-target gobinary` on its own reports nothing, because the strategy is
+still disabled.
+
+### How do I inspect the same data by hand?
+
+```bash
+go version -m path/to/binary
+```

docs/references/subcommands/analyze.md

@@ -138,6 +138,7 @@ In addition to the [standard flags](#specifying-fossa-project-details), the anal
 |-----------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
 | [`--detect-vendored`](./analyze/detect-vendored.md)                               | Enable the vendored source identification engine. For more information, see the [C and C++ overview](../strategies/languages/c-cpp/c-cpp.md).                                                    |
 | [`--detect-dynamic './some-binary`](./analyze/detect-dynamic.md)                  | Analyze the binary at the provided path for dynamically linked dependencies. For more information, see the [C and C++ overview](../strategies/languages/c-cpp/c-cpp.md).                         |
+| [`--enable-go-binary-analysis`](../strategies/languages/golang/gobinary.md)       | Report Go modules read from the buildinfo embedded in compiled Go binaries. Opt-in. Combine with `--unpack-archives` to reach binaries nested inside archives.                                   |
 | [`--static-only-analysis`](../strategies/README.md#static-and-dynamic-strategies) | Do not use third-party tools when analyzing projects.                                                                                                                                            |
 | `--strict`                                                                        | Enforces strict analysis to ensure the most accurate results by rejecting fallbacks. When run with `--static-only-analysis`, the most optimal static strategy will be applied without fallbacks. |
 

extlib/millhone/src/cmd/analyze_container.rs

@@ -14,7 +14,10 @@ use tar::{Archive, Entry};
 use tracing::{debug, info, info_span, warn};
 use typed_builder::TypedBuilder;
 
-use super::go_buildinfo::{is_candidate_binary, scan_go_buildinfo, GoBuildInfo, GoModule};
+use super::go_buildinfo::{
+    is_candidate_binary, scan_go_buildinfo, DiscoveredGoBinary, BINARY_PREFIX_LEN,
+    MIN_GO_BINARY_SIZE,
+};
 
 #[derive(Debug, Parser, Getters)]
 #[getset(get = "pub")]
@@ -25,11 +28,6 @@ pub struct Subcommand {
 }
 
 const JAR_OBSERVATION: &str = "v1.discover.binary.jar";
-const GO_BINARY_OBSERVATION: &str = "v1.discover.binary.go";
-
-/// Only sniff regular files at least this large; Go binaries are never tiny.
-/// (u64 because that's what tar header sizes are.)
-const MIN_GO_BINARY_SIZE: u64 = 4096;
 
 /// Magic bytes identifying a gzip stream.
 const GZIP_MAGIC: [u8; 2] = [0x1f, 0x8b];
@@ -61,29 +59,6 @@ struct OciManifest {
 #[derive(Debug, PartialEq, Eq, Serialize, Hash)]
 struct LayerPath(PathBuf);
 
-/// A Go binary discovered in a layer, with the module list parsed from its
-/// embedded buildinfo.
-#[derive(Debug, PartialEq, Eq, Serialize, Clone)]
-struct DiscoveredGoBinary {
-    kind: &'static str,
-    path: PathBuf,
-    go_version: String,
-    main_module: Option<GoModule>,
-    modules: Vec<GoModule>,
-}
-
-impl DiscoveredGoBinary {
-    fn new(path: PathBuf, info: GoBuildInfo) -> Self {
-        DiscoveredGoBinary {
-            kind: GO_BINARY_OBSERVATION,
-            path,
-            go_version: info.go_version,
-            main_module: info.main_module,
-            modules: info.modules,
-        }
-    }
-}
-
 #[derive(Debug, PartialEq, Eq, Serialize, TypedBuilder)]
 struct ContainerAnalysis {
     /// Jars and fingerprints associated with each layer in a jar file.
@@ -232,7 +207,7 @@ fn maybe_go_binary(entry: &mut Entry<'_, impl Read>, path: &Path) -> Option<Disc
     // Sniffing consumes the entry's leading bytes (an `Entry` is a forward-only
     // reader), so stitch the prefix back onto the front of the stream: the
     // buildinfo scan needs offsets relative to the start of the file.
-    let mut prefix = [0u8; 64];
+    let mut prefix = [0u8; BINARY_PREFIX_LEN];
     if let Err(e) = entry.read_exact(&mut prefix) {
         debug!(?path, "skipped: failed to read file prefix: {e:?}");
         return None;