lovell / detect-libc Goto Github PK

In a docker image that has been stripped of all non-essential files, the libc detection comes back with ''.

Other than the node app installed under /app, the entire rest of the image consists of this:

-rw-r--r--   1 root     root          108 Nov 22 00:26 /etc/passwd
-rw-r--r--   1 root     root           36 Nov 22 00:26 /etc/group
-rw-r--r--   1 root     root          388 Nov 22 00:26 /etc/ld.so.preload
-rw-r--r--   1 root     root          259 Nov 22 00:26 /etc/nsswitch.conf
-rwxr-xr-x   1 root     root      1738176 Jun 18 11:09 /lib/x86_64-linux-gnu/libc.so.6
-rw-r--r--   1 root     root        22952 Jun 18 11:09 /lib/x86_64-linux-gnu/libnss_dns.so.2
-rw-r--r--   1 root     root        84856 Jun 18 11:09 /lib/x86_64-linux-gnu/libresolv.so.2
-rwxr-xr-x   1 root     root       137384 Jun 18 11:09 /lib/x86_64-linux-gnu/libpthread.so.0
-rw-r--r--   1 root     root      1051056 Jun 18 11:09 /lib/x86_64-linux-gnu/libm.so.6
-rw-r--r--   1 root     root        31784 Jun 18 11:09 /lib/x86_64-linux-gnu/librt.so.1
-rw-r--r--   1 root     root        90096 Dec 25  2014 /lib/x86_64-linux-gnu/libgcc_s.so.1
-rw-r--r--   1 root     root        14664 Jun 18 11:09 /lib/x86_64-linux-gnu/libdl.so.2
-rw-r--r--   1 root     root        47712 Jun 18 11:09 /lib/x86_64-linux-gnu/libnss_files.so.2
-rw-r--r--   1 root     root      1008120 Dec 25  2014 /usr/lib/x86_64-linux-gnu/libstdc++.so.6
-rwxr-xr-x   1 root     root       140928 Jun 18 11:09 /lib64/ld-linux-x86-64.so.2
-rwxr-xr-x   1 root     root     35197069 Nov 22 00:26 /bin/node

From reading the code, I expect the filesystem based detection to match the one entry under /lib, but it doesn't.

I'm trying to reproduce this with tests now, but wanted to open this sooner rather than later.

Related to mapbox/node-pre-gyp#325.

From my point of view we need to consider two dimensions in regard to the integration tests:

• Node.js versions (0.10.x, 0.12.x, 4.x, 6.x, 8.x)
• Linux distributions and their versions (e.g. Alpine Linux v3.6, Alpine Linux v3.4, CentOS Linux 7 (Core), ...)

I think we can test both dimensions independently without creating a full blown matrix. This approach should be enough to proof reliability of lovell/detect-libc. In case you agree, we just need to define the set of Linux distributions and versions we want to support.

Note: even though we do not need a full-blown matrix, we need to run the tests on every selected distribution/distri-version for Node.js 0.10.x and 8.x/latest. Reason is the fallback to filesystem detection method on Node.js 0.10.x. Still we can use a sparse test matrix :-)... (at least from my point of view).

Pre-release versions of the future Fedora Rawhide 38 are now including development/unreleased versions of glibc with a trailing patch version e.g. 2.36.9000

https://packages.fedoraproject.org/pkgs/glibc/glibc/fedora-rawhide.html

We'll need a container image for testing this, but these are currently not yet made available - see https://github.com/fedora-cloud/docker-brew-fedora/branches

This was originally reported at lovell/sharp#3423

Thanks to @core-process for this excellent suggestion.

Can initially be based on the great work in https://github.com/core-process/detect-libc-async/tree/master/tests/docker

Allow to detect system libc on systems that don't have getconf or ldd commands, like NodeOS. This would be done by reading the symlink at /lib/libc.so.

ldd --version on alpine:3.6 returns with exitcode 1 and prints information on stderr. This szenario should be supported by the library.

This module is downloaded about a million times every day (!), and its logic is perhaps run hundreds of times every second.

Making changes to something this highly depended upon is a little frightening, but if we can reduce the time/energy cost then I think it will be worthwhile.

The logic hasn't changed in over four years and (rather amazingly) it all still works, but the ecosystem around it has improved so it's time for this module to do likewise.

Goals:

• Expose functionality via both async and sync functions
• Reduce require-time CPU and I/O cost, defer as much logic as possible until functions are called
• Only spawn a child process where absolutely necessary, and consolidate into a single call
• The requirements of both @mapbox/node-pre-gyp and prebuild become first-class concerns (perhaps others? Parcel? Electron?)
• Support Node.js 8+, drop support for Node.js 6 and earlier
• Migrate CI to GitHub Actions, add more recent OS and Node.js versions to integration tests
• Use Node.js 12+ native report feature where available, allows family detection and glibc version (but not musl version)

Progress:

• Readme - https://github.com/lovell/detect-libc/tree/main
• Implement new API
• Unit tests - https://github.com/lovell/detect-libc/actions/runs/1708403127
• Integration tests (26 combinations) - https://github.com/lovell/detect-libc/actions/runs/1708403128
• JSDocs
• TypeScript type definitions
• Verify prebuild will work
• Verify @mapbox/node-pre-gyp will work

This package currently assumes that musl uses semver, but it looks like this might not always be the case.

https://gitlab.alpinelinux.org/alpine/aports/-/commit/bd758e828afaa840262f174077c151cd9c5ded04

# apk list musl
musl-1.2.3_git20230411-r0 x86_64 {musl} (MIT) [installed]
# node -p "require('detect-libc').versionSync()"
1.2.3_git20230411

Originally reported at lovell/sharp#3641

Most of the functions depend on functions that are expensive to call, I even opened an issue on NodeJS to talk more about it nodejs/node#48204.

So, if more than one library depends on this library and the results from it, they will call the expensive functions twice, do you see some issue with caching it?

Also, some libraries uses a faster way of checking if musl is supported, like https://github.com/swc-project/cli/blob/master/src/swcx/index.ts#L78, I don't know why they use getReport but did you see any issue with this method?

@lovell We could reduce run-time of the integration tests significantly, if we decide to pre-build the docker images. In case we want to do this, we need a docker registry; e.g. we could use a free quay.io account for that.