nuxinl / cloudlibc Goto Github PK
View Code? Open in Web Editor NEWCloudABI's standard C library
License: BSD 2-Clause "Simplified" License
CloudABI's standard C library
License: BSD 2-Clause "Simplified" License
I'm entertaining the idea of using clouldabi in WebAssembly envirnment. I forked the project at https://github.com/yurydelendik/cloudlibc and it works for simple example now. So far I identified following items that will be nice to upstream:
#define LG_PAGE 16
for wasmIf there is an interest in accepting to above changes, I will be willing to submit pull requests.
I'm curious what you think about the idea to release localtime_l and mktime_l as two functions in a independent shared library. Until today, i have not found a good library that does what these functions do, purely converting between timezones based on the IANA timezone database.
Because such a library does not exists, i integrated your localtime_l code inside my own program, but that requires (as you do now) a new release as soon as the IANA database is updated. If these functions would exist in a independent shared library, that update mechanism would be far easier, by just updating the shared library.
I also think a lot of C developers would be interested in these two functions alone without having to use the full cloudlibc library.
$ dpkg --listfiles x86-64-unknown-cloudabi-c-runtime
/.
$ dpkg --listfiles x86-64-unknown-cloudabi-cxx-runtime
/.
Am I doing this right?
I'm assuming that at some point calls would be added or possibly removed, and one might need preprocessor definitions to understand which version of cloudlibc is available.
I found the following in include/uv.h
but that file is for event handling:
//
// Version-checking macros and functions.
//
#define UV_VERSION_MAJOR 1
#define UV_VERSION_MINOR 15
#define UV_VERSION_PATCH 0
#define UV_VERSION_IS_RELEASE 0
#define UV_VERSION_SUFFIX "cloudlibc"
#define UV_VERSION_HEX \
(UV_VERSION_MAJOR << 16 | UV_VERSION_MINOR << 8 | UV_VERSION_PATCH)
__BEGIN_DECLS
unsigned int uv_version(void);
const char *uv_version_string(void);
__END_DECLS
Is this official versioning for the entire library?
Hello, I am trying to figure out if I can build 32-bit binaries with the CloudABI toolchain on macOS. I can build and run 64-bit binaries with x86_64-unknown-cloudabi-cc, but when I use the i686 variant, I get an error about a missing header file, stdio.h. I get the same behavior in Docker as well.
Do I need to install some additional LLVM components, or pass some new flags to the compiler, to get this to work?
Same error for the aarch64 and armv6 eabihf compilers.
Could cloudlibc offer a dassert()
macro that accepts a file descriptor for emitting assertion errors? Is this even possible, given that assert()
tends to hook deep into C++ throw
?
Alternatively, NuxiNL/cloudabi-utils#11 would be fantastic!
I suspect this may be possible with dup2()
, though I'm not sure how portable that function is.
How feasible would it be to support CloudABI on DragonflyBSD?
Are there any prerequisites to supporting CloudABI that DragonflyBSD currently lacks?
What sort of changes would need to be made to accomplish this?
Hi! This is really exciting work and I look forward to trying it out.
I followed the instructions on https://nuxi.nl/cloudabi/freebsd/ but I get segfault when running helloworld..
Running ./_obj/cloudlibc-unittests also gives segfault..
FreeBSD PC1 11.0-CURRENT FreeBSD 11.0-CURRENT #0 5a410d3(HEAD): Wed Apr 6 11:14:22 PDT 2016 root@PC1:/usr/obj/usr/home/johannes/dev/freebsd/sys/GENERIC-NODEBUG amd64
As observed here: NuxiNL/debian-cloudabi#2
root@88e39bb79e41:/# cloudabi-run -e /usr/x86_64-unknown-cloudabi/bin/cloudlibc-unittests << EOF
%TAG ! tag:nuxi.nl,2015:cloudabi/
---
tmpdir: !file
path: tmp-unittest
logfile: !fd stdout
EOF
WARNING: Attempting to start executable using emulation.
Keep in mind that this emulation provides no actual sandboxing.
Though this is likely no problem for development and testing
purposes, using this emulator in production is strongly
discouraged.
-> drand48::bounds
-> fdopen::bad
-> localtime_l::santiago
-> fflush::eagain
-> wcsrtombs::ascii_null_ok
-> wmemset::example
-> call_once::example
-> clock_nanosleep::monotonic_relative
-> fread::zero
-> unlinkat::examples
-> uv_ip4_addr::einval
-> wctomb::euro
-> scandirat::even_files
Test failed
--
Statement: ASSERT_EQ(0, mkdirat(state->tmpdir, test->__name))
Expected: 0 == 0 == (0)
Actual: -1 == 0xffffffff == (mkdirat(state->tmpdir, test->__name))
Location: src/libc/testing/testing_execute.c:82
Errno: 20, File exists
Aborted (core dumped)
This is usually an indicator that two TEST()
s are defined with the same name, causing them to race for the same unit testing data directory. A git grep
doesn't seem to indicate this is the case. We should add some diagnostics to testing_execute
to print which directory it was trying to create.
I think, it should be 6, as with 5 the terminating nul char is not taken into account in string like "fr_FR"
cloudlibc/src/libc/locale/newlocale.c
Line 34 in 33041bf
which would be fine if it wasn't checked for it
cloudlibc/src/libc/locale/newlocale.c
Line 84 in 33041bf
cloudlibc offers engineers a tremendous gatekeeper for I/O, writing applications just once in terms of a simple API, and flexibly reconfiguring applications for many different workflows for runtime. Let's extend the same gatekeeping to calls for the current date and time. This is not so much a security measure, as a safety measure:
Gating the system clock makes it much easier to test complex applications for different clock values, such as several years in the future when important SSL certificates expire, and when 2038 breaks legacy applications. While developers are advised to program in terms of an injectable clock, most applications are not written this way. By intercepting these calls, we offer utility for integration testing this part of the total application input space.
For many systems, there will come a day when the true date breaks the application, and a convenient way to frequently reset the date to a historical value, will be the only way to resurrect these applications.
When I run the following command:
cloudabi-run /usr/x86_64-unknown-cloudabi/bin/cloudlibc-unittests << EOF
%TAG ! tag:nuxi.nl,2015:cloudabi/
---
tmpdir: !file
path: tmp-unittest
logfile: !fd stdout
nthreads: !!int 1
EOF
one of the test cases fails with the following error:
[passing tests omitted]
-> setvbuf::iolbf_pipe
Test failed
--
Statement: ASSERT_EQ(0, ioctl(fds[0], FIONREAD, &nbytes))
Expected: 0 == 0 == (0)
Actual: -1 == 0xffffffff == (ioctl(fds[0], FIONREAD, &nbytes))
Location: src/libc/stdio/setvbuf_test.c:100
Errno: 52, Function not implemented
Aborted
Extra info:
# Linux version
vagrant@vagrant:~$ uname -a
Linux vagrant 4.7.0-cloudabi+ #1 SMP Fri Jul 21 12:03:48 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
# CloudABI version
vagrant@vagrant:~$ apt-cache show x86-64-unknown-cloudabi-cloudlibc
Package: x86-64-unknown-cloudabi-cloudlibc
Version: 0.78-1
Architecture: all
Maintainer: [email protected]
Description: cloudlibc for x86_64-unknown-cloudabi
Description-md5: 5b87b85f313f06f0c58f75cc46309a30
Homepage: https://github.com/NuxiNL/cloudlibc
Depends: x86-64-unknown-cloudabi-cloudabi, x86-64-unknown-cloudabi-compiler-rt
Filename: x86-64-unknown-cloudabi-cloudlibc_0.78-1_all.deb
Size: 1196348
SHA256: adecbabc32e77944d5c3c463fa73cf0d1b687a89fade4d3d0e91d6467a3ddca9
Package: x86-64-unknown-cloudabi-cloudlibc
Status: install ok installed
Installed-Size: 7719
Maintainer: [email protected]
Architecture: all
Version: 0.78-1
Depends: x86-64-unknown-cloudabi-cloudabi, x86-64-unknown-cloudabi-compiler-rt
Description: cloudlibc for x86_64-unknown-cloudabi
Description-md5: 5b87b85f313f06f0c58f75cc46309a30
Is this a known limitation?
I first ran into this when using python:
File "selectors.py", line 376, in select
OSError: [Errno 52] Function not implemented
Then I confirmed it using cloudlibc-unittests:
$ cloudabi-run /usr/x86_64-unknown-cloudabi/bin/cloudlibc-unittests << EOF
> %TAG ! tag:nuxi.nl,2015:cloudabi/
> ---
> tmpdir: !file
> path: tmp-unittest
> logfile: !fd stdout
> nthreads: !!int 8
> EOF
-> argdata_serialize::seq
-> mblen::euro
-> fgetpos::eoverflow
-> fputws::success
-> wcsncpy::example1
-> mbstowcs::zero
-> malloc::zero
-> poll::file
-> twalk::empty
-> ntohl::example
-> strndup::null
Test failed
--
Statement: ASSERT_EQ(1, poll(&pfd, 1, -1))
Expected: 1 == 0x1 == (1)
Actual: -1 == 0xffffffff == (poll(&pfd, 1, -1))
Location: src/libc/poll/poll_test.c:168
Errno: 52, Function not implemented
$ uname -a
Linux pav 4.7.0-cloudabi+ #1 SMP Sat Apr 8 10:56:21 CDT 2017 x86_64 x86_64 x86_64 GNU/Linux
$ (cd ~/projects/linux-cloudabi/; git log -n1)
commit 477884e20e3c68d3524a42864900a5d3bbb0e1b4
Author: Ed Schouten <[email protected]>
Date: Sun Feb 26 22:02:46 2017 +0100
Add missing cloudabi_vdso.lds linker script.
Reported by: Dan Connolly
Could we get .DEB packages, to automate the process of installing CloudABI toolchains, including the cloudabi-run utility? That would make it much easier for Debian/Ubuntu users to access CloudABI!
YAML files are supposed to be able to begin with ---
, and yamllint recommends this rule to ensure that many YAML files behave well when concatenated together. However, CloudABI's YAML parsing setup requires its YAML files to begin like %TAG ! tag:nuxi.nl,2015:cloudabi/
, and then a document delimiter and capability spec document can follow.
Would it be possible to enhance the capability spec so that ---
can lead the very beginning of .YML files, matching how other applications process YAML files?
Hey, I'm glad to report mild success with CloudABI, building some very simple C and Rust apps! Unfortunately, when I try to use sprintf, I'm getting build errors. Am I doing something wrong?
My "issue-115" branch of my tonixxx project, in the examples/fewer C project directory:
https://github.com/mcandre/tonixxx/tree/issue-115/examples/fewer
$ CC=x86_64-unknown-cloudabi-cc CXX=x86_64-unknown-cloudabi-c++ cmake .
$ cmake --build . --config Release
[ 33%] Linking C executable bin/fewer
/usr/bin/x86_64-unknown-cloudabi-ld: error: undefined symbol: sprintf
>>> referenced by fewer.c
>>> CMakeFiles/fewer.dir/lib/fewer.c.o:(render_boi)
clang: error: linker command failed with exit code 1 (use -v to see invocation)
CMakeFiles/fewer.dir/build.make:120: recipe for target 'bin/fewer' failed
make[2]: *** [bin/fewer] Error 1
CMakeFiles/Makefile2:227: recipe for target 'CMakeFiles/fewer.dir/all' failed
make[1]: *** [CMakeFiles/fewer.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2
This is weird, as I am able to access other cloudlibc entities such as dprintf()
and fopen()
just fine in my main.c
source file. Perhaps cmake is getting confused about where libc comes from when dealing with the separate file fewer.c
?
Note: I am running these commands from an Ubuntu 18.04 Bionic Beaver instance in VirtualBox on macOS. When I try to build this CMake project directly on my macOS host, I can't even get the initial CMake cache to complete:
$ CC=x86_64-unknown-cloudabi-cc CXX=x86_64-unknown-cloudabi-c++ cmake .
-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Check for working C compiler: /usr/local/bin/x86_64-unknown-cloudabi-cc
-- Check for working C compiler: /usr/local/bin/x86_64-unknown-cloudabi-cc -- broken
CMake Error at /usr/local/Cellar/cmake/3.13.2/share/cmake/Modules/CMakeTestCCompiler.cmake:52 (message):
The C compiler
"/usr/local/bin/x86_64-unknown-cloudabi-cc"
is not able to compile a simple test program.
It fails with the following output:
Change Dir: /Users/andrew/go/src/github.com/mcandre/tonixxx/examples/fewer/CMakeFiles/CMakeTmp
Run Build Command:"/usr/local/bin/gmake" "cmTC_9b5a3/fast"
/usr/local/bin/gmake -f CMakeFiles/cmTC_9b5a3.dir/build.make CMakeFiles/cmTC_9b5a3.dir/build
gmake[1]: Entering directory '/Users/andrew/go/src/github.com/mcandre/tonixxx/examples/fewer/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_9b5a3.dir/testCCompiler.c.o
/usr/local/bin/x86_64-unknown-cloudabi-cc -o CMakeFiles/cmTC_9b5a3.dir/testCCompiler.c.o -c /Users/andrew/go/src/github.com/mcandre/tonixxx/examples/fewer/CMakeFiles/CMakeTmp/testCCompiler.c
Linking C executable cmTC_9b5a3
/usr/local/Cellar/cmake/3.13.2/bin/cmake -E cmake_link_script CMakeFiles/cmTC_9b5a3.dir/link.txt --verbose=1
/usr/local/bin/x86_64-unknown-cloudabi-cc -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/cmTC_9b5a3.dir/testCCompiler.c.o -o cmTC_9b5a3
/usr/local/bin/x86_64-unknown-cloudabi-ld: error: unknown argument: -search_paths_first
clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[1]: *** [CMakeFiles/cmTC_9b5a3.dir/build.make:87: cmTC_9b5a3] Error 1
gmake[1]: Leaving directory '/Users/andrew/go/src/github.com/mcandre/tonixxx/examples/fewer/CMakeFiles/CMakeTmp'
gmake: *** [Makefile:121: cmTC_9b5a3/fast] Error 2
CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:3 (project)
Update
I spent some more time configuring my CMakeLists.txt so that the compiler options are closer to the cloudabi compiler defaults, but I am still getting this strange error. Clearly, fewer.c
is #include
ing the <stdio.h>
file, so I am scratching my head wondering what part of this system is dropping the ball.
One thing I've noticed is that cmake tends to compile executables in multiple stages, compared to using x86_64-unknown-cloudabi-cc
directly. Perhaps the compiler is dropping information about where libraries are defined, when object files are built and linked, versus building an executable all at once?
A new release of time zone information has been made in september 2019:
https://www.iana.org/time-zones
Lacking the traditional int argc, char **argv
entrypoint parameters for querying command line arguments, could program.h provide another way to obtain the name of the program? This is often useful for displaying accurate help command syntax for downstream users.
cloudlibc has a fantastic number of ports available. Motion to spread support even wider!
Could we get a Windows port of the cloudlibc runtime and an example cmake project illustrating how to build CloudABI binaries from Windows development hosts? So that we can grow the deployment space for CloudABI apps!
I saw the awesome talk on CloudABI and think it would be a good idea to begin offering dependency injection for the system clock and for random number generation, for even richer application tests. Could we begin designing an API towards that end?
Porting applications from libc to cloudlibc is hard work, but one problem is particularly pernicious: cloudlibc toolchains do not warn the programmer when he or she forgets basic steps for correctly wiring up the new entrypoint.
<void|int> main([int argc[, char **argv]])
signature, as opposed to the recommended void program_main(const argdata_t *ad)
signature.argdata::env::argdata()
.When this happens, the user observes the resulting CloudABI executable emit no output. It would be better for the runtime to present a dummy warning that the entrypoint was misconfigured. Best of all would be to detect this at compile time, and refuse to produce a binary with the wrong entrypoint.
I'm used to fprintf()
intelligently flushing my output streams, e.g. when writing brief text blurbs to stderr in a Read Eval Print Loop application. However, CloudABI appears to wait until the program is terminating to flush. Could we get the flushing behavior closer to a typical GNU/libc fprintf()
implicit flush?
As a workaround, I am doing this for important user feedback sections:
fprintf(console, "blahblahblah\n");
#if defined(__CloudABI__)
fflush(console);
#endif
The posted installation steps for manually installing the CloudABI toolchain with the cloudabi-run utility in Debian/Ubuntu, omits some important requirements:
Without these steps, cloudabi-utils fails to compile. Many of these additional steps are documented in the Dockerfile, but have not been copied to the plain vanilla Debian/Ubuntu documentation.
Could we move these commands to a standalone shell script, and have both the Dockerfile and the documentation point to that, to keep everything in sync?
That's would be great if cloundlibc would support for win32.
Hello, I am trying to write my first CloudABI program, following along with the tutorials. I have a Clang-based CloudABI toolchain installed, along with cloudabi-run.
hello.c:
#include <stdio.h>
int main(void) {
dprintf(1, "Hello World!\n");
}
cloudabi.yml:
%TAG ! tag:nuxi.nl,2015:cloudabi/
---
- !fd stdout
Trace:
$ x86_64-unknown-cloudabi-cc -o hello hello.c
$ cloudabi-run -e hello <cloudabi.yml
WARNING: Attempting to start executable using emulation.
Keep in mind that this emulation provides no actual sandboxing.
Though this is likely no problem for development and testing
purposes, using this emulator in production is strongly
discouraged.
So Hello World is never printed; no segmentation fault occurs, no error message about stdout access is presented. Furthermore, execution exits with a zero exit code, indicating "success".
Update
I looked at some more examples and see that file descriptor 1
appears to no longer work out of the box as stdout in CloudABI. I changed my code to:
#include <stdio.h>
#include <stdlib.h>
int m(int stdout) {
dprintf(stdout, "Hello World!\n");
return EXIT_SUCCESS;
}
#ifdef __CloudABI__
#include <argdata.h>
#include <program.h>
void program_main(const argdata_t *ad) {
int stdout;
argdata_get_fd(ad, &stdout);
#else
#include <unistd.h>
void main() {
int stdout = STDOUT_FILENO;
#endif
exit(m(stdout));
}
And am now able to build and run my lil app! This one is a polyglot, so it compiles and runs with either plain vanilla Clang or with CloudABI. Also, CloudABI appears to work with more modern Clang/LLVM/LLD versions, including v6.0. Could we update the documentation to reflect this?
Could we update the different per-OS tutorials to fix the stdout file descriptor part?
A larger question is why are guarded? I thought CloudABI was meant to protect sensitive components that can break a system. Do stdout/stderr somehow contribute to an increased attack surface?
Some microkernels (NOVA) keep segment registers fixed across context switches for the sake of performance. The problem is that is breaks compatibility with binaries that abuse FS and GS for thread local storage.
Would there be an interest in replacing the inline TLS dereferencing code that uses %fs
with a function call than can be placed in the vDSO?
The Rust tutorial for developing CloudABI programs requires the developer to write a substantial amount of unsafe code by hand in order to be able to use cloudabi::fd's with typical Rust writing functions and macros. Could we include this console write implementation in the crate by default, so that developers can use cloudabi in a more safe and convenient manner?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
ð Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ððð
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google â€ïž Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.