testcontainers / testcontainers-rs Goto Github PK
View Code? Open in Web Editor NEWA library for integration-testing against docker containers from within Rust.
Home Page: https://rust.testcontainers.org
License: Apache License 2.0
A library for integration-testing against docker containers from within Rust.
Home Page: https://rust.testcontainers.org
License: Apache License 2.0
Currently, users can request a mapped host port from a container through Container#get_host_port
.
This API allows users to pass whichever port the want and the function will simply return an Option
if the port was not mapped.
It would be interesting to explore an API where implementations of Image
could somehow statically specify, which particular ports will be exposed. They should "know" that because we know which docker image we will be starting.
The advantage I am trying to gain from that is increased discoverability:
If I am starting a postgres
container through testcontainers
, I don't want to need to know that postgres exposes port 5432
. Instead, it would be nice if I could discover that through intelli-sense on the container that I just started.
Design suggestions are highly welcome!
How can I define environment variables when starting an image?
Many images can be customized through env vars, for example, Postgres permits to define user, password db name, db init arguments and so on:
> docker run -e POSTGRES_PASSWORD=mysecretpassword -d postgres
For what I see in the Cli (here) this is not possible with the current implementation.
Currently the rustfmt version is not fixed.
Because this is a library, we should fix the version of Rust to use (ie, we should not use rust-toolchain
file).
Avoid issues where different versions of rustfmt are used by different developers and yield different results.
Add +stable
to the githook.
The docker daemon can be hosted on a remote machine, hence in addition to the port
of a container, we should also expose the host
on which the docker container was started.
Probably be done through clippy?
Hi there!
I have a somewhat custom docker container that changes frequently, therefore I would like to provide a docker file instead. How can I do this?
Cheers!
At the moment we wait for 100ms for the logs to be ready. Apparently, this is not enough as the tests on || keep failing randomly.
Increasing the timeout to something like 500ms should not hurt that much because the container starts up in the meantime anyway. It is basically just a matter of where we spend the time waiting: In the loop that polls the logs or in the sleep before that.
I'm currently building my code with bazel instead of cargo. Using testcontainers, the containers appear to be left alive after the test finishes, I.E. they don't seem to be being dropped and cleaned up. Note that manually drop
ing the container
and docker
variables before exiting the function does not appear to fix it.
version: testcontainers v0.10.0
OS: Ubuntu 20.04 inside WSL2 host
Docker engine: 19.03.12
Code:
use testcontainers as tc;
use testcontainers::Docker;
#[must_use]
fn setup_container(client: &tc::clients::Cli) -> tc::Container<'_, tc::clients::Cli, tc::images::redis::Redis> {
client.run(tc::images::redis::Redis::default())
}
#[test]
fn test_new() -> RedisResult<()> {
let docker = tc::clients::Cli::default();
let container = setup_container(&docker);
let port = container.get_host_port(6379).unwrap();
let mut cache = TensorCache::new(Url::parse(&format!{"redis://localhost:{}/", port}).unwrap())?;
let test_str = "are you there?";
let response: String = redis::cmd("PING").arg(test_str).query(&mut cache.conn)?;
assert_eq!(response, test_str);
Ok(())
}
docker ps
:
P.S. There are actually two test cases, one of which I omitted in the above code, which is why you observe two docker containers at a time. I've been able to reproduce it with only the test case above, so this should not be the cause of the bug.
Right now it's possible to get mapped port via get_host_port
method, and connect to services as localhost:<mapped port>
.
It might be useful to implement get_ip_address
method and return NetworkSettings.IPAddress
from docker inspect <id>
, so it's possible to connect from one docker container to another using that IP.
Currently, our rust-toolchain
file says stable
.
This can cause your CI to fail because stable
is a moving target and components like rustfmt might have different opinions about how code should be formatted for different versions.
Figure out what the MSRV for our project is an pin it to that.
Hi there!
Im not sure if this is the right place to ask these specific questions, but I just can't figure out how to solve this, trying to get this running since days.
So Im trying to get my Integration tests to run in the CI using Github Actions. Before I started a docker container manually and executed tests in the container, doing the cleanup manually. With Testcontainers this is no longer required but the containers need to be able to be started.
So locally everything works fine.
The CI however spits out an interesting failure message that I cant make sense of:
https://github.com/Geigerkind/LegacyPlayersV3/runs/777848531
(Scroll down to line 651) "Cannot run dynamic test fn out-of-process"
The testcotainer is a GenericImage, downloading a MariadDB and linking a volume.
https://github.com/Geigerkind/LegacyPlayersV3/blob/dev/Backend/src/tests/test_container.rs
A container is then started in each integration test function, e.g.
https://github.com/Geigerkind/LegacyPlayersV3/blob/dev/Backend/src/modules/account/tests/get.rs
Does anyone have an idea how I can fix this?
Cheers!
Bitcoind allows to specify the default type for addresses generated through getnewaddress
with the -addresstype
option.
We should allow users to configure this option in the args and default to bech32.
testcontainers-rs/src/clients/cli.rs
Line 232 in 40dcfb8
Do not remove docker network if KEEP_CONTAINERS=true
. Otherwise it is impossible to start conrainers that was created during test:
> docker start {container_name}
Error response from daemon: network {network_id} not found
Following the discussion in #141 (comment), it would be nice to provide an additional async API (or replace the existing one?). This would allow an easier integration with libraries like https://github.com/softprops/shiplift or https://github.com/fussybeaver/bollard to ease future introduction of additional features into this library.
Opening this issue to initiate a discussion.
I can't make sense of why the build in #139 is breaking. Here is the first error when cargo build
is run:
$ cargo build
Updating crates.io index
Compiling smallvec v1.0.0
Compiling percent-encoding v2.1.0
error[E0658]: use of unstable library feature 'alloc': this library is unlikely to be stabilized in its current form or name (see issue #27783)
--> /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/smallvec-1.0.0/lib.rs:38:1
|
38 | extern crate alloc;
| ^^^^^^^^^^^^^^^^^^^
...
Note that this branch compiles and all tests passed when cargo build
and cargo test
are run on my local machine
When running cargo test
on my Linux box, I'm getting this test failure:
failures:
---- dynamodb_local_create_table stdout ----
thread 'dynamodb_local_create_table' panicked at '
expected: result[ok]
but was: result[error]<HttpDispatch(HttpDispatchError { message: "Error during dispatch: connection error: Connection reset by peer (os error 104)" })>
', /home/walterh/.cargo/registry/src/github.com-1ecc6299db9ec823/spectral-0.6.0/src/lib.rs:343:9
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
failures:
dynamodb_local_create_table
Not sure if this shows up on CI or not. If I'm the only person getting this, feel free to close it.
The quoted test fail.
Seems to be a Mac OS X issue only.
---- dynamodb_local_create_table stdout ----
thread 'dynamodb_local_create_table' panicked at '
expected: result[ok]
but was: result[error]<HttpDispatch(HttpDispatchError { message: "parsed HTTP message from remote is incomplete" })>
', /Users/froyer/.cargo/registry/src/github.com-1ecc6299db9ec823/spectral-0.6.0/src/lib.rs:343:9
The official postgres docker image does not start if the "POSTGRES_PASSWORD" environment variable is not specified; consequently, this command always fails:
docker.run(images::postgres::Postgres::default());
I think it'd be good to have it, including checking in Travis as we have a number of external contributors.
Using Docker::stop
to stop a container is nice for testing failure conditions. It would be nice if there was a Docker::start
so that I can restart the container and test that the issue goes away.
Currently I fail to move container into variable because of Container<'a, ..., ...>
lifetime. This lifetime depends on let docker = clients::Cli::default();
lifetime, which I can't move into generic option of another variable.
Also, I fail to move container initialization into lazy static
macro.
Any ideas how to avoid those issues and create docker containers only once for all tests?
Some images will depend on files in the repo, we should have a way to reference them within the image, by allowing mountpoints into the container
The expectation is that testcontainers
should tear down a container after a test is completed, but when running the following test in isolation using cargo test <TESTNAME>
#[test]
fn test() {
let docker = testcontainers::clients::Cli::default();
docker.run(testcontainers::images::coblox_bitcoincore::BitcoinCore::default());
}
the docker container remains up after the test has passed.
Interestingly, this doesn't happen if the test is run using cargo test
.
Having panicking APIs is more convenient and this library is designed to be used within tests only so panicked should be fine.
At the moment, you have to import two crates in order to use an image.
testcontainers
This can lead to problems when the versions of the transitive dependency on testcontainers
in the image crate doesn't match the one you are depending on explicitly.
There are several ways on how to proceed with this:
start
/ run
/ deploy
on the Image
trait that uses testcontainers under the hood to start the image. This way, you don't have to depend on testcontainers in the user code.testcontainers
from every image so that one can import from the transitive dependency.testcontainers
is always a test-dependency, this would be secondary issue.At the moment, it is possible to run into problems with testcontainers-rs where the docker containers is killed to early.
To reproduce this, simply spawn a new container, connect to it and move the client to a different thread. As soon as the scope where the container is defined ends, the underlying docker container will be dropped and your client will fail when trying to use the connection.
We should make use of Rust's lifetimes to avoid this issue.
Hey,
the current approach of this library assumes that cargo test
is executed on the host system running docker. For having a reproducible environment, it makes sense to run the tests themselves inside a docker container. To be able to have those tests spin up and access other containers, we'd need to:
docker-cli
installed)docker inspect
has itMy proposed approach would be to create another Client
and implement 2 and 3 as enumerated above. Obviously, to be able to use it, the library users needs to make sure 1 is properly set up.
What do you think about this approach? Would you be willing to accept a PR implementing the above?
Releasing new versions of testcontainers and all it's little subcrates can be quite difficult. In particular, figuring out what the next version should be is not trivial.
We should create a guide that documents how versioning works within the testcontainers-rs
ecosystem.
I think the whole meta-crate approach is over-engineered.
There is no actual need to split the functionality up that way and it introduces WAY too much complexity, esp. in terms of releasing and versioning.
Hello, thanks for your work, and sorry to bother your issues with a question :\
Is there a way to use a docker-compose.yml
file in order to setup our integration tests stack ?
Similar to: https://www.testcontainers.org/modules/docker_compose/
Or is it limited to only images defined in the images
folder (redis etc.).
If not possible atm, is it a feature you'd like to have in the future ?
Thank you.
I'm not sure if this is a problem with testcontainers or Rust Analyzer, but I've noticed that if I run the tests from within VSCode then my containers don't ever shut down. If I run the exact same test from the command line, with the exact command line that VSCode is using then everything works exactly as expected.
This is the case regardless of whether the test passes or fails.
Pressing this "Run Test" button:
Comes up with:
> Executing task in folder service: cargo test --test get_by_id -- get_unknown_by_id --exact --nocapture <
And running that exact command in the terminal then correctly shuts things down.
I only noticed this when I found I had 30 containers still running from the last 2 days.
Cheers
At the moment, each Image
implementation has to provide a separate implementation for specifying port mappings:
testcontainers-rs/src/core/image.rs
Line 91 in 0eb3114
This is not ideal because it creates duplicated code between all images.
We should move this functionality to RunArgs
:
testcontainers-rs/src/core/docker.rs
Line 22 in 3d80acb
It would be good to have an equivalent of the GenericContainer Java class to easily use custom images.
In fact, at the moment, for using an image not already provided by testcontainers-rs, it is required to create a new implementation of the Image trait which involves lots of boilerplate.
Is there any plan to add it?
Apparently, depending on testcontainers brings in a dependency on openssl. That shouldn't be the case because we don't do any networks requests (yet).
Investigate and fix.
I'm fairly unfamiliar with Rust's options for docker clients, but from what I can see based on a cursory search of crates.io, I think this library's api is a lot more ergonomic than the alternatives out there. I know that this crate is not intended for production use or use outside of integration tests, but is there any code that could be extracted into another crate and used as the start of a general purpose docker client library? Any thoughts on what that might look like?
As mentioned in https://github.com/testcontainers/testcontainers-rs/pull/87/files#r281005504, we should remove pulling docker containers in the before_script
section. It's not worth the duplication.
Would it be possible to provide a minimal usage example directly in the rustdocs? Hints on how to get started are provided, but having a minimal complete example would go a long way (new user here).
This allows us to build containers dedicated for testing locally using a buildscript. See #174 for more context.
This should be added by default because it is useful for debugging.
With the addition of the dynamodb image, we now have two images that require an "additional wait period" setting. The first one is bitcoind.
In both cases, this is exposed through an environment variable.
We should provide an abstraction that allows images to easily opt-into this behavior without implementing it completely themselves.
The documentation of the crate is currently in a very early stage. The following things are missing:
readme
key in Cargo.toml)Hello! I'm Richard; I created the initial Java version of Testcontainers, and I'm very happy to see the idea taking off in Rust as well ๐.
We were wondering if you'd like to join us over on the Testcontainers Slack team to chat and to see if we can help out in any way. We're very hands-off when it comes to language forks, but we want to be supportive and friendly!
Please say hi (myself, @bsideup and @kiview), either on Slack or indeed email/any other channel if you prefer.
I have the feeling it is more annoying than it actually helps. Mainly because we don't have that many tests and automatically testing the lib is hard. Therefore, test coverage is not a very useful metric for this library.
I have a strange bug when using the postgres image, the connection is closed unexpectedly while I am using it.
I created here a small reproducer (launch the tests to see it):
https://github.com/ufoscout/rust-pg-issue
What happens?
while using a connection to the postgres image, randomly rust-postgres returns this error:
Err(Error(Io(Custom { kind: UnexpectedEof, error: StringError("unexpected EOF") })))
I initially thought that it is a rust-postgres issue; however, this error does not happen if I use a Postgres docker image launched manually from the command line instead of the testcontainers one.
The error appears randomly. If you reproduce it through the above-linked reproducer, I'll notice that sometimes it happens after 2 loops, sometimes after 3 or 4 or whatever number of loops.
It seems to me that the image is stopped asynchronously while still in use.
Environment: Ubuntu LTS 18.04 64 bits
Rustc: 1.32.0 (9fda7c223 2019-01-16)
We should use/provide the latest parity image.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.