Giter VIP home page Giter VIP logo

docker-client's Introduction

Build Status Maven Central API Coverage

Docker Client

A Docker HTTP client for the Java VM written in Groovy

This client library aims at supporting all existing Docker api endpoints, which effectively allows to use it in place of the official Docker client binary.

See the supported-api.md for details about the current api coverage.

All platforms are natively supported, which includes:

Consider the client as a thin wrapper to perform HTTP requests, minimizing the need to manually configure TLS or auth encoding in your code. Most commonly known environment variables will work as expected, e.g. DOCKER_HOST, DOCKER_TLS_VERIFY, or DOCKER_CERT_PATH. Due to its thin layer this client might feel a bit less convenient, though, while it gives you a bit more freedom to access the engine api and some less popular endpoints.

The client also includes Docker Compose version 3 support as part of the docker stack --compose ... command. See the docker stack deploy docs for details. Please note that you'll need at least Java 8 when using that feature.

Plain Usage

For use in Gradle, ensure that you declared Maven Central as repository:

repositories {
  mavenCentral()
}

Then, you need to add the dependency, but please ensure to use the latest version:

dependencies {
  implementation("de.gesellix:docker-client:2021-02-20T21-57-11")
}

The tests in DockerClientImplSpec and DockerClientImplIntegrationSpec should give you an idea how to use the docker-client.

The default Docker host is expected to be available at unix:///var/run/docker.sock. When using Docker Machine the existing environment variables as configured by eval "$(docker-machine env default)" should be enough.

Even for the native packages Docker for Mac and Docker for Windows you'll be able to rely on the default configuration. For Mac the default is the same as for Linux at unix:///var/run/docker.sock, while Windows uses the named pipe at //./pipe/docker_engine.

You can override existing DOCKER_* environment variables with Java system properties like this:

System.setProperty("docker.host", "192.168.99.100")
System.setProperty("docker.cert.path", "/Users/${System.getProperty('user.name')}/.docker/machine/machines/default")

Please note that the raw responses (including headers) from the Docker daemon are returned, with the actual response body being available in the content attribute. Some endpoints return a stream, which is then available in stream. For some cases, like following the logs or events stream, you need to provide a callback which is called for every response line, see example 3 below.

Example 1: docker info

A basic example connecting to a Docker daemon running in a VM (boot2docker/machine) looks like this:

System.setProperty("docker.cert.path", "/Users/${System.getProperty('user.name')}/.docker/machine/machines/default")
def dockerClient = new DockerClientImpl(System.getenv("DOCKER_HOST"))
def info = dockerClient.info().content

Example 2: docker run

Running a container being available on the host via HTTP port 4712 can be achieved like this:

System.setProperty("docker.cert.path", "/Users/${System.getProperty('user.name')}/.docker/machine/machines/default")
def dockerClient = new DockerClientImpl(System.getenv("DOCKER_HOST"))
def image = "busybox"
def tag = "latest"
def cmds = ["sh", "-c", "ping 127.0.0.1"]
def containerConfig = ["Cmd"         : cmds,
                       "ExposedPorts": ["4711/tcp": [:]],
                       "HostConfig"  : ["PortBindings": [
                               "4711/tcp": [
                                       ["HostIp"  : "0.0.0.0",
                                        "HostPort": "4712"]]
                       ]]]
def result = dockerClient.run(image, containerConfig, tag).content

Example 3: docker logs --follow

def callback = new DockerAsyncCallback() {
    def lines = []

    @Override
    def onEvent(Object line) {
        println line
        lines << line
    }
}

dockerClient.logs("foo", [tail: 1], callback)

// callback.lines will now collect all log lines
// you might implement it as a fifo instead of the List shown above

Example 4: docker stack deploy --compose-file docker-stack.yml example

def dockerClient = new DockerClientImpl()
dockerClient.initSwarm()

def namespace = "example"
def composeStack = getClass().getResourceAsStream('docker-stack.yml')
String workingDir = Paths.get(getClass().getResource('docker-stack.yml').toURI()).parent

def deployConfig = new DeployConfigReader(dockerClient).loadCompose(namespace, composeStack, workingDir)

def options = new DeployStackOptions()
dockerClient.stackDeploy(namespace, deployConfig, options)

Usage with Gradle Docker Plugin

My personal focus implementing this Docker client was to leverage the Docker engine API in our Gradle scripts. A convenient integration in Gradle is possible by using the Gradle Docker Plugin, which will be developed along with the Docker client library.

Contributing and Future Plans

If something doesn't work as expected or if you have suggestions, please create an issue. Pull requests are welcome as well!

The developer api is quite rough, but that's where you can certainly help: I'll add a more convenient layer on top of the raw interface so that for 80% of common use cases the Docker client should help you to get it working without digging too deep into the source code. So, all I need is an indication about how you'd like that convenience layer to look like. Feel free to create an issue where we can discuss how your use case could be implemented!

Buy Me a Coffee at ko-fi.com

Publishing/Release Workflow

See RELEASE.md

License

MIT License

Copyright 2015-2021 Tobias Gesellchen (@gesellix)

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

docker-client's People

Contributors

dependabot[bot] avatar dvallant avatar gesellix avatar github-actions[bot] avatar gradle-update-robot avatar imago-storm avatar nikoglyph avatar sgrell avatar tobad357 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

docker-client's Issues

DOCKER_TLS_VERIFY=1 will fail always the TLS check

In de.gesellix.docker.client.protocolhandler.DockerURLHandler:

 if ((dockerTlsVerify && Boolean.valueOf(dockerTlsVerify) == FALSE) || "0".equals(dockerTlsVerify)) {

will always produce true and thus fail the check for TLS, because Boolean.valueOf("1") == FALSE. You should check that if its defined and it equals to 0 (or false or no as string, but not the Boolean.valueOf which returns false on everything except ignore-case "true").

Log:

20:15:08.039 [INFO] [de.gesellix.docker.client.DockerClientImpl] using docker at 'tcp://127.0.0.1:2376'
20:15:08.074 [DEBUG] [de.gesellix.docker.client.protocolhandler.DockerURLHandler] dockerTlsVerify=1
20:15:08.078 [INFO] [de.gesellix.docker.client.protocolhandler.DockerURLHandler] assume 'http'
20:15:08.079 [DEBUG] [de.gesellix.docker.client.protocolhandler.DockerURLHandler] selected dockerHost at 'http://127.0.0.1:2376'

If you don't have DOCKER_TLS_VERIFY environment variable defined at all, you will correctly get the certPath, etc. defined.

add v1.21 support (Docker 1.9)

new endpoints:

  • /volumes
  • /networks
  • check changed endpoints (incompatibilities?)

note to self:

  • unit tests
  • integration tests
  • implementation
  • docs/roadmap

Migrate to Kotlin

After the underlying libs (docker-filesocket and docker-compose-v3) have already been migrated, the time has come to convert the main repository.

My philosophy behind this Docker client is to keep the Docker engine facing side as flexible as possible (to keep our code as stable as possible for different engine api versions). Types might be less convenient, so I have no idea whether Kotlin is counter productive.

My main motivation to introduce Kotlin is the library's client facing side. A strongly typed api certainly helps to use a library given a capable IDE. That way, the lack of documentation might be compensated to a certain degree.

Things to keep in mind:

There's no static modifier in Kotlin. static someMethod(){} could be converted to @JvmStatic fun someMethod(), but it would also be valid to simply remove the static modifier and make someMethod() an instance method.
We should focus on the production code first. Test code can also be converted to Kotlin, but with low priority. I'm a fan of the Groovy Spock framework and there's no equivalent in Kotlin, yet (I didn't find any). Spek seems to be the most similar alternative as of today.
See docker-client/docker-filesocket@620cc55 for a first try at converting from Groovy to Kotlin.

Lack of documentation

Great lib, but the lack of documentation difficult the usage and even the evolution by community.

Docker Swarm mode

Does the library support the APIs introduced in docker swarm 1.12 ?
i mean the new:
swarm, services , node , task ans so on ...APIs

Cheers

Error in DeployConfigReader usage of Collections.sort

Line 114, 116 use Collection.sort and then assign to a variable
The problem is Collections.sort is an in place sorting so the variables will always end up as null

hosts : Collections.sort(convertExtraHosts(service.extraHosts)),
should be
hosts : convertExtraHosts(service.extraHosts),

same for env variables

Allow relative bind mount directories

When doing testing it would be very useful to have docker-client changing relative bind mount paths to absolute paths as docker only allows absolute paths while people using docker-client may be running it from different project dirs

My suggestion would be that when using loadCompose and option could be to convert the relative volume paths to absolute ones based on the work dir

Example when workdir is /path/to/project/root

    volumes:
      - "./build/data:/data" 

becomes

    volumes:
      - "/path/to/project/root/./build/data:/data" 

add v1.24 support (Docker 1.12)

The major change is the merge of the old remote api with the Swarm api.

new endpoints:

  • GET /nodes
  • GET /nodes/(id)
  • POST /swarm/init
  • POST /swarm/join
  • POST /swarm/leave
  • POST /swarm/update
  • GET /services
  • POST /services/create
  • DELETE /services/(id or name)
  • GET /services/(id or name)
  • POST /services/(id or name)/update
  • GET /swarm/tasks
  • GET /tasks/(task id)

note to self:

  • unit tests
  • integration tests
  • implementation
  • docs/roadmap

prepare JDK9 compatibility

We already considered using OkHttp in #26 as possible alternative to the current sun.net.* dependencies.

See jdep output for details:

$ <jdk9>/bin/jdeps -jdkinternals ./docker-client.jar 
docker-client.jar -> java.base
   de.gesellix.docker.client.protocolhandler.DockerContentHandlerFactory (docker-client.jar)
      -> sun.net.www.content.text.plain                     JDK internal API (java.base)
   de.gesellix.docker.client.protocolhandler.urlstreamhandler.HttpOverNamedPipeClient (docker-client.jar)
      -> sun.net.NetworkClient                              JDK internal API (java.base)
      -> sun.net.www.MessageHeader                          JDK internal API (java.base)
      -> sun.net.www.http.HttpClient                        JDK internal API (java.base)
      -> sun.net.www.http.PosterOutputStream                JDK internal API (java.base)
   de.gesellix.docker.client.protocolhandler.urlstreamhandler.HttpOverNamedPipeClient$_printHeaders_closure1 (docker-client.jar)
      -> sun.net.www.MessageHeader                          JDK internal API (java.base)
   de.gesellix.docker.client.protocolhandler.urlstreamhandler.HttpOverNamedPipeURLConnection (docker-client.jar)
      -> sun.net.www.http.HttpClient                        JDK internal API (java.base)
      -> sun.net.www.protocol.http.Handler                  JDK internal API (java.base)
      -> sun.net.www.protocol.http.HttpURLConnection        JDK internal API (java.base)
   de.gesellix.docker.client.protocolhandler.urlstreamhandler.HttpOverUnixSocketClient (docker-client.jar)
      -> sun.net.NetworkClient                              JDK internal API (java.base)
      -> sun.net.www.http.HttpClient                        JDK internal API (java.base)
   de.gesellix.docker.client.protocolhandler.urlstreamhandler.HttpOverUnixSocketURLConnection (docker-client.jar)
      -> sun.net.www.http.HttpClient                        JDK internal API (java.base)
      -> sun.net.www.protocol.http.Handler                  JDK internal API (java.base)
      -> sun.net.www.protocol.http.HttpURLConnection        JDK internal API (java.base)

Warning: JDK internal APIs are unsupported and private to JDK implementation that are
subject to be removed or changed incompatibly and could break your application.
Please modify your code to eliminate dependency on any JDK internal APIs.
For the most recent update on JDK internal API replacements, please check:
https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool

Allow docker pull by digest

Great library!

I would like to be able to pull an image specified by its digest, the equivalent of (by example):

docker pull nginx@sha256:b555f8c64ab4e85405e0d8b03f759b73ce88deb802892a3b155ef55e3e832806

In fact I would like to do this from the gradle-docker-plugin. Any plans for adding support for digests?

dockerignore broken?

given

node_modules/
.tmp/
release/
target/

includes those directories in the build context tar file.

use plain urlconnection where possible

Relevant use case: docker attach would need a streaming response, but the currently used httpclient consumes the response and closes the inputstream.
Since we're only using a quite simple request/response pattern, we might be able to use plain Java classes and minimal external dependencies.

docker.tls.verify has no effect

I want to connect to non-TLS docker host. For that purpose, I am setting environment variable docker.tls.verify to 0. Since no docker.cert.path is provided, docker-client takes defaultDockerCertPath which exists (I can not delete that folder) and tries to establish secure connection.

Here is the code snippet:

System.setProperty("docker.tls.verify", "0")
def dockerClient = new DockerClientImpl(endpoint)
dockerClient.info().content

Here are the logs from docker-client run:

21:25:37.381 [main] DEBUG d.g.docker.engine.DockerClientConfig - defaultDockerCertPath=/home/<username>/.docker
21:25:37.385 [main] DEBUG d.g.docker.engine.DockerClientConfig - certsPathExists=true, isTlsPort=true
21:25:37.390 [main] DEBUG d.g.docker.engine.DockerClientConfig - assume 'https'
21:25:37.391 [main] DEBUG d.g.docker.engine.DockerClientConfig - selected dockerHost at '[certPath:/home/<username>/.docker, protocol:https, host:192.168.5.160, port:2376]'
21:25:37.455 [main] INFO  d.g.docker.client.DockerClientImpl - using docker at 'tcp://192.168.5.160:2376'
21:25:37.621 [main] INFO  d.g.d.c.system.ManageSystemClient - docker info
java.io.FileNotFoundException: /home/<username>/.docker/key.pem (No such file or directory)

verify to behave like the official cli

e.g. check the environment variable support and default behaviours: https://docs.docker.com/engine/reference/commandline/cli/#environment-variables

List of relevant properties:

  • DOCKER_API_VERSION The API version to use (e.g. 1.19)
  • DOCKER_CONFIG The location of your client configuration files.
  • DOCKER_CERT_PATH The location of your authentication keys.
  • DOCKER_DRIVER The graph driver to use.
  • DOCKER_HOST Daemon socket to connect to.
  • DOCKER_NOWARN_KERNEL_VERSION Prevent warnings that your Linux kernel is unsuitable for Docker.
  • DOCKER_RAMDISK If set this will disable ‘pivot_root’.
  • DOCKER_TLS_VERIFY When set Docker uses TLS and verifies the remote.
  • DOCKER_CONTENT_TRUST When set Docker uses notary to sign and verify images. Equates to --disable-content-trust=false for build, create, pull, push, run.
  • DOCKER_CONTENT_TRUST_SERVER The URL of the Notary server to use. This defaults to the same URL as the registry.
  • DOCKER_TMPDIR Location for temporary Docker files.

Support for ECC client certs provided by Docker EE UCP

Docker Universal Control Plane allows user to download client bundle which contains TLS cert files to connect docker CLI to it. The bundle contains client keypairs in ECC (Elliptic curve cryptography) algorithm format. Using these certs causes

java.security.spec.InvalidKeySpecException: java.security.InvalidKeyException: Invalid RSA private key .

working with the DockerRunTask

When I have a simple 'echo' command in my execInContainer task, I get a null response. Here is the debug:

10:34:34.184 [INFO] [de.gesellix.docker.client.DockerClientImpl] docker start exec 'db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a'
10:34:34.185 [INFO] [de.gesellix.docker.client.DockerClientImpl] docker inspect exec 'db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a'
10:34:34.188 [DEBUG] [de.gesellix.docker.client.OkDockerClient] GET http://2f7661722f72756e2f646f636b65722e736f636b.socket/exec/db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a/json using proxy: DIRECT
10:34:34.189 [DEBUG] [de.gesellix.docker.client.filesocket.UnixSocket] connect via '/var/run/docker.sock'...
10:34:34.190 [DEBUG] [de.gesellix.docker.client.OkDockerClient] response: Response{protocol=http/1.1, code=200, message=OK, url=http://2f7661722f72756e2f646f636b65722e736f636b.socket/exec/db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a/json}
10:34:34.190 [DEBUG] [de.gesellix.docker.client.OkDockerClient] status: [text:OK, code:200, success:true]
10:34:34.190 [DEBUG] [de.gesellix.docker.client.OkDockerClient] headers: 
Content-Type: application/json
Server: Docker/1.12.0 (linux)
Date: Fri, 12 Aug 2016 16:34:34 GMT
Content-Length: 382

10:34:34.194 [DEBUG] [de.gesellix.docker.client.OkDockerClient] POST http://2f7661722f72756e2f646f636b65722e736f636b.socket/exec/db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a/start using proxy: DIRECT
10:34:34.195 [DEBUG] [de.gesellix.docker.client.filesocket.UnixSocket] connect via '/var/run/docker.sock'...
10:34:34.198 [DEBUG] [de.gesellix.docker.client.OkDockerClient] response: Response{protocol=http/1.1, code=200, message=OK, url=http://2f7661722f72756e2f646f636b65722e736f636b.socket/exec/db91e0a4c9eec744f26c9185e638ba0347b7cdfccc26842888ebf87316544e6a/start}
10:34:34.198 [DEBUG] [de.gesellix.docker.client.OkDockerClient] status: [text:OK, code:200, success:true]
10:34:34.198 [DEBUG] [de.gesellix.docker.client.OkDockerClient] headers: 
Content-Type: application/vnd.docker.raw-stream

10:34:34.218 [INFO] [org.gradle.api.Task] de.gesellix.docker.client.DockerResponse(status:[text:OK, code:200, success:true], headers:Content-Type: application/vnd.docker.raw-stream
, contentType:application/vnd.docker.raw-stream, mimeType:application/vnd.docker.raw-stream, contentLength:-1, stream:de.gesellix.docker.client.rawstream.RawInputStream@39da5e49, content:null)
10:34:34.218 [DEBUG] [org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter] Finished executing task ':execInContainer'

My gradle task looks like this:

plugins {
  id "de.gesellix.docker" version "2016-07-24T01-05-06"
}

import de.gesellix.gradle.docker.tasks.*
import java.util.logging.Logger

task rmImage(type: DockerRmiTask) {
  imageId = "atom"
}

task buildImage(type: DockerBuildTask) {
  dependsOn rmImage
  imageName = "atom"
  buildContextDirectory = file(".")
}

task stopContainer(type: DockerStopTask) {
  dependsOn buildImage
  containerId = "atom"
}

task rmContainer(type: DockerRmTask) {
  dependsOn stopContainer
  containerId = "atom"
}

task runContainer(type: DockerRunTask) {
  dependsOn rmContainer
  imageName = "atom"
  containerName = "atom"
}

task execInContainer(type: DockerExecTask) {
  dependsOn runContainer

  containerId = "atom"
  commandLine = 'echo "it works!"'

  doLast {
    logger.info("$result")
  }
}

Am I missing something? I can't figure out why it fails to even return the simple echo command.

Can I use it in jenkins pipeline?

I want to make a pipeline job to pull images and create containers on remote docker。jenkins‘s pipeline is written in groovy,so can i import the api into a jenkins pipeline job

Does docker service need X-Registry-Auth header?

Hello,

Trying to create a service with an image from a private repository but docker rejects it. Apparently this has to do with the registry authentication. The ManageServiceClient.createService does not let you pass base64 authentication details as ManageImageClient.pull does. Is this by design or a limitation?

Thanks

Java implementation for Docker Compose

Since Docker Compose ain't an api, but a pure cli tool (docker/compose#230), we need to decide how to support the docker-compose.yml file format. A minimum is necessary to implement the docker stack deploy command.

Options:

Side note: I didn't find any existing implementation for Java/Groovy/Kotlin, yet.

add docker exec support

it would be user to add support for executing commands in a running container, i.e, docker exec

Use same logic as docker client when finding default docker host

Hello. First - thank you for this and the Gradle plugin - it's working quite well for us.

I think the default dockerHost is a bit odd - with default options Docker daemon won't be listening there. What about switching to the same logic the Docker client uses? I.e. first check DOCKER_HOST environment variable, otherwise assume the default unix socket?

If you are happy with this I don't mind doing a Pull Request.

Unable to delete certificate files in 'docker.cert.path'

We are using de.gesellix:docker-client:2017-07-19T21-12-05 to deploy services on a Docker swam cluster.

We have a process that accepts information on the Docker endpoint and the service to deploy as input. The process writes out the pem files to a temporary directory, configures the 'docker.cert.path' to point to this location before calling the DockerClient APIs which do all the work.
Finally the process needs to remove the pem files by simply deleting the temporary directory (the directory is created with a unique path every time allowing parallel runs to not clobber each other). I found that the files are not getting deleted after the DockerClient APIs are invoked. If I comment out the call to the APIs, then the deletion goes through fine. I suspect that the file streams to the pem files are opened somewhere in the docker-client code but they are never closed. This prevents the files from getting deleted.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.