kubernetes-client / gen Goto Github PK
View Code? Open in Web Editor NEWCommon generator scripts for all client libraries
License: Apache License 2.0
gen's People
Contributors
Stargazers
Watchers
Forkers
brendandburns mbohlool krabhishek8260 noironetworks guoshimin tg123 josedonizetti lewisheadden nicolaferraro spiffxp sivagao craigtracey roycaihw technosophos tomplus voyagermesh dservgun dmonroy olitheolix kubeflow-incubator juliantaylor nikhita itowlson marie-yau davidxia mbovo yue9944882 tpetr lholmquist rlenferink ackintosh openapitools jonschoning binarycrayon jay-rob johscheuer pickware fabiokung frincon scottilee sataqiu palnabarun janosi t-skgm sheldonkwok qiaoxingli ityuhui fabianvf edwinpjacques drauyueng shnkr tony-shi wavezhang manshuxu elonzh szab100 lou-lan littlebaibai dsyer abelsromero isabella232 mu-l karunasagark mrbobbytables global-localhost global19 global19-atlassian-net sfowl yliaog openai theyoungester dedsec-9 saeger alberthuang24 gm1957 junm-cloudnatix jpinkney ccravens itaru2622 alex-bezek sshyran olivermking alpha412826 rakhithjk loft-sh knkgun rotty3000 arrikto max-l-f atguo alexrogalskiy uesyn fombico tryweirdier sayakmukhopadhyay spacether qaz123ly jacobhenner daimg bop0hzgen's Issues
custom_objects_spec misses deletecollection operations
The custom object openapi spec misses deletecollection operations for cluster-scoped and namespaced custom objects. These operations are supported in apiserver. We should add them to the spec and regenerate the clients.
cc @scottilee
Typescript generator config class name is possibly outdated
While working on a fix for devfile/api#1036 that includes the fix for #236, I came across the following error:
Error:
org.openapitools.codegen.GeneratorNotFoundException: Can't load config class with name 'typescript'
Available:
ada
ada-server
android
apache2
apex
aspnetcore
bash
c
clojure
cwiki
cpp-qt5-client
cpp-qt5-qhttpengine-server
cpp-pistache-server
cpp-restbed-server
cpp-restsdk
cpp-tizen
csharp
csharp-netcore
csharp-dotnet2
csharp-nancyfx
dart
dart-jaguar
eiffel
elixir
elm
erlang-client
erlang-proper
erlang-server
flash
go
go-experimental
go-server
go-gin-server
graphql-schema
graphql-nodejs-express-server
groovy
kotlin
kotlin-server
kotlin-spring
haskell-http-client
haskell
java
jaxrs-cxf-client
java-inflector
java-msf4j
java-pkmst
java-play-framework
java-undertow-server
java-vertx
jaxrs-cxf
jaxrs-cxf-extended
jaxrs-cxf-cdi
jaxrs-jersey
jaxrs-resteasy
jaxrs-resteasy-eap
jaxrs-spec
javascript
javascript-flowtyped
javascript-closure-angular
jmeter
lua
mysql-schema
nodejs-server-deprecated
objc
openapi
openapi-yaml
perl
php
php-laravel
php-lumen
php-slim
php-silex
php-symfony
php-ze-ph
powershell
python
python-experimental
python-flask
python-aiohttp
python-blueplanet
r
ruby
ruby-on-rails
ruby-sinatra
rust
rust-server
scalatra
scala-akka
scala-finch
scala-httpclient-deprecated
scala-gatling
scala-lagom-server
scala-play-server
scalaz
spring
dynamic-html
html
html2
swift2-deprecated
swift3-deprecated
swift4
typescript-angular
typescript-angularjs
typescript-aurelia
typescript-axios
typescript-fetch
typescript-inversify
typescript-jquery
typescript-node
typescript-rxjs
fsharp-giraffe-server
at org.openapitools.codegen.CodegenConfigLoader.forName (CodegenConfigLoader.java:48)
at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput (CodegenConfigurator.java:528)
at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:670)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
Caused by: java.lang.ClassNotFoundException: typescript
at org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy.loadClass (SelfFirstStrategy.java:50)
at org.codehaus.plexus.classworlds.realm.ClassRealm.unsynchronizedLoadClass (ClassRealm.java:271)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass (ClassRealm.java:247)
at org.codehaus.plexus.classworlds.realm.ClassRealm.loadClass (ClassRealm.java:239)
at java.lang.Class.forName0 (Native Method)
at java.lang.Class.forName (Class.java:264)
at org.openapitools.codegen.CodegenConfigLoader.forName (CodegenConfigLoader.java:46)
at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput (CodegenConfigurator.java:528)
at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:670)
at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo (DefaultBuildPluginManager.java:137)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:210)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:156)
at org.apache.maven.lifecycle.internal.MojoExecutor.execute (MojoExecutor.java:148)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:117)
at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject (LifecycleModuleBuilder.java:81)
at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build (SingleThreadedBuilder.java:56)
at org.apache.maven.lifecycle.internal.LifecycleStarter.execute (LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:305)
at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:192)
at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:105)
at org.apache.maven.cli.MavenCli.execute (MavenCli.java:972)
at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:293)
at org.apache.maven.cli.MavenCli.main (MavenCli.java:196)
at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke (NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke (DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke (Method.java:498)
at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced (Launcher.java:282)
at org.codehaus.plexus.classworlds.launcher.Launcher.launch (Launcher.java:225)
at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode (Launcher.java:406)
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.045 s
[INFO] Finished at: 2023-02-28T20:19:14Z
[INFO] ------------------------------------------------------------------------
Error: Failed to execute goal org.openapitools:openapi-generator-maven-plugin:4.0.3:generate (default) on project client-typescript: Code generation failed. See above for the full exception. -> [Help 1]
Error:
Error: To see the full stack trace of the errors, re-run Maven with the -e switch.
Error: Re-run Maven using the -X switch to enable full debug logging.
Error:
Error: For more information about the errors and possible solutions, please read the following articles:
Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Error: Process completed with exit code 1.
It seems that the name for typescript is missing from the generators list and possibly outdated. I have created a folk https://github.com/michael-valdron/kubernetes-client-gen/tree/ts-class-fix that changes the config class name from typescript to typescript-node, this seems to have worked in my use case.
If this is an appropriate fix for this I can create a PR from this folk.
Create a SECURITY_CONTACTS file.
As per the email sent to kubernetes-dev[1], please create a SECURITY_CONTACTS
file.
The template for the file can be found in the kubernetes-template repository[2].
A description for the file is in the steering-committee docs[3], you might need
to search that page for "Security Contacts".
Please feel free to ping me on the PR when you make it, otherwise I will see when
you close this issue. :)
Thanks so much, let me know if you have any questions.
(This issue was generated from a tool, apologies for any weirdness.)
[1] https://groups.google.com/forum/#!topic/kubernetes-dev/codeiIoQ6QE
[2] https://github.com/kubernetes/kubernetes-template-project/blob/master/SECURITY_CONTACTS
[3] https://github.com/kubernetes/community/blob/master/committee-steering/governance/sig-governance-template-short.md
Dubious Ownership error for /source/openapi-generator
When running the generator inside the container, the following error is reported, and generation fails.
/source/openapi-generator /
fatal: detected dubious ownership in repository at '/source/openapi-generator'
To add an exception for this directory, call:
git config --global --add safe.directory /source/openapi-generator
If it helps, I'm running the generator using java-crd-cmd.sh:
java-crd-cmd.sh -n com.vmware.tanzu.apps.config -p com.vmware.tanzu.acs -l 2 -h true -o /tmp/gen-output -g trueFor now, I've worked around the issue by adding RUN git config --system --add safe.directory '/source/openapi-generator' in my fork. I'm happy to contribute the change, but I wasn't sure if this would be the best fix.
Protobuf generation broken?
Trying to generate Java protobuf isn't working for me. I used the version of protoc in the install.sh script.
Running generate.sh java /tmp/java
I get:
Munging proto file packages
Generating code for java
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/authorization/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/authorization/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/authorization/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/authorization/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/authorization/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/authorization/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/imagepolicy/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/imagepolicy/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/imagepolicy/v1alpha1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/certificates/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/certificates/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/certificates/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/extensions/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/extensions/v1beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/extensions/v1beta1/generated.proto: warning: Import k8s.io/api/policy/v1beta1/generated.proto but not used.
k8s.io/api/autoscaling/v2beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/autoscaling/v2beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/autoscaling/v2beta1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/autoscaling/v2beta1/generated.proto: warning: Import k8s.io/api/core/v1/generated.proto but not used.
k8s.io/api/autoscaling/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto but not used.
k8s.io/api/autoscaling/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto but not used.
k8s.io/api/autoscaling/v1/generated.proto: warning: Import k8s.io/apimachinery/pkg/util/intstr/generated.proto but not used.
k8s.io/api/autoscaling/v1/generated.proto: warning: Import k8s.io/api/core/v1/generated.proto but not used.
[libprotobuf WARNING google/protobuf/compiler/parser.cc:546] No syntax specified for the proto file: k8s.io/api/admission/v1alpha1/generated.proto. Please use 'syntax = "proto2";' or 'syn
tax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
k8s.io/api/admission/v1alpha1/generated.proto:1:1: Expected top-level statement (e.g. "message").
Dockerize generators
Generators are using maven right now. Dockerize them to have a more consistent way of generating clients across platforms.
Error installing dotnet-sdk-5.0
Hello, I am trying to run java-crd-cmd.sh on the repo, and I see it fails installing dotnet-sdk-5.0:
KUBERNETES_CRD_GROUP_PREFIX: {REMOVED}
OPENAPI_MODEL_LENGTH: 2
PACKAGE_NAME: {REMOVED}
GENERATE_APIS: true
CLIENT_VERSION: 5.0-SNAPSHOT
OUTPUT_DIR: /tmp/gen-output
HIDE_GENERATION_TIMESTAMP: true
LIBRARY: okhttp-gson
rendering settings file to /tmp/settings
reading input openapi specs
--- Building docker image kubernetes-java-client-gen-with-openapi-generator:v1...
[+] Building 1.5s (17/24)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 2.67kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/maven:3.5-jdk-8-slim 0.3s
=> [internal] load build context 0.0s
=> => transferring context: 90.63kB 0.0s
=> [ 1/20] FROM docker.io/library/maven:3.5-jdk-8-slim@sha256:ec1edb56d6803521e65f83769063bfe34e4e5264b7e13e90f2c38823ea37503c 0.0s
=> CACHED [ 2/20] RUN apt-get update && apt-get -qq -y install libunwind8 libicu57 libssl1.0 liblttng-ust0 libcurl3 libuuid1 libkrb5-3 zlib1g 0.0s
=> CACHED [ 3/20] RUN curl -sL https://deb.nodesource.com/setup_14.x | bash - 0.0s
=> CACHED [ 4/20] RUN apt-get update && apt-get -y install nodejs libunwind8-dev && rm -rf /var/lib/apt/lists/* 0.0s
=> CACHED [ 5/20] RUN apt-get update && apt-get -y install git python-pip && pip install urllib3==1.24.2 0.0s
=> CACHED [ 6/20] RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg 0.0s
=> CACHED [ 7/20] RUN mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/ 0.0s
=> CACHED [ 8/20] RUN curl https://packages.microsoft.com/config/debian/9/prod.list > prod.list 0.0s
=> CACHED [ 9/20] RUN mv prod.list /etc/apt/sources.list.d/microsoft-prod.list 0.0s
=> CACHED [10/20] RUN chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg 0.0s
=> CACHED [11/20] RUN chown root:root /etc/apt/sources.list.d/microsoft-prod.list 0.0s
=> CACHED [12/20] RUN apt-get update 0.0s
=> ERROR [13/20] RUN apt-get install -yy -q dotnet-sdk-5.0 1.0s
------
> [13/20] RUN apt-get install -yy -q dotnet-sdk-5.0:
#16 0.224 Reading package lists...
#16 0.769 Building dependency tree...
#16 0.903 Reading state information...
#16 0.984 The following additional packages will be installed:
#16 0.984 aspnetcore-runtime-5.0 aspnetcore-targeting-pack-5.0 dotnet-apphost-pack-5.0
#16 0.984 dotnet-host dotnet-hostfxr-5.0 dotnet-runtime-5.0 dotnet-runtime-deps-5.0
#16 0.985 dotnet-targeting-pack-5.0 netstandard-targeting-pack-2.1
#16 1.004 The following NEW packages will be installed:
#16 1.004 aspnetcore-runtime-5.0 aspnetcore-targeting-pack-5.0 dotnet-apphost-pack-5.0
#16 1.004 dotnet-host dotnet-hostfxr-5.0 dotnet-runtime-5.0 dotnet-runtime-deps-5.0
#16 1.005 dotnet-sdk-5.0 dotnet-targeting-pack-5.0 netstandard-targeting-pack-2.1
#16 1.007 0 upgraded, 10 newly installed, 0 to remove and 47 not upgraded.
#16 1.007 Need to get 95.5 MB of archives.
#16 1.007 After this operation, 396 MB of additional disk space will be used.
#16 1.007 WARNING: The following packages cannot be authenticated!
#16 1.007 dotnet-runtime-deps-5.0 dotnet-host dotnet-hostfxr-5.0 dotnet-runtime-5.0
#16 1.007 aspnetcore-runtime-5.0 dotnet-targeting-pack-5.0
#16 1.007 aspnetcore-targeting-pack-5.0 dotnet-apphost-pack-5.0
#16 1.007 netstandard-targeting-pack-2.1 dotnet-sdk-5.0
#16 1.013 E: There were unauthenticated packages and -y was used without --allow-unauthenticated
------
executor failed running [/bin/sh -c apt-get install -yy -q dotnet-sdk-5.0]: exit code: 100
When I checkout the repo at 6694faf24cea3884e9c51fb316fc206a53dfcec5, it is working.
However, when I checkout the repo at 59b81c6360ce1a453594d6e6570f7d70ff9bfb86, I get the same error. So I think the cause is due to merging #201 - link new csharp generator
Support ad-hoc client generation for CRDs
as 1.15's out, users can publish their own openapi spec w/ CRD integration. we should probably refactor the scripts to support ad-hoc client generations.
failed generating with latest openapi-generator ver 5.x
when I tried gen with latest openapi-generator v5.3.0, the following error raised in gen.
The reason is very simple.
gen/openapi/{{language}}.xml is still using older 'language' perspective and no replaced 'generatorName' perspective.
the below patch passed the openapi-generator process in case of python client.
I think all other language clients have the same issue.
diff --git a/openapi/python.xml b/openapi/python.xml
--- a/openapi/python.xml
+++ b/openapi/python.xml
<configuration>
<language>python</language>
+ <generatorName>python</generatorName>
<gitUserId>kubernetes-client</gitUserId>error message:
export OPENAPI_GENERATOR_COMMIT="v5.3.0"
generate client for python (by scripts/update-clients.sh in kubernetes-client/python )
:
[INFO] --------------------< io.kubernetes:client-python >---------------------
[INFO] Building client-python 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
:
Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/maven-artifact/3.3.1/maven-artifact-3.3.1.jar (55 kB at 95 kB/s)
[ERROR] A generator name (generatorName) is required.
[WARNING] /output_dir/swagger.json [0:0]: unexpected error in Open-API generation
org.apache.maven.plugin.MojoExecutionException: The generator requires 'generatorName'. Refer to documentation for a list of options.
at org.openapitools.codegen.plugin.CodeGenMojo.execute (CodeGenMojo.java:550)
:
at org.codehaus.plexus.classworlds.launcher.Launcher.main (Launcher.java:347)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.openapitools:openapi-generator-maven-plugin:5.3.0:generate (default) on project client-python: Code generation failed. See above for the full exception. -> [Help 1]client-generator.sh fails when running docker container as current user
openapi/client-generator.sh's final docker run command fails for me.
repro steps
- Clone kubernetes-client/gen and kubernetes-client/java.
$ git clone [email protected]:kubernetes-client/gen ~/gen
$ git clone [email protected]:kubernetes-client/java ~/java- Add
set -exto the top of~/gen/openapi/java.sh. - Generate Java client code as described here and here.
$ cd ~/java && ~/gen/openapi/java.sh kubernetes-new ./settings
+ set -o errexit
+ set -o nounset
+ set -o pipefail
+ ARGC=2
+ '[' 2 -ne 2 ']'
+ OUTPUT_DIR=kubernetes-new
+ SETTING_FILE=./settings
+ mkdir -p kubernetes-new
++ dirname /home/dxia/gen/openapi/java.sh
+ SCRIPT_ROOT=/home/dxia/gen/openapi
+ pushd /home/dxia/gen/openapi
++ pwd
+ SCRIPT_ROOT=/home/dxia/gen/openapi
+ popd
+ pushd kubernetes-new
++ pwd
+ OUTPUT_DIR=/home/dxia/java/kubernetes-new
+ popd
+ source /home/dxia/gen/openapi/client-generator.sh
++ set -o errexit
++ set -o nounset
++ set -o pipefail
+ source ./settings
++ export KUBERNETES_BRANCH=release-1.12
++ KUBERNETES_BRANCH=release-1.12
++ export CLIENT_VERSION=3.0-SNAPSHOT
++ CLIENT_VERSION=3.0-SNAPSHOT
++ export PACKAGE_NAME=io.kubernetes.client
++ PACKAGE_NAME=io.kubernetes.client
+ SWAGGER_CODEGEN_COMMIT=5d263e1c9cdd395d93adf061c63d5ef58a8e9ec5
+ CLIENT_LANGUAGE=java
+ CLEANUP_DIRS=(docs src/test/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/models src/main/java/io/kubernetes/client/auth gradle)
+ kubeclient::generator::generate_client /home/dxia/java/kubernetes-new
+ : docs
+ : release-1.12
+ : 3.0-SNAPSHOT
+ : io.kubernetes.client
+ : java
+ SWAGGER_CODEGEN_USER_ORG=swagger-api
+ SWAGGER_CODEGEN_COMMIT=5d263e1c9cdd395d93adf061c63d5ef58a8e9ec5
+ USERNAME=kubernetes
+ REPOSITORY=kubernetes
+ local output_dir=/home/dxia/java/kubernetes-new
+ pushd /home/dxia/java/kubernetes-new
++ pwd
+ local output_dir=/home/dxia/java/kubernetes-new
+ popd
++ dirname /home/dxia/gen/openapi/client-generator.sh
+ SCRIPT_ROOT=/home/dxia/gen/openapi
+ pushd /home/dxia/gen/openapi
++ pwd
+ local SCRIPT_ROOT=/home/dxia/gen/openapi
+ popd
+ mkdir -p /home/dxia/java/kubernetes-new
+ echo '--- Building docker image...'
--- Building docker image...
+ '[' kubernetes '!=' kubernetes ']'
+ image_name=kubernetes-java-client-gen:v1
+ docker build /home/dxia/gen/openapi -t kubernetes-java-client-gen:v1 --build-arg SWAGGER_CODEGEN_USER_ORG=swagger-api --build-arg SWAGGER_CODEGEN_COMMIT=5d263e1c9cdd395d93adf061c63d5ef58a8e9ec5 --build-arg GENERATION_XML_FILE=java.xml
Sending build context to Docker daemon 121.9kB
Step 1/26 : FROM maven:3.5-jdk-8-slim
---> a045f84d6a3d
Step 2/26 : ARG SWAGGER_CODEGEN_COMMIT
---> Using cache
---> a255c0bdc80a
Step 3/26 : ARG GENERATION_XML_FILE
---> Using cache
---> 1522badfd7f4
Step 4/26 : ARG SWAGGER_CODEGEN_USER_ORG=swagger-api
---> Using cache
---> ed138cfa431b
Step 5/26 : RUN apt-get update && apt-get -y install git python-pip && pip install urllib3
---> Using cache
---> fd741fd3053d
Step 6/26 : RUN apt-get update && apt-get -qq -y install libunwind8 libicu57 libssl1.0 liblttng-ust0 libcurl3 libuuid1 libkrb5-3 zlib1g
---> Using cache
---> bf6ee9a53cf4
Step 7/26 : RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
---> Using cache
---> 3a91b572f130
Step 8/26 : RUN apt-get update && apt-get -y install nodejs libunwind8-dev && rm -rf /var/lib/apt/lists/*
---> Using cache
---> e04e17dfb6d4
Step 9/26 : RUN npm install -g autorest
---> Using cache
---> 5cb60ba1c7af
Step 10/26 : RUN mkdir /source && cd /source && git clone -n https://github.com/${SWAGGER_CODEGEN_USER_ORG}/swagger-codegen.git && cd swagger-codegen && git checkout $SWAGGER_CODEGEN_COMMIT
---> Using cache
---> edafb1d1fb91
Step 11/26 : RUN mkdir /.npm && chmod -R go+rwx /.npm && chmod -R go+rwx /root && umask 0 && cd /source/swagger-codegen && mvn install -DskipTests -Dmaven.test.skip=true -pl modules/swagger-codegen-maven-plugin -am && cp -r /root/.m2/* /usr/share/maven/ref
---> Using cache
---> a47047f92da2
Step 12/26 : RUN mkdir -p /node_modules && chmod -R go+rwx /node_modules
---> Using cache
---> db6685b5676a
Step 13/26 : RUN npm install @microsoft.azure/autorest.csharp @microsoft.azure/autorest.modeler
---> Using cache
---> 08b8ba5e7deb
Step 14/26 : RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg
---> Using cache
---> 8aafca706b37
Step 15/26 : RUN mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
---> Using cache
---> b26721396501
Step 16/26 : RUN curl https://packages.microsoft.com/config/debian/9/prod.list > prod.list
---> Using cache
---> 99f1ad3b2e4b
Step 17/26 : RUN mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
---> Using cache
---> 83f246907714
Step 18/26 : RUN chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
---> Using cache
---> 170952156af4
Step 19/26 : RUN chown root:root /etc/apt/sources.list.d/microsoft-prod.list
---> Using cache
---> 9f2b7d08c703
Step 20/26 : RUN apt-get update
---> Using cache
---> 57f783b3bf71
Step 21/26 : RUN apt-get install -yy -q dotnet-hosting-2.0.8
---> Using cache
---> 8e4de4892426
Step 22/26 : COPY generate_client_in_container.sh /generate_client.sh
---> Using cache
---> f2c4e602f742
Step 23/26 : COPY preprocess_spec.py /
---> Using cache
---> 6bb47c4b413a
Step 24/26 : COPY custom_objects_spec.json /
---> Using cache
---> eba2359717c6
Step 25/26 : COPY ${GENERATION_XML_FILE} /generation_params.xml
---> Using cache
---> f6c2f8db399e
Step 26/26 : ENTRYPOINT mvn-entrypoint.sh /generate_client.sh
---> Using cache
---> 1b514d271cdc
Successfully built 1b514d271cdc
Successfully tagged kubernetes-java-client-gen:v1
+ CLEANUP_DIRS_STRING='docs src/test/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/models src/main/java/io/kubernetes/client/auth gradle'
+ echo '--- Running generator inside container...'
--- Running generator inside container...
++ id -u
+ docker run -u 4768 -e 'CLEANUP_DIRS=docs src/test/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/apis src/main/java/io/kubernetes/client/models src/main/java/io/kubernetes/client/auth gradle' -e KUBERNETES_BRANCH=release-1.12 -e CLIENT_VERSION=3.0-SNAPSHOT -e CLIENT_LANGUAGE=java -e PACKAGE_NAME=io.kubernetes.client -e SWAGGER_CODEGEN_USER_ORG=swagger-api -e SWAGGER_CODEGEN_COMMIT=5d263e1c9cdd395d93adf061c63d5ef58a8e9ec5 -e USERNAME=kubernetes -e REPOSITORY=kubernetes -v /home/dxia/java/kubernetes-new:/output_dir kubernetes-java-client-gen:v1 /output_dir
Can not write to /root/.m2/copy_reference_file.log. Wrong volume permissions? Carrying on ...
Cannot find swagger-codegen-maven-plugin versionexpected result
The script above succeeds and generates Java client code.
other info
I think the ultimate error is Cannot find swagger-codegen-maven-plugin version which is caused by the docker run -u $(id -u) introduced in this PR. If I rerun ~/gen/openapi/java.sh kubernetes-new ./settings without -u $(id -u) as shown here, the script succeeds.
platform info
$ uname -a
Linux guc3-heliosjenkins-b-49ey.guc3.spotify.net 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04 LTS
Release: 14.04
Codename: trustydocker version
$ docker version
Client:
Version: 17.05.0-ce
API version: 1.29
Go version: go1.7.5
Git commit: 89658be
Built: Thu May 4 22:06:06 2017
OS/Arch: linux/amd64
Server:
Version: 17.05.0-ce
API version: 1.29 (minimum version 1.12)
Go version: go1.7.5
Git commit: 89658be
Built: Thu May 4 22:06:06 2017
OS/Arch: linux/amd64
Experimental: false
$ docker info
Containers: 9
Running: 1
Paused: 0
Stopped: 8
Images: 2453
Server Version: 17.05.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 2717
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Swarm: locked
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 9048e5e50717ea4497b757314bad98ea3763c145
runc version: 9c2d8d184e5da67c95d601382adf14862e4f2228
init version: 949e6fa
Security Options:
apparmor
Kernel Version: 3.16.0-77-generic
Operating System: Ubuntu 14.04 LTS
OSType: linux
Architecture: x86_64
CPUs: 8
Total Memory: 29.2GiB
Name: myhost.com
ID: T27Q:JS26:C367:AKIT:NSWI:GCAM:AKST:6BRN:HMYG:OPBZ:2CAM:2PUA
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: davidxia
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit supportpreprocess_spec.py: Add conditional for crd_mode or metadata without $ref key
It seems like the api-server returns an invalid CRD List type, as commented here: kubernetes-client/java#921 (comment). As a result, the API specification returned by the api-server for a CRD List type may look like this (extract):
{
"metadata": {
"description": "ListMeta describes metadata that synthetic resources must have, including lists and various status objects. A resource may have only one of {ObjectMeta, ListMeta}.",
"type": "object",
"properties": {
"continue": {
"description": "continue may be set if the user set a limit on the number of items returned, and indicates that the server has more data available. The value is opaque and may be used to issue another request to the endpoint that served this list to retrieve the next set of available objects. Continuing a consistent list may not be possible if the server configuration has changed or more than a few minutes have passed. The resourceVersion field returned when using this continue value will be identical to the value in the first response, unless you have received this token from an error message.",
"type": "string"
},
"remainingItemCount": {
"description": "remainingItemCount is the number of subsequent items in the list which are not included in this list response. If the list request contained label or field selectors, then the number of remaining items is unknown and the field will be left unset and omitted during serialization. If the list is complete (either because it is not chunking or because this is the last chunk), then there are no more remaining items and this field will be left unset and omitted during serialization. Servers older than v1.15 do not set this field. The intended use of the remainingItemCount is *estimating* the size of a collection. Clients should not rely on the remainingItemCount to be set or to be exact.",
"type": "integer",
"format": "int64"
},
"resourceVersion": {
"description": "String that identifies the server's internal version of this object that can be used by clients to determine when objects have changed. Value must be treated as opaque by clients and passed unmodified back to the server. Populated by the system. Read-only. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#concurrency-control-and-consistency",
"type": "string"
},
"selfLink": {
"description": "selfLink is a URL representing this object. Populated by the system. Read-only.\n\nDEPRECATED Kubernetes will stop propagating this field in 1.20 release and the field is planned to be removed in 1.21 release.",
"type": "string"
}
}
}
}instead of the usual:
{
"metadata": {
"$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ListMeta"
}
}This was compensated for in deac82e, but d5a4e0d introduces a change that breaks the crd_mode. It works with the previous commit.
I think this is the problematic line:
gen/openapi/preprocess_spec.py
Line 396 in d5a4e0d
Which results in the following error running preprocess_spec.py in crd_mode:
if v["properties"]["metadata"]["$ref"] == "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ListMeta":
KeyError: '$ref'
Error generating c# codes
Trying to follow the python-crd-cmd.sh and see if c# models can be generated, but failed with a mysterious error. Wish someone has experiences of this repo can shed some lights
[INFO] NOTE: AutoRest core version selected from configuration: ~3.1.0.
[INFO] Network Enabled: true
[INFO] Starting @autorest/core from /tmp/.autorest/@[email protected]/node_modules/@autorest/core
[INFO] Loading AutoRest core '/tmp/.autorest/@[email protected]/node_modules/@autorest/core/dist' (3.1.3)
[INFO] VERBOSE [0.44 s]: No configuration found at 'file:///'.
[INFO] VERBOSE [0.45 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/default-configuration.md'
[INFO] VERBOSE [0.47 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/inspect.md'
[INFO] VERBOSE [0.47 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/directives.md'
[INFO] VERBOSE [0.48 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/pipeline.md'
[INFO] VERBOSE [0.5 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/loader-openapi.md'
[INFO] VERBOSE [0.5 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/miscellaneous.md'
[INFO] VERBOSE [0.51 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/patches.md'
[INFO] VERBOSE [0.53 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-powershell.md'
[INFO] VERBOSE [0.54 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-azureresourceschema.md'
[INFO] VERBOSE [0.55 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-csharp.md'
[INFO] VERBOSE [0.56 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-go.md'
[INFO] VERBOSE [0.57 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-java.md'
[INFO] VERBOSE [0.58 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-nodejs.md'
[INFO] VERBOSE [0.59 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-php.md'
[INFO] VERBOSE [0.6 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-python.md'
[INFO] VERBOSE [0.61 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-ruby.md'
[INFO] VERBOSE [0.62 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-typescript.md'
[INFO] VERBOSE [0.63 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-validators.md'
[INFO] VERBOSE [0.65 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-az.md'
[INFO] VERBOSE [0.66 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-terraform.md'
[INFO] VERBOSE [0.67 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/plugin-azure-functions.md'
[INFO] VERBOSE [0.69 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/graphs.md'
[INFO] VERBOSE [0.71 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/core/resources/help-configuration.md'
[INFO] INFORMATION: > Loading AutoRest extension '@autorest/csharp' (latest->3.0.0-beta.20210712.1)
[INFO] VERBOSE [0.92 s]: > Including extension configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/csharp/readme.md'
[INFO] VERBOSE [0.92 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/csharp/readme.md'
[INFO] INFORMATION: > Loading AutoRest extension '@autorest/modelerfour' (4.19.3->4.19.3)
[INFO] VERBOSE [0.99 s]: > Including extension configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/modelerfour/readme.md'
[INFO] VERBOSE [0.99 s]: > Including configuration file 'file:///tmp/.autorest/@[email protected]/node_modules/@autorest/modelerfour/readme.md'
[INFO] Error: write EPIPE
[INFO] at afterWriteDispatched (internal/stream_base_commons.js:156:25)
[INFO] at writeGeneric (internal/stream_base_commons.js:147:3)
[INFO] at Socket._writeGeneric (net.js:787:11)
[INFO] at Socket._write (net.js:799:8)
[INFO] at writeOrBuffer (internal/streams/writable.js:358:12)
[INFO] at Socket.Writable.write (internal/streams/writable.js:303:10)
[INFO] at Writable.write [as _write] (/tmp/.autorest/@[email protected]/node_modules/@autorest/core/dist/src_lib_autorest-core_ts.js:1989:31)
[INFO] at doWrite (internal/streams/writable.js:377:12)
[INFO] at clearBuffer (internal/streams/writable.js:529:7)
[INFO] at onwrite (internal/streams/writable.js:430:7)
[INFO] at afterWrite (internal/streams/writable.js:466:5)
[INFO] at afterWriteTick (internal/streams/writable.js:453:10)
[INFO] at processTicksAndRejections (internal/process/task_queues.js:81:21)
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 7.510 s
[INFO] Finished at: 2021-07-12T23:20:51Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.1.1:exec (generate) on project client-csharp: Result of /bin/sh -c cd / && node --max-old-space-size=8192 /usr/bin/autorest --input-file=/output_dir/swagger.json --csharp --namespace=k8s --package-version=0.0.1 --output-folder=/output_dir --add-credentials --debug --verbose execution is: '1'. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionExceptionComplete support for Rust
Tracking issue to ensure we complete the process to add a rust client.
Ref #192
Does gen support C/C++ ?
Hi, all
I have not found any generator for C/C++. Does gen support these two programming lauguages ?
Thanks.
Create an OWNERS file
ref: kubernetes/community#1721
This repo is listed in sigs.yaml as belonging to sig-api-machinery, but it lacks an OWNERS file. Can you please add one with the appropriate list of approvers and reviewers? eg:
For more information see the OWNERS docs
/sig api-machinery
/assign @lavalamp @deads2k
sig-api-machinery chairs
/assign @brendandburns @mbohlool
as folks who've been merging PR's recently
[Java] generics not being set
client-generator.sh fails when trying to generate models for kubernetes-client/java
While working on a CRD using kubernetes-client/java, I noticed that the model generation fails. As per the instructions in kubernetes-client/java, there is a container within a container. The outer container invokes java-crd-cmd.sh and this container is run with host networking enabled. The inner container runs the openapi-generator/client-generator.sh script. This inner container seems to invoke maven commands that tries to download the maven plugin and fails. On debugging, the inner container doesn't seem to have network access. Error detail:
--- Generating client ...
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< io.kubernetes:client-java >----------------------
[INFO] Building client-java 1.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/apache/maven/plugins/maven-clean-plugin/2.5/maven-clean-plugin-2.5.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 20.615 s
[INFO] Finished at: 2021-02-24T08:57:04Z
[INFO] ------------------------------------------------------------------------
[ERROR] Plugin org.apache.maven.plugins:maven-clean-plugin:2.5 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-clean-plugin:jar:2.5: Could not transfer artifact org.apache.maven.plugins:maven-clean-plugin:pom:2.5 from/to central (https://repo.maven.apache.org/maven2): repo.maven.apache.org: Temporary failure in name resolution: Unknown host repo.maven.apache.org: Temporary failure in name resolution -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/PluginResolutionException
Support get_api_resources() (GET /apis/{group}/{version}) in the custom objects API
The built-in APIs include get_api_resources(), but the Custom Objects API provides no equivalent. The autogenerated API libraries should expose a method to GET /apis/{group}/{version} for arbitrary combinations of group and version to ensure parity with the built-in APIs.
PR incoming.
Resources starting with "io.k8s" being considered builtin prevents Volume Snapshot models generation
I tried to generate models for Volume Snapshots, as follows:
$ bash ./python-crd-cmd.sh -n io.k8s.storage.snapshot < swagger.jsonThis produced no models.
I inspected the code and it turns out that resources starting with "io.k8s" are considered built:
gen/openapi/preprocess_spec.py
Lines 307 to 316 in 291f9a6
This does not hold for (at least) Volume Snapshot resources which are well-known CRDs.
I tweaked a few lines of the script and here is a list of the Resources that were initially ignored, in my case:
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshot
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshotClass
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshotClassList
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshotContent
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshotContentList
io.k8s.storage.snapshot.v1alpha1.VolumeSnapshotList
Allow client generation against certain specific Kubernetes patch release
Ref: kubernetes-client/python#631 (comment)
Currently the generator only requires specifying a Kubernetes branch to get OpenAPI spec from, e.g. spec on release-1.12 branch. The generated client can be non-deterministic when there are some openapi changes cherry-picked into release branches (e.g. kubernetes-client/python#649), depending on whether the cherry-pick or the client generation happens first.
I suggest we can add an optional parameter to pass to generator, which specifies a patch version. If the target branch and optional parameter together forms a valid semver, the preprocess should fetch the patch release source code to get the spec instead of getting from the branch.
/kind feature
/assign
v2beta2 HorizontalPodAutoscalerSpec missing from proto generate.sh
https://github.com/kubernetes-client/gen/blob/master/proto/generate.sh is missing HorizontalPodAutoscalerSpec v2beta2.
Is there a reason for this?
Happy to do a small PR to fix this.
Improve kubernetes-client/gen documentation for {CLIENT_LANGUAGE}.sh
It's not clear that the PACKAGE_NAME in the settings files for python.sh (or any language).sh file needs to be specified as "client". Providing an example of the settings file for python-client, python.sh:
Example settings file for python.sh:
export KUBERNETES_BRANCH="master"
export CLIENT_VERSION="8.0.0b1"
export PACKAGE_NAME="client"
Note: For generating the client for any language, the PACKAGE_NAME should be "client".
You can use the latest version for the CLIENT_VERSION. It's displayed here for
the python-client ( https://github.com/kubernetes-client/python ), and similarly
for other language clients.
Will generate a PR to check this in.
Add versioning and changelog to gen package
As all of our clients depends on this repo and we occasionally add features that affect all languages, we should:
- Version this repo
- have a changelog
- document the gen release in the generated code (the same way we document kubernetes and swagger codegen hash keys)
Generated Java code is referencing IoK8sApimachineryPkgApisMetaV1DeleteOptionsV2 which is missing
I'm trying to generate CRD models for my CRD, like so:
$ bash java-crd-cmd.sh -n com.tsystemsmms.cmcc.cmccoperator -p com.tsystemsmms.cmcc.cmccoperator.client -l 2 -h true -o /tmp/cmcc-operator-22816/generated -g true
KUBERNETES_CRD_GROUP_PREFIX: com.tsystemsmms.cmcc.cmccoperator
OPENAPI_MODEL_LENGTH: 2
PACKAGE_NAME: com.tsystemsmms.cmcc.cmccoperator.client
GENERATE_APIS: true
CLIENT_VERSION: 5.0-SNAPSHOT
OUTPUT_DIR: /tmp/cmcc-operator-22816/generated
HIDE_GENERATION_TIMESTAMP: true
LIBRARY: okhttp-gson
rendering settings file to /tmp/settings
reading input openapi specs
The resulting class com.tsystemsmms.cmcc.cmccoperator.client.apis.CmccoperatorCmccTsystemsmmsComV1alpha1Api has two references that cannot be resolved:
import com.tsystemsmms.cmcc.cmccoperator.client.models.IoK8sApimachineryPkgApisMetaV1DeleteOptionsV2;
import com.tsystemsmms.cmcc.cmccoperator.client.models.IoK8sApimachineryPkgApisMetaV1StatusV2;
How to I get the generator to generate these classes?
I found exactly one reference to these classes, in this issue: kubernetes-client/java#1710 but there's no workaround or resolution.
C# failed after PR #57
PR #57
ping @aogier
bug1
umask 0
which will cause /usr/local/bin/mvn-entrypoint.sh: line 39: /generate_client.sh: Permission denied
bug2
need root to install autorest
[INFO] AutoRest code generation utility [version: 2.0.4280; node: v8.11.3]
[INFO] (C) 2018 Microsoft Corporation.
[INFO] https://aka.ms/autorest
[INFO] Network Enabled: true
[INFO] Unhandled rejection Error: EACCES: permission denied, mkdir '/.npm'
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11.321 s
[INFO] Finished at: 2018-06-26T10:50:35Z
[INFO] Final Memory: 9M/72M
[INFO] ------------------------------------------------------------------------
---Done.
---Done.
IntOrString type is not defined in typescript-fetch generated code
It looks like typescript-fetch.xml maps the IntToString type to be imported from ../../types, I'm not sure what this refers to, but the resulting code has unresolved imports (and the import mapping seems to be ignored as far as I can tell).
OperationID duplication in custom object openapi spec
Custom object openapi spec has duplicated operationId for delete and deletecollection operations, which causes the client generation to append _0 to dedup tomplus/kubernetes_asyncio#99 (comment). The operationId should be deleteCollection... for deletecollection operations.
Sed command is not compatible for OSX in kubernetes-client/gen/openapi/haskell.sh
This is related to https://stackoverflow.com/questions/4247068/sed-command-with-i-option-failing-on-mac-but-works-on-linux where OSX don't support -i flag in the script, I think there can be 2 different approach:
- First improve the docs so that those who wants to use the script aware that we are using sed to override the value on the
cabalfile. - Give and additional if else condition for this script.
Failure for centos
Hi,
I had an issue when trying to run the script:
[root@kube openapi]# ./python.sh ./test/ test.sh
/home/charles/gen/openapi/test
--- Building docker image...
Sending build context to Docker daemon 22.28 MB
Step 1 : FROM maven:3.5-jdk-8-slim
---> 414ea0973ca6
...
Step 16 : ENTRYPOINT mvn-entrypoint.sh /generate_client.sh
---> Using cache
---> 83647fbb35bb
Successfully built 83647fbb35bb
--- Running generator inside container...
/generate_client.sh: line 41: pushd: /output_dir: Permission denied
it took me some time to understand why....
It seems due to SELinux that prevent to write on the output directory used inside the container (the one we give in the docker run .... -v)
If I deactivate SE linux with
setenforce 0
then it works 100% fine.
Otherwise changing the mount option in the call to docker run in client-generator.sh from
-v "${output_dir}:/output_dir" \
to
-v "${output_dir}:/output_dir:Z" \
also fix the issue.
I m not expert enough to see the potential negative impact of adding the :Z option and maybe there is even another way to fix it...
unversioned kubernetes spec
Hi.
This change kubernetes/kubernetes#84654 removed a name of version from swagger.json, which may appear in generated code. For example the Python Client is affected. Now we'll get 'OpenAPI spec version: unversioned' in each file if code is generated by openapi-generator from release branches.
To make it work again we have to switch to use git tags to fetch spec and update version info.
$ curl -s https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.14/api/openapi-spec/swagger.json | jq .info.version
"v1.14.11"$ curl -s https://raw.githubusercontent.com/kubernetes/kubernetes/release-1.15/api/openapi-spec/swagger.json | jq .info.version
"unversioned"
cc: @roycaihw
Got Max retries exceeded with url: /kubernetes/kubernetes/release-1.14/api/openapi-spec/swagger.json
I got the following error:
$ ${GEN_ROOT}/gen/openapi/java.sh kubernetes ./settings
--- Building docker image kubernetes-java-client-gen:v1...
Sending build context to Docker daemon 143.9kB
Step 1/26 : FROM maven:3.5-jdk-8-slim
---> a045f84d6a3d
Step 2/26 : ARG SWAGGER_CODEGEN_COMMIT
---> Using cache
---> c4c0a39c22ce
Step 3/26 : ARG GENERATION_XML_FILE
---> Using cache
---> ec19cbb39e86
Step 4/26 : ARG SWAGGER_CODEGEN_USER_ORG=swagger-api
---> Using cache
---> 4b6d772f4057
Step 5/26 : RUN apt-get update && apt-get -y install git python-pip && pip install urllib3
---> Using cache
---> a2a20cf3ba81
Step 6/26 : RUN apt-get update && apt-get -qq -y install libunwind8 libicu57 libssl1.0 liblttng-ust0 libcurl3 libuuid1 libkrb5-3 zlib1g
---> Using cache
---> 24be2cc95919
Step 7/26 : RUN curl -sL https://deb.nodesource.com/setup_8.x | bash -
---> Using cache
---> e59ac3b89f21
Step 8/26 : RUN apt-get update && apt-get -y install nodejs libunwind8-dev && rm -rf /var/lib/apt/lists/*
---> Using cache
---> 1bc73d8794ea
Step 9/26 : RUN npm install -g autorest
---> Using cache
---> 06d3009eac02
Step 10/26 : RUN mkdir /source && cd /source && git clone -n https://github.com/${SWAGGER_CODEGEN_USER_ORG}/swagger-codegen.git && cd swagger-codegen && git checkout $SWAGGER_CODEGEN_COMMIT
---> Using cache
---> d477602a1ee6
Step 11/26 : RUN mkdir /.npm && chmod -R go+rwx /.npm && chmod -R go+rwx /root && umask 0 && cd /source/swagger-codegen && mvn install -DskipTests -Dmaven.test.skip=true -pl modules/swagger-codegen-maven-plugin -am && cp -r /root/.m2/* /usr/share/maven/ref
---> Using cache
---> 0f3181b9b0a8
Step 12/26 : RUN mkdir -p /node_modules && chmod -R go+rwx /node_modules
---> Using cache
---> 84acc5beb3b5
Step 13/26 : RUN npm install @microsoft.azure/autorest.csharp @microsoft.azure/autorest.modeler
---> Using cache
---> 1617776bc33e
Step 14/26 : RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.asc.gpg
---> Using cache
---> 402db4094b4e
Step 15/26 : RUN mv microsoft.asc.gpg /etc/apt/trusted.gpg.d/
---> Using cache
---> 566248fed782
Step 16/26 : RUN curl https://packages.microsoft.com/config/debian/9/prod.list > prod.list
---> Using cache
---> 845076f45e3e
Step 17/26 : RUN mv prod.list /etc/apt/sources.list.d/microsoft-prod.list
---> Using cache
---> 7e8808324ff9
Step 18/26 : RUN chown root:root /etc/apt/trusted.gpg.d/microsoft.asc.gpg
---> Using cache
---> f1841a985bc4
Step 19/26 : RUN chown root:root /etc/apt/sources.list.d/microsoft-prod.list
---> Using cache
---> 0c8dda5527cb
Step 20/26 : RUN apt-get update
---> Using cache
---> e68bed6d6b4b
Step 21/26 : RUN apt-get install -yy -q dotnet-hosting-2.0.8
---> Using cache
---> 6291ba40751f
Step 22/26 : COPY swagger-codegen/generate_client_in_container.sh /generate_client.sh
---> Using cache
---> 09256aa48da8
Step 23/26 : COPY preprocess_spec.py /
---> Using cache
---> 293dc7183779
Step 24/26 : COPY custom_objects_spec.json /
---> Using cache
---> 37817bacd59e
Step 25/26 : COPY ${GENERATION_XML_FILE} /generation_params.xml
---> Using cache
---> 171c1dbd761b
Step 26/26 : ENTRYPOINT ["mvn-entrypoint.sh", "/generate_client.sh"]
---> Using cache
---> 0768743170bb
Successfully built 0768743170bb
Successfully tagged kubernetes-java-client-gen:v1
--- Running generator inside container...
/source/swagger-codegen /
/
--- Downloading and pre-processing OpenAPI spec
Traceback (most recent call last):
File "//preprocess_spec.py", line 351, in <module>
sys.exit(main())
File "//preprocess_spec.py", line 339, in main
with pool.request('GET', spec_url, preload_content=False) as response:
File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 68, in request
**urlopen_kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/request.py", line 89, in request_encode_url
return self.urlopen(method, url, **extra_kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/poolmanager.py", line 326, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 670, in urlopen
**response_kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 670, in urlopen
**response_kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 670, in urlopen
**response_kw)
File "/usr/local/lib/python2.7/dist-packages/urllib3/connectionpool.py", line 641, in urlopen
_stacktrace=sys.exc_info()[2])
File "/usr/local/lib/python2.7/dist-packages/urllib3/util/retry.py", line 399, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /kubernetes/kubernetes/release-1.14/api/openapi-spec/swagger.json (Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:661)'),))
When trying to generate API for release-1.14. This is the details of my OS:
$ uname -a
Darwin ip-10-10-26-39.ap-southeast-1.compute.internal 18.2.0 Darwin Kernel Version 18.2.0: Thu Dec 20 20:46:53 PST 2018; root:xnu-4903.241.1~1/RELEASE_X86_64 x86_64 i386 MacBookPro14,1 Darwin
Fail to build the docker image
./openapi/typescript-fetch.sh ./out config will end up with a failure during the build of the docker image.
Here is the output.
Step 3/24 : RUN curl -sL https://deb.nodesource.com/setup_14.x | bash -
---> Running in 439ad1ba66ce
## Installing the NodeSource Node.js 14.x repo...
## Populating apt-get cache...
+ apt-get update
Hit:1 http://security.debian.org/debian-security stretch/updates InRelease
Ign:2 http://deb.debian.org/debian stretch InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch Release
Reading package lists...
## Installing packages required for setup: apt-transport-https lsb-release gnupg...
+ apt-get install -y apt-transport-https lsb-release gnupg > /dev/null 2>&1
## Confirming "stretch" is supported...
+ curl -sLf -o /dev/null 'https://deb.nodesource.com/node_14.x/dists/stretch/Release'
## Adding the NodeSource signing key to your keyring...
+ curl -s https://deb.nodesource.com/gpgkey/nodesource.gpg.key | gpg --dearmor | tee /usr/share/keyrings/nodesource.gpg >/dev/null
## Creating apt sources list file for the NodeSource Node.js 14.x repo...
+ echo 'deb [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_14.x stretch main' > /etc/apt/sources.list.d/nodesource.list
+ echo 'deb-src [signed-by=/usr/share/keyrings/nodesource.gpg] https://deb.nodesource.com/node_14.x stretch main' >> /etc/apt/sources.list.d/nodesource.list
## Running `apt-get update` for you...
+ apt-get update
Hit:1 http://security.debian.org/debian-security stretch/updates InRelease
Ign:2 http://deb.debian.org/debian stretch InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch Release
Ign:5 https://deb.nodesource.com/node_14.x stretch InRelease
Ign:6 https://deb.nodesource.com/node_14.x stretch Release
Ign:7 https://deb.nodesource.com/node_14.x stretch/main Sources
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Ign:7 https://deb.nodesource.com/node_14.x stretch/main Sources
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Ign:7 https://deb.nodesource.com/node_14.x stretch/main Sources
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Ign:7 https://deb.nodesource.com/node_14.x stretch/main Sources
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Ign:7 https://deb.nodesource.com/node_14.x stretch/main Sources
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Err:7 https://deb.nodesource.com/node_14.x stretch/main Sources
server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
Ign:9 https://deb.nodesource.com/node_14.x stretch/main amd64 Packages
Ign:10 https://deb.nodesource.com/node_14.x stretch/main all Packages
Reading package lists...
W: The repository 'https://deb.nodesource.com/node_14.x stretch Release' does not have a Release file.
E: Failed to fetch https://deb.nodesource.com/node_14.x/dists/stretch/main/source/Sources server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
E: Some index files failed to download. They have been ignored, or old ones used instead.
Error executing command, exiting
The command '/bin/sh -c curl -sL https://deb.nodesource.com/setup_14.x | bash -' returned a non-zero code: 1
Looks like nodejs setup is failed due to the server certificate verification failure. I guess some changes happened recently, but the base image maven:3.5-jdk-8-slim is too old to catch up with the changes.
Required arrays with null value
Looks like there is a problem with 1.8 spec going forward [TODO:ref issue] that an array that marked as required can be null. Assuming an empty array is an acceptable array in these situation (and it look like it is as kubernetes return empty array in some of those cases, TODO: add ref issues) we should consider fixing it in the main repo (TODO: create an issue) and also here as a quick fix until main repo issue has been fixed. This is my brain dump on the issue in case of anyone wants to work on this:
- we should first research the problem and see if that is the case for all required arrays or a subset of them
- as a quick fix, we can remove those arrays (or subset of them based on previous findings) from required list in a preprocessing step in this repo.
- A regeneration of python (and other languages) clients should fix the issues (TODO: ref those issues)
- Next, there should be a more permanent fix in the main repo.
generating rust protos fail
when running generate.sh rust kube in proto folder:
Downloading proto files for master
Munging proto file packages
Generating code for
k8s.io/apimachinery/pkg/apis/meta/v1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/authorization/v1beta1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/authorization/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/authorization/v1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/authorization/v1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/imagepolicy/v1alpha1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/imagepolicy/v1alpha1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/core/v1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/certificates/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/certificates/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/certificates/v1beta1/generated.proto:24:1: warning: Import k8s.io/api/core/v1/generated.proto is unused.
k8s.io/api/extensions/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/extensions/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/autoscaling/v2beta2/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/autoscaling/v2beta2/generated.proto:28:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/autoscaling/v2beta2/generated.proto:24:1: warning: Import k8s.io/api/core/v1/generated.proto is unused.
k8s.io/api/autoscaling/v2beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/autoscaling/v2beta1/generated.proto:28:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/autoscaling/v2beta1/generated.proto:24:1: warning: Import k8s.io/api/core/v1/generated.proto is unused.
k8s.io/api/autoscaling/v1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/autoscaling/v1/generated.proto:28:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/autoscaling/v1/generated.proto:24:1: warning: Import k8s.io/api/core/v1/generated.proto is unused.
k8s.io/api/policy/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/policy/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/authentication/v1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/authentication/v1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/admission/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/authentication/v1beta1/generated.proto:25:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/authentication/v1beta1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/apps/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/apps/v1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/apps/v1beta2/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/batch/v1/generated.proto:26:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/batch/v1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
k8s.io/api/batch/v1beta1/generated.proto:27:1: warning: Import k8s.io/apimachinery/pkg/runtime/generated.proto is unused.
k8s.io/api/batch/v1beta1/generated.proto:28:1: warning: Import k8s.io/apimachinery/pkg/runtime/schema/generated.proto is unused.
[libprotobuf WARNING google/protobuf/compiler/parser.cc:651] No syntax specified for the proto file: k8s.io/api/batch/v2alpha1/generated.proto. Please use 'syntax = "proto2";' or 'syntax = "proto3";' to specify a syntax version. (Defaulted to proto2 syntax.)
k8s.io/api/batch/v2alpha1/generated.proto:1:1: Expected top-level statement (e.g. "message").using latest protobuf:
$ protoc --version
libprotoc 3.17.3Support client generation for CRD
Following @mbohlool 's advice kubernetes/kube-openapi#13 (comment), I have managed to generate swagger.json for my CRD. In the spirit of not reinventing the wheel, I was hoping to generate Java client using this repo. But I noticed that the swagger.json file is hardcoded to Kubernetes repos swagger.json https://github.com/kubernetes-client/gen/blob/master/openapi/preprocess_spec.py#L302 .
Do you have any suggestion how can I go about generating Java client from my CRD's swagger.json ?
Delete `client-gen-with-openapi-generator` container after it has run.
Running ghcr.io/kubernetes-client/java/crd-model-gen:v1.0.6 /generate.sh to create model classes from CRD yaml utilises this project. Of the several shell scripts that gets triggered, one of them is openapi/openapi-generator/client-generator.sh. This shell script builds and runs a container from openapi/openapi-generator/Dockerfile during runtime of the script. The problem is after the created container has been run, it doesn't clean up after itself and keeps the created container. Thus a new container is built and left after running each time the CRD build script is triggered. This quickly creates lots of stopped containers.
The proposal is to add an --rm argument during the execution of the built container so that it gets removed after it is stopped.
I don't think that the container needs to be kept after it is stopped.
This will require a very small PR which I am more than happy to provide.
Update openapi-generator for typescript to v4.1.3
With the 4.1.3 version of oepnapi-generator they have fixed a major issue OpenAPITools/openapi-generator#3872 that was a main painful point in debugging errors. I would like to see this fix integrated into the @kubernetes/client-node library, but I saw that we first need to update the openapi-generator version in this project.
Line 49 in 7959939
Consider including proto files in language repositories...
Module '"."' has no exported member
my setting file
#!/bin/bash
export KUBERNETES_BRANCH="v1.25.5"
export CLIENT_VERSION="1.0.1"
export PACKAGE_NAME="k8s-ts-client"gen code by
./typescript-fetch.sh gen-code setting
then
> cd gen-code
> npm install
> kubernetes-client-typescript-fetch@1.0.1 prepare
> npm run build
> kubernetes-client-typescript-fetch@1.0.1 build
> tsc
src/models/V1HTTPGetAction.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1HTTPGetAction.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1HTTPGetAction.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1HTTPGetAction.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1NetworkPolicyPort.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1NetworkPolicyPort.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1NetworkPolicyPort.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1NetworkPolicyPort.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1PodDisruptionBudgetSpec.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1PodDisruptionBudgetSpec.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1PodDisruptionBudgetSpec.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1PodDisruptionBudgetSpec.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDaemonSet.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1RollingUpdateDaemonSet.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDaemonSet.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDaemonSet.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDeployment.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1RollingUpdateDeployment.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDeployment.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateDeployment.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateStatefulSetStrategy.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1RollingUpdateStatefulSetStrategy.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateStatefulSetStrategy.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1RollingUpdateStatefulSetStrategy.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
src/models/V1ServicePort.ts:17:5 - error TS2305: Module '"."' has no exported member 'IntOrString'.
17 IntOrString,
~~~~~~~~~~~
src/models/V1ServicePort.ts:18:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSON'.
18 IntOrStringFromJSON,
~~~~~~~~~~~~~~~~~~~
src/models/V1TCPSocketAction.ts:19:5 - error TS2305: Module '"."' has no exported member 'IntOrStringFromJSONTyped'.
19 IntOrStringFromJSONTyped,
~~~~~~~~~~~~~~~~~~~~~~~~
src/models/V1TCPSocketAction.ts:20:5 - error TS2305: Module '"."' has no exported member 'IntOrStringToJSON'.
20 IntOrStringToJSON,
~~~~~~~~~~~~~~~~~
Found 32 errors.Missing IoK8sApimachineryPkgApisMetaV1ObjectMeta in generated classes
After following the instructions in generate-model-from-third-party-resources.md the classes generated have references to a io.kubernetes.client.models.IoK8sApimachineryPkgApisMetaV1ObjectMeta
which I cannot find anywhere?
The cluster version is 1.15 (minikube) and it contains a number of CRDs from the prometheus-operator (https://github.com/coreos/prometheus-operator).
Here is a typical snippet from the generated code:
ComCoreosMonitoringV1PodMonitor.java:
public static final String SERIALIZED_NAME_METADATA = "metadata";
@SerializedName(SERIALIZED_NAME_METADATA)
private IoK8sApimachineryPkgApisMetaV1ObjectMeta metadata = null;
The commands I used:
kc get --raw="/openapi/v2" > openapi.json
docker run -i --rm yue9944882/java-model-gen < openapi.json | tar -xzf - -C ./tmp
Should have a `getCustomObjectsForAllNamespaces` method
To match the behavior for non-custom objects.
preprocess_spec.py generates invalid Swagger json
When I tried to use https://github.com/Trax-air/swagger-parser on the json generated by calling preprocess_spec.py release-1.6 kubernetes.json, it complained that the json was invalid:
ValueError: swagger_kubernetes.json is not a valid swagger2.0 file: {'schema': {'type': 'object'}} is not valid under any of the given schemas
Failed validating 'oneOf' in schema['properties']['paths']['patternProperties']['^/']['properties']['post']['properties']['responses']['patternProperties']['^([0-9]{3})$|^(default)$']:
{'oneOf': [{'$ref': '#/definitions/response'},
{'$ref': '#/definitions/jsonReference'}]}
On instance['paths']['/apis/{group}/{version}/{plural}']['post']['responses']['201']:
{'schema': {'type': 'object'}}
I double-checked this with https://hub.docker.com/r/swaggerapi/swagger-validator/, which suggests that it is invalid. swagger-codegen still accepts it so apparently swagger-codegen's validation isn't as strict, but this still makes it hard to use other tools on it.
migrate from swagger-codegen to openapi-generator
This repo uses the swagger-codegen, which is not actively maintained, at least from a my point of view. It makes lots of hacks like post-generation patching and it’s very hard to maintain these scripts.
For instance - here is a list of PRs which are ready to merge but are not merged yet and had to be applied as patches in Python generator:
- swagger-api/swagger-codegen#7905 (Mar 25, waiting)
- swagger-api/swagger-codegen#8061 (Apr 24, waiting)
- swagger-api/swagger-codegen#8204 (May 17, waiting)
- swagger-api/swagger-codegen#8401 (Jul 8, finally merged)
- swagger-api/swagger-codegen#8797 (Oct 9, waiting)
A lot of contributors moved from the swagger-codegen to the new project - openapi-generator. Basically it’s a fork of swagger-codegen but it’s actively developed and it gains popularity very fast.
All above PRs are already ported and merged to the openapi-generator:
- OpenAPITools/openapi-generator#1387 (merged)
- OpenAPITools/openapi-generator#107 (merged)
- OpenAPITools/openapi-generator#519 (merged)
- OpenAPITools/openapi-generator#1211 (merged)
What do you think about changing generator? Is it possible? I’ll be happy to work on it...
preprocesing swagger.json makes operationId duplicated
The script preprocess_spec.py removes "tags" from operationId downloaded from K8s spec:
gen/openapi/preprocess_spec.py
Line 102 in 09f7470
and it makes that some operationId's become duplicated but they should be unique.
The latest version of openapi-generator (>4.x) has additional validations which complains about that:
-attribute paths.'/apis/node.k8s.io/v1beta1/runtimeclasses/{name}'(delete).operationId is repeated
-attribute paths.'/apis/rbac.authorization.k8s.io/'(get).operationId is repeated
-attribute paths.'/apis/extensions/v1beta1/namespaces/{namespace}/deployments/{name}'(put).operationId is repeated
[~300 similar errors]
We can disable this validation or fix our script by removing this replacing step - is this replacement necessary?
missing support for custom object status/scale api
since kubernetes 1.11 the custom object status/scale api is in beta:
https://kubernetes.io/docs/tasks/access-kubernetes-api/custom-resources/custom-resource-definitions/#status-subresource
But the generator does not support it as it is missing in openapi/custom_objects_spec.json
If this is not intentional due to the beta status of the api it would be very helpful to have this added so clients can generate code for it.
Extend proto generation to other APIs
Support client generation for custom resources
Currently the java_crd_cmd.sh script only generates models for CRD but not the API clients. It would be great to also generate API clients for custom resources so we don't need to use GenericKubernetesApi or CustomObjectsApi in our controller code.
The generator knows how to generate API client for native kubernetes resources, so we can update preprocess_spec.py to keep the API endpoints for custom resources in CRD_MODE.
kubernetes-client/gen/openapi/python.sh - 'no such file or directory' exec error
An error occurred when executing kubernetes-client/gen/openapi/python.sh
Following the instructions under the “Generating a client”
( https://github.com/kubernetes-client/gen ):
${CLIENT_LANGUAGE}.sh OUTPUT_DIR SETTING_FILE
Below is an example run:
/root/kubernetes/gen/openapi/python.sh /root/output_client/kubernetes /root/kubernetes/python/settings
The /root/kubernetes/python/settings files is as follows:
export KUBERNETES_BRANCH="master"
export CLIENT_VERSION="8.0.0b1"
export PACKAGE_NAME="kubernetes"
The stdout from the above run is shown below, revealing an execution find error in the last line:
...
---Done.
--- Patching generated code ...
find: '/root/output_client/kubernetes/client/': No such file or directory
...
I will submitt a PR to the kubernetes-client/gen repo to remedy the execution error above.
The fix involved modifying: /root/kubernetes/gen/openapi/python.sh
Python generator is auto-generating wrong links in README
There are some links that are being wrongly generated in the README, specifically the link to install the kubernetes python client package using the git link.
Generated link: pip install git+https://github.com/kubernetes-kubernetes.client/python.git
Expected link: pip install git+https://github.com/kubernetes-client/python.git
References:
crd-model-gen should enforce that enums are UPPERCASE and don't contain special characters
While working on a CRD, i had two enum patterns that didn't work out:
- Enums with lowercase or mixed case names
- Enums with special characters in the names
Example:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
...
spec:
...
versions:
- name: v1alpha1
...
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
...
type:
type: string
# First pattern with lowercase or mixed case names
enum:
- apple
- banana
- WaterMelon
default: apple
description: "Type of fruit"
...
...
fileName:
type: string
# Second pattern with names containing special characters
enum:
- log4j.properties
- web.xml
- server.xml
...
Technically these are not valid patterns for enum names in Java (atleast the second pattern wouldn't even compile) and crd-model-gen happily does the right thing by transforming these names to UPPERCASE and replacing special characters with "_" putting the values in the enum like below:
// for the first pattern
public static enum TypeEnum {
APPLE("apple"),
BANANA("banana"),
WATERMELON("WaterMelon");
// for the second pattern
public static enum FileNameEnum {
LOG4J_PROPERTIES("log4j.properties"),
WEB_XML("web.xml"),
SERVER_XML("server.xml");
The problem however happens when loading a CR with such enum values using Yaml class that is provided with java implementation of kubernetes-client.
SnakeYaml started supporting a loader option to ignore case while loading enums starting version 1.28 and I have a corresponding change here to consume the same. However i am not able to justify that the second pattern should even be supported. And that brings to me to the thought, should crd-model-gen enforce that enum names are always UPPERCASE and don't container special characters other than "_".
Thoughts?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.