Automatically Create CNAME records for containers served by Traefik

License: MIT License

Dockerfile 6.76% Python 93.24%

docker cloudflare cname dns alpine traefik

docker-traefik-cloudflare-companion's Issues

Set proxy status for domain records

The records that are being automatically created in cloudflare have the proxy status set to false, but I'd like for the new subdomains to be proxied by default to hide my IP. Is this possible at the moment? Didn't see it in the docs.

an A, AAAA or CNAME record already exists with that host

Not sure if really a bug but anyway:

I've recently updated my setup to use proxied dns from cloudflare. The first node was done manually. But I hoped to automate the second one a bit by changing the DOMAIN1_PROXIED to TRUE and restarting the container. At first it did nothing (no result and nothing interesting in the logs) but after recreating it through portainer, it spat out the following

2020-10-16 08:02:23,553 - Python Cloudflare API v4 - DEBUG - Response: 400, application/json, b'{"result":null,"success":false,"errors":[{"code":81053,"message":"An A, AAAA or CNAME record already exists with that host."}],"messages":[]}'


2020-10-16 08:02:23,553 - Python Cloudflare API v4 - DEBUG - Response: error 81053 An A, AAAA or CNAME record already exists with that host.

and:

[DEBUG] Container List Discovery Loop


[DEBUG] Called check_container_t2 for: <Container: ca997d9b92>


[DEBUG] Starting event watch routines


--- Logging error ---


Traceback (most recent call last):


  File "/usr/lib/python3.8/logging/__init__.py", line 1081, in emit


    msg = self.format(record)


  File "/usr/lib/python3.8/logging/__init__.py", line 925, in format


    return fmt.format(record)


  File "/usr/lib/python3.8/logging/__init__.py", line 664, in format


    record.message = record.getMessage()


  File "/usr/lib/python3.8/logging/__init__.py", line 369, in getMessage


    msg = msg % self.args


TypeError: not all arguments converted during string formatting


Call stack:


  File "/usr/sbin/cloudflare-companion", line 253, in <module>


    logger.debug("Time:", t)


Message: 'Time:'


guments: ('1602835343',)


[DEBUG] Called check_container_t2 for: <Container: 2ef775c29e>

The first is understandable, The second might be worth looking into.

Also if this is working as intended (might be dangerous to just overwrite existing records, or even impossible with the python API) It might be an idea to include a FORCE_UPDATE_RECORDS parameter for people like me that want all their records updated regardless of current settings

Exclude certain containers based on docker-compose label

Description of the feature
It is possible to exclude certain subdomain trees from Cloudflare companion by adding these to the environment variables of Cloudflare companion like

DOMAIN1_EXCLUDED_SUB_DOMAINS	(optional specify sub domain trees to be ignored in lables) ex: DOMAIN1_EXCLUDED_SUB_DOMAINS=int would not create a CNAME for *.int.example.com

Would it also be possible to exclude specific containers from being added to Cloudflare by adding an additional cloudflare-exclude=yes label?

Motivation: I am using a split DNS and I would like Traefik to listen to all subdomains on my LAN (hence all containers have a traefik router label applied), but I only want a few of the containers actually published as subdomain on cloudflare.

No CNAME's records added

Hello :)
Docker swarm + dtcc 6.0.1

[INFO] ** [zabbix] Starting Zabbix Agent
zabbix_agentd [743]: Warning: EnableRemoteCommands parameter is deprecated, use AllowKey=system.run[*] or DenyKey=system.run[*] instead
Starting Zabbix Agent [alpine]. Zabbix 5.0.2 (revision {ZABBIX_REVISION}).
Press Ctrl+C to exit.


[debug] Swarm Mode: true
[debug] Refresh Entries: false
[debug] Traefik Version: 1
[debug] Default TTL: 1

Nothing happens.

environment:
      TRAEFIK_VERSION: 1
      CF_EMAIL: email
      CF_TOKEN: token
      DOMAIN1: domain
      DOMAIN1_ZONE_ID: zoneId
      DOMAIN1_PROXIED: "true"
      TARGET_DOMAIN: target
      SWARM_MODE: "true"
      REFRESH_ENTRIES: "false"
      CONTAINER_LOG_LEVEL: DEBUG

Also it would be nice if script will listen docker sock for not just "start" event, but for "update" (if it possible) too, because docker stack deploy trigger "update" (Updating service ...) if service already exist.

Disabling Log Shipping Functionality

Summary

today at 06:49:42 /var/run/s6/etc/cont-init.d/05-logging: line 11: /assets/cron//logrotate: No such file or directory
today at 06:49:42 2021-08-09-06:49:42 [NOTICE] ** [logging] Disabling Log Shipping Functionality

Steps to reproduce

Start container, see logs.

What is the expected correct behavior?

For log rotate to work.

Relevant logs and/or screenshots

today at 06:49:41 2021-08-09-06:49:41 [NOTICE] ** [container-init] Detected Container that has been restarted - Cleaning '/tmp/state' files
today at 06:49:41 [cont-init.d] 00-startup: exited 0.
today at 06:49:41 [cont-init.d] 01-timezone: executing... 
today at 06:49:41 [cont-init.d] 01-timezone: exited 0.
today at 06:49:41 [cont-init.d] 02-permissions: executing... 
today at 06:49:41 [cont-init.d] 02-permissions: exited 0.
today at 06:49:41 [cont-init.d] 03-monitoring: executing... 
today at 06:49:41 2021-08-09-06:49:41 [NOTICE] ** [monitoring] Container configured for monitoring with 'zabbix modern'
today at 06:49:41 [cont-init.d] 03-monitoring: exited 0.
today at 06:49:41 [cont-init.d] 04-scheduling: executing... 
today at 06:49:41 2021-08-09-06:49:41 [NOTICE] ** [scheduling] Disabling Scheduling
today at 06:49:41 [cont-init.d] 04-scheduling: exited 0.
today at 06:49:41 [cont-init.d] 05-logging: executing... 
today at 06:49:42 /var/run/s6/etc/cont-init.d/05-logging: line 11: /assets/cron//logrotate: No such file or directory
today at 06:49:42 2021-08-09-06:49:42 [NOTICE] ** [logging] Disabling Log Shipping Functionality
today at 06:49:42 [cont-init.d] 05-logging: exited 0.
today at 06:49:42 [cont-init.d] 06-messaging: executing... 
today at 06:49:42 2021-08-09-06:49:42 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
today at 06:49:42 [cont-init.d] 06-messaging: exited 0.
today at 06:49:42 [cont-init.d] 10-cloudflare-companion: executing... 
today at 06:49:42 [cont-init.d] 10-cloudflare-companion: exited 0.
today at 06:49:42 [cont-init.d] 99-container: executing... 
today at 06:49:42 [cont-init.d] 99-container: exited 0.
today at 06:49:42 [cont-init.d] done.
today at 06:49:42 [services.d] starting services
today at 06:49:42 [services.d] done.

Environment

Docker, latest

  # Cloudflare-Companion - Automatic CNAME DNS Creation
  cfcompanion:
    container_name: CF-Companion
    image: tiredofit/traefik-cloudflare-companion:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL # Same as traefik
      # - CF_TOKEN=$CLOUDFLARE_API_TOKEN # Scoped api token not working. Error 10000.
      - CF_TOKEN=$CLOUDFLARE_API_KEY # Same as traefik
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=$CLOUDFLARE_ZONEID # Copy from Cloudflare Overview page
      - DOMAIN1_PROXIED=TRUE
    depends_on:
      - dnscrypt-proxy
    # depends_on: 
    #   dnscrypt-proxy:
    #     condition: service_healthy
    labels:
      # Add hosts specified in rules here to force cf-companion to create the CNAMEs
      # Since cf-companion creates CNAMEs based on host rules, this a workaround for non-docker/external apps
      - "traefik.http.routers.cf-companion-rtr.rule=Host(`netdata.$DOMAINNAME`)" 
      #- "traefik.http.routers.cf-companion-rtr.rule=Host(`impi.$DOMAINNAME`)" #TLS middlewares not yet implemented https://github.com/containous/traefik/issues/6684
      #- "traefik.http.routers.cf-companion-rtr.rule=Host(`webmin.$DOMAINNAME`)" #TLS middlewares not yet implemented https://github.com/containous/traefik/issues/6684
    networks:
      - isonet

Possible fixes

Looks like line 11 has an errant "/" (/assets/cron//logrotate)

Scoped api token not working. Error 10000

As the title says, workaround by using Global API Key.

Support for Traefik V2 labels

Would it be possible to trigger a CNAME creation/update based on the traefik.http.routers.router1.rule=Host(`traefik.example.com`) label? The traefik.frontend.rule label is not used in version 2 of Traefik

Version tag on docker hub for 6.7.0?

Summary

Latest has been updated but no versioned tag exists. I prefer those for this container, thanks.

EDIT: Seems the action failed: https://github.com/tiredofit/docker-traefik-cloudflare-companion/actions/runs/1500618594

Cloudflare - no CNAME's added

I've configured DTCC as described in the example compose file but none of my containers are getting corresponding cnames in cloudflare.
`[s6-init] making user provided files available at /var/run/s6/etc...exited 0.

[s6-init] ensuring user provided files have correct perms...exited 0.

[fix-attrs.d] applying ownership & permissions fixes...

[fix-attrs.d] 00-functions: applying...

[fix-attrs.d] 00-functions: exited 0.

[fix-attrs.d] 01-s6: applying...

[fix-attrs.d] 01-s6: exited 0.

[fix-attrs.d] 02-zabbix: applying...

[fix-attrs.d] 02-zabbix: exited 0.

[fix-attrs.d] 03-logrotate: applying...

[fix-attrs.d] 03-logrotate: exited 0.

[fix-attrs.d] done.

[cont-init.d] executing container initialization scripts...

[cont-init.d] 00-startup: executing...

[cont-init.d] 00-startup: exited 0.

[cont-init.d] 01-timezone: executing...

[NOTICE] ** [timezone] Setting timezone to 'America/Chicago'

[cont-init.d] 01-timezone: exited 0.

[cont-init.d] 02-permissions: executing...

[cont-init.d] 02-permissions: exited 0.

[cont-init.d] 03-zabbix: executing...

[cont-init.d] 03-zabbix: exited 0.

[cont-init.d] 04-cron: executing...

[NOTICE] ** [cron] Disabling Cron

[cont-init.d] 04-cron: exited 0.

[cont-init.d] 05-smtp: executing...

[NOTICE] ** [smtp] Disabling SMTP Features

[cont-init.d] 05-smtp: exited 0.

[cont-init.d] 10-cloudflare-companion: executing...

/var/run/s6/etc/cont-init.d/10-cloudflare-companion: line 4: prepare_service: command not found

[NOTICE] ** [traefik-cloudflare-companion] Setting Traefik 2.x Mode

[cont-init.d] 10-cloudflare-companion: exited 0.

[cont-init.d] 99-container: executing...

[cont-init.d] 99-container: exited 0.

[cont-init.d] done.

[services.d] starting services

[services.d] done.

[INFO] ** [zabbix] Starting Zabbix Agent`

All containers are accessible with wildcard cert.

Enchancement: Support for Docker Secrets

I moved my CF token to docker secrets from .env. But cf-companion does not appear to work with secrets.

Enchancement: Support for Docker Secrets

I would like to request an update to add in feature support Docker Secrets passing values to the container.

Image fails to pull on any LXC instance of Docker

Summary

Whenever the image is pulled down from Docker hub, the following error is encountered when trying to pull the image:
failed to register layer: ApplyLayer exit status 1 stdout: stderr: lchown /run/fail2ban: invalid argument
Most of my reading into this, implies that this is an issue with rootless mode specifically. All of the other containers I was pulling did not have any issues, this was the only one that had any issues in rootless mode. Something during the pull is calling a UID and GID beyond 65565, which is the max uid and gid values on an LXC container in rootless mode.

Steps to reproduce

Attempt to pull the docker image from hub.docker.com on an LXC Container with Docker on it. (I have tried it on both Debian and Almalinux with the same results)

What is the expected correct behavior?

The image should pull without erroring out.

Relevant logs and/or screenshots

[root@docker3 etc]# sudo docker pull tiredofit/traefik-cloudflare-companion
Using default tag: latest
latest: Pulling from tiredofit/traefik-cloudflare-companion
213ec9aee27d: Pull complete 
309f922759ec: Extracting [==================================================>]  46.13MB/46.13MB
45d9207b0216: Download complete 
af2e2319950c: Download complete 
18e4eeca72f6: Download complete 
failed to register layer: ApplyLayer exit status 1 stdout:  stderr: lchown /run/fail2ban: invalid argument

Environment

Image version / tag: Latest (6.8.5)
Guest OS: Almalinux 8.6
Host OS: Proxmox 7.2-7

Allow records to be updated if they already exist

Currently, records that already exist will not be updated. They may need updating if various attributes change (TTL, proxy status, target CNAME).

PR #17 addresses this.

DOMAIN2 ... DOMAIN3 ... generating errors

Summary

I have several domains that I am hosting on Docker. Whenever I add a second, third, ..., eighth domain, I am getting an error

Steps to reproduce

      - DOMAIN1=$DOMAIN1
      - DOMAIN1_ZONE_ID=$CLOUDFLARE_ZONEID_DOMAIN1 
      - DOMAIN1_PROXIED=TRUE
      - DOMAIN2=$DOMAIN2
      - DOMAIN2_ZONE_ID=$CLOUDFLARE_ZONEID_DOMAIN2 # Copy from Cloudflare Overview page
      - DOMAIN2_PROXIED=TRUE
      - DOMAIN3=$DOMAIN3
      - DOMAIN3_ZONE_ID=$CLOUDFLARE_ZONEID_DOMAIN3
      - DOMAIN3_PROXIED=TRUE```

### What is the expected *correct* behavior?
```2022-03-11.19:08:36 [NOTICE] ** [container-init] Detected Container that has been restarted - Cleaning '/tmp/state' files
2022-03-11.19:08:36 [NOTICE] ** [scheduling] Disabling Scheduling
2022-03-11.19:08:36 [STARTING] ** [traefik-cloudflare-companion] [1] Starting Traefik Cloudflare Companion```

<!-- Describe what should be seen instead. -->
```2022-03-11.19:08:30 [STARTING] ** [traefik-cloudflare-companion] [610] Starting Traefik Cloudflare Companion
Traceback (most recent call last):
  File "/usr/sbin/cloudflare-companion", line 466, in <module>
    sync_mappings(get_initial_mappings(traefik_included_hosts, traefik_excluded_hosts), doms)
  File "/usr/sbin/cloudflare-companion", line 354, in sync_mappings
    if point_domain(k, domain_infos):
  File "/usr/sbin/cloudflare-companion", line 168, in point_domain
    records = cf.zones.dns_records.get(domain_info['zone_id'], params={u'name': name})
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 672, in get
    return self._base.call_with_auth('GET', self._parts,
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 126, in call_with_auth
    return self._call(method, headers, parts,
  File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 502, in _call
    raise CloudFlareAPIError(code, message)
CloudFlare.exceptions.CloudFlareAPIError: Authentication error```

I don't know why it's showing an Authentication error because the ```CF_TOKEN``` is working when I just use DOMAIN1. I've confirmed the Zone IDs for all of my other domains.

### Relevant logs and/or screenshots
<!-- Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise. -->

Please see above

### Environment
https://pastebin.com/P3rvUFHR

- Image version / tag:  v6.8.2
- Host OS: Ubuntu 20.04 LTS
Docker 20.10.12 (API: 1.41)

<details>
<summary>Any logs | docker-compose.yml</summary>
</details>

<!-- Include anything additional -->
### Possible fixes
<!-- If you can, provide details to the root cause that might be responsible for the problem. -->

Thanks!

Different TARGET_DOMAIN for subdomain

I like to define two TARGET_DOMAIN entries. One is pointing to the internet reachable address of my server the other is pointing to my local IP address. The idea is that i can configure a *.local.domain.tld pointing to the server LAN IP for internal services but also configure a *.domain.tld for services that need to be reached from the WWW.

Note that I don't want/need to reach a service by both options (although possible):

1004 DNS Validation Error

I'm getting this error in the log when trying to create the cnames for domain.xyz and www.domainname.xyz

container rule value: Host(domainname.xyz,www.domainname.xyz)
extracted_domains from rule: [u'domainname.xyz,www.domainname.xyz']
Found Container: 647e66db859869694c080e371cff0b9bba33e21850d439c5fe3b86a41eb818f7 with Hostname domainname.xyz,www.domainname.xyz
** domainname.xyz,www.domainname.xyz - 1004 DNS Validation Error

organizr container label is Host(domainname.xyz,www.domainname.xyz)

Logging

Description of the feature

Is there a different logging level where it wont show this error?

Don't work with SWARM_MODE

Hi, thank for your great solution. Please, explain why container does not watching swarm cluster and updating cloudflare cnames?

version: '3.5'

services:
  traefik-cloudflare-companion:
    image: tiredofit/traefik-cloudflare-companion
    networks:
      - {{ monitoring_network_name }}
      - {{ traefik_network_name }}
      - internal
    environment:
      - TIMEZONE={{ TZ }}
      - TRAEFIK_VERSION="2"
      - CF_EMAIL={{ cf_api_email }}
      - CF_TOKEN={{ cf_api_key }}
      - TARGET_DOMAIN={{ app_domain_name }}
      - DOMAIN1={{ app_domain_name }}
      - DOMAIN1_ZONE_ID={{ cf_api_zone }}
      - DOMAIN1_PROXIED="TRUE"
      - SWARM_MODE="TRUE"
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro

networks:
  {{ monitoring_network_name }}:
    driver: overlay
    attachable: true
    name: {{ monitoring_network_name }}
  {{ traefik_network_name }}:
    external: true
    name: {{ traefik_network_name }}
  internal:
    external: false

exec: python3: not found

After upgrading my docker containers today I started to receive a python3 error. Below is the section of my docker compose file as well as the relevant log lines. The last section just keeps repeating. Everything goes back to working if I change from latest to v5.0.0

  cf-companion:
    container_name: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    security_opt:
      - no-new-privileges:true
    restart: unless-stopped
    networks:
      - socket_proxy
    depends_on:
      - socket-proxy
    # volumes:
    #  - /var/run/docker.sock:/var/run/docker.sock:ro # Use Docker Socket Proxy instead for improved security
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL # Same as traefik
      # - CF_TOKEN=$CLOUDFLARE_API_TOKEN # Scoped api token not working. Error 10000.
      - CF_TOKEN=$CLOUDFLARE_API_KEY # Same as traefik
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=$CLOUDFLARE_ZONEID # Copy from Cloudflare Overview page
      - DOMAIN1_PROXIED=TRUE
      - DOCKER_HOST=tcp://socket-proxy:2375
      - CONTAINER_LOG_LEVEL=DEBUG
    secrets: # not working
      - cloudflare_email
      - cloudflare_api_key
      - cloudflare_zoneid

today at 9:31 PM [cont-init.d] 99-container: executing...
today at 9:31 PM [DEBUG] ** [container] Getting ready to start '03-zabbix' service
today at 9:31 PM [DEBUG] ** [container] Skipping '04-cron' service for startup routines
today at 9:31 PM [DEBUG] ** [container] Skipping '05-smtp' service for startup routines
today at 9:31 PM [DEBUG] ** [container] Getting ready to start '10-cloudflare-companion' service
today at 9:31 PM [cont-init.d] 99-container: exited 0.
today at 9:31 PM [cont-init.d] done.
today at 9:31 PM [services.d] starting services
today at 9:31 PM [DEBUG] ** [traefik-cloudflare-companion] Checking to see if container initialization scripts have completed
today at 9:31 PM [DEBUG] ** [traefik-cloudflare-companion] Checking to see if service has initialized
today at 9:31 PM [DEBUG] ** [zabbix] Checking to see if container initialization scripts have completed
today at 9:31 PM [DEBUG] ** [zabbix] Checking to see if service has initialized
today at 9:31 PM [services.d] done.
today at 9:31 PM ./run: line 11: exec: python3: not found
today at 9:31 PM [INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
today at 9:31 PM [INFO] ** [zabbix] Starting Zabbix Agent
today at 9:31 PM Starting Zabbix Agent [alpine]. Zabbix 5.0.3 (revision {ZABBIX_REVISION}).
today at 9:31 PM Press Ctrl+C to exit.
today at 9:31 PM
today at 9:31 PM [DEBUG] ** [traefik-cloudflare-companion] Checking to see if container initialization scripts have completed
today at 9:31 PM [DEBUG] ** [traefik-cloudflare-companion] Checking to see if service has initialized
today at 9:31 PM [INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
today at 9:31 PM ./run: line 11: exec: python3: not found
today at 9:31 PM [DEBUG] ** [traefik-cloudflare-companion]

Healthcheck Option

Is there a possibility to use healthchecks like:

    healthcheck:
      test: wget --spider http://localhost || exit 1
      interval: "30s"
      timeout: "5s"
      retries: 3
      start_period: "15s"

REFRESH_ENTRIES is false but companion tries to post existing CNAME

I'm currently using traefik poll api workflow and REFRESH_ENTRIES=false.
And during each poll the companion is trying to post existing entries instead of avoiding them.

Resulting in many errors like this

[ERROR] ** abc.domain.tld - 81053 An A, AAAA, or CNAME record with that host already exists.
[ERROR] ** xyz.domain.tld - 81053 An A, AAAA, or CNAME record with that host already exists.
[ERROR] ** abc.domain.tld - 81053 An A, AAAA, or CNAME record with that host already exists.
[ERROR] ** xyz.domain.tld - 81053 An A, AAAA, or CNAME record with that host already exists.

I can try to make a PR in the next few days to resolve this issue and avoid posting records that already exists.

Rate limited. Please wait and consider throttling your request speed

Hi,

Recently I'm getting this error in docker logs

2022-05-07T07:48:40.115540769Z 2022-05-07-07:48:40 [STARTING] ** [traefik-cloudflare-companion] [20] Starting Traefik Cloudflare Companion
2022-05-07T07:48:40.838016615Z Traceback (most recent call last):
2022-05-07T07:48:40.839233444Z   File "/usr/sbin/cloudflare-companion", line 466, in <module>
2022-05-07T07:48:40.840670399Z     sync_mappings(get_initial_mappings(traefik_included_hosts, traefik_excluded_hosts), doms)
2022-05-07T07:48:40.841077265Z   File "/usr/sbin/cloudflare-companion", line 354, in sync_mappings
2022-05-07T07:48:40.841842197Z     if point_domain(k, domain_infos):
2022-05-07T07:48:40.842125288Z   File "/usr/sbin/cloudflare-companion", line 168, in point_domain
2022-05-07T07:48:40.842756780Z     records = cf.zones.dns_records.get(domain_info['zone_id'], params={u'name': name})
2022-05-07T07:48:40.843058265Z   File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 672, in get
2022-05-07T07:48:40.843978952Z     return self._base.call_with_auth('GET', self._parts,
2022-05-07T07:48:40.844305153Z   File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 126, in call_with_auth
2022-05-07T07:48:40.844964418Z     return self._call(method, headers, parts,
2022-05-07T07:48:40.845231219Z   File "/usr/lib/python3.9/site-packages/CloudFlare/cloudflare.py", line 502, in _call
2022-05-07T07:48:40.845985004Z     raise CloudFlareAPIError(code, message)
2022-05-07T07:48:40.849372463Z CloudFlare.exceptions.CloudFlareAPIError: Rate limited. Please wait and consider throttling your request speed

I'm not sure why or what else I can provide so you can reproduce it. I'm sure it's on my end but I have no idea what exactly is causing this error.

Edit: I'm using this tag tiredofit/traefik-cloudflare-companion:6.8.1

[RPi][ARMv7] Fatal Error on Startup - Python error: pyinit_main: can't initialize time

Summary

Docker container fails to start;

After initialization it shoots the following errors in the Docker Logs:

Current thread 0x76fc7390 (most recent call first):
<no Python frame>
7889-11-12-01:56:40 [STARTING] ** [monitoring] [1] Starting Zabbix Agent (modern) 5.4.2
7889-09-19-00:35:36 [STARTING] ** [traefik-cloudflare-companion] [2] Starting Traefik Cloudflare Companion
Fatal Python error: pyinit_main: can't initialize time
Python runtime state: core initialized
PermissionError: [Errno 1] Operation not permitted

Steps to reproduce

Run on RaspberryPi3 (Armv7).

What is the expected correct behavior?

Should start

Relevant logs and/or screenshots

Recreating cf-companion ... done
[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] 00-functions: applying...
[fix-attrs.d] 00-functions: exited 0.
[fix-attrs.d] 01-s6: applying...
[fix-attrs.d] 01-s6: exited 0.
[fix-attrs.d] 02-zabbix: applying...
[fix-attrs.d] 02-zabbix: exited 0.
[fix-attrs.d] 03-logrotate: applying...
[fix-attrs.d] 03-logrotate: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-startup: executing...
[cont-init.d] 00-startup: exited 0.
[cont-init.d] 01-timezone: executing...
[cont-init.d] 01-timezone: exited 0.
[cont-init.d] 02-permissions: executing...
[cont-init.d] 02-permissions: exited 0.
[cont-init.d] 03-monitoring: executing...
7889-11-19-08:08:24 [NOTICE] ** [monitoring] Container configured for monitoring with 'zabbix modern'
[cont-init.d] 03-monitoring: exited 0.
[cont-init.d] 04-scheduling: executing...
7889-09-14-17:59:20 [NOTICE] ** [scheduling] Disabling Scheduling
[cont-init.d] 04-scheduling: exited 0.
[cont-init.d] 05-logging: executing...
/var/run/s6/etc/cont-init.d/05-logging: line 11: /assets/cron//logrotate: No such file or directory
7889-09-08-06:06:48 [NOTICE] ** [logging] Disabling Log Shipping Functionality
[cont-init.d] 05-logging: exited 0.
[cont-init.d] 06-messaging: executing...
7889-10-01-02:12:08 [NOTICE] ** [messaging] Container configured to route mail via SMTP to 'postfix-relay'
[cont-init.d] 06-messaging: exited 0.
[cont-init.d] 10-cloudflare-companion: executing...
[cont-init.d] 10-cloudflare-companion: exited 0.
[cont-init.d] 99-container: executing...
[cont-init.d] 99-container: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
7889-10-24-08:29:12 [STARTING] ** [traefik-cloudflare-companion] [1] Starting Traefik Cloudflare Companion
Fatal Python error: pyinit_main: can't initialize time
Python runtime state: core initialized
PermissionError: [Errno 1] Operation not permitted

Current thread 0x76fc7390 (most recent call first):
<no Python frame>
7889-11-12-01:56:40 [STARTING] ** [monitoring] [1] Starting Zabbix Agent (modern) 5.4.2
7889-09-19-00:35:36 [STARTING] ** [traefik-cloudflare-companion] [2] Starting Traefik Cloudflare Companion
Fatal Python error: pyinit_main: can't initialize time
Python runtime state: core initialized
PermissionError: [Errno 1] Operation not permitted

Current thread 0x76f68390 (most recent call first):
<no Python frame>
7889-10-30-03:34:48 [STARTING] ** [traefik-cloudflare-companion] [3] Starting Traefik Cloudflare Companion
Fatal Python error: pyinit_main: can't initialize time
Python runtime state: core initialized
PermissionError: [Errno 1] Operation not permitted

Environment

Image version / tag: latest (6.5.1?)
Host OS: Raspbian GNU/Linux 10 (buster)

  cf-companion:
    container_name: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL # Same as traefik
      - CF_TOKEN=$CLOUDFLARE_API_KEY # Same as traefik
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=$CLOUDFLARE_ZONEID # Copy from Cloudflare Overview page
      - DOMAIN1_PROXIED=TRUE

Possible fixes

I read online that libseccomp2 should be updated, but I'm not sure.
linuxserver/docker-calibre-web#119

A working workaround exists here:
https://blog.samcater.com/fix-workaround-rpi4-docker-libseccomp2-docker-20/

Disable zabbix-agent per default

Description of the feature
The Zabbix-Agent comes with this container as your base image (https://github.com/tiredofit/docker-alpine) configures this.

The problem with this base-implementation is that this container makes DNS requests for zabbix-agent every minute. This is spamming the DNS server and actually unnecessary in an environment without Zabbix.

I was able to disable zabbix via CONTAINER_ENABLE_MONITORING: "FALSE" environment variable and believe this should be the default or better documented in this container (since I had to dig out the base image to find this variable). Or: The default zabbix config could be to not do a DNS request for zabbix-agent every minute?

Benftits of feature
No more zabbix-agent DNS spam

Create "A" record if it does not exist for TARGET_DOMAIN ?

Hey, great product!

Description of the feature
To be able to create the A record for the target_domain if it does not exist.
Maybe have to introduce another variable for TARGET_NAME_IP (optional) - so that it is able to add it OR update it.

Benftits of feature
The companion would take care of creating the main A record so we don't need to add these manually.

Container stalls after "Starting Zabbix Agent"

I've got the following log

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] 00-functions: applying... 
[fix-attrs.d] 00-functions: exited 0.
[fix-attrs.d] 01-s6: applying... 
[fix-attrs.d] 01-s6: exited 0.
[fix-attrs.d] 02-zabbix: applying... 
[fix-attrs.d] 02-zabbix: exited 0.
[fix-attrs.d] 03-logrotate: applying... 
[fix-attrs.d] 03-logrotate: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-startup: executing... 
[cont-init.d] 00-startup: exited 0.
[cont-init.d] 01-timezone: executing... 
[NOTICE] ** [timezone] Timezone: Setting to 'America/New_York' from 'Etc/GMT'
[cont-init.d] 01-timezone: exited 0.
[cont-init.d] 02-permissions: executing... 
[cont-init.d] 02-permissions: exited 0.
[cont-init.d] 03-zabbix: executing... 
[cont-init.d] 03-zabbix: exited 0.
[cont-init.d] 04-cron: executing... 
[NOTICE] ** [cron] Disabling Cron
[cont-init.d] 04-cron: exited 0.
[cont-init.d] 05-smtp: executing... 
[NOTICE] ** [smtp] Disabling SMTP Features
[cont-init.d] 05-smtp: exited 0.
[cont-init.d] 10-cloudflare-companion: executing... 
[NOTICE] ** [traefik-cloudflare-companion] Setting Traefik 2.x Mode
[cont-init.d] 10-cloudflare-companion: exited 0.
[cont-init.d] 99-container: executing... 
[cont-init.d] 99-container: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
[INFO] ** [zabbix] Starting Zabbix Agent

My docker-compose looks like this:

    image: tiredofit/traefik-cloudflare-companion:latest
    container_name: cloudflare-companion
    networks:
     - traefik_proxy
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL
      - CF_TOKEN=$CLOUDFLARE_API_KEY
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=$CLOUDFLARE_ZONEID
      - DOMAIN1_PROXIED=FALSE
    restart: always
    deploy:
      placement:
        constraints:
          - "node.role==manager"

Logging into the container and executing the items in /etc/cont-init.d/ shows only a single issue with "03-zabbix" as shown below

mkdir: can't create directory '': No such file or directory
chown: unknown user 
chown: unknown user

My cursory glance is showing its failing to create a logfile somewhere along the line and then dropping out of the whole thing.

Feature request: Delete CNAME on container stop

Heya,

I'm unsure if this is possible at all.

Would it be feasible to delete CNAMEs on containers that no longer exist?

Would be useful for test containers or limited time/scope mini-projects that once their usefulness is done can just be deleted. But obviously currently, the CNAME still stays up on CF.

Thanks!

ERROR message for each service.

Does cloudflare-companion do any tracking or checks before it looks to create DNS records?

I keep getting this in my logs for each service whenever I restart my docker-compose:
today at 18:48:36 [ERROR] ** dozzle.host.com - 81053 An A, AAAA, or CNAME record with that host already exists.

My config:

  cf-companion:
    container_name: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      TRAEFIK_VERSION: 2
      CF_EMAIL: $CLOUDFLARE_EMAIL
      CF_TOKEN: $CLOUDFLARE_API_KEY
      TARGET_DOMAIN: $DOMAINNAME
      DOMAIN1: $DOMAINNAME
      DOMAIN1_ZONE_ID: $CLOUDFLARE_ZONE_ID
      DOMAIN1_PROXIED: "TRUE"
      PUID: $PUID
      PGID: $PGID
      TZ: $TZ
      UMASK: $UMASK

It's not a major issue, just not sure if it should be an ERROR

Sorry if I've missed something!

ignore spercific subdomains

I use a wildcard cname for *.lan.example.com on my cloudflare this points to an an internal address.
when I run the container it creates enties for each of these internal services.
a way to ignore specific subdomains would be great.

Enable flag in label

Similar to Traefik, add a flag to enable cloudfare-companion for individual containers based on an additional label.

labels:
cloudflare.enable=True

It would also be useful to have an environment variable to set the default behavior.

Constant Authentication Errors

Summary

When I add a new domain to a container through traefik, cloudflare-companion errors out repeatedly with authentication failures to cloudflare. I'm not sure if something has changed with Cloudflare's API, but both of my other services which rely on the same API keys/tokens do not have the same issue as this does.

Steps to reproduce

(Input your api keys and tokens, and then attempt to start the container with a host set to a subset of the domain.)

What is the expected correct behavior?

It should not error and should create the subdomain automatically.

Relevant logs and/or screenshots

cloudflare-companion    | 2022-08-14.15:59:38 [STARTING] ** [traefik-cloudflare-companion] [19] Starting Traefik Cloudflare Companion
cloudflare-companion    | Traceback (most recent call last):
cloudflare-companion    |   File "/usr/sbin/cloudflare-companion", line 466, in <module>
cloudflare-companion    |     sync_mappings(get_initial_mappings(traefik_included_hosts, traefik_excluded_hosts), doms)
cloudflare-companion    |   File "/usr/sbin/cloudflare-companion", line 354, in sync_mappings
cloudflare-companion    |     if point_domain(k, domain_infos):
cloudflare-companion    |   File "/usr/sbin/cloudflare-companion", line 168, in point_domain
cloudflare-companion    |     records = cf.zones.dns_records.get(domain_info['zone_id'], params={u'name': name})
cloudflare-companion    |   File "/usr/lib/python3.10/site-packages/CloudFlare/cloudflare.py", line 695, in get
cloudflare-companion    |     return self._base.call_with_auth('GET', self._parts,
cloudflare-companion    |   File "/usr/lib/python3.10/site-packages/CloudFlare/cloudflare.py", line 139, in call_with_auth
cloudflare-companion    |     return self._call(method, headers, parts,
cloudflare-companion    |   File "/usr/lib/python3.10/site-packages/CloudFlare/cloudflare.py", line 525, in _call
cloudflare-companion    |     raise CloudFlareAPIError(code, message)
cloudflare-companion    | CloudFlare.exceptions.CloudFlareAPIError: Authentication error

Environment

Image version / tag: 6.8.5
Host OS: Proxmox VE 7.2-7
Guest OS: Almalinux 8.6 x86_64

docker-compose.yml

version: "3.7"
networks:
  internal:
    driver: bridge
  services:
    driver: bridge
  keycloakDB:
    driver: bridge
volumes:
  traefik-logs:
services:
  cloudflare-ddns:
    image: timothyjmiller/cloudflare-ddns:latest
    container_name: cloudflare-ddns
    security_opt:
      - no-new-privileges:true
    network_mode: "host"
    environment:
      - PUID=1000
      - PGID=1000
    volumes:
      - ./CloudflareDDNS/config.json:/config.json:ro
    restart: unless-stopped
  cloudflare-companion:
    image: tiredofit/traefik-cloudflare-companion:latest
    container_name: cloudflare-companion
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TRAEFIK_VERSION=2
      - CF_EMAIL=<Redacted>
      - CF_TOKEN=<Redacted>
      - TARGET_DOMAIN=ddns.domain.tld
      - DOMAIN1=domain.tld
      - DOMAIN1_PROXIED=true
      - DOMAIN1_ZONE_ID=<Redacted>
      #- DOCKER_HOST=tcp://198.51.100.32:2376
      #- DOCKER_CERT_PATH=/docker-certs
      #- DOCKER_TLS_VERIFY=1
    networks:
      - internal
      - services
    restart: unless-stopped
  KCPostgres:
    image: bitnami/postgresql:14
    container_name: keycloakDB
    restart: unless-stopped
    networks:
      - keycloakDB
    volumes:
      - ./postgres/data/postgres_data:/bitnami/postgresql/
    ports:
      - 5432:5432
    env_file:
      - ./postgres/KCpostgres.env
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U <Redacted>"]
      interval: 10s
      timeout: 5s
      retries: 5
  KCPBackup:
      image: bitnami/postgresql:14
      container_name: KCDBBackup
      networks: 
        - keycloakDB
      volumes:
        - ./postgres/data/backup/:/dump/
      restart: unless-stopped
      environment:
        - PGHOST=KCPostgres
        - PGUSER=<redacted>
        - PGPASSWORD=<redacted>
        - BACKUP_NUM_KEEP=7
        - BACKUP_FREQUENCY=1d
      entrypoint: |
        bash -c 'bash -s <<EOF
        trap "break;exit" SIGHUP SIGINT SIGTERM
        sleep 2m
        while /bin/true; do
          pg_dump -Fc > /dump/dump_\`date +%d-%m-%Y"_"%H_%M_%S\`.psql
          (ls -t /dump/dump*.psql|head -n $$BACKUP_NUM_KEEP;ls /dump/dump*.psql)|sort|uniq -u|xargs rm -- {}
          sleep $$BACKUP_FREQUENCY
        done
        EOF'
      depends_on:
        KCPostgres:
          condition: service_healthy
  traefik:
      image: traefik:vacherin
      container_name: traefik
      restart: unless-stopped
      security_opt:
        - no-new-privileges:true
      networks:
        - services
      ports:
        - 80:80
        - 443:443
        - 8080:8080
      environment:
        - CF_API_EMAIL=<Redacted>
        #- CF_API_KEY=<Redacted>
        - CF_DNS_API_TOKEN=<Redacted>
      volumes:
        - /etc/localtime:/etc/localtime:ro
        - /var/run/docker.sock:/var/run/docker.sock:ro
        - ./Traefik/data/traefik.yml:/traefik.yml:ro
        - ./Traefik/data/acme.json:/acme.json
        - ./Traefik/data/config.yml:/config.yml:ro
        - ./Traefik/data/log:/var/log/traefik
      labels:
        - traefik.enable=true
        - traefik.http.routers.traefik.entrypoints=http
        - traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https
        - traefik.http.middlewares.redirect-to-https.redirectscheme.port=443
        - traefik.http.routers.redirs.rule=hostregexp(`{host:.+}`)
        - traefik.http.routers.redirs.entrypoints=http
        - traefik.http.routers.redirs.middlewares=redirect-to-https
        - traefik.http.routers.dashboard.rule=Host(`traefik.domain.tld`)
        - traefik.http.services.dashboard.loadbalancer.server.port=8080

Possible fixes

./run: line 11: exec: python: not found

Latest tag has error:

[INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
./run: line 11: exec: python: not found

4.2.0 and 4.2.1 is absolutely fine.

Add multiple excluded subdomains in docker-compose

Hey!

It seems we can't add multiple subdomains to exclude in docker-compose with the variable DOMAIN1_EXCLUDED_SUB_DOMAINS

When enter one subdomain, there is no problem. The CNAME record will not be added in the DNS zone.
But if adding multiple subdomains, with comma separated, all subdomains will be added by a CNAME in the DNS zone.

Example of working excluded subdomains docker-compose:

  cf-companion:
    container_name: cf-companion
    hostname: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL
      - CF_TOKEN=$CLOUDFLARE_API_KEY
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=XXXXXXXXXXXXX
      - DOMAIN1_PROXIED=TRUE
      - DOMAIN1_EXCLUDED_SUB_DOMAINS=phpmyadmin

Example of not working excluded subdomains docker-compose (if wanted to use more than one subdomain excluded):

  cf-companion:
    container_name: cf-companion
    hostname: cf-companion
    image: tiredofit/traefik-cloudflare-companion:latest
    restart: unless-stopped
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL=$CLOUDFLARE_EMAIL
      - CF_TOKEN=$CLOUDFLARE_API_KEY
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=XXXXXXXXXXXXX
      - DOMAIN1_PROXIED=TRUE
      - DOMAIN1_EXCLUDED_SUB_DOMAINS=phpmyadmin,influxdb

Already tried with a space between subdomains, with a semicolon, by specifying multiple times DOMAIN1_EXCLUDED_SUB_DOMAINS (only the last definition is working, of course), with double quotes. None are working.

Thanks for the help!

Authentication error with zone DNS API token

For minimal permissions I have created an API token locked down to zone DNS edits using the "Edit zone DNS" template:

When launching this service I get an Authentication error:

cloudflare-companion | [INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
cloudflare-companion | Traceback (most recent call last):
cloudflare-companion |   File "/usr/sbin/cloudflare-companion", line 276, in <module>
cloudflare-companion |     init(doms)
cloudflare-companion |   File "/usr/sbin/cloudflare-companion", line 211, in init
cloudflare-companion |     check_container_t2(c, doms)
cloudflare-companion |   File "/usr/sbin/cloudflare-companion", line 176, in check_container_t2
cloudflare-companion |     point_domain(extracted_domains[0], doms)
cloudflare-companion |   File "/usr/sbin/cloudflare-companion", line 91, in point_domain
cloudflare-companion |     records = cf.zones.dns_records.get(dom['zone_id'], params={u'name': name})
cloudflare-companion |   File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 672, in get
cloudflare-companion |     return self._base.call_with_auth('GET', self._parts,
cloudflare-companion |   File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 126, in call_with_auth
cloudflare-companion |     return self._call(method, headers, parts,
cloudflare-companion |   File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 502, in _call
cloudflare-companion |     raise CloudFlareAPIError(code, message)
cloudflare-companion | CloudFlare.exceptions.CloudFlareAPIError: Authentication error

Am I missing other permissions?

Base container not handle very well drop all caps and run as non-root

Summary

The base image of the container make some tests and run code to "fix" something how is not a problem.

The code is on file /package/admin/s6-overlay-3.0.0.2/libexec/preinit inside the container.

Steps to reproduce

Se the logs from:

version: '3.9'

services:
  dns-cloudflare:
    image: tiredofit/traefik-cloudflare-companion
    cap_drop:
      - ALL
    user: "1000:1000"
    environment:
      - "TRAEFIK_VERSION=2"
      - "SWARM_MODE=TRUE"
      - "DOCKER_HOST=tcp://export-docker-sock:2375"
      - "CF_TOKEN=${CF_TOKEN}"

      - "TARGET_DOMAIN=${CF_LOADBALANCER}"
      - "DOMAIN1=${CF_DOMAIN}"
      - "DOMAIN1_ZONE_ID=${CF_ZONE_ID}"
    deploy:
      replicas: 1
    networks:
      - traefik-net
      - docker-net

networks:
  traefik-net:
    external:
      name: traefik-net
  docker-net:
    external:
      name: docker-ro-net

What is the expected correct behavior?

Container not need to set uid or gid. The container can run as any user and drop all caps.

Relevant logs and/or screenshots

This is the log:

s6-overlay-suexec: fatal: unable to setgid to root: Operation not permitted

With cap add setuid and setgid:

s6-chown: fatal: unable to chown /run: Operation not permitted
s6-overlay-suexec: fatal: child failed wth exit code 111

Environment

Image version / tag: latest
Host OS: Linux

Any logs | docker-compose.yml

s6-overlay-suexec: fatal: unable to setgid to root: Operation not permitted
s6-chown: fatal: unable to chown /run: Operation not permitted
s6-overlay-suexec: fatal: child failed wth exit code 111

Possible fixes

Well, there is somethings:

List caps used by container(not need this)
Change entrypoint to run /usr/sbin/cloudflare-companion direct
Change the entrypoint to not create files and/or set group/user(or disable by env)

Randomly stopped working, log is full of errors

I had this working for quite some time, but it seems to have failed recently and I'm not sure why. The logs are full of errors like these and recreating the container doesn't fix it.

urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory')),
    sock.connect(self.unix_socket),
  File "/usr/lib/python3.8/site-packages/docker/transport/unixconn.py", line 43, in connect,
    self.connect(),
  File "/usr/lib/python3.8/http/client.py", line 950, in send,
    self.send(msg),
  File "/usr/lib/python3.8/http/client.py", line 1010, in _send_output,
    self._send_output(message_body, encode_chunked=encode_chunked),
    self.endheaders(body, encode_chunked=encode_chunked),
  File "/usr/lib/python3.8/http/client.py", line 1250, in endheaders,
  File "/usr/lib/python3.8/http/client.py", line 1301, in _send_request,
    self._send_request(method, url, body, headers, encode_chunked),
  File "/usr/lib/python3.8/http/client.py", line 1255, in request,
    conn.request(method, url, **httplib_request_kw),
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 392, in _make_request,
    httplib_response = self._make_request(,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 670, in urlopen,
    raise value.with_traceback(tb),
  File "/usr/lib/python3.8/site-packages/urllib3/packages/six.py", line 734, in reraise,
    raise six.reraise(type(error), error, _stacktrace),
  File "/usr/lib/python3.8/site-packages/urllib3/util/retry.py", line 403, in increment,
    retries = retries.increment(,
  File "/usr/lib/python3.8/site-packages/urllib3/connectionpool.py", line 726, in urlopen,
    resp = conn.urlopen(,
  File "/usr/lib/python3.8/site-packages/requests/adapters.py", line 439, in send,
Traceback (most recent call last):

Support for TCP Header with Host SNI

Hello,

When setting a HostSNI prefix instead of Host on a TCP router (as opposed to HTTP) for something like Nextcloud, the subdomains are not properly scraped for creation. Samples of successful and unsuccessful below. Thanks for any insight!

Compose:

# Local Cloud Storage
  nextcloud:
    image: linuxserver/nextcloud
    container_name: nextcloud
    restart: always
    networks:
      - $TRAEFIK_NETWORK
    security_opt:
      - no-new-privileges
    volumes:
      - ${DOCKERDIR}/nextcloud:/config
      - ${STORAGEDIR}/backups/nextcloud:/data
    environment:
      - TZ
      - PUID
      - PGID
    labels:
      - "traefik.enable=true"
      ## TCP Routers
      - "traefik.tcp.routers.nextcloud-tcp.entrypoints=https"
      - "traefik.tcp.routers.nextcloud-tcp.rule=HostSNI(`cloud.$DOMAINNAME`)"
      - "traefik.tcp.routers.nextcloud-tcp.tls=true"
      - "traefik.tcp.routers.nextcloud-tcp.tls.passthrough=true"
      ## TCP Services
      - "traefik.tcp.routers.nextcloud-tcp.service=nextcloud-tcp-svc"
      - "traefik.tcp.services.nextcloud-tcp-svc.loadbalancer.server.port=443"


# Services & Media Dashboard
  organizr:
    image: organizrtools/organizr-v2
    container_name: organizr
    restart: always
    networks:
      - $TRAEFIK_NETWORK
    security_opt:
      - no-new-privileges
    volumes:
      - ${DOCKERDIR}/organizr:/config
    environment:
      - TZ
      - PUID
      - PGID
    labels:
      - "traefik.enable=true"
      ## HTTP Routers
      - "traefik.http.routers.organizr-rtr.entrypoints=https"
      - "traefik.http.routers.organizr-rtr.rule=Host(`org.$DOMAINNAME`)"
      - "traefik.http.routers.organizr-rtr.tls=true"
      ## Middlewares
      - "traefik.http.routers.organizr-rtr.middlewares=chain-authelia@file"
      ## HTTP Services
      - "traefik.http.routers.organizr-rtr.service=organizr-svc"
      - "traefik.http.services.organizr-svc.loadbalancer.server.port=80"

Organizr log:

today at 8:20 AM container rule value:  Host(`org.domain.com`)
today at 8:20 AM extracted_domains from rule:  [u'org.domain.com']
today at 8:20 AM Found Container: 1319414219a6dbfaf20323092c7b72feff709c17ebb94a020b14202a216e6023 with Hostname org.domain.com

Nextcloud log:

today at 11:33 AM container rule value:  HostSNI(`cloud.domain.com`)
today at 11:33 AM extracted_domains from rule:  []

script 10-cloudflare-companion did not pass completion check

Summary

I built the local docker image on armV7 succesfully a couple of weeks ago. When I update the local docker build with the newest version, it build successfully, but doesn't launch.

Steps to reproduce

What is the expected correct behavior?

no errors

Relevant logs and/or screenshots

**********************************************************************************************************************
**********************************************************************************************************************
****                                                                                                              ****
****       ERROR - Some initialization scripts haven't completed - All services are now halted                    ****
****             - The following scripts in '/etc/cont-init.d' did not pass their completion check                ****
****                                                                                                              ****
**********************************************************************************************************************
**********************************************************************************************************************

10-cloudflare-companion

[cont-init.d] 06-messaging: exited 0.
[cont-init.d] 10-cloudflare-companion: executing...
foreground: warning: unable to spawn /var/run/s6/etc/cont-init.d/10-cloudflare-companion: No such file or directory
[cont-init.d] 10-cloudflare-companion: exited 127.
[cont-init.d] 99-container: executing...

However, the file does exist

bash-5.1# cat /var/run/s6/etc/cont-init.d/10-cloudflare-companion
#!/command/with-contenv bash

source /assets/functions/00-container
prepare_service single
PROCESS_NAME="traefik-cloudflare-companion"

Sanity Test

sanity_var TARGET_DOMAIN "Target Domain"
sanity_var DOMAIN1 "Domain 1"
sanity_var DOMAIN1_ZONE_ID "Domain 1 Zone ID"

liftoff

### Environment

- Image version / tag:  docker built locally
- Host OS: Linux 5.10.63-v7+ #1496 SMP Wed Dec 1 15:58:11 GMT 2021 armv7l GNU/Linux

<details>
  cf-companion:
    container_name: cf-companion
    image: cf-comp:latest
    restart: always
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    environment:
      - TIMEZONE=Europe/Brussels
      - TRAEFIK_VERSION=2
      - CF_EMAIL=${CF_API_EMAIL}
      - CF_TOKEN=${CF_GLOBAL_API_KEY}
      - TARGET_DOMAIN=${DOMAIN_NAME}
      - DOMAIN1=${DOMAIN_NAME}
      - DOMAIN1_ZONE_ID=${CF_ZONE_ID}
      - DOMAIN1_PROXIED=TRUE
      - </details>

<!-- Include anything additional -->
    
### Possible fixes
<!-- If you can, provide details to the root cause that might be responsible for the problem. -->

Consider using pythons logging modules instead of prints

Implementing and using pythons logging module instead of prints feels a lot more flexible and more pythonic.

It also give cleaner code as we can remove a lot of if CONTAINER_LOG_LEVEL == "DEBUG":

See mbjurstrom@ac36ae6 for inspiration.

Looks much cleaner to my eyes. But not sure what your opinion is about that.
If there is any interest I am happy to supply a PR without the rest of the of the additions I added in that branch. Most of them is in existing open PRs from me or other people.

Support for defaultRule

Instead of defining Host label under each service, i have a default rule in the traefik.yml configuration as shown below:

providers:
  docker:
    endpoint: "unix:///var/run/docker.sock"
    exposedByDefault: false
    swarmMode: false
    network: traefik_2
    defaultRule: "Host(`{{ index .Labels \"com.docker.compose.service\"}}.domain.com`)"

Is their any work around for this scenario or possibility to add support?

Support Dynamic Configs

Description of the feature
Traefik supports dynamic configurations via providers: https://doc.traefik.io/traefik/getting-started/configuration-overview/#the-dynamic-configuration. I use this to proxy other internal services running on different internal hosts to be accessible externally.

However, the urls that configured dynamically do not get registered with cloudflare. It works locally, but not accessible via cloudflare. A work-around is to manually insert CNAME records in Cloudflare's DNS config, however it's just a hassle everytime I want a new service online

Benftits of feature
Dynamic Configurations are supposed to work just like static urls, but cloudflare companion does not support it.

Additional context
No additional context.

Support for internal domain

use case

I have a few services on my internal network I’d like the ability to generate SSL certs for but not have them exposed behind the gateway.

Use case — I’m running an internal dns server and using an Unraid box. The box has multiple services running, some internal and some external only. All requests are being done via my TLD for simplicity — internal dns handles the routing and it never hits the external dns server. Some services are being exposed to the internet.
All these containers have the routing handled via Traefik

As of right now this container is almost perfect for me

Feature

A label that can be added to containers to generate an ssl certificate but not add an external dns entry

Support for alternative traefik host declaration

The following syntax is valid within traefik:

"Host(`subdomain.domain.com`,`subdomain2.domain.com`,`subdomain3.domain.com`)"

However this syntax is not valid within cloudflare companion.

#15 addresses this

Support for Raspberry Pi

Can you add support for raspberry?

Create A-records instead

Instead of CNAMEs, I'd like to create A-Records, as with Cloudflare free tier, CNAMEs get flattened to reveal destination IP, bypassing the proxy function.

Authentication Error

I have verified the token to work via cloudfare-ddns docker container I use.

cf-companion:
container_name: cf-companion
image: tiredofit/traefik-cloudflare-companion:latest
restart: always
security_opt:
- no-new-privileges:true
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- TIMEZONE=$TZ
- TRAEFIK_VERSION=2
#- CF_EMAIL=$CLOUDFLARE_EMAIL# Same as traefik
- CF_TOKEN=$CF_APITOKEN
#- CF_TOKEN=$CLOUDFLARE_API_KEY # Same as traefik
- TARGET_DOMAIN=$DOMAINNAME
- DOMAIN1=$DOMAINNAME
- DOMAIN1_ZONE_ID=$CF_ZONES # Copy from Cloudflare Overview page
- DOMAIN1_PROXIED=TRUE

CloudFlare.exceptions.CloudFlareAPIError: Authentication error
[INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion
Traceback (most recent call last):
File "/usr/sbin/cloudflare-companion", line 276, in
init(doms)
File "/usr/sbin/cloudflare-companion", line 211, in init
check_container_t2(c, doms)
File "/usr/sbin/cloudflare-companion", line 176, in check_container_t2
point_domain(extracted_domains[0], doms)
File "/usr/sbin/cloudflare-companion", line 91, in point_domain
records = cf.zones.dns_records.get(dom['zone_id'], params={u'name': name})
File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 672, in get
return self._base.call_with_auth('GET', self._parts,
File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 126, in call_with_auth
return self._call(method, headers, parts,
File "/usr/lib/python3.8/site-packages/CloudFlare/cloudflare.py", line 502, in _call
raise CloudFlareAPIError(code, message)

No CNAME's added

Hello.
I'm using Docker Swarm mode, Traefik 1.7.24 and latest image of docker-traefik-cloudflare-companion.
Here's my compose file:

version: '3.5'

services:
  cloudflare-companion:
    image: tiredofit/traefik-cloudflare-companion:latest
    environment:
      TRAEFIK_VERSION: 1
      CF_EMAIL: [email protected]
      CF_TOKEN: my-custom-cf-token
      DOMAIN1: mydomain.com
      DOMAIN1_ZONE_ID: my-zone-id
      DOMAIN1_PROXIED: "true"
      TARGET_DOMAIN: target.mydomain.com
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
    networks: 
      - traefik-net
      - internal
    deploy:
      replicas: 1
      placement:
       constraints:
         - node.role == manager
      restart_policy:
        condition: none

networks:
  traefik-net:
    name: traefik-net
    attachable: true
  internal:
    external: false

After start container logs:

[s6-init] making user provided files available at /var/run/s6/etc...exited 0.
[s6-init] ensuring user provided files have correct perms...exited 0.
[fix-attrs.d] applying ownership & permissions fixes...
[fix-attrs.d] 00-functions: applying... 
[fix-attrs.d] 00-functions: exited 0.
[fix-attrs.d] 01-s6: applying... 
[fix-attrs.d] 01-s6: exited 0.
[fix-attrs.d] 02-zabbix: applying... 
[fix-attrs.d] 02-zabbix: exited 0.
[fix-attrs.d] 03-logrotate: applying... 
[fix-attrs.d] 03-logrotate: exited 0.
[fix-attrs.d] done.
[cont-init.d] executing container initialization scripts...
[cont-init.d] 00-startup: executing... 
[cont-init.d] 00-startup: exited 0.
[cont-init.d] 01-timezone: executing... 
[cont-init.d] 01-timezone: exited 0.
[cont-init.d] 02-permissions: executing... 
[cont-init.d] 02-permissions: exited 0.
[cont-init.d] 03-zabbix: executing... 
[cont-init.d] 03-zabbix: exited 0.
[cont-init.d] 04-cron: executing... 
[NOTICE] ** [cron] Disabling Cron
[cont-init.d] 04-cron: exited 0.
[cont-init.d] 05-smtp: executing... 
[NOTICE] ** [smtp] Disabling SMTP Features
[cont-init.d] 05-smtp: exited 0.
[cont-init.d] 10-cloudflare-companion: executing... 
[NOTICE] ** [traefik-cloudflare-companion] Setting Traefik 1.x Mode
[cont-init.d] 10-cloudflare-companion: exited 0.
[cont-init.d] 99-container: executing... 
[cont-init.d] 99-container: exited 0.
[cont-init.d] done.
[services.d] starting services
[services.d] done.
[INFO] ** [zabbix] Starting Zabbix Agent
[INFO] ** [traefik-cloudflare-companion] Starting Traefik Cloudflare Companion

Then i start service with compose file:

version: '3'

services:
  landing:
    image: my-company/my-image:version
    networks: 
      - traefik-net
    deploy:
      replicas: 1
      labels:
      - traefik.frontend.rule=Host:app.mydomain.com
      - traefik.port=80
      - traefik.enable=true
      placement:
       constraints:
         - node.role == manager
    restart: always

networks:
  traefik-net:
    external: true

Companion logs is empty (no new entries). No new records in Cloudflare.
What am i doing wrong?

domain-specific TARGET_DOMAIN?

I have two zones in Cloudflare, so 2 domains and separate DNS settings.
In traefik I specify dedicated subdomains for each domain.

cfcompanion's config allows to specifiy several domains DOMAIN1, DOMAIN2...
However, the TARGET_DOMAIN seems to be used for all domains specified.
All CNAME records are created in that TARGET_DOMAIN.

So in my domain1.com DNS settings I get

CNAME    mysubdomain.domain2.com       domain1.com
CNAME    othersubdomain                domain1.com

Instead of one CNAME in domain1.com and the other in domain2.com

Bug ? Or am I not getting something?

Possible to disable debug logging?

Recently I'm seeing a ton of extra log output and I don't recall specifying debug logging anywhere. How can I revert this to an info or warn type of level? Sample output:

today at 9:43 PM 2020-10-12 21:43:26,370 - Python Cloudflare API v4 - DEBUG - Response: error 81053 An A, AAAA or CNAME record already exists with that host.
today at 9:43 PM [ERROR] ** php.xxxxxxxx.com - 81053 An A, AAAA or CNAME record already exists with that host.
today at 9:43 PM 2020-10-12 21:43:26,381 - Python Cloudflare API v4 - DEBUG - Call: emulated curl command ...
today at 9:43 PM curl -X GET "https://api.cloudflare.com/client/v4/zones/af7b02f6760430b6eaaf6489002e9251/dns_records?name=redis.xxxxxxxx.com" \
today at 9:43 PM -H "User-Agent: python-cloudflare/2.8.13/python-requests/2.23.0/python/3.8.5" \
today at 9:43 PM -H "Content-Type: application/json" \
today at 9:43 PM -H "X-Auth-Email: [email protected]" \
today at 9:43 PM -H "X-Auth-Key: REDACTED"

Config:

# Cloudflare-Companion - Automatic container CNAME DNS Creation
  cloudflare-companion:
    image: tiredofit/traefik-cloudflare-companion
    container_name: cloudflare-companion
    restart: unless-stopped
    security_opt:
      - no-new-privileges:true
    networks:
      - $SOCKET_NETWORK
    depends_on:
      - socket-proxy
    environment:
      - TIMEZONE=$TZ
      - TRAEFIK_VERSION=2
      - CF_EMAIL_FILE=/run/secrets/cf_email
      - CF_TOKEN_FILE=/run/secrets/cf_token
      - TARGET_DOMAIN=$DOMAINNAME
      - DOMAIN1=$DOMAINNAME
      - DOMAIN1_ZONE_ID=$CF_API_ZONE_ID
      - DOMAIN1_PROXIED=TRUE
      - DOCKER_HOST=tcp://socket-proxy:2375
    secrets:
      - cf_email
      - cf_token

Support for creating multiple CNAMES per container

With Traefik 2 you can create multiple subdomains for a single container, like so

      - "traefik.http.routers.foobar-rtr.rule=Host(`foo.$DOMAIN_NAME`,`bar.$DOMAIN_NAME`)"

or like so

      - "traefik.http.routers.foobar-rtr.rule=Host(`foo.$DOMAIN_NAME`)"
      - "traefik.http.routers.foobar-rtr.rule=Host(`bar.$DOMAIN_NAME`)"

It seems that only one subdomain per container is seen by this tool and only a single CNAME is created. It would be nice if this tool would catch and create all of the CNAMES defined in a container's compose file with either method of formatting in these examples.

tiredofit / docker-traefik-cloudflare-companion Goto Github PK

docker-traefik-cloudflare-companion's Issues

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Environment

Possible fixes

Summary

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Environment

Summary

Steps to reproduce

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Environment

Possible fixes

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Environment

Possible fixes

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Environment

Possible fixes

Summary

Steps to reproduce

What is the expected correct behavior?

Relevant logs and/or screenshots

Sanity Test

Recommend Projects

Recommend Topics

Recommend Org