boredazfcuk / docker-icloudpd Goto Github PK
View Code? Open in Web Editor NEWAn Alpine Linux container for the iCloud Photos Downloader command line utility
An Alpine Linux container for the iCloud Photos Downloader command line utility
Hi,
Any chance to support Pushover notifications?
Thanks...
I'm hitting this issue after setting up the Docker container. I have set up the 2FA keyring and it's successful:
2020-12-16 19:04:02 INFO Two factor authentication cookie generated. Sync should now be successful.
but after restarting the Docker I am seeing this:
2020-12-16 19:04:19 INFO Check 2FA Cookie
2020-12-16 19:04:19 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes
I really appreciate you putting this together. It allows me to download my pictures to my local nas automatically and I've been wanting to make sure I wasn't locked in to another photo web service again.
Thanks again!
When trying to run this on raspbian I get the following error:
standard_init_linux.go:178: exec user process caused "exec format error"
Any idea why?
Thanks for getting this up and running
Hi,
There seems to be a lot of network traffic, mainly uploads (and down) from the container.
This is slightly concerning, especially as I’ve left the container running after it failed to locate the required cookie to work.
Why is it doing that, and where is all that traffic going ?
Hi there,
please keep up the great work! I’m trying to run your code on a NAS from Synology. My problem is that Synology always runs docker containers interactively, so your script is always in „config mode“ trying to renew the auth. cookie.
Would you be so kind to extend this check so that we can deactivate config mode also by setting an env e.g. „disableSetup=1“.
Thanks again!
Noting that my configured Prowl notifications are not being sent, I checked the source and noticed a bug.
The Notify() function uses notify_api_key in the curl call to send the notification, but this variable is never set based on prowl_api_key / pushbullet_api_key. This should probably be done in ConfigureNotifications().
Hello!
I've been running icloudpd for months without any issues. Sadly icloudpd currently fails checking for new files. I've tried to remove the 2 factor cookie, just in case, but without any success.
Can you please help me out?
The log says:
2020-08-22 13:25:48 INFO ***** boredazfcuk/icloudpd container for icloud_photo_downloader started *****
2020-08-22 13:25:48 INFO Alpine Linux v3.11
2020-08-22 13:25:48 INFO Local user: docker:1026
2020-08-22 13:25:48 INFO Local group: users:100
2020-08-22 13:25:48 INFO Apple ID: xxx
2020-08-22 13:25:48 INFO Apple ID Password: xxxx
2020-08-22 13:25:48 INFO Authentication Type: 2FA
2020-08-22 13:25:48 INFO Cookie path: /config/xxx
2020-08-22 13:25:48 INFO Cookie expiry notification period: 7
2020-08-22 13:25:48 INFO iCloud directory: /home/docker/iCloud
2020-08-22 13:25:48 INFO Folder structure: {:%Y/%m/%d}
2020-08-22 13:25:48 INFO Directory permissions: 750
2020-08-22 13:25:48 INFO File permissions: 640
2020-08-22 13:25:48 INFO Syncronisation interval: 86400
2020-08-22 13:25:48 INFO Time zone: Europe/Berlin
2020-08-22 13:25:48 INFO Additional command line options: --until-found 500
2020-08-22 13:25:48 INFO Interactive only mode set, bypassing 2FA cookie generation
2020-08-22 13:25:48 INFO Telegram notifications enabled
2020-08-22 13:25:48 INFO Telegram token xxx
2020-08-22 13:25:48 INFO Telegram chat id xxx
2020-08-22 13:25:48 INFO Sending Telegram startup notification
2020-08-22 13:25:49 INFO Telegram notification sent successfully
2020-08-22 13:25:49 INFO Set owner, docker, on iCloud directory, if required
2020-08-22 13:25:51 INFO Set group, users, on iCloud directory, if required
2020-08-22 13:25:51 INFO Correct owner on icloudpd temp directory, if required
2020-08-22 13:25:51 INFO Correct group on icloudpd temp directory, if required
2020-08-22 13:25:51 INFO Correct owner on config directory, if required
2020-08-22 13:25:51 INFO Correct group on config directory, if required
2020-08-22 13:25:51 INFO Set 750 permissions on iCloud directories, if required
2020-08-22 13:25:52 INFO Set 640 permissions on iCloud files, if required
2020-08-22 13:25:52 INFO Sync user docker
2020-08-22 13:25:52 INFO Check 2FA Cookie
2020-08-22 13:25:52 INFO Cookie exists, check expiry date
2020-08-22 13:25:52 INFO Valid two factor authentication cookie found. Days until expiration: 89
2020-08-22 13:25:52 INFO Check download directory mounted correctly
2020-08-22 13:25:52 INFO Check for new files...
2020-08-22 13:25:58 ERROR Check failed - Exit code: 1
2020-08-22 13:25:58 INFO Sending Telegram startup notification
2020-08-22 13:25:58 INFO Telegram notification sent successfully
2020-08-22 13:25:58 INFO Web cookie expires: 2020-10-21 @ 11:20:38
2020-08-22 13:25:58 INFO Two factor authentication cookie expires: 2020-11-20 @ 11:20:49
2020-08-22 13:25:58 INFO Syncronisation complete for docker
2020-08-22 13:25:58 INFO Next syncronisation at 13:25
Hi,
Pushbullet notification always fails.
2020-11-10 14:42:42 INFO Sending Pushbullet startup notification,
2020-11-10 14:42:42 ERROR Pushbullet notification failed for Apple ID XXXXXX,
2020-11-10 14:46:46 INFO Pushbullet notifications enabled,
2020-11-10 14:46:46 INFO Notification period: 14,
2020-11-10 14:46:46 INFO Sending Pushbullet startup notification,
2020-11-10 14:46:46 ERROR Pushbullet notification failed for Apple ID XXXXXX
Hi,
I love your container and it make my life a lot easier ;-). However for the notification via webhook it would be great to have a way configure the path of the request.
At the moment it's hardcoded (I guess for homeassist): notification_url="${webhook_scheme}://${webhook_server}:${webhook_port}/api/webhook/${webhook_id}"
Would be great to have something to be flexible with the webhook service like:
webhook_path="/hook/myPath/"
notification_url="${webhook_scheme}://${webhook_server}:${webhook_port}${webhook_path}${webhook_id}"
Hi,
It will be great if there will be another option to schedule by time of the day instead of intervals (like cron for example).
I have couple of accounts that i want to sync, and i don't want them to correspond.
Thanks...
No changes to configuration. Now getting the following error. Gives "No Reason"
2020-06-13 17:21:53 INFO Set 750 permissions on iCloud directories, if required
2020-06-13 17:21:53 INFO Set 640 permissions on iCloud files, if required
2020-06-13 17:21:53 INFO Sync user remote_fs_user
2020-06-13 17:21:53 INFO Check 2FA Cookie
2020-06-13 17:21:53 INFO Cookie exists, check expiry date
2020-06-13 17:21:53 INFO Valid two factor authentication cookie found. Days until expiration: 89
2020-06-13 17:21:53 INFO Check download directory mounted correctly
2020-06-13 17:21:53 INFO Checking for new files using insecure password method. Please store password in the iCloud keyring to prevent password leaks
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 209, in authenticate
req = self.session.post(
File "/usr/lib/python3.8/site-packages/requests/sessions.py", line 578, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 100, in request
self._raise_error(code, reason)
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 122, in _raise_error
raise api_error
pyicloud_ipd.exceptions.PyiCloudAPIResponseError: Unknown reason
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/icloudpd", line 11, in
load_entry_point('icloudpd==1.5.0', 'console_scripts', 'icloudpd')()
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/icloudpd-1.5.0-py3.8.egg/icloudpd/base.py", line 253, in main
File "/usr/lib/python3.8/site-packages/icloudpd-1.5.0-py3.8.egg/icloudpd/authentication.py", line 29, in authenticate
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 193, in init
self.authenticate()
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/base.py", line 216, in authenticate
raise PyiCloudFailedLoginException(msg, error)
pyicloud_ipd.exceptions.PyiCloudFailedLoginException: ('Invalid email/password combination.', PyiCloudAPIResponseError('Unknown reason'))
2020-06-13 17:21:55 ERROR Check failed - Exit code: 1
2020-06-13 17:21:55 INFO Web cookie expires: 2020-08-12 @ 15:18:48
Hello,
What I have to set in Value for the convert_heic_to_jpeg variable environnent ?
Second question, what if I set folder structure to {:%Y/%m} and 2 pictures have the same name ?
Thank you
Hi, is it possible/doable to have webhooks as a notification service or notification to an mqtt broker? Loving the container!
Weird situation where it continues to recognize that there are files to download, but never seems to download anything. the directories get created, but no files appear. This had been working fine for a long time.
icloudpd, version 1.4.4
2020-02-09 08:33:51 INFO Web cookie expires: 2020-02-28 @ 14:37:14
2020-02-09 08:33:51 INFO Two factor authentication cookie expires: 2020-03-29 @ 14:37:37
2020-02-09 08:33:51 INFO Syncronisation complete for usernamehere
2020-02-09 08:33:51 INFO Next syncronisation at 14:33
2020-02-09 14:33:51 INFO Check 2FA Cookie
2020-02-09 14:33:51 INFO Cookie exists, check expiry date
2020-02-09 14:33:51 INFO Valid two factor authentication cookie found. Days until expiration: 48
2020-02-09 14:33:51 INFO Check download directory mounted correctly
2020-02-09 14:33:51 INFO Check for new files...
2020-02-09 14:45:41 INFO Check successful
2020-02-09 14:45:41 INFO Check detected 72 files requiring download. Verifying list accuracy
2020-02-09 14:45:41 INFO Ignoring 28 files which have already been downloaded
2020-02-09 14:45:41 INFO New files detected: 44
2020-02-09 14:45:41 INFO Starting download of 44 new files for user: usernamehere
2020-02-09 14:45:42 DEBUG Authenticating...
2020-02-09 14:50:00 INFO Web cookie expires: 2020-02-28 @ 14:37:14
2020-02-09 14:50:00 INFO Two factor authentication cookie expires: 2020-03-29 @ 14:37:37
2020-02-09 14:50:00 INFO Syncronisation complete for usernamehere
2020-02-09 14:50:00 INFO Next syncronisation at 20:50
2020-02-09 20:50:00 INFO Check 2FA Cookie
2020-02-09 20:50:00 INFO Cookie exists, check expiry date
2020-02-09 20:50:00 INFO Valid two factor authentication cookie found. Days until expiration: 48
2020-02-09 20:50:00 INFO Check download directory mounted correctly
2020-02-09 20:50:00 INFO Check for new files...
2020-02-09 21:01:47 INFO Check successful
2020-02-09 21:01:47 INFO Check detected 72 files requiring download. Verifying list accuracy
2020-02-09 21:01:47 INFO Ignoring 28 files which have already been downloaded
2020-02-09 21:01:47 INFO New files detected: 44
2020-02-09 21:01:47 INFO Starting download of 44 new files for user: usernamehere
2020-02-09 21:01:48 DEBUG Authenticating...
2020-02-09 21:06:07 INFO Web cookie expires: 2020-02-28 @ 14:37:14
2020-02-09 21:06:07 INFO Two factor authentication cookie expires: 2020-03-29 @ 14:37:37
2020-02-09 21:06:07 INFO Syncronisation complete for usernamehere
2020-02-09 21:06:07 INFO Next syncronisation at 03:06
Is it possible to not print the plaintext password in the log? Just leave it blank.
I have auto_delete: "True"
enabled and the logs show;
Deleting any files found in 'Recently Deleted'...
today at 13:08 2020-11-13 13:08:47 INFO Deleting /home/robmint/iCloud/2020/IMG_0047.JPG!
today at 13:08 2020-11-13 13:08:47 INFO Deleting /home/robmint/iCloud/2020/IMG_0037.JPG!
today at 13:08 2020-11-13 13:08:47 INFO Deleting /home/robmint/iCloud/2020/IMG_0041.JPG!
today at 13:08 2020-11-13 13:08:48 INFO New files downloaded: 1
today at 13:08 2020-11-13 13:08:48 INFO Sending Telegram downloaded files notification
today at 13:08 2020-11-13 13:08:48 INFO Telegram downloaded files notification sent successfully
today at 13:08 2020-11-13 13:08:48 INFO Number of files deleted: 14
today at 13:08 2020-11-13 13:08:48 INFO Sending Telegram deleted files notification
today at 13:08 2020-11-13 13:08:48 INFO Telegram deleted files notification sent successfully
today at 13:08 2020-11-13 13:08:48 INFO Synchronisation complete for robmint
but when I check the filesystem, the files are not deleted and have different names with a string appended to the end;
IMG_0047-973486.JPG
IMG_0037-978773.JPG
IMG_0041-974132.JPG
There are no other files in the folder that share the same name (i.e. IMG_0047.JPG
, IMG_0037.JPG
or IMG_0041.JPG
).
Is it icloudpd that's renaming the files? If so, how can I ensure Recently Deleted files actually get deleted.
Thanks.
When executing: sync-icloud.sh for my second docker container there is no question about saving the password in keyring? Any idea's to force this question so i can store the password in keyring?
Everything else is working fine after enter the 2FA.
I already running another docker contrainer where i succesfull stored the password in keyring.
The docker containers are running on Synology NAS.
Hi,
I’ve just set up a Icloudpd container for my other half on my QNAP and after it starts, it eventually can’t progress any further than the following.
2021-01-24 22:20:29 INFO Check 2FA Cookie
2021-01-24 22:20:29 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes
2021-01-24 22:25:29 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes
So I tried to run the docker exec -it icloudpd /usr/local/bin/sync-icloud.sh --Generate2FACookie option from the command line, but it doesn’t prompt the device or me to do anything, eventually returning the same as the Docker container does .
2021-01-24 22:30:19 INFO Check 2FA Cookie
2021-01-24 22:30:19 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes
2021-01-24 22:35:19 ERROR Cookie does not exist. Please run container interactively to generate - Retry in 5 minutes
Any ideas what I am missing ?
If I run the container with the following docker-compose file, I get permissions errors. The user, user_id, group and group_id are the same as on the host. If I correct the local permissions with chown to match the docker-compose file, I still get permission errors.
If I run docker exec -it icloudpd /usr/local/bin/sync-icloud.sh
, the script runs through without any interaction and says the 2FA cookie has been set but it is not.
docker-compose;
icloudpd:
container_name: icloudpd
image: boredazfcuk/icloudpd
restart: unless-stopped
networks:
- default
volumes:
- /home/mint/Docker/media-server/config/icloudpd:/config
- /media/mint/htpc09/Media/Photos/iCloud:/home/mint/iCloud
environment:
user: mint
user_id: 501
group: docker
group_id: 999
apple_id: [email protected]
apple_password: "usekeyring"
authentication_type: "2FA"
notification_days: 14
command_line_options: '--folder-structure={:%Y} --recent 50 --auto-delete'
synchronisation_interval: 21600
TZ: Europe/London
The only way I could get the 2FA cookie was to run the above docker-compose without the user, user_id, group and group_id env variables, so instead it uses the defaults. I was then able to run the docker exec
command and successfully get the 2FA cookie. After this I reinstated the user, user_id, group and group_id env variables and chown the local folders to match. This results in the following log but no photos are downloaded;
logs
today at 21:32 2020-08-22 21:32:00 INFO Group ID available, creating group
today at 21:32 addgroup: gid '999' in use
today at 21:32 2020-08-22 21:32:00 INFO User ID available, creating user
today at 21:32 adduser: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Correct owner on config directory, if required
today at 21:32 find: unknown user mint
today at 21:32 2020-08-22 21:32:00 INFO Correct group on config directory, if required
today at 21:32 find: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Using password stored in keyring
today at 21:32 2020-08-22 21:32:00 INFO Set owner, mint, on iCloud directory, if required
today at 21:32 find: unknown user mint
today at 21:32 2020-08-22 21:32:00 INFO Set group, docker, on iCloud directory, if required
today at 21:32 find: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Correct owner on icloudpd temp directory, if required
today at 21:32 find: unknown user mint
today at 21:32 2020-08-22 21:32:00 INFO Correct group on icloudpd temp directory, if required
today at 21:32 find: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Correct owner on config directory, if required
today at 21:32 find: unknown user mint
today at 21:32 2020-08-22 21:32:00 INFO Correct group on config directory, if required
today at 21:32 find: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Correct owner on keyring directory, if required
today at 21:32 find: unknown user mint
today at 21:32 2020-08-22 21:32:00 INFO Correct group on keyring directory, if required
today at 21:32 find: unknown group docker
today at 21:32 2020-08-22 21:32:00 INFO Set 750 permissions on iCloud directories, if required
today at 21:32 2020-08-22 21:32:00 INFO Set 640 permissions on iCloud files, if required
today at 21:32 2020-08-22 21:32:00 INFO Sync user mint
today at 21:32 chown: invalid user: ‘mint:docker’
today at 21:32 2020-08-22 21:32:00 INFO Check 2FA Cookie
today at 21:32 2020-08-22 21:32:00 INFO Cookie exists, check expiry date
today at 21:32 2020-08-22 21:32:00 INFO Valid two factor authentication cookie found. Days until expiration: 89
today at 21:32 2020-08-22 21:32:00 INFO Check download directory mounted correctly
today at 21:32 2020-08-22 21:32:00 INFO Check for new files using password stored in keyring...
today at 21:32 su: unknown user mint
today at 21:32 cat: /tmp/icloudpd/icloud_check_exit_code: No such file or directory
today at 21:32 ash: out of range
today at 21:32 2020-08-22 21:32:00 INFO Check successful
today at 21:32 grep: /tmp/icloudpd/icloudpd_check.log: No such file or directory
today at 21:32 ash: out of range
today at 21:32 ash: out of range
today at 21:32 2020-08-22 21:32:00 INFO No new files detected. Nothing to download
today at 21:32 ash: out of range
today at 21:32 2020-08-22 21:32:00 INFO Web cookie expires: 2020-10-21 @ 20:24:44
today at 21:32 2020-08-22 21:32:00 INFO Two factor authentication cookie expires: 2020-11-20 @ 20:25:06
today at 21:32 2020-08-22 21:32:00 INFO Next syncronisation at 03:32
I'm not sure if it is relevant but if I docker exec
into the container and run ls -l
it shows the /config folder has permissions of mint:ping
. The user:group I set in the env variables is mint:docker
.
Any advice appreciated. Thanks.
I authenticated my user and stored the keyring file in /config/python_keyring/keyring_pass.cfg
but when starting the download container I get the following error.
What could be wrong?
kisiel_1 | 2020-08-02 18:13:06 INFO Apple ID password: usekeyring
kisiel_1 | 2020-08-02 18:13:06 INFO Authentication Type: 2FA
kisiel_1 | 2020-08-02 18:13:06 INFO Cookie path: /config/ituneskisielnetpl
kisiel_1 | 2020-08-02 18:13:06 INFO Cookie expiry notification period: 14
kisiel_1 | 2020-08-02 18:13:06 INFO Download destination directory: /home/kisiel/iCloud
kisiel_1 | 2020-08-02 18:13:06 INFO Folder structure: {:%Y/%m/%d}
kisiel_1 | 2020-08-02 18:13:06 INFO Directory permissions: 750
kisiel_1 | 2020-08-02 18:13:06 INFO File permissions: 640
kisiel_1 | 2020-08-02 18:13:06 INFO Syncronisation interval: 21600
kisiel_1 | 2020-08-02 18:13:06 INFO Time zone: Europe/London
kisiel_1 | 2020-08-02 18:13:06 INFO Additional command line options:
kisiel_1 | 2020-08-02 18:13:06 INFO Using password stored in keyring
kisiel_1 | 2020-08-02 18:13:06 INFO Set owner, kisiel, on iCloud directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Set group, kisiel, on iCloud directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct owner on icloudpd temp directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct group on icloudpd temp directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct owner on config directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct group on config directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct owner on keyring directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Correct group on keyring directory, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Set 750 permissions on iCloud directories, if required
kisiel_1 | 2020-08-02 18:13:06 INFO Set 640 permissions on iCloud files, if required
kisiel_1 | 2020-08-02 18:13:07 INFO Sync user kisiel
kisiel_1 | 2020-08-02 18:13:07 INFO Check 2FA Cookie
kisiel_1 | 2020-08-02 18:13:07 INFO Cookie exists, check expiry date
kisiel_1 | 2020-08-02 18:13:07 INFO Valid two factor authentication cookie found. Days until expiration: 89
kisiel_1 | 2020-08-02 18:13:07 INFO Check download directory mounted correctly
kisiel_1 | 2020-08-02 18:13:07 INFO Check for new files using password stored in keyring...
kisiel_1 | iCloud Password:
kisiel_1 | /usr/lib/python3.8/getpass.py:91: GetPassWarning: Can not control echo on the terminal.
kisiel_1 | passwd = fallback_getpass(prompt, stream)
kisiel_1 | Warning: Password input may be echoed.
kisiel_1 | Aborted!
kisiel_1 | 2020-08-02 18:13:07 ERROR Check failed - Exit code: 1
kisiel_1 | 2020-08-02 18:13:07 INFO Web cookie expires: 2020-10-01 @ 17:07:47
Hi,
Can you please add a variable for the until-found option?
Thanks...
Hi,
I've created another container and added my Prowl api key in this time, , but when running it, it returns an error in the console. Are you able to confirm this feature is working, and if so, what could the failure be as i've just used the same api key on another system and it worked ? Thanks.
2021-01-18 17:49:04 WARNING Additional command line options is depreceated. Please specify all options using the dedicated variables: --folder-structure={:%Y}",
2021-01-18 17:49:04 INFO Convert HEIC to JPEG: False
2021-01-18 17:49:04 INFO Prowl notifications enabled
2021-01-18 17:49:04 INFO Prowl api key 4581181d251myprowkapikey66758b21f
2021-01-18 17:49:04 INFO Notification period: 14
2021-01-18 17:49:04 INFO Sending Prowl startup notification
2021-01-18 17:49:09 ERROR Prowl notification failed for Apple ID myappleidemailaddress
Hi
I started getting this sort of errors
any idea what might cause them?
/home/user/iCloud/2018/12/29/IMG_9490-2184363.JPG
/home/user/iCloud/2018/12/29/IMG_9489-2373439.JPG
/home/user/iCloud/2018/12/29/IMG_9488-2415572.JPG
/home/user/iCloud/2018/12/29/IMG_9485-2347916.JPG
/home/user/iCloud/2018/12/29/IMG_9484-2159835.JPG
/home/user/iCloud/2018/12/29/IMG_9483-2295044.JPG
/home/user/iCloud/2018/12/29/IMG_9482-2288166.JPG
/home/user/iCloud/2018/12/29/IMG_9481-1983629.JPG
Traceback (most recent call last):
File "/usr/bin/icloudpd", line 11, in <module>
load_entry_point('icloudpd==1.6.2', 'console_scripts', 'icloudpd')()
File "/usr/lib/python3.8/site-packages/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.8/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.8/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/icloudpd-1.6.2-py3.8.egg/icloudpd/base.py", line 562, in main
File "/usr/lib/python3.8/site-packages/icloudpd-1.6.2-py3.8.egg/icloudpd/base.py", line 456, in download_photo
FileNotFoundError: [Errno 2] No such file or directory: '/home/user/iCloud/2018/12/29/IMG_9480.JPG'
2020-10-31 14:49:02 ERROR Check failed - Exit code: 1
Just as the title says, support for docker-compose launching.
Hi,
Does this container allow me to access Shared Albums?
Thanks!
I do not have the "convert_heic_to_jpeg" flag set and manually converted one of the .HEIC files to .JPG to see if on a re-sync it would re-download the original .HEIC and it did.
My guestion is if I set the "convert_heic_to_jpeg" flag to "True" will it be smart enough to check existing files against the .JPG name (instead of the .HEIC name) and not download every time?
Also, can I manually convert all my existing .HEIC files to .JPG and move on from here with the "convert_heic_to_jpeg" flag or should I let the program do it?
Hello community.
I have the docker image seemingly connected to icloud as I see the list of files blowing by the console indicating they are seen and downloading. The only problem is that I do not see any of the files "created" in the assigned download directory that I have specified. The ".mounted" file exists and is detected but none of the files appear to actually get saved into the directory. My guess is this is a file permission issue but I do not see any errors in the logs or console to indicate actual permission errors. Any ideas? This is on Synology system . Thanks. (I am currently setting up my "user" environment variable to my synology username that is running the docker container, and trying all combinations of "user_id", "group" and "group_id" which dont seem to be doing anything.
I see in the docker readme that it requires either a "named volume mapped to /config" or ".mounted" file in /home/${user}/iCloud. I'm confused on what this means?
Any information would be very helpful!
Thanks :)
As title.
I really like your tool, especially the automatic conversion of heic to jpeg. Although, I noticed differences in file size between pictures converted by macos photos and your script. Likely due to level of compressibility. I was thinking it might be nice to add a feature to adjust quality vs file size? I prefer high quality over compression, others might not.
You are you using the libheif package, right?
Hi @boredazfcuk
QQ - I tired your example of ‘ --env command_line_options="--folder-structure={:%Y}’ but it returned an error ?
Does it still work for you ?
Looking at the link you shared for these, it’s not clear what the other date options are ?
Ideally I would like the sort by just the year like you suggested..
Hi, when authenticating I am getting this weird error
please help
Traceback (most recent call last):
File "/usr/bin/icloudpd", line 11, in <module>
load_entry_point('icloudpd==1.5.0', 'console_scripts', 'icloudpd')()
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/icloudpd-1.5.0-py3.8.egg/icloudpd/base.py", line 292, in main
File "/usr/lib/python3.8/posixpath.py", line 336, in normpath
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType
2020-08-02 12:55:25 INFO Please ignore the 'expected str, bytes or os.PathLike object, not NoneType' error. This is a fault with the underlying application Python application
Apologize if this is expected or I'm misunderstanding it, but this doesn't make sense to me.
iCloud
chris
iCloud is mapped to my volume, I confirmed this logging into the container and going to /home/iCloud and executing "touch test123" which is now a file exposed to my server in the "icloud_backup" directory. /home/chris is empty. Obviously I get the error ".mounted file is not present. "
iCloud
chris
The problem is, the volume is still mapping to /home/iCloud. So the script is checking the correct place, " while [ ! -f "${download_path}/.mounted" ]; do" which corresponds to /home/chris/iCloud, but my volume is mapped to /home/iCloud
I'm afraid to try to continue. If the backup works, I assume it will fill the container, and not my volume like it should.
Trying to make a long story short. Volume is mapping to /home/iCloud, expected behavior is /home/chris/iCloud and I don't know enough about docker to find out what it's doing with that volume entry.
thank you,
Chris
I've rebuilt the container due to unexpected circumstances on my server, and am getting errors trying to download now. I've created .mounted in my iCloud directory, but I'm getting the following when I start the container:
2020-04-10 15:55:05 INFO Alpine Linux v3.11
2020-04-10 15:55:05 INFO Local user: manderso:1000
2020-04-10 15:55:05 INFO Local group: users:100
2020-04-10 15:55:05 INFO Apple ID: /*************
2020-04-10 15:55:05 INFO Apple ID Password: **********
2020-04-10 15:55:05 INFO Authentication Type: 2FA
2020-04-10 15:55:05 INFO Cookie path: /config/*************
2020-04-10 15:55:05 INFO Cookie expiry notification period: 7
2020-04-10 15:55:05 INFO iCloud directory: /home/manderso/iCloud
2020-04-10 15:55:05 INFO Folder structure: {:%Y/%m/%d}
2020-04-10 15:55:05 INFO Directory permissions: 750
2020-04-10 15:55:05 INFO File permissions: 640
2020-04-10 15:55:05 INFO Syncronisation interval: 43200
2020-04-10 15:55:05 INFO Time zone: America/New_York
2020-04-10 15:55:05 INFO Additional command line options:
2020-04-10 15:55:05 INFO Set owner, manderso, on iCloud directory, if required
2020-04-10 15:55:06 INFO Set group, users, on iCloud directory, if required
2020-04-10 15:55:08 INFO Correct owner on icloudpd temp directory, if required
2020-04-10 15:55:08 INFO Correct group on icloudpd temp directory, if required
2020-04-10 15:55:08 INFO Correct owner on config directory, if required
2020-04-10 15:55:08 INFO Correct group on config directory, if required
2020-04-10 15:55:08 INFO Set 750 permissions on iCloud directories, if required
2020-04-10 15:55:09 INFO Set 640 permissions on iCloud files, if required
2020-04-10 15:55:11 INFO Sync user manderso
2020-04-10 15:55:11 INFO Check 2FA Cookie
2020-04-10 15:55:11 INFO Cookie exists, check expiry date
2020-04-10 15:55:11 INFO Valid two factor authentication cookie found. Days until expiration: 90
2020-04-10 15:55:11 INFO Check download directory mounted correctly
2020-04-10 15:55:11 INFO Check for new files...
ash: syntax error: unexpected word
2020-04-10 15:55:11 ERROR Check failed - Exit code: 2
2020-04-10 15:55:11 INFO Web cookie expires: 2020-06-09 @ 19:49:21
2020-04-10 15:55:11 INFO Two factor authentication cookie expires: 2020-07-09 @ 19:49:33
2020-04-10 15:55:11 INFO Syncronisation complete for manderso
2020-04-10 15:55:11 INFO Next syncronisation at 03:55
Can you suggest what I should try next?
Thanks!
Notify "failure" code for Pushover looks like this:
Notify "failure" "iCloudPD container failure" "-2" "iCloudPD failed to download new files for Apple ID: ${apple_id} - Exit code ${check_exit_code}"
Where -2 is the priority.
From Pushover API:
priority - send as -2 to generate no notification/alert, -1 to always send as a quiet notification, 1 to display as high-priority and bypass the user's quiet hours, or 2 to also require confirmation from the user
I think "generate no notification/alert" dosen't go well together with "failure" :)
If anything "failure" would be more like priority 1 "high-priority and bypass the user's quiet hours".
An other option would be to remove "priority" from Pushover notifications.
Do what you like. 😁
Hello,
I have a question/issue. I created the container and the cookies and everything works up to this point perfectly. Thank you for this.
But the script just downloaded the pictures of the last 6 month and not all of them. Is there an argument I am missing? Or is there a somewhat another way to download all photos in the iCloud?
Thanks for the help.
curo1305
My docker commands are
touch /media/volume/home/myUser/Pictures/iCloud/.mounted
docker create \
--name icloudpd \
--hostname icloudpd_machine \
--network host \
--restart=always \
--env user=myUser \
--env user_id=1001 \
--env group=myUser \
--env group_id=1001 \
--env [email protected] \
--env authentication_type=2FA \
--env command_line_options="--folder-structure={:%Y} --recent 50 --auto-delete" \
--env TZ=Europe/London \
--env recent_only=False \
--volume /media/volume/docker/icloud:/config \
--volume /media/volume/home/myUser/Pictures/iCloud:/home/myUser/iCloud \
boredazfcuk/icloudpd
docker exec -it icloudpd /usr/local/bin/sync-icloud.sh
I can run both docker containers but when I run the second, it doesnt allow me to input the code from my iphone. I don't get the option to press 0 or 1. It basically just says cookie is generated and sync should be successful but the cookie doesnt appear in the file...
Hello,
I'm trying to convert HEIC to JPG. I used the the variable convert_heic_to_jpeg
:. According to logs, HEIC files had been converted to JPG but not the result expected in folders :
2020-11-12 18:17:40 INFO /home/user/iCloud/2019/12/IMG_0530.HEIC already exists.
2020-11-12 18:17:40 INFO All photos have been downloaded!
2020-11-12 18:17:40 INFO Convert HEIC files to JPEG
2020-11-12 18:17:40 INFO Convert HEIC to JPEG...
2020-11-12 18:17:40 INFO Synchronisation complete for user
Hi, would it be possible to incorporate telegram as a notification service for when the cookie is due to expire? Admittedly I'm not sure how feasible this is but I have it as a notification service for other docker containers and would be nice to incorporate this too.
Hey there, I am getting the following error message in my log. Do you have any troubleshooting suggestions? Thank you!
Traceback (most recent call last):
File "/usr/bin/icloudpd", line 11, in
load_entry_point('icloudpd==1.5.0', 'console_scripts', 'icloudpd')()
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 722, in call
return self.main(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/icloudpd-1.5.0-py3.8.egg/icloudpd/base.py", line 276, in main
File "/usr/lib/python3.8/site-packages/pyicloud_ipd/services/photos.py", line 194, in albums
folder['fields']['albumNameEnc']['value']).decode('utf-8')
KeyError: 'albumNameEnc'
2020-08-19 20:33:26 ERROR Check failed - Exit code: 1
Recently updated, tried to go interactive and refresh my token. Now getting this:
2020-04-29 21:21:35 INFO Great, you're all set up. The script can now be run without user interaction until 2SA expires.
You can set up email notifications for when the two-step authentication expires.
(Use --help to view information about SMTP options.)
Traceback (most recent call last):
File "/usr/bin/icloudpd", line 11, in <module>
load_entry_point('icloudpd==1.5.0', 'console_scripts', 'icloudpd')()
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 722, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.8/site-packages/click-6.7-py3.8.egg/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/usr/lib/python3.8/site-packages/icloudpd-1.5.0-py3.8.egg/icloudpd/base.py", line 292, in main
File "/usr/lib/python3.8/posixpath.py", line 336, in normpath
path = os.fspath(path)
TypeError: expected str, bytes or os.PathLike object, not NoneType
Any ideas?
Error: Invalid value for "-d" / "--directory": Path "/home//iCloud" does not exist.
ERROR Check failed - Exit code: 2
what am i missing?
Hi
First many thanks for posting the time/effort and instructions you've done around this.
I've just created an instance on my QNAP NAS (via its container station app) and when I completed the configuration and did the build, it run for the first time and on the console, i did not see a
"ERROR Apple ID password set to 'usekeyring' but keyring file does not exist. Container must be run interactively to add a password to the system keyring - Restart in 5mins"
... but it went on to prompted me with the following.
...
2021-01-18 12:21:25 WARNING Using Apple ID password from variable. This password will be visible in the process list of the host. Please add your password to the system keyring instead
2021-01-18 12:21:40 INFO Correct owner on config directory, if required
2021-01-18 12:21:40 INFO Correct group on config directory, if required
2021-01-18 12:07:06 INFO Generate 2FA cookie with password: mypassword
2021-01-18 12:07:06 WARNING Checking for new files using insecure password method. Please store password in the iCloud keyring to prevent password leaks
0: SMS to ********66
1: Enter two-factor authentication code
Please choose an option: [0]: 273684
Error: 273684 is not in the valid range of 0 to 1.
Please choose an option: [0]: 1
Please enter two-factor authentication code: 283684
2021-01-18 12:08:42 INFO Two factor authentication cookie generated. Sync should now be successful.
Yet I've not run the sync-icloud.sh or built/run a seperate instance yet) as I had the authorisation code there on my phone so I entered it in and it suggests it was successful ?
Is this message correct or do I still need to run the sync-icloud.sh etc ?
example:
my apple id format is: [email protected]
generated cookie file is: firstname_lastnameprovidercom
healthcheck.sh expects firstnamelastnameprovidercom
(i checked with a print statement)
therefore the healthcheck fails.
manually renaming the cookie file doesn't help either because it just gets regenerated.
I've been running with synchronisation_interval=10800
(3 hours) without issue. I tried to set this to a shorter period with synchronisation_interval=900
(15 minutes) and I started getting a 2FA popup on my laptop. To fix, I changed synchronisation_interval back to 10800 and ran docker exec -it icloudpd /usr/local/bin/sync-icloud.sh
again to reset the cookie.
My question is, is there a minimum synchronisation_interval
before this happens? Thanks.
Noticing the directories are getting created with 'root' ownership and then photos fail to download. At the end, all is reported as successful!
2020-02-14 22:04:49 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1684.JPG
2020-02-14 22:22:32 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1684.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:22:32 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1683.JPG
2020-02-14 22:40:17 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1683.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:40:17 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1682.JPG
2020-02-14 22:58:02 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1682.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:58:02 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1681.JPG
2020-02-14 23:15:46 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-14 23:33:36 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-14 23:51:26 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:09:16 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:27:07 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:27:12 INFO Could not download IMG_1681.JPG! Please try again later.
2020-02-15 00:27:12 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1680.JPG
2020-02-15 00:44:57 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
2020-02-15 01:02:47 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
2020-02-15 01:20:37 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
2020-02-14 22:04:42 DEBUG Looking up all photos and videos from album All Photos...
2020-02-14 22:04:42 INFO Downloading 3316 original photos and videos to /home/username/iCloud/ ...
2020-02-14 22:04:49 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1684.JPG
2020-02-14 22:22:32 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1684.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:22:32 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1683.JPG
2020-02-14 22:40:17 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1683.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:40:17 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1682.JPG
2020-02-14 22:58:02 ERROR IOError while writing file to /home/username/iCloud/2020/02/13/IMG_1682.JPG! You might have run out of disk space, or the file might be too large for your OS. Skipping this file...
2020-02-14 22:58:02 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1681.JPG
2020-02-14 23:15:46 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-14 23:33:36 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-14 23:51:26 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:09:16 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:27:07 ERROR Error downloading IMG_1681.JPG, retrying after 5 seconds...
2020-02-15 00:27:12 INFO Could not download IMG_1681.JPG! Please try again later.
2020-02-15 00:27:12 INFO Downloading /home/username/iCloud/2020/02/13/IMG_1680.JPG
2020-02-15 00:44:57 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
2020-02-15 01:02:47 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
2020-02-15 01:20:37 ERROR Error downloading IMG_1680.JPG, retrying after 5 seconds...
(....)
2020-02-15 10:18:28 INFO All photos have been downloaded!
2020-02-15 10:18:29 INFO Web cookie expires: 2020-02-28 @ 14:37:14
2020-02-15 10:18:29 INFO Two factor authentication cookie expires: 2020-03-29 @ 14:37:37
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.