[Feature] Unraid Community / All-Inclusive Image about immich HOT 90 CLOSED

No problem, I will try to merge the images next week

from immich.

@SmoochSmackers Thank you for your very detailed thoughts and ideas. On behalf of the team, I will answer your questions.

1. When I first started building Immich, I structure the app to be microservices and make them scalable. In order to scale with Docker, you either need Docker Swarm or a simpler and straightforward approach is to use Docker Compose. The number of containers is the number of underlying technology you need for the solution to work. If you are a Windows user, and you install software that required a database, it will install the database as well, but the services are often hidden from your eyes so you don't notice them. With self-hosting, you install every yourself and you know exactly what technologies are required to run, so it might take some time to get used to. Lastly, the point of us not building the all-in-one solution is the level of maintenance and complexity involved that we don't have the resources to spare. We rather use it to develop and maintain the core functionality of the application.
2. Immich is still under active development, so I encourage you not to rely on an auto-updating tool but to update the Docker's containers manually after reading the Release notes in cases there are breaking changes that need user involvement to perform a successful update
3. As you can see that the motion of self-hosting solutions is moving toward Docker and containers more and more. And it is a learning experience for us all, you start off with the convenience of running your services from a click of a button, then eventually your needs grow and you want to be able to control, configure, and understand how the underlying technology that you run the services you are most depended on work. So there is no harm in trying to learn and to know more about those information/technologies. I would encourage all self-hosters to do that for the sake of securing their data/resources. We provide information to run and set up Immich on Unraid so it takes some learning but I think it will be worthwhile for all parties involved.

Cheer!

from immich.

Sorry if this is a silly question, but since you closed this issue, does this mean that there will never be an official AIO Immich image? Only the third party one you linked to?

My number one issue right now, as an extremely novice self-hoster, is the way that I have to use docker compose on my Unraid machine to use it. The three issues I have with this is that:
1.) It spins up 5 separate containers.
2.) I cannot mass update it along with the rest of my docker containers and have to keep an eye on the github page for any new releases.
3.) I think that not having a single image available in Unraid's Community App store creates an artificial hurtle that limits Immich's reach. It took quite a lot of time, and build up of frustration with other photo hosting apps, to finally push me to learn how to use docker compose. I feel there are many like me out there, but with less time on their hands, who are not ready to take the plunge. It's a matter of getting more users/supporters by lowering the bar to entry.

To be clear, I'm not writing this in anger or frustration. To the contrary! I absolutely love this software and understand the ungodly amount of work you all must have in front of you, tackling such a project! I just wanted to give my perspective as a newbie trying their darndest to rip as many of Google's tendrils from my life as possible and taking back control of my data.

Thank you all for your amazing work, I've been looking for a solid google photos replacement for years now. From my experience trying to self-host Photoview, Photoprism, etc., Immich is far and away the best one I've come across, the only one that truly has a google photos-like experience with the mobile app, and the only one that has runs rock solid with almost no issues with its core functionalites. In fact, I love this project so much that I created a github account to better follow your progress and so that I could donate.

from immich.

@alextran1502 Thank you very much for your detailed reply! In regards to the Windows comparison, I'd honestly never thought of it in terms of everything an exe is doing in the background, so it makes a lot of sense that multiple containers would be a necessity. Regardless of how you need to do it, I'm just glad you're doing it!

Just spit-balling, but maybe, way down the line when you're not in active pre-production development, you could maybe revisit this, or even somehow put all the different containers on the unraid app store, such that one is a "core" for basic photo backup, another serves as an addon for utilizing the mobile app, another serves as an addon for machine learning, etc. No idea how possible any of that is, but I just wanted to throw it out there for consideration much later down the line.

Keep up the incredible work!! Thanks again!

from immich.

I think we're good with you keeping it in a separate repo for now. We just get the occasional question about it, so now everyone will know to go ask you instead. 😁

from immich.

@JamesTeague There is no image that contains the whole application yet. I am working on the branch deployment/containerize-app-into-single-dockerfile to accomplish that goal.

It is correct that the Postgres port is 5432. You can access it on your local machine as well as throughout your local network with the VM IPV4

from immich.

I actually already have Redis up and running, but how should that be configured? According to 'Step 1' from https://github.com/alextran1502/immich, there is only the database configuration.

from immich.

@Mattyfaz Thank you so much for posting the guide!! Do you want to post it on Reddit or I can include the link in the next monthly update?

from immich.

Thanks a lot @Mattyfaz for your guide 👍

I have to admit that this screenshot :

Is the reason why people hope someone would achieve to merge all the images together ! ;)

Personnaly I'll wait for this moment as I always have the feeling that it would make my unraid server a mess

EDIT : typo

from immich.

I managed to install altran1502/immich-server:release on Unraid without docker-compose

@martabal how did manage to override the entrypoint? unraid webpage Extra Parameters doesn't seems to help
Edit:
Nevermind I got it working, I needed to use "Post Argument" commnad

from immich.

Maybe we should also post here : https://discourse.linuxserver.io/t/request-immich/4482/2

Linuxserver's team would be a BIG step for Immich integration on Unraid and it would offer a really big visibility IMO.

from immich.

Agreed, it would be nice to have all in one container to simplify things. However, docker folders is pretty useful to put all the current containers together, can always collapse the folder to make it look more tidy.

from immich.

@sohail579 We don't have any concrete plan when to put resource into this yet. Right now it is heavily depend on the community to provide support for a single Docker image.

from immich.

Here is the answer from a LSIO moderator :
https://discourse.linuxserver.io/t/request-immich/4482/5?u=guilhem

I hope the immich contributors will make it possible :)

from immich.

What is going to happen if I follow Immich guide and point it at a directory already full of existing photos and media?

Nothing, you need to give Immich an empty directory to work with and upload files via the API. There is an open issue for what you're looking for: #34. If you need to do a bulk import you can use the CLI https://github.com/immich-app/cli

from immich.

@blaine07

So I have been using a different app on my phone to upload photos to Unraid forever. What is going to happen if I follow Immich guide and point it at a directory already full of existing photos and media?

It won't pick them up (someone please correct me if I am wrong), if you'd rather not re-upload via your mobile an alternative is to copy the images from Unraid to your PC and upload via the WebUI as shown in the image below.

from immich.

Is it possibly because this Machine-Learning 1 is continuing to restart/bootloop?

Yes looks like your machine cannot run the Machine Learning container

from immich.

There is now a single AIO image at https://github.com/martabal/docker-immich. It isn't supported by the Immich team though.

from immich.

Would it be possible to migrate from the normal docker-compose setup to the all-in-one docker preserving the database and uploaded pictures?

from immich.

@guim31 Yep, I plan to do it this week. But you can already install it on unraid manually.

from immich.

Is there an existing Docker image on Docker Hub already or would one need to get it into a registry?

from immich.

Also there seems to be a dependency on Postgres running in port 5432. Did I read that right? If so, that port should probably be configurable defaulting to 5432.

from immich.

@alextran1502 Regarding the port that is true, but there will be people running Postgres in a container not using port 5432. It's generally not a good assumption that the database will be exposed on that port. So in the case of Unraid it would help users to know that this template requires their own postées instance but it shouldn't also dictate the port they have to expose on the container. Or if for some reason they are running multiple instances of the database, 1 has to be running on a different port.

Once the container is in the registry though I wouldn't mind helping get the template set up and tested. Or trying to contribute on other things as well.

from immich.

@JamesTeague There is no image that contains the whole application yet. I am working on the branch deployment/containerize-app-into-single-dockerfile to accomplish that goal.

Isn't the immich image sufficient if using an existing database?

from immich.

@JamesTeague There is no image that contains the whole application yet. I am working on the branch deployment/containerize-app-into-single-dockerfile to accomplish that goal.

Isn't the immich image sufficient if using an existing database?

There is a message queue that utilizes Redis for passing the message across the background processes, so you would need Redis as well.

Now that I am adding a web interface that is using SvelteKit, which has its own built-in Nodejs server, this adds an additional complexity to the process of putting everything into a single Docker image.

from immich.

@JamesTeague There is no image that contains the whole application yet. I am working on the branch deployment/containerize-app-into-single-dockerfile to accomplish that goal.

Isn't the immich image sufficient if using an existing database?

I think so, most people probably want to decouple that and Redis anyway.

from immich.

@JamesTeague There is no image that contains the whole application yet. I am working on the branch deployment/containerize-app-into-single-dockerfile to accomplish that goal.

The branch doesn't exist anymore. It would be very nice to have a single container to deploy. I already have postgres and redis dockers running and don't want to deploy other instances.

from immich.

You can use environment variables to point your Immich services at your existing redis and postgres containers, check the .env file

from immich.

Is it possible to get rid of the .env file and to use only the environment variables ? It would allow to run the services with docker run

from immich.

Sure, you can skip the .env file and pass the variables in any other way you like.

from immich.

For now Immich works with Unraid with the docker-compose plugin but in my understanding to run with the docker run command, it would be easier to integrate the entrypoint in the Dockerfile. Is there any particular reason to put it outside the file?

from immich.

yes the custom entrypoint is the main problem .. in my case, even in advanced docker mode in unraid i don't see where to put the custom entrypoint for each image.

from immich.

There is the --entrypoint argument but it is for overwriting the default entrypoint of the image. But there is no entrypoint by default in the images. So it is currently impossible to run Immich images with docker run

from immich.

The images are built on top of the node base image, so there is a default entrypoint defined that you can override. A command like docker run --entrypoint /bin/sh altran1502/immich-server:release ./start-server.sh works just fine (it doesn't have the necessary config, but it will start the right executable). That said, I agree that having entrypoints defined in the images is nicer, and I've suggested it to the team.

from immich.

For a release on Unraid, it would be a lot better to avoid arguments other than path, device, port or environment variable. Other arguments has to be passed with the extra-parameters field which is not very user-friendly.
And of course to have only one image.

from immich.

Sorry for my useless message but I have to say I would really love ab easy unraid install !!!

from immich.

I managed to install altran1502/immich-server:release on Unraid without docker-compose. But for now there is too many images to publish it for Unraid community. Another thing I noticed with my install is that I created a specific user on Postgres to Immich but it didn't work, I had to pass the default postgres user to work.

from immich.

Thanks @martabal for your feedback.
I hope someone with some Docker skills could merge all those images in one !

from immich.

No problem, I will try to merge the images next week

You’d be a hero here. Looking forward to this! :-)

from immich.

No problem, I will try to merge the images next week

Yea thanks, keen to give this a go :)

from immich.

Would we need to install a bunch of individual "apps" (containers) one by one to get this working on unraid without docker-compose? It looks like there's 7 containers, is that right?

I'm hesitant to use compose on unraid as it doesn't play well with their Web UI and that is my primary interface to the server.

from immich.

Currently, yes, you would need to set up the individual containers manually. For Redis and Postgres you could use templates, and the machine-learning container is not required (but you would miss out on object detection/tagging).

from immich.

I got this working by installing the unraid plugin for docker-compose. This is an OK workaround for now, but it makes a bunch of unmanaged containers in the main docker window.

1. Install docker compose manager from the app store
2. Add a new "stack" (find the button at the bottom of the docker tab) and call it "immich"
3. Edit the docker compose of the immich stack pasting in the contents docker/docker-compose.yml and save changes
4. Edit the environment of the immich stack pasting in the contents docker/.env, editing to suit your environment and save changes
5. Click the "Docker Up" button for the stack

Enjoy

from immich.

I got this working by installing the unraid plugin for docker-compose

Yeah, It works but is not ideal for many unraid users.

from immich.

Yeah Linuxserver's work is amazing, I hope they will create an Immich image. Btw, this issue doesn't appear in the Kanban. Hope this isn't a bad thing.

from immich.

I got this working by installing the unraid plugin for docker-compose. This is an OK workaround for now, but it makes a bunch of unmanaged containers in the main docker window.

1. Install docker compose manager from the app store

2. Add a new "stack" (find the button at the bottom of the docker tab) and call it "immich"

3. Edit the docker compose of the immich stack pasting in the contents `docker/docker-compose.yml` and save changes

4. Edit the environment of the immich stack pasting in the contents `docker/.env`, editing to suit your environment and save changes

5. Click the "Docker Up" button for the stack

Enjoy

I'm sorry could you share your .env and .yaml files? For the life of me, I can't get it working on unraid.

from immich.

I'm sorry could you share your .env and .yaml files? For the life of me, I can't get it working on unraid.

I got it working the other day so quickly put a rough guide online to help others: https://mfaz.dev/posts/immich-unraid/

from immich.

@Mattyfaz Thank you so much for posting the guide!! Do you want to post it on Reddit or I can include the link in the next monthly update?

@alextran1502 Feel free to post wherever/include in your update 😊

from immich.

awesome write up but I also think i will wait for a unified image. is there a view on how long this will take?

from immich.

I have successfully created an image bundling, web, machine-learning, nginx and server images. It works but I have some bugs with machine-learning. I tested it on Unraid and it works. However, this is just a draft, I don't plan to update this image frequently. You can try the container by pulling martabal/immich:latest, and you can give a look to the Dockerfile here

It uses almost 1 gb of RAM 😧 (but all official images combined use 1 gb of RAM)

from immich.

Nice work! It's good to have an initial approach (even if just a draft) to drive discussion further - for example, it just made me realise that doing this fully will be dependent on #647, since all the services will only be reachable on localhost.

from immich.

I'm sorry could you share your .env and .yaml files? For the life of me, I can't get it working on unraid.

I got it working the other day so quickly put a rough guide online to help others: https://mfaz.dev/posts/immich-unraid/

@Mattyfaz Thankyou for this awesome write up I had it up and running in minuets. I have a question though. I endered my location for the media upload folder but normally on unraid i would expect to see a folder in the appdata with all of the appdata.. where is this all now? where are all the thumbnails saved? am i just filling up my docker image now?

from immich.

Thumbnails and such are saved in your library folder. The database data is (by default) stored in a docker volume, which probably is ending up in your docker img right now. That shouldn't be that much data though.

from immich.

Thumbnails and such are saved in your library folder. The database data is (by default) stored in a docker volume, which probably is ending up in your docker img right now. That shouldn't be that much data though.

@bo0tzz your are everywhere :) and your advise and help is appreciated and awesome.. should my database in that case not be in my app data so that i am never going to loose it? or wlll the database rebuild itself from the media folder if needed ever?

from immich.

Quick update, the container runs very well, finally no bug observed. It uses between 800 and 900 mb of RAM. I pushed a version for arm64 for everyone using a raspberry pi. I will do my best to keep them up to date.

@bo0tzz In the long term do you plan to merge images or keep them sperated?

from immich.

@sohail579

where is this all now? where are all the thumbnails saved? am i just filling up my docker image now?

In your upload location - If you followed my guide you can see I set the upload location as mnt/user/images/immich.

Within that directory you can see a folder with a uuid, and inside that is 3 folders: encoded-video, original, and thumb. :)

from immich.

@sohail579

where is this all now? where are all the thumbnails saved? am i just filling up my docker image now?

In your upload location - If you followed my guide you can see I set the upload location as mnt/user/images/immich.

Within that directory you can see a folder with a uuid, and inside that is 3 folders: encoded-video, original, and thumb. :)

thanks @Mattyfaz - on the other note I ubderstand that the database is stored in the docker image. shouldn't this be in the app data?

from immich.

@sohail579

I ubderstand that the database is stored in the docker image. shouldn't this be in the app data?

Ideally - I've just tried and managed to get it working with an existing Postgres container separate from Immich so all data will be in my appdata now. Here is what I did.

.Env File
Update the database values to match your postgres container.

# DB_HOSTNAME=immich_postgres
DB_HOSTNAME=192.168.0.25
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# Optional Database settings:
DB_PORT=5432

Compose file:
You'll need to comment out all reference of database in this file.
Comment out the whole database service:

#  database:
#    container_name: immich_postgres
#    image: postgres:14
#    env_file:
#      - .env
#    environment:
#      POSTGRES_PASSWORD: ${DB_PASSWORD}
#      POSTGRES_USER: ${DB_USERNAME}
#      POSTGRES_DB: ${DB_DATABASE_NAME}
#      PG_DATA: /var/lib/postgresql/data
#    volumes:
#      - pgdata:/var/lib/postgresql/data
#    restart: always

Then for each other service remove the dependency on databse:

depends_on:
  - redis
# - database

Finally remove the volume at the bottom:

# volumes:
#	pgdata: 

Doing this I've successfully got it running.

from immich.

@bo0tzz In the long term do you plan to merge images or keep them sperated?

We had some discussions with the contributor team and did a few experiments ourselves with merging images. For now, we have decided not to build an all-in-one image.

Running multiple services inside one docker image is an unusual way of doing things, and would come with some issues (especially if including the Redis and Postgres dependencies). Merging the current images into one would also take time and energy away from developing other Immich features, as would providing support for the resulting mega-image.

In addition, the current images work just fine, also on Unraid (especially with @Mattyfaz's great guide). At the moment, the only reason for creating an all-in-one image would be to have it work with Unraid's Docker templates, but docker-compose fills a similar role to those already.

from immich.

So I have been using a different app on my phone to upload photos to Unraid forever. What is going to happen if I follow Immich guide and point it at a directory already full of existing photos and media?

from immich.

not sure I got this.. you still havent said anywhere to use a file location in appdata, isnt is as simple as replacing pgdata with a location in the appdata folder?

from immich.

That should also work, yes.

from immich.

That should also work, yes.

would that need to be everywhere where it says pgdata or do you just create it as a variable in the env file?

from immich.

Just this line, you would replace it with something like - /mnt/user/appdata/immich:/var/lib/postgresql/data.

from immich.

what about the last 2 lins? it also makes a reference to pgdata there wouldnt that also need to be the same?

from immich.

The

volumes:
  pgdata:

instructs Docker to create the pgdata volume. If you're not using it, you can remove that part, but leaving it won't do any harm.

from immich.

The

volumes:
  pgdata:

instructs Docker to create the pgdata volume. If you're not using it, you can remove that part, but leaving it won't do any harm.

got you thanks I thought it was trying to mount the volume :)

from immich.

I'm sorry could you share your .env and .yaml files? For the life of me, I can't get it working on unraid.

I got it working the other day so quickly put a rough guide online to help others: https://mfaz.dev/posts/immich-unraid/

Pardon my idiocracy, but how do I update everything once it's setup? I do have it working but... curious?

Thanks

from immich.

Also, with docker compose plugin installed on Unraid is it normal for it to say this at bottom of the docker tab:

Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/compose.manager/php/compose_manager_main.php on line 32

from immich.

Are those of you who have followed the Unraid Guide seeing this, below, spamming your syslogs since Immich was installed?

from immich.

Is it possibly because this Machine-Learning 1 is continuing to restart/bootloop?

from immich.

I disable JUST that container and Syslog spam stops in Unraid; I just have no idea how important it is that piece runs nor how to fix just that piece? Is there anything I can do or?

from immich.

Is it possibly because this Machine-Learning 1 is continuing to restart/bootloop?

Yes looks like your machine cannot run the Machine Learning container

So what can I do and how can I fix it? It's a old-ish Dell T610 server, FWIW

from immich.

Google says my Dell T610 doesn't have AVX support; so thats probably the reason my machine learning portion won't stay running: #300

Still not sure of the side effects of it not running or if that will cause me other issues. Also still unsure if docker compose plugin installed on Unraid if it is normal for it to say this at bottom of the docker tab:

Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/compose.manager/php/compose_manager_main.php on line 32

from immich.

Wouldn't it be better to use environment variables rather than using hardcoded urls ? This would allow for a single image and users to rename their containers as they wish. #647

from immich.

Every single time in Unraid I "docker compose down" then "Update Stack" MORE of my Docker Image is full. How can I "clean up" after this configuration? @Mattyfaz ? Anyone?

from immich.

https://docs.docker.com/engine/reference/commandline/system_prune/ should do the trick.

from immich.

https://docs.docker.com/engine/reference/commandline/system_prune/ should do the trick.

That will work in Unraid?

from immich.

https://docs.docker.com/engine/reference/commandline/system_prune/ should do the trick.

I ran it; eventually the unraid command window wen't away. Didn't see that it did anything; docker image still larger than expected?

Did it from ssh and it deleted 284mb :scratches head:

Yep, that was a bad idea; deleted containers I had that weren't running...a few I can't seem to get recreated right. DOH LOL

from immich.

Every single time in Unraid I "docker compose down" then "Update Stack" MORE of my Docker Image is full. How can I "clean up" after this configuration? @Mattyfaz ? Anyone?

Not sure sorry, I don’t think I have the same issue. I don’t click “docker compose down”, I just click “update stack” and it does it all for you.

from immich.

Every single time in Unraid I "docker compose down" then "Update Stack" MORE of my Docker Image is full. How can I "clean up" after this configuration? @Mattyfaz ? Anyone?

Install user scripts and create this as a script it will remove any dangling images

#!/bin/bash

remove_orphaned_images="yes" # select "yes" or "no" to remove any orphaned images
remove_unconnected_volumes="yes" # select "yes" or "no" to remove any unconnected volumes

#Do not make changes below this line#

echo "##################################################################################"
echo "Cleanup before starting (if requested in script)"
echo "##################################################################################"
echo
if [ "$remove_orphaned_images" == "yes" ] ; then
echo "Removing orphaned images..."
echo
docker image prune -af
else
echo "Not removing orphaned images (this can be set in script if you want to)"
fi
echo
echo "---------------------------------------------------------------------------------"
echo
if [ "$remove_unconnected_volumes" == "yes" ] ; then
echo "Removing unconnected docker volumes"
echo
docker volume prune -f
else
echo "Not removing unconnected docker volumes (this can be set in script if you want to)"
fi
echo
echo "##################################################################################"
echo "List of Image, Container and docker volume size."
echo "##################################################################################"
echo
#docker system df
docker system df --format 'There are \t {{.TotalCount}} \t {{.Type}} \t taking up ......{{.Size}}'
echo
echo "##################################################################################"
echo "List of containers showing size and virtual size"
echo "##################################################################################"
echo
echo "First size is the writable layers of the container (Virtual size is writable and read only layers)"
echo
docker container ls -a --format '{{.Size}} \t Is being taken up by ......... {{.Image}}'
echo
echo "##################################################################################"
echo "List of containers in size order"
echo "##################################################################################"
echo
docker image ls --format "{{.Repository}} {{.Size}}" |
awk '{if ($2~/GB/) print substr($2, 1, length($2)-2) *1000 "MB - " $1 ; else print $2 " - " $1 }' |
sed '/^0/d' |
sort -nr
echo
echo "##################################################################################"
echo "List of docker volumes, the container which they are connected to their size"
echo "##################################################################################"
echo
volumes=$(docker volume ls --format '{{.Name}}')
for volume in $volumes
do
name=(docker ps -a --filter volume="$volume" --format '{{.Names}}' | sed 's/^/ /')
size=(du -sh $(docker volume inspect --format '{{ .Mountpoint }}' $volume) | cut -f -1)
echo "ID" "$volume"
echo "This volume connected to" $name "has a size of" $size
echo ""
done
echo
echo "##################################################################################"
echo
echo "Done. Scroll up to view results"
exit

from immich.

Not sure if this has been discussed but there are other templates (at least 1 I know of) like TubeArchivist that require multiple dockers to work correctly that currently work with the Unraid Community Applications Plugin. Would this approach be more appropriate?

from immich.

It looks like TubeArchivist doesn't do anything special, they just have a template for each of their containers. At that point, docker-compose is still the better option :)

from immich.

I second the request, a single container for all elements would be great! I am managing Home Assistant addons, which is not compatible with stacks of several containers.

from immich.

Updated the immich docs to link to the community all-in-one github repository (#1301).

https://immich.app/docs/install/all-in-one

from immich.

Hello !
Do you want that I maintain the all-in-one Dockerfile on a separate repo or do you want me to create a pull request ?

from immich.

@martabal do you think you could provide an unraid template ? I don't really know how your image could land on Unraid's community software... maybe it should be requested idk.
I know that everything could be working right know as it is, but like many other Unraid user, I use this system because of the simplicity to install Docker containers with just few clicks.

Thanks a lot for your work, and THANKS a lot @alextran1502

from immich.

Would it be possible to migrate from the normal docker-compose setup to the all-in-one docker preserving the database and uploaded pictures?

Immich persists data to the upload folder and postgres data volumes. As long as you start up the AIO with those same two volumes it should be possible.

from immich.

@jrasm91 Is it ok that my image is published on unraid community apps ?

from immich.

I don't have a problem with it, just as long as it's clear it's not officially supported by Immich dev team and they should ask you question if they run into issues 😁

from immich.