Comments (5)
Hello @DuckY-Y, thanks for reporting this issue. I'll have a look and keep you updated about the progress.
from marqo.
@DuckY-Y Thanks for reporting this issue. We've established the root cause and are working to include a fix for this in a release this week. Please also note that this issue is limited to the built-in Vespa and does not affect Marqo upgrades when an external Vespa is used.
from marqo.
Alright this is quite interesting:
I've got no idea why I had to use sudo to remove it, nor why the volume breaks it.
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYT
ES=200000" marqoai/marqo:latest
docker: Error response from daemon: Conflict. The container name "/marqo" is already in use by container "0500beaa9a57f85f73938dceff24bd66422f0f5207f59bebbe6dc9d0f3a4c7ca". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
tesh@teshtesh:~/echo/marqo$ docker stop marqo
Error response from daemon: No such container: marqo
tesh@teshtesh:~/echo/marqo$ docker rm marqo
Error response from daemon: No such container: marqo
tesh@teshtesh:~/echo/marqo$ docker system prune -a
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all images without at least one container associated to them
- all build cache
Are you sure you want to continue? [y/N] y
Deleted Images:
untagged: marqoai/marqo:latest
untagged: marqoai/marqo@sha256:5d10b06ddb62665cc8ff57ffdedacecfad1660ac2a1a4235e87ab1c61a6c99ac
deleted: sha256:7c72260329e6a2e35b4ac15ea43a2f406d73790bb7608fe0a7bb58fbffa62972
deleted: sha256:8fe0b3bb1c3b7bc0036a8b1ad4bc0dd8f291d2db40596e7fd35e41f4def59f3f
deleted: sha256:9d23b2afd5cb9e915a2860f5eb3d9b1941464c60ee5135bc3c4e8eb618753a87
deleted: sha256:c5d97c85aade4862e21f3c87e2a9cb9b2ec919250a2c63a4093196ce0a79866d
deleted: sha256:d14dda496eabe09ed872509375143cd69e6e59146064ced681249a4f9706382a
deleted: sha256:015e8bef8822807d13f9ff1818b7c3fe4abda56cb6e64a78ba126ccd5d1624cf
deleted: sha256:bc85585b10cbc490f0ca3517cd54047352f23631d021339e63e161292b48c24f
deleted: sha256:e0989deaabec72ee6cfb5cae20a048a75cdd1de431ad2fcdb2259e19cb2e443b
deleted: sha256:841d1cdd5c74335c90915a6c86949fb4ee4b62fad7493b46a271d0e0ad3438b7
deleted: sha256:f7f9bcc29f0efbf484bcdd47a2c5df767c0f62e38d18e2211f3d7b3654b5fb0b
deleted: sha256:c277aebc6b7e9f5bb994b6ae77adf309c8b30b9045dfedb4a5565bfefe940691
deleted: sha256:6c34b849aeaeca6858f8afb0cb1b12fd1f90c9abd7f362db7593a2d44e0d940f
deleted: sha256:57e5642d63e332a0c106f2ce09691507e650c14a59bc581d0e36e942f929aebc
deleted: sha256:5054496fea8c678afcb66adb1a46947cf243901a6fc523bb45323ce716ed3d50
deleted: sha256:97bf8faa5fe984fa17fe7bd74f26bb182504bb7a24e411dc957efcebe7c7a645
deleted: sha256:11743e4defd3a6fc2aa8eda0856b5fab270fbc95fff3b91a507b896d8a8c52e6
deleted: sha256:7bd4e6645b5eb8eee49206422d76b0e2cdb01e815c3d1b82fd792c4f5eb10a6b
deleted: sha256:42700eeecb16d2f738af93b12d7e435e3fc7c834d464cf07d91d127a520c2e49
deleted: sha256:3e1520d4703898e2fc04106c2cbe13b9f83b448c58505fbb0aace5790b80a4d2
deleted: sha256:0945b96ec308a61b7293b23dd5016cfccb6463f64d2df670936e303a3401d478
deleted: sha256:a130314df5ced9df85970d096ca4d24d37e2d3c7ee6768b188326632aaef308f
Total reclaimed space: 9.703GB
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYTES=200000" marqoai/marqo:latest
docker: Error response from daemon: Conflict. The container name "/marqo" is already in use by container "0500beaa9a57f85f73938dceff24bd66422f0f5207f59bebbe6dc9d0f3a4c7ca". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
tesh@teshtesh:~/echo/marqo$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
62d55c0007a6 ghcr.io/open-webui/open-webui:main "bash start.sh" 3 days ago Up 3 days (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
ba0316eaef61 ollama/ollama:latest "/bin/ollama serve" 3 days ago Up 3 days 0.0.0.0:11434->11434/tcp, :::11434->11434/tcp ollama
ac44cf4e2db8 penpotapp/frontend:latest "/bin/bash /entrypoi…" 4 weeks ago Up 5 days 0.0.0.0:9001->80/tcp, :::9001->80/tcp taiga-docker-penpot-frontend-1
21221a13e849 penpotapp/backend:latest "/bin/bash run.sh" 4 weeks ago Up 5 days
taiga-docker-penpot-backend-1
1666c63232d9 sj26/mailcatcher:latest "mailcatcher --foreg…" 4 weeks ago Up 5 days 1025/tcp, 0.0.0.0:1080->1080/tcp, :::1080->1080/tcp taiga-docker-penpot-mailcatch-1
7fa9a1df8d83 postgres:15 "docker-entrypoint.s…" 4 weeks ago Up 5 days 5432/tcp taiga-docker-penpot-postgres-1
0d115dc00139 redis:7 "docker-entrypoint.s…" 4 weeks ago Up 5 days 6379/tcp taiga-docker-penpot-redis-1
77e3b4d9e251 penpotapp/exporter:latest "node app.js" 4 weeks ago Up 5 days
taiga-docker-penpot-exporter-1
2a9f10c43949 mongo-express "/sbin/tini -- /dock…" 5 weeks ago Up 5 days 0.0.0.0:8081->8081/tcp, :::8081->8081/tcp monger-mongo-express-1
608a840af444 mongo "docker-entrypoint.s…" 5 weeks ago Up 5 days 0.0.0.0:27017->27017/tcp, :::27017->27017/tcp monger-mongo-1
tesh@teshtesh:~/echo/marqo$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
ghcr.io/open-webui/open-webui main 95fa326eca5f 4 days ago 3.56GB
ollama/ollama latest 018439a5a146 5 days ago 465MB
postgres 15 e3e85a8d0b0f 5 weeks ago 440MB
mongo latest ff65a94ec485 7 weeks ago 795MB
mongo-express latest 0eec10060d2f 7 weeks ago 182MB
penpotapp/exporter latest 32028df821b9 2 months ago 1.23GB
penpotapp/frontend latest 4f343eac55bf 2 months ago 268MB
penpotapp/backend latest b05ca261add7 2 months ago 1.1GB
redis 7 9509c4dd19fb 2 months ago 116MB
sj26/mailcatcher latest 7ab5eb1bdf54 10 months ago 95.7MB
tesh@teshtesh:~/echo/marqo$ sudo docker rm marqo
Error response from daemon: cannot remove container "/marqo": container is running: stop the container before removing or force remove
tesh@teshtesh:~/echo/marqo$ sudo docker stop marqo
marqo
tesh@teshtesh:~/echo/marqo$ sudo docker rm marqo
marqo
tesh@teshtesh:~/echo/marqo$ sudo docker run --restart unless-stopped --name marqo -it -p 8882:8882 -e "MARQO_MAX_DOC_BYTES=200000" marqoai/marqo:latest
External vector store not configured. Using local vector store
Waiting for vector store to start
Marqo did not find an existing vector store. Setting up vector store...
Vector store is available. Vector store setup complete
Starting Marqo throttling
Called Marqo throttling start command
Marqo throttling is now running
/usr/local/lib/python3.8/site-packages/transformers/generation_utils.py:24: FutureWarning: Importing `GenerationMixin` from `src/transformers/generation_utils.py` is deprecated and will be removed in Transformers v5. Import as `from transformers import GenerationMixin` instead.
warnings.warn(
INFO:ModelsForStartup:pre-loading ['hf/e5-base-v2', 'open_clip/ViT-B-32/laion2b_s34b_b79k'] onto devices=['cpu']
INFO:marqo.tensor_search.index_meta_cache:Starting index cache refresh thread
###########################################################
###########################################################
###### STARTING DOWNLOAD OF MARQO ARTEFACTS################
###########################################################
###########################################################
INFO:CUDA device summary:Found devices [{'id': -1, 'name': ['cpu']}]
INFO:SetBestAvailableDevice:Best available device set to: cpu
loading for: model_name=hf/e5-base-v2 and properties={'name': 'intfloat/e5-base-v2', 'dimensions': 768, 'tokens': 512, 'type': 'hf', 'model_size': 0.438, 'text_query_prefix': 'query: ', 'text_chunk_prefix': 'passage: ', 'notes': ''}
INFO:marqo.tensor_search.index_meta_cache:Last index cache refresh at 1718612120.0507038
config.json: 100%|█████████████████████████████████████████████████████████████████████| 650/650 [00:00<00:00, 54.2kB/s]
model.safetensors: 100%|█████████████████████████████████████████████████████████████| 438M/438M [00:25<00:00, 17.3MB/s]
tokenizer_config.json: 100%|███████████████████████████████████████████████████████████| 314/314 [00:00<00:00, 44.3kB/s]
vocab.txt: 100%|█████████████████████████████████████████████████████████████████████| 232k/232k [00:00<00:00, 1.19MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████| 711k/711k [00:00<00:00, 5.86MB/s]
special_tokens_map.json: 100%|█████████████████████████████████████████████████████████| 125/125 [00:00<00:00, 56.6kB/s]
INFO:marqo.s2_inference.s2_inference:loaded hf/e5-base-v2 on device cpu with normalization=True at time=2024-06-17 08:15:19.813664.
INFO:ModelsForStartup:hf/e5-base-v2 cpu run succesfully!
loading for: model_name=open_clip/ViT-B-32/laion2b_s34b_b79k and properties={'name': 'open_clip/ViT-B-32/laion2b_s34b_b79k', 'dimensions': 512, 'note': 'open_clip models', 'type': 'open_clip', 'pretrained': 'laion2b_s34b_b79k'}
open_clip_pytorch_model.bin: 100%|███████████████████████████████████████████████████| 605M/605M [00:23<00:00, 25.3MB/s]
INFO:marqo.s2_inference.s2_inference:loaded open_clip/ViT-B-32/laion2b_s34b_b79k on device cpu with normalization=True at time=2024-06-17 08:15:49.375963.
INFO:ModelsForStartup:open_clip/ViT-B-32/laion2b_s34b_b79k cpu run succesfully!
INFO:ModelsForStartup:0.027287817001342772 for hf/e5-base-v2 and cpu
INFO:ModelsForStartup:0.04055120944976807 for open_clip/ViT-B-32/laion2b_s34b_b79k and cpu
INFO:ModelsForStartup:completed loading models
INFO:marqo.connections:Took 1.476ms to connect to redis and load scripts.
INFO:CachePatchModels:completed prewarming patch models
###########################################################
###########################################################
###### !!COMPLETED SUCCESSFULLY!!! ################
###########################################################
###########################################################
Version: 2.8.1
__ __ ___ _ __ ___ ___ ___ ___ ______ ___ ___ ___ ____ ____ ___ ___ __
| |__| | / _]| | / ] / \ | | | / _] | | / \ | | | / || \ / \ / \ | |
| | | | / [_ | | / / | || _ _ | / [_ | || | | _ _ || o || D ) || || |
| | | || _]| |___ / / | O || \_/ || _] |_| |_|| O | | \_/ || || /| Q || O ||__|
| ` ' || [_ | / \_ | || | || [_ | | | | | | || _ || \| || | __
\ / | || \ || || | || | | | | | | | || | || . \ || || |
\_/\_/ |_____||_____|\____| \___/ |___|___||_____| |__| \___/ |___|___||__|__||__|\_|\__,_| \___/ |__|
_____ _ __ _
|_ _|__ _ __ ___ ___ _ __ ___ ___ __ _ _ __ ___| |__ / _| ___ _ __ | |__ _ _ _ __ ___ __ _ _ __ ___
| |/ _ \ '_ \/ __|/ _ \| '__| / __|/ _ \/ _` | '__/ __| '_ \ | |_ / _ \| '__| | '_ \| | | | '_ ` _ \ / _` | '_ \/ __|
| | __/ | | \__ \ (_) | | \__ \ __/ (_| | | | (__| | | | | _| (_) | | | | | | |_| | | | | | | (_| | | | \__ \
|_|\___|_| |_|___/\___/|_| |___/\___|\__,_|_| \___|_| |_| |_| \___/|_| |_| |_|\__,_|_| |_| |_|\__,_|_| |_|___/
INFO: Started server process [2052]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8882 (Press CTRL+C to quit)
INFO:marqo.tensor_search.index_meta_cache:Last index cache refresh at 1718612180.5601022
from marqo.
Yep confirmed through testing that the volume I created in like 2.3 or 2.5 (can't remember) is breaking the vector store.
from marqo.
Related Issues (20)
- [ENHANCEMENT] Disk persistent storage configuration HOT 1
- Usage on Google Cloud Platform
- [ENHANCEMENT] Marqo on Azure HOT 1
- ONNX Support - CPU HOT 1
- [BUG] Cannot pull docker image on latest 2.1.0 HOT 1
- Marqo Integration into cacheGPT HOT 1
- Error Status: 400 after using add_documents() to add the images from docker to index
- [ENHANCEMENT] Wildcard support HOT 3
- Marqo encountered an unexpected internal error, status_code: 500[BUG] HOT 1
- Support OpenAPI spec generation [ENHANCEMENT]
- [ENHANCEMENT] Allow Pydantic > 2 HOT 3
- Unable to delete index [BUG] HOT 5
- [ENHANCEMENT] Nicer error message if not enough memory HOT 1
- [ENHANCEMENT] Highlighting Exact Matches HOT 2
- [BUG] This is a test HOT 1
- [BUG] HOT 2
- [BUG] Error ingesting simple wiki using simple_wiki_demo script HOT 1
- [BUG] Performing a lexical search on a structured index always results in a _score of 0.0 HOT 2
- Can't find any configuration for OpenAI Embeddings HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from marqo.