Hi,
I have downloaded the repo but facing problem while setting up the back-end as per the instructions provided in the readme or the YouTube tutorial. Please help me for the same.

Thanks

While running a application getting attributeError

After set the value of _async_session getting below error

Hi Sourab,
For some reason the program is unable to establish connection. I tried to change the port, but that didnt work. I can see the document in the bucket (https://investors.mongodb.com/static-files/10f57c7a-a481-4663-bbc5-1cc79e36b147). Curious what am I missing? Yesterday it worked just fine. Thanks for the help.

Hi @sourabhdesai , you have made a great platform. Can I deploy it on EC2?

Hello, when I run the chat test, I have the following message:

(Chat🦙) message Hi
====== Final Message ======
Sorry, I either wasn't able to understand your question or I don't have an answer for it.
(Chat🦙)

and in the backend terminal I have the following:

INFO: 127.0.0.1:33006 - "GET /api/conversation/a2d5e482-a251-4d36-8c63-66bde3418919/message?user_message=Hi HTTP/1.1" 200 OK
STARTING TURN 1
---------------

Tried sending SubProcess event (source=MessageSubProcessSourceEnum.AGENT_STEP) after channel was closed
Traceback (most recent call last):
File "/workspaces/sec-insights/backend/app/chat/messaging.py", line 106, in async_on_event
await self._send_chan.send(
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/anyio/streams/memory.py", line 213, in send
self.send_nowait(item)
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/anyio/streams/memory.py", line 195, in send_nowait
raise ClosedResourceError
anyio.ClosedResourceError

Thank you

While running some of the steps mentioned in the README, i encountered module not found error.

Should this get fixed?

PS: I updated my PYTHONPATH to fix it, but thought of mentioning it for others.

I saw the youtube video that allowed for custom data but is possible to still use the UI with custom data. I want to legal docs instead of sec filings

Hello

I am finding how to use my own pdf files for analysis

but I can't find it

This template currently focuses solely on using it for conversations with documents in PDF format. However, since the majority of enterprise data is structured, could we also consider supporting a text-to-SQL use case? In this scenario, rather than uploading two separate documents, users could opt to upload only one document and select specific tables they wish to interact with for data analysis. They could then use the document to augment the knowledge, facilitating the creation of more robust completions. If someone want to use this how we can approach this idea? I have tried it but seems like am not able to reach to the actual goal. @sourabhdesai , @antimatter15 @kikirizki @eltociear @siddarthreddygsr do you have any idea on how to approach this?

I followed the instructions on the latest PR merge, but still bumping into a few issues with codespaces. The first make run worked, but then on following further instructions, it starts throwing issues. The only change I have made is the the .env adding OPENAI_API_KEY

make seed_db_local

 ✔ Container localstack_main  Running                                                                                                                  0.0s 
docker compose start localstack
echo "Waiting for localstack to start..."
Waiting for localstack to start...
sleep 3
# Check that S3_ASSET_BUCKET_NAME is set
if [ -z llama-app-web-assets-local ]; then \
        echo "S3_ASSET_BUCKET_NAME is not set. Please set it and try again."; \
        exit 1; \
fi
awslocal s3 mb s3://llama-app-web-assets-local
make[1]: awslocal: No such file or directory
make[1]: *** [Makefile:44: setup_localstack] Error 127
make[1]: Leaving directory '/workspaces/sec-insights/backend'
make: *** [Makefile:81: seed_db_local] Error 2

make refresh_db

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/__main__.py", line 4, in <module>
    main(prog="alembic")
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 630, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 624, in main
    self.run_cmd(cfg, options)
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 601, in run_cmd
    fn(
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/command.py", line 385, in upgrade
    script.run_env()
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/workspaces/sec-insights/backend/alembic/env.py", line 98, in <module>
    run_migrations_online()
  File "/workspaces/sec-insights/backend/alembic/env.py", line 92, in run_migrations_online
    asyncio.run(run_async_migrations())
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/workspaces/sec-insights/backend/alembic/env.py", line 83, in run_async_migrations
    async with connectable.connect() as connection:
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/base.py", line 127, in __aenter__
    return await self.start(is_ctxmanager=True)  # type: ignore
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/engine.py", line 268, in start
    await greenlet_spawn(self.sync_engine.connect)
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn
    result = context.throw(*sys.exc_info())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3264, in connect
    return self._connection_cls(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
    self._dbapi_connection = engine.raw_connection()
                             ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3288, in raw_connection
    return self.pool.connect()
           ^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect
    return _ConnectionFairy._checkout(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1267, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
    rec = pool._do_get()
          ^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 284, in _do_get
    return self._create_connection()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
    return _ConnectionRecord(self)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
    self.__connect()
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
    with util.safe_reraise():
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 637, in connect
    return dialect.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 615, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 917, in connect
    await_only(creator_fn(*arg, **kw)),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connection.py", line 2092, in connect
    return await connect_utils._connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connect_utils.py", line 895, in _connect
    raise last_error
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connect_utils.py", line 881, in _connect
    return await _connect_addr(
           ^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connect_utils.py", line 773, in _connect_addr
    return await __connect_addr(params, timeout, True, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connect_utils.py", line 825, in __connect_addr
    tr, pr = await compat.wait_for(connector, timeout=timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/compat.py", line 56, in wait_for
    return await asyncio.wait_for(fut, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/tasks.py", line 479, in wait_for
    return fut.result()
           ^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/asyncpg/connect_utils.py", line 692, in _create_ssl_connection
    do_ssl_upgrade = await pr.on_data
                     ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/asyncio/selector_events.py", line 995, in _read_ready__data_received
    data = self._sock.recv(self.max_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ConnectionResetError: [Errno 104] Connection reset by peer
make[2]: *** [Makefile:15: migrate] Error 1
make[2]: Leaving directory '/workspaces/sec-insights/backend'
make[1]: *** [Makefile:29: confirmed_refresh_db] Error 2
make[1]: Leaving directory '/workspaces/sec-insights/backend'
make: *** [Makefile:21: refresh_db] Error 2

Then going back to try make run again

2023-09-30 08:58:36,454 [MainThread  ] [DEBUG]  Skipping migrations
INFO:     Will watch for changes in these directories: ['/workspaces/sec-insights/backend']
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [36511] using StatReload
INFO:     Started server process [37826]
INFO:     Waiting for application startup.
Connected to the database on attempt 1.
2023-09-30 08:58:39,096 INFO sqlalchemy.engine.Engine select pg_catalog.version()
2023-09-30 08:58:39,097 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-09-30 08:58:39,097 INFO sqlalchemy.engine.Engine select current_schema()
2023-09-30 08:58:39,097 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-09-30 08:58:39,098 INFO sqlalchemy.engine.Engine show standard_conforming_strings
2023-09-30 08:58:39,098 INFO sqlalchemy.engine.Engine [raw sql] {}
2023-09-30 08:58:39,098 INFO sqlalchemy.engine.Engine BEGIN (implicit)
2023-09-30 08:58:39,101 INFO sqlalchemy.engine.Engine SELECT pg_catalog.pg_class.relname 
FROM pg_catalog.pg_class JOIN pg_catalog.pg_namespace ON pg_catalog.pg_namespace.oid = pg_catalog.pg_class.relnamespace 
WHERE pg_catalog.pg_class.relname = %(table_name)s AND pg_catalog.pg_class.relkind = ANY (ARRAY[%(param_1)s, %(param_2)s, %(param_3)s, %(param_4)s, %(param_5)s]) AND pg_catalog.pg_table_is_visible(pg_catalog.pg_class.oid) AND pg_catalog.pg_namespace.nspname != %(nspname_1)s
2023-09-30 08:58:39,101 INFO sqlalchemy.engine.Engine [generated in 0.00016s] {'table_name': 'alembic_version', 'param_1': 'r', 'param_2': 'p', 'param_3': 'f', 'param_4': 'v', 'param_5': 'm', 'nspname_1': 'pg_catalog'}
2023-09-30 08:58:39,103 INFO sqlalchemy.engine.Engine COMMIT
ERROR:    Traceback (most recent call last):
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
    async with self.lifespan_context(app) as maybe_state:
  File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/workspaces/sec-insights/backend/app/main.py", line 77, in lifespan
    raise Exception(
Exception: Database is not up to date. Please run `poetry run alembic upgrade head`

ERROR:    Application startup failed. Exiting.

Not sure what the missing step is. I presume to keep the first make run terminal open per this line. Or do I rather ctrl+c to exit and run everything in that terminal? i think I've probably tried both. In any case I get stuck with above in both scenarios once I've got the Exception: Database is not up to date. Please run poetry run alembic upgrade head error

I'm encountering this error in both CodeSpace and on my local machine.

File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/aiobotocore/endpoint.py", line 285, in _send
    return await self.http_session.send(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/aiobotocore/httpsession.py", line 253, in send
    raise EndpointConnectionError(endpoint_url=request.url, error=e)
botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://localhost:4566/llama-app-backend-local"

In The Given project is there any specific reason for using custom chat engine by building a chat engine out of query engine and AnyIO memory stream when there is in built chat engine

When I try to run the cron service seeing the following errors.

Oct 5 03:45:33 PM  - Downloading filing for 1682852 10-Q
Oct 5 03:45:33 PM  Error downloading filing for symbol=1682852 & file_type=10-Q: 500 Server Error: Internal Server Error for url: https://efts.sec.gov/LATEST/search-index
Oct 5 03:45:33 PM  - Downloading filing for 200406 10-K
Oct 5 03:45:33 PM  Error downloading filing for symbol=200406 & file_type=10-K: 500 Server Error: Internal Server Error for url: https://efts.sec.gov/LATEST/search-index

Getting the error below when running the docker compose up:

backend-llama-app-fastapi-1  | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
backend-llama-app-fastapi-1  |  Is the server running on host "127.0.0.1" and accepting
backend-llama-app-fastapi-1  |  TCP/IP connections on port 5432?
backend-llama-app-fastapi-1  | 
backend-llama-app-fastapi-1  | (Background on this error at: https://sqlalche.me/e/20/e3q8)
backend-llama-app-fastapi-1  | 
backend-llama-app-fastapi-1  | ERROR:    Application startup failed. Exiting.

This was part of step 6 in the backend/ReadMe file. Am I missing something?

Not a defect, but an ask. The backend components are tightly coupled together and assume everything runs locally. Any attempt to allow for deploying the backend to AWS or GCP or Azure and allowing for respective cloud native services to be used?

I am finding that the app is not returning answers when the information is clearly in the document. Using JNJ 10-K for 2023.
Example: What was the ending accounts receivable balance
Generated Sub Query #1
What is the ending accounts receivable balance for Johnson & Johnson (JNJ) in the 10-K filing for 2022?:
The ending accounts receivable balance for Johnson & Johnson (JNJ) in the 10-K filing for 2022 is not provided in the available data.
Final answer: I apologize, but it seems that I don't have access to the ending accounts receivable balance for Johnson & Johnson (JNJ) in the selected SEC documents. Is there anything else I can assist you with?

Very hopeful that this project can truly answer at least simple questions over SEC filings.

The application seems to be working fine and can do the right Q&As like below. However the PDF control doesn't seem to load, and the error message from the console is:

instrument.js:142  Mixed Content: The page at 'https://special-doodle-q7qp4969w9grc9jpg-3000.app.github.dev/conversation/cbcef1a5-bbd4-4582-9d1f-a7012979e874' was loaded over HTTPS, but requested an insecure resource 'http://llama-app-web-assets-local.s3-website.localhost.localstack.cloud:4566/sec-edgar-filings/0001018724/10-K/0001018724-23-000004/filing-details.pdf'. This request has been blocked; the content must be served over HTTPS.

any ideas?

poetry run python -m alembic upgrade head
/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/bin/python: No module named alembic.main; 'alembic' is a package and cannot be directly executed
make: *** [Makefile:15: migrate] Error 1

Hello @sourabhdesai, First off, thank you for your contribution to the community. This project is truly awesome.

I was exploring the live UI and noticed that it features filings for a select group of companies. I want to know if it is possible to locally deploy an app that can answer queries for other SEC filings.

I understand from looking at the code that the backend might be equipped to handle this, but I'm unclear on whether the front end supports this feature or not. If it's not already implemented, do you think it's feasible to run the app locally and tailor it for the filings of different companies?

Would appreciate your insights. Looking forward to hearing from you!

Encountering this error when running make seed_db_local command local environment.

awslocal s3api put-bucket-cors --bucket llama-app-web-assets-local --cors-configuration file://./localstack-cors-config.json
echo "LocalStack S3 bucket website is ready. Open http://llama-app-web-assets-local.s3-website.localhost.localstack.cloud:4566 in your browser to verify."
LocalStack S3 bucket website is ready. Open http://llama-app-web-assets-local.s3-website.localhost.localstack.cloud:4566 in your browser to verify.
make[1]: Leaving directory '/home/jay/Documents/sec-insights-3/sec-insights/backend'
python scripts/seed_db.py --ciks '["0001018724", "1326801"]'  --filing_types '["10-K"]'
Traceback (most recent call last):
  File "/home/jay/Documents/sec-insights-3/sec-insights/backend/scripts/seed_db.py", line 7, in <module>
    from app.core.config import settings
ModuleNotFoundError: No module named 'app'
make: *** [Makefile:86: seed_db_local] Error 1

I'm encountering an issue when trying to retrieve information from a document. The error message reads 'Failed to load index with ID {index_id}.

INFO:     Application startup complete.
INFO:     127.0.0.1:48006 - "POST /api/conversation/ HTTP/1.1" 200 OK
INFO:     127.0.0.1:33524 - "GET /api/conversation/1eecef0b-17af-48e8-9d5f-95fcbabc5f58 HTTP/1.1" 200 OK
INFO:     127.0.0.1:38606 - "GET /api/conversation/1eecef0b-17af-48e8-9d5f-95fcbabc5f58/message?user_message=what%20is%20this%20document%3F HTTP/1.1" 200 OK
Failed to load indices from storage. Creating new indices. If you're running the seed_db script, this is normal and expected.
Error in message publisher
Traceback (most recent call last):
  File "/home/jay/Documents/Pixl/AI/blue_pond/sec-insights-4/sec-insights/backend/app/chat/engine.py", line 151, in build_doc_id_to_index_map
    indices = load_indices_from_storage(
              ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/loading.py", line 71, in load_indices_from_storage
    raise ValueError(f"Failed to load index with ID {index_id}")
ValueError: Failed to load index with ID e06dc2c1-bfa6-41d6-be82-cdbfa64d4d31

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/tenacity/__init__.py", line 382, in __call__
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/embeddings/openai.py", line 172, in get_embeddings
    data = openai.Embedding.create(input=list_of_text, model=engine, **kwargs).data
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/openai/api_resources/embedding.py", line 33, in create
    response = super().create(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 149, in create
    ) = cls.__prepare_create_request(
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/openai/api_resources/abstract/engine_api_resource.py", line 83, in __prepare_create_request
    raise error.InvalidRequestError(
openai.error.InvalidRequestError: Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.embedding.Embedding'>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jay/Documents/Pixl/AI/blue_pond/sec-insights-4/sec-insights/backend/app/api/endpoints/conversation.py", line 149, in event_publisher
    await task
  File "/usr/lib64/python3.11/asyncio/futures.py", line 290, in __await__
    return self.result()  # May raise too.
           ^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/usr/lib64/python3.11/asyncio/tasks.py", line 267, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "/home/jay/Documents/Pixl/AI/blue_pond/sec-insights-4/sec-insights/backend/app/chat/messaging.py", line 131, in handle_chat_message
    chat_engine = await get_chat_engine(
                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/Documents/Pixl/AI/blue_pond/sec-insights-4/sec-insights/backend/app/chat/engine.py", line 270, in get_chat_engine
    doc_id_to_index = await build_doc_id_to_index_map(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/Documents/Pixl/AI/blue_pond/sec-insights-4/sec-insights/backend/app/chat/engine.py", line 170, in build_doc_id_to_index_map
    index = VectorStoreIndex.from_documents(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/base.py", line 102, in from_documents
    return cls(
           ^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/vector_store/base.py", line 46, in __init__
    super().__init__(
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/base.py", line 71, in __init__
    index_struct = self.build_index_from_nodes(nodes)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/vector_store/base.py", line 241, in build_index_from_nodes
    return self._build_index_from_nodes(nodes)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/vector_store/base.py", line 229, in _build_index_from_nodes
    self._add_nodes_to_index(
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/vector_store/base.py", line 201, in _add_nodes_to_index
    embedding_results = self._get_node_embedding_results(nodes, show_progress)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/indices/vector_store/base.py", line 111, in _get_node_embedding_results
    ) = self._service_context.embed_model.get_queued_text_embeddings(show_progress)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/embeddings/base.py", line 218, in get_queued_text_embeddings
    embeddings = self._get_text_embeddings(cur_batch_texts)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/llama_index/embeddings/openai.py", line 323, in _get_text_embeddings
    return get_embeddings(
           ^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/tenacity/__init__.py", line 289, in wrapped_f
    return self(f, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/tenacity/__init__.py", line 379, in __call__
    do = self.iter(retry_state=retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jay/.cache/pypoetry/virtualenvs/llama-app-backend-ObpdDQgr-py3.11/lib/python3.11/site-packages/tenacity/__init__.py", line 326, in iter
    raise retry_exc from fut.exception()
tenacity.RetryError: RetryError[<Future at 0x7f03c9eb0e90 state=finished raised InvalidRequestError>]

hey,

I watched the YouTube video https://www.youtube.com/watch?v=2O52Tfj79T4 and tried replicating the entire setup locally.
When I try to download sec filings using sec-edgar-downloader and then convert them to PDF - I don't see images being populated. HTML or. PDF files.

I'm guessing the downloader is picking up the name of the image asset and not appending or downloading the image assets.

Can someone help me understand this better? or how can I download the sec-filings including images?

@sourabhdesai

I'm following the instructions and the video step by step but I get this error even after the recent proposed fix.

git log output

commit 3e5d162e27687bda9d223851ff6c2b6e2b02ac69 (HEAD -> main, origin/main, origin/HEAD)
Author: Sourabh Desai <[email protected]>
Date:   Mon Sep 11 14:44:07 2023 -0700

    Bugfix/issue 13 (#23)
    
    * fixed typo in README
    
    * fixed docker compose bug #13
    
    * made requested changes

error log

backend-llama-app-fastapi-1  | Attempt 27: Database is not yet available. Error: [Errno 111] Connect call failed ('127.0.0.1', 5432)
backend-llama-app-fastapi-1  | Attempt 28: Database is not yet available. Error: [Errno 111] Connect call failed ('127.0.0.1', 5432)
backend-llama-app-fastapi-1  | Attempt 29: Database is not yet available. Error: [Errno 111] Connect call failed ('127.0.0.1', 5432)
backend-llama-app-fastapi-1  | Attempt 30: Database is not yet available. Error: [Errno 111] Connect call failed ('127.0.0.1', 5432)
backend-llama-app-fastapi-1  | ERROR:    Traceback (most recent call last):
backend-llama-app-fastapi-1  |   File "/app/app/db/wait_for_db.py", line 10, in check_database_connection
backend-llama-app-fastapi-1  |     await db.execute(text("SELECT 1"))
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/ext/asyncio/session.py", line 439, in execute
backend-llama-app-fastapi-1  |     result = await greenlet_spawn(
backend-llama-app-fastapi-1  |              ^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 190, in greenlet_spawn
backend-llama-app-fastapi-1  |     result = context.throw(*sys.exc_info())
backend-llama-app-fastapi-1  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2246, in execute

backend-llama-app-fastapi-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
backend-llama-app-fastapi-1  |     return await anext(self.gen)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/app/app/main.py", line 68, in lifespan
backend-llama-app-fastapi-1  |     await check_database_connection()
backend-llama-app-fastapi-1  |   File "/app/app/db/wait_for_db.py", line 16, in check_database_connection
backend-llama-app-fastapi-1  |     raise ValueError(
backend-llama-app-fastapi-1  | ValueError: Couldn't connect to database after 30 attempts.
backend-llama-app-fastapi-1  | 
backend-llama-app-fastapi-1  | 
backend-llama-app-fastapi-1  | ERROR:    Application startup failed. Exiting.
backend-db-1                 | 2023-09-11 22:31:10.634 UTC [27] LOG:  checkpoint starting: time
backend-db-1                 | 2023-09-11 22:31:10.748 UTC [27] LOG:  checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s)

howdy - just watching the demo tutorial on YT (thanks!)

just wanted to mention - in case it could be helpful i think you could automate some of the dev setup steps in the devcontainer so that users would not need to manually run stuff like pyenv, poetry etc type stuff once the container is built.

you can use the post create command option in devcontainers to automate as much as possible.

here is an example of what i mean on another project i helped on a while back:

https://github.com/LAION-AI/Open-Assistant/blob/main/.devcontainer/backend-dev/post_create_command.sh

the .devcontainer for the backend-dev then just runs that script: https://github.com/LAION-AI/Open-Assistant/blob/709bb99d3e03dc62abc11892500352cb2da31a29/.devcontainer/backend-dev/devcontainer.json#L9

Steps to reproduce:

1. run poetry shell
2. run poetry lock
3. confirm poetry.lock file has updates. For example, latest for me is:

[[package]]
name = "llama-index"
version = "0.8.42"
description = "Interface between LLMs and your data"
optional = false
python-versions = "*"
files = [
    {file = "llama_index-0.8.42-py3-none-any.whl", hash = "sha256:08720554ceaef169e1a151f4fd982c0555703e85b23bf0dbf290d9dc0605655d"},
    {file = "llama_index-0.8.42.tar.gz", hash = "sha256:14cca43188d9afd058d74c5574e745dc937e724a20576a9ef81967b9f2659b03"},
]

4. run poetry install and steps 5-7 in /backend README
5. start a chat, ask a question, result is infinite loop of the same question and this server log error:

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 552, in _prepare_and_execute
    self._rows = await prepared_stmt.fetch(*parameters)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/asyncpg/prepared_stmt.py", line 176, in fetch
    data = await self.__bind_execute(args, 0, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/asyncpg/prepared_stmt.py", line 241, in __bind_execute
    data, status, _ = await self.__do_execute(
                      ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/asyncpg/prepared_stmt.py", line 230, in __do_execute
    return await executor(protocol)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "asyncpg/protocol/protocol.pyx", line 201, in bind_execute
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 287, in __await__
    yield self  # This tells Task to wait for completion.
    ^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 339, in __wakeup
    future.result()
  File "/opt/homebrew/Cellar/[email protected]/3.11.5/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
asyncpg.exceptions.InvalidTextRepresentationError: invalid input value for enum "MessageSubProcessSourceEnum": "AGENT_STEP"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2112, in _exec_insertmany_context
    dialect.do_execute(
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 922, in do_execute
    cursor.execute(statement, parameters)
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 586, in execute
    self._adapt_connection.await_(
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 125, in await_only
    return current.driver.switch(awaitable)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/util/_concurrency_py3k.py", line 185, in greenlet_spawn
    value = await result
            ^^^^^^^^^^^^
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 564, in _prepare_and_execute
    self._handle_exception(error)
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 515, in _handle_exception
    self._adapt_connection._handle_exception(error)
  File "/Users/joshuasabol/.local/share/virtualenvs/HealthKey-ZlgcgJrM/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/asyncpg.py", line 803, in _handle_exception
    raise translated_error from error
sqlalchemy.dialects.postgresql.asyncpg.AsyncAdapt_asyncpg_dbapi.Error: <class 'asyncpg.exceptions.InvalidTextRepresentationError'>: invalid input value for enum "MessageSubProcessSourceEnum": "AGENT_STEP"

The above exception was the direct cause of the following exception:
...
...
...

Steps to Resolt

Simple solution is to revert back to version = "0.8.13".

Request

Can we update SEC-Insights to support the latest llama-index version?

I ran the entire repository in Github CodeSpaces, and everything works perfectly. However, when opening a Conversation the frontend gives the error " https://llama-app-web-assets-local.s3-website.localhost.localstack.cloud:4566/sec-edgar-filings/0001018724/10-K/0001018724-21-000004/filing-details.pdf net::ERR_CONNECTION_REFUSED. It seems the routing doesn't work properly?

In addition, I had to change the NEXT_PUBLIC_BACKEND_URL=http://localhost:8000, to the actual URL of the Github Codespace. Not sure if that's required but that's the only way it worked for me.

Anyone else got this problem?

I am trying to duplicate the SEC-INSIGHTS app in codespace. I seem to be successful in getting both front end and backed to run but am not able to open in browser. I get this error message after running 'make run' in backend

2024-01-23 23:10:43,202 INFO sqlalchemy.engine.Engine COMMIT
[nltk_data] Downloading package punkt to /tmp/llama_index...
[nltk_data] Unzipping tokenizers/punkt.zip.
INFO: Application startup complete.
INFO: 10.240.4.209:0 - "GET /?vscodeBrowserReqId=1706051469969 HTTP/1.1" 404 Not Found
INFO: 10.240.4.209:0 - "GET /?vscodeBrowserReqId=1706051520714 HTTP/1.1" 404 Not Found

Hi,

I'm getting the error below when running make seed_db_local. I've destroyed and recreated the db multiple times (including make migrate) and I still get the two errors below.

I cannot find this object inside the db and I'm not sure if it is a pgvector object or something else...

Any advise to fix this?

thanks

Mateus

Failed to load indices from storage. Creating new indices.
Traceback (most recent call last):
File "/Users/mateusfantini/dev/sec-insights/backend/app/chat/engine.py", line 151, in build_doc_id_to_index_map
indices = load_indices_from_storage(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/mateusfantini/Library/Caches/pypoetry/virtualenvs/llama-app-backend-5VBsNyJv-py3.11/lib/python3.11/site-packages/llama_index/indices/loading.py", line 71, in load_indices_from_storage
raise ValueError(f"Failed to load index with ID {index_id}")

sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "data_pg_vector_store" does not exist
LINE 1: INSERT INTO data_pg_vector_store (text, metadata_, node_id, ...
^

[SQL: INSERT INTO data_pg_vector_store (text, metadata_, node_id, embedding) SELECT p0::VARCHAR, p1::JSON, p2::VARCHAR, p3::VECTOR(1536) FROM (VALUES (%(text__0)s, %(metadata___0)s, %(node_id__0)s, %(embedding__0)s, 0), (%(text__1)s, %(metadata___1)s, %(no ... 22172 characters truncated ... en_counter) ORDER BY sen_counter RETURNING data_pg_vector_store.id, data_pg_vector_store.id AS id__1]
[parameters: {'metadata___0': '{"page_label": "1", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_content ... (1383 characters truncated) ... id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "ref_doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164"}', 'node_id__0': 'a4ef423f-5151-480b-8d83-531528233eb6', 'embedding__0': '[-0.022870471701025963,-0.020472979173064232,-0.007333905901759863,-0.02161785028874874,-0.009192637167870998,0.020284412428736687,-0.031544551253318 ... (32572 characters truncated) ... -0.009859355166554451,0.011542988941073418,-0.01328049786388874,0.001522004371508956,0.012176034972071648,-0.007845730520784855,-0.02987438626587391]', 'text__0': 'Table\tof\tContents\nUNITED\tSTATES\nSECURITIES\tAND\tEXCHANGE\tCOMMISSION\nWashington,\tD.C.\t20549\n\t\n__________________________________\nFORM\ ... (561 characters truncated) ... tcharter)\nDelaware\n\t\n91-1646860\n(State\tor\tother\tjurisdiction\tof\nincorporation\tor\torganization)\n\t\n(I.R.S.Employer\nIdentification\tNo.)', 'metadata___1': '{"page_label": "1", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "ref_doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164"}', 'node_id__1': '23256ba6-403b-42cd-8d1d-a349ace9b7ca', 'embedding__1': '[-0.017658807337284088,0.009382538497447968,-0.015335644595324993,-0.029952215030789375,-0.010025556199252605,0.01793537475168705,-0.0236603133380413 ... (32534 characters truncated) ... 10457693133503199,-0.006903805769979954,-0.013095451518893242,0.00010776393173728138,-0.008677291683852673,-0.01140839233994484,-0.01604088954627514]', 'text__1': '410\tTerry\tAvenue\tNorth\nSeattle\n,\t\nWashington\n\t\n98109-5210\n(\n206\n)\t\n266-1000\n(Address\tand\ttelephone\tnumber,\tincluding\tarea\tcode, ... (594 characters truncated) ... e\tby\tcheck\tmark\tif\tthe\tregistrant\tis\tnot\trequired\tto\tfile\treports\tpursuant\tto\tSection\t13\tor\tSection\t15(d)\tof\tthe\tExchange\tAct.', 'metadata___2': '{"page_label": "1", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "ref_doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164"}', 'node_id__2': '9bbca51d-605f-4dca-aa87-4bbd879871a7', 'embedding__2': '[-0.028036287054419518,-0.0018036260735243559,-0.01268805656582117,-0.022423459216952324,-0.01582176983356476,0.009463814087212086,-0.034401208162307 ... (32570 characters truncated) ... 2,-0.003008364699780941,0.017270242795348167,-0.02905300445854664,0.004063381347805262,0.0089693833142519,0.007729826029390097,-0.044902630150318146]', 'text__2': 'Yes\t\t\n☐\n\t\t\t\t\nNo\n\t\t\n☒\nIndicate\tby\tcheck\tmark\twhether\tthe\tregistrant\t(1)\thas\tfiled\tall\treports\trequired\tto\tbe\tfiled\tby\tS ... (739 characters truncated) ... tlarge\taccelerated\tfiler,\tan\taccelerated\tfiler,\ta\tnon-accelerated\tfiler,\ta\tsmaller\treporting\tcompany,\tor\tan\temerging\tgrowth\tcompany.', 'metadata___3': '{"page_label": "1", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "ref_doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164"}', 'node_id__3': '81a01724-28f5-4597-8c87-7ed6468de4c1', 'embedding__3': '[-0.026984572410583496,-0.005932071711868048,-0.013684640638530254,-0.027273103594779968,-0.01509981881827116,0.002528086304664612,-0.021639868617057 ... (32521 characters truncated) ... 0.0030931271612644196,0.013547244481742382,-0.02034834586083889,-0.001504486077465117,0.00701406504958868,-0.006254952400922775,-0.04855574294924736]', 'text__3': 'See\tthe\ndefinitions\tof\t“large\taccelerated\tfiler,”\t“accelerated\tfiler,”\t“smaller\treporting\tcompany,”\tand\t“emerging\tgrowth\tcompany”\tin\ ... (838 characters truncated) ... taudit\treport.☒\nIndicate\tby\tcheck\tmark\twhether\tthe\tregistrant\tis\ta\tshell\tcompany\t(as\tdefined\tin\tRule\t12b-2\tof\tthe\tExchange\tAct).', 'metadata___4': '{"page_label": "1", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164", "ref_doc_id": "f2d8b2ad-e2af-425f-ab89-34ccff0e1164"}', 'node_id__4': '7681c7c3-853b-4a18-a661-ef55e11d4c37', 'embedding__4': '[-0.03121873550117016,-0.0164028313010931,-0.020123908296227455,-0.03507661446928978,-0.029276113957166672,0.0076883998699486256,-0.03321607783436775 ... (32587 characters truncated) ... .0019392005633562803,0.03411898389458656,-0.013789870776236057,-0.01167624443769455,-0.00868707150220871,-0.008215096779167652,-0.012640714645385742]', 'text__4': 'Yes\t\t\n☐\n\t\t\t\tNo\t\t\n☒\nAggregate\tmarket\tvalue\tof\tvoting\tstock\theld\tby\tnon-affiliates\tof\tthe\tregistrant\tas\tof\tJune\t30,\t2021\n$ ... (530 characters truncated) ... filed\twith\tthe\tSecurities\tand\tExchange\tCommission\twithin\t120\tdays\tafter\tthe\tend\tof\tthe\tfiscal\tyear\tto\twhich\tthis\tReport\nrelates.', 'metadata___5': '{"page_label": "2", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635", "doc_id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635", "ref_doc_id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635"}', 'node_id__5': 'ade9667a-5b36-443c-9633-0754abb4b761', 'embedding__5': '[-0.009512137621641159,-0.003550972556695342,-0.012673824094235897,-0.02387174218893051,-0.015977684408426285,0.03460928797721863,-0.0318199619650840 ... (32599 characters truncated) ... 0.010541209019720554,0.011123446747660637,-0.029301447793841362,-0.01079170685261488,0.001871961634606123,-0.01431221328675747,-0.025279944762587547]', 'text__5': 'Table\tof\tContents\nAMAZON.COM,\tINC.\nFORM\t10-K\nFor\tthe\tFiscal\tYear\tEnded\tDecember\t31,\t2021\nINDEX\n\t\n\t\n\t\nPage\nPART\tI\nItem\t1.\nB ... (970 characters truncated) ... \tCompensation\n70\nItem\t12.Security\tOwnership\tof\tCertain\tBeneficial\tOwners\tand\tManagement\tand\tRelated\tShareholder\tMatters\n70\nItem\t13.', 'metadata___6': '{"page_label": "2", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635", "doc_id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635", "ref_doc_id": "05a1cfcd-0100-4cce-a0d9-54e2a7a02635"}', 'node_id__6': 'bc917fe3-f336-4b4e-8957-11f7552ba66c', 'embedding__6': '[-0.004647449124604464,-0.001603020471520722,-0.009434671141207218,-0.03720754757523537,-0.022894801571965218,0.018408091738820076,-0.017443658784031 ... (32528 characters truncated) ... 0.011922628618776798,0.012174219824373722,-0.023048551753163338,0.00359565787948668,-0.00179782893974334,-0.007736430037766695,-0.030274810269474983]', 'text__6': 'Certain\tRelationships\tand\tRelated\tTransactions,\tand\tDirector\tIndependence\n70\nItem\t14.Principal\tAccountant\tFees\tand\tServices\n70\nPART\tIV\nItem\t15.Exhibits,\tFinancial\tStatement\tSchedules\n71\nItem\t16.Form\t10-K\tSummary\n73\nSignatures\n74\n2', 'metadata___7': '{"page_label": "3", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "0738125f-edac-46b1-b53a-896156b83e74", "doc_id": "0738125f-edac-46b1-b53a-896156b83e74", "ref_doc_id": "0738125f-edac-46b1-b53a-896156b83e74"}', 'node_id__7': 'ac7a7485-a561-45a8-b85b-8870df476ece', 'embedding__7': '[-0.018580947071313858,-0.0006651829462498426,-0.006592362187802792,-0.02414027974009514,-0.004536496941000223,0.027918994426727295,-0.02520049549639 ... (32562 characters truncated) ... .012151694856584072,-0.003214626107364893,-0.012892486527562141,0.0009047507774084806,0.008937068283557892,-0.02479271963238716,-0.02721218392252922]', 'text__7': 'Table\tof\tContents\nAMAZON.COM,\tINC.\nPART\tI\nItem\t1.\nBusiness\nThis\tAnnual\tReport\ton\tForm\t10-K\tand\tthe\tdocuments\tincorporated\therein\ ... (898 characters truncated) ... ,\tand\temployees.We\thave\torganized\tour\toperations\tinto\tthree\tsegments:\tNorth\tAmerica,\tInternational,\tand\tAmazon\tWeb\tServices\t(“AWS”).', 'metadata___8': '{"page_label": "3", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "0738125f-edac-46b1-b53a-896156b83e74", "doc_id": "0738125f-edac-46b1-b53a-896156b83e74", "ref_doc_id": "0738125f-edac-46b1-b53a-896156b83e74"}', 'node_id__8': 'c2045825-2016-43a5-9531-dbc75c02897e', 'embedding__8': '[-0.005483160726726055,0.004950684495270252,-0.014100519940257072,-0.028416724875569344,-0.004505830351263285,0.023685097694396973,-0.010568649508059 ... (32537 characters truncated) ... 034476174041628838,-0.007023299112915993,-0.0035318699665367603,0.005911164451390505,0.01451841276139021,-0.013588263653218746,-0.024520885199308395]', 'text__8': 'These\tsegments\treflect\tthe\tway\tthe\nCompany\tevaluates\tits\tbusiness\tperformance\tand\tmanages\tits\toperations.Information\ton\tour\tnet\tsal ... (975 characters truncated) ... hip\tprogram\tthat\tincludes\tfast,\tfree\tshipping\ton\tmillions\tof\titems,\taccess\tto\taward-winning\tmovies\tand\tseries,\tand\nother\tbenefits.', 'metadata___9': '{"page_label": "3", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "0738125f-edac-46b1-b53a-896156b83e74", "doc_id": "0738125f-edac-46b1-b53a-896156b83e74", "ref_doc_id": "0738125f-edac-46b1-b53a-896156b83e74"}', 'node_id__9': '518add46-489f-4594-a9e2-b155aef612cd', 'embedding__9': '[-0.0031864107586443424,0.003576404880732298,-0.02449793741106987,-0.015012157149612904,-0.015543808229267597,0.03167523071169853,-0.0214619282633066 ... (32590 characters truncated) ... 0.01017832662910223,-0.006187443155795336,-0.017656423151493073,0.002637270838022232,0.005540367215871811,-0.012892546132206917,-0.02162981778383255]', 'text__9': 'We\tfulfill\tcustomer\torders\tin\ta\tnumber\tof\tways,\tincluding\tthrough:\tNorth\tAmerica\tand\tInternational\tfulfillment\tnetworks\tthat\twe\top ... (852 characters truncated) ... ad\tset\tof\ton-\ndemand\ttechnology\tservices,\tincluding\tcompute,\tstorage,\tdatabase,\tanalytics,\tand\tmachine\tlearning,\tand\tother\tservices.', 'metadata___10': '{"page_label": "3", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "0738125f-edac-46b1-b53a-896156b83e74", "doc_id": "0738125f-edac-46b1-b53a-896156b83e74", "ref_doc_id": "0738125f-edac-46b1-b53a-896156b83e74"}', 'node_id__10': '6ba5f67e-ce66-495f-b1ce-c1dd60e2c915', 'embedding__10': '[-0.015770666301250458,0.010859748348593712,-0.0218866765499115,-0.026748834177851677,-0.0009699935326352715,0.03937093913555145,-0.02690208330750465 ... (32536 characters truncated) ... 18553588539362,-0.007857469841837883,-0.009905427694320679,-0.0055970544926822186,-0.0032321508042514324,-0.008170933462679386,-0.024756602942943573]', 'text__10': 'Content\tCreators\nWe\tserve\tauthors\tand\tindependent\tpublishers\twith\tKindle\tDirect\tPublishing,\tan\tonline\tservice\tthat\tlets\tindependent\ ... (393 characters truncated) ... vices\tto\tsellers,\tvendors,\tpublishers,\tauthors,\tand\tothers,\tthrough\tprograms\tsuch\tas\tsponsored\tads,\tdisplay,\tand\tvideo\tadvertising.3', 'metadata___11': '{"page_label": "4", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1383 characters truncated) ... _id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7", "doc_id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7", "ref_doc_id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7"}', 'node_id__11': '69a6697f-5743-4a67-bdfd-b83fec0c9401', 'embedding__11': '[-0.02492537535727024,0.006510151084512472,-0.009779167361557484,-0.016658738255500793,0.00041668626363389194,0.03685832768678665,-0.0162405278533697 ... (32596 characters truncated) ... 011319577693939209,-0.011082591488957405,-0.01748121902346611,-0.008433922193944454,0.006621674168854952,-0.014958013780415058,-0.015738673508167267]', 'text__11': 'Table\tof\tContents\nCompetition\nOur\tbusinesses\tencompass\ta\tlarge\tvariety\tof\tproduct\ttypes,\tservice\tofferings,\tand\tdelivery\tchannels.Th ... (979 characters truncated) ... thosting,\tomnichannel\tsales,\tinventory\tand\tsupply\nchain\tmanagement,\tadvertising,\tfulfillment,\tcustomer\tservice,\tand\tpayment\tprocessing;', 'metadata___12': '{"page_label": "4", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_content ... (1736 characters truncated) ... _id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7", "doc_id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7", "ref_doc_id": "5e52eb1e-7e70-4184-a0cd-dc510235a3e7"}', 'node_id__12': '29b6121b-f82b-4240-8a7a-dd28223984ba' ... 1040 parameters truncated ... 'embedding__272': '[-0.02633950486779213,-0.01672605611383915,-0.022910932078957558,-0.037055473774671555,-0.028073959052562714,0.020746225491166115,-0.0150050465017557 ... (32583 characters truncated) ... 0030621858313679695,0.010514289140701294,-0.03592606261372566,-0.003825211199000478,0.006312606856226921,-0.005273951217532158,-0.021781519055366516]', 'text__272': 'Table\tof\tContents\nOther\tInformation\nNot\tapplicable.Item\t9C.Disclosure\tRegarding\tForeign\tJurisdictions\tthat\tPrevent\tInspections\nNot\tapp ... (797 characters truncated) ... tis\tset\tforth\tin\tour\tProxy\tStatement\trelating\tto\tour\n2022\tAnnual\tMeeting\tof\tShareholders\tand\tis\tincorporated\therein\tby\treference.', 'metadata___273': '{"page_label": "70", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_conten ... (1387 characters truncated) ... _id": "666c2725-da6d-40bf-a7e0-a161c4518465", "doc_id": "666c2725-da6d-40bf-a7e0-a161c4518465", "ref_doc_id": "666c2725-da6d-40bf-a7e0-a161c4518465"}', 'node_id__273': '954b793c-742b-4d8a-a3c9-e046ac1021bf', 'embedding__273': '[-0.02962600067257881,-0.010689308866858482,-0.020194653421640396,-0.04442554712295532,-0.023719636723399162,0.0255359448492527,-0.031159773468971252 ... (32604 characters truncated) ... 0028001414611935616,0.016292955726385117,-0.009848425164818764,-0.0006071177776902914,0.005818912759423256,-0.01375685166567564,-0.02051755227148533]', 'text__273': 'To\tthe\textent\tpermissible\tunder\tNasdaq\trules,\twe\tintend\tto\tdisclose\tamendments\tto\tour\tCode\nof\tBusiness\tConduct\tand\tEthics,\tas\twe ... (1054 characters truncated) ... tis\tincluded\tin\tour\tProxy\tStatement\trelating\tto\tour\t2022\tAnnual\tMeeting\tof\tShareholders\tand\tis\tincorporated\therein\tby\nreference.70', 'metadata___274': '{"page_label": "71", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1387 characters truncated) ... id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "ref_doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0"}', 'node_id__274': 'f449d061-c5d9-4cb7-b375-d9de7b11c6fe', 'embedding__274': '[-0.03576786443591118,-0.013389434665441513,0.0025495686568319798,-0.020114384591579437,-0.011689722537994385,0.012630274519324303,-0.021525213494896 ... (32498 characters truncated) ... ,0.0010001766495406628,0.000522342452313751,-0.02040998637676239,-0.012616838328540325,0.002223734511062503,0.007356463000178337,-0.0532621368765831]', 'text__274': 'Table\tof\tContents\nPART\tIV\nItem\t15.Exhibits,\tFinancial\tStatement\tSchedules\n(a)\n\tList\tof\tDocuments\tFiled\tas\ta\tPart\tof\tThis\tReport: ... (961 characters truncated) ... tstatements\tor\tthe\tnotes\tthereto,\tor\tbecause\tit\tis\nnot\trequired.(3)\t\nIndex\tto\tExhibits\nSee\texhibits\tlisted\tunder\tPart\t(b)\tbelow.', 'metadata___275': '{"page_label": "71", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1741 characters truncated) ... id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "ref_doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0"}', 'node_id__275': '671cbf3c-8054-4255-8972-062749498bf6', 'embedding__275': '[-0.025430046021938324,-0.015719158574938774,-0.005737018305808306,0.0007476433529518545,-0.01939464919269085,0.01113496907055378,-0.0289156567305326 ... (32548 characters truncated) ... -0.007195007521659136,0.01378647517412901,-0.005309793632477522,0.004377358593046665,0.008571621030569077,-0.022473379969596863,-0.02526729367673397]', 'text__275': '(b)\t\nExhibits:\nExhibit\tNumber\nDescription\n3.1\nRestated\tCertificate\tof\tIncorporation\tof\tthe\tCompany\t(incorporated\tby\treference\tto\tth ... (464 characters truncated) ... d\tForm\tof\t2.500%\tNote\tdue\t2022\t(incorporated\tby\treference\tto\tthe\tCompany’s\tCurrent\tReport\ton\tForm\t8-K,\nfiled\tNovember\t29,\t2012).', 'metadata___276': '{"page_label": "71", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1741 characters truncated) ... id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "ref_doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0"}', 'node_id__276': '922a1a79-cc4d-4c95-9ee0-c1adccb6552c', 'embedding__276': '[-0.02938547357916832,-0.008914917707443237,-0.014057375490665436,-0.02465997263789177,-0.010979842394590378,0.00805453211069107,-0.04023956879973411 ... (32534 characters truncated) ... 0.0060756453312933445,0.020755145698785782,-0.016201721504330635,-0.00682351877912879,0.02344219572842121,-0.00885535217821598,-0.021178720518946648]', 'text__276': '4.2\nOfficers’\tCertificate\tof\tAmazon.com,\tInc.,\tdated\tas\tof\tDecember\t5,\t2014,\tcontaining\tForm\tof\t2.600%\tNote\tdue\t2019,\tForm\tof\t3. ... (810 characters truncated) ... g\tForm\tof\t5.200%\tNote\tdue\t2025\t(incorporated\tby\treference\tto\tthe\nCompany’s\tCurrent\tReport\ton\tForm\t8-K,\tfiled\tDecember\t20,\t2017).', 'metadata___277': '{"page_label": "71", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1387 characters truncated) ... id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0", "ref_doc_id": "0a0bc66b-6198-4894-bbf2-1000be6e94c0"}', 'node_id__277': '42025de0-bb05-4519-8e82-e913e33f35e7', 'embedding__277': '[-0.021043267101049423,-0.005917990580201149,-0.023138009011745453,-0.025013694539666176,-0.011883900500833988,0.018537789583206177,-0.03702081367373 ... (32581 characters truncated) ... 0.015142389573156834,0.02254929021000862,-0.005363500211387873,-0.00696878507733345,0.009939759969711304,-0.018756847828626633,-0.025068458169698715]', 'text__277': '4.5\nOfficers’\tCertificate\tof\tAmazon.com,\tInc.,\tdated\tas\tof\tJune\t3,\t2020,\tcontaining\tForm\tof\t0.400%\tNote\tdue\t2023,\tForm\tof\t0.800% ... (127 characters truncated) ... tand\tForm\tof\t2.700%\tNote\tdue\t2060\t(incorporated\tby\treference\nto\tthe\tCompany’s\tCurrent\tReport\ton\tForm\t8-K,\tfiled\tJune\t3,\t2020).71', 'metadata___278': '{"page_label": "72", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1387 characters truncated) ... id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "ref_doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa"}', 'node_id__278': '9795b5e8-84b2-4c66-a7db-2f426fdd2538', 'embedding__278': '[-0.023282980546355247,-0.005994346458464861,-0.013472813181579113,-0.02840251475572586,-0.023446369916200638,0.022738350555300713,-0.035155940800905 ... (32580 characters truncated) ... 0.00959231611341238,0.018422149121761322,-0.00914299488067627,-0.009571892209351063,0.007781417574733496,-0.012553747743368149,-0.021335924044251442]', 'text__278': 'Table\tof\tContents\n4.6\nOfficers’\tCertificate\tof\tAmazon.com,\tInc.,\tdated\tas\tof\tMay\t12,\t2021,\tcontaining\tForm\tof\t0.250%\tNote\tdue\t20 ... (781 characters truncated) ... ed\tand\trestated)\t(incorporated\tby\treference\tto\tthe\tCompany’s\tQuarterly\tReport\ton\tForm\t10-Q\tfor\tthe\nQuarter\tended\tMarch\t31,\t2013).', 'metadata___279': '{"page_label": "72", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1741 characters truncated) ... id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "ref_doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa"}', 'node_id__279': '0ee707e8-ebd7-4c16-b1e8-0b3c2e442bca', 'embedding__279': '[-0.021323321387171745,-0.0071167596615850925,-0.015958793461322784,-0.038441285490989685,-0.021013310179114342,0.014705273322761059,-0.0182771328836 ... (32568 characters truncated) ... 0.0106818787753582,0.006193467881530523,-0.0369047112762928,-0.0060822684317827225,-0.005856500007212162,-0.013963944278657436,-0.008403976447880268]', 'text__279': '10.3†\nForm\tof\tIndemnification\tAgreement\tbetween\tthe\tCompany\tand\teach\tof\tits\tDirectors\t(incorporated\tby\treference\tto\tExhibit\t10.1\tt ... (674 characters truncated) ... rm\t10-K\tfor\tthe\tYear\tended\tDecember\t31,\t2001).10.7†\nForm\tof\tGlobal\tRestricted\tStock\tUnit\tAward\tAgreement\tfor\tExecutive\tOfficers\n.', 'metadata___280': '{"page_label": "72", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1741 characters truncated) ... id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "ref_doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa"}', 'node_id__280': '94eb16eb-f6c0-4572-a3d1-c2663f1ccf39', 'embedding__280': '[-0.016197150573134422,-0.012619493529200554,-0.008098575286567211,-0.014095024205744267,-0.015577292069792747,0.01635885238647461,-0.011844671331346 ... (32565 characters truncated) ... .004827478434890509,0.006609569303691387,0.005777477752417326,-0.00273040565662086,0.0019134735921397805,-0.022166648879647255,-0.026074446737766266]', 'text__280': '10.8\nAmended\tand\tRestated\tCredit\tAgreement,\tdated\tas\tof\tJune\t23,\t2020,\n\tas\tamended\tby\tthe\tFirst\tAmendment\tthereto,\tdated\tas\tof\ ... (846 characters truncated) ... on\tof\tBrian\tT.\tOlsavsky,\tSenior\tVice\tPresident\tand\tChief\tFinancial\tOfficer\tof\tAmazon.com,\tInc.,\tpursuant\tto\t18\tU.S.C.Section\t1350.', 'metadata___281': '{"page_label": "72", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1387 characters truncated) ... id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa", "ref_doc_id": "728aa349-0b99-4d8c-b509-146313b3a9aa"}', 'node_id__281': '308f4f11-822c-4083-8ebe-d51169614d6a', 'embedding__281': '[-0.02487175166606903,-0.0048425570130348206,-0.017734641209244728,-0.029602792114019394,-0.01039477065205574,0.010428563691675663,-0.035901833325624 ... (32597 characters truncated) ... 017724502831697464,0.021046368405222893,-0.031143760308623314,-0.006579524837434292,0.009164700284600258,-0.009043044410645962,-0.030603069812059402]', 'text__281': '101\nThe\tfollowing\tfinancial\tstatements\tfrom\tthe\tCompany’s\tAnnual\tReport\ton\tForm\t10-K\tfor\tthe\tyear\tended\tDecember\t31,\t2021,\tformat ... (803 characters truncated) ... its\tsubsidiaries\ton\ta\tconsolidated\tbasis.The\tCompany\tagrees\tto\tfurnish\ta\tcopy\tof\tsuch\tagreements\tto\tthe\nCommission\tupon\trequest.72', 'metadata___282': '{"page_label": "73", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "node_conten ... (1033 characters truncated) ... id": "af6430f7-4860-43fd-a729-a44c1b7e902f", "doc_id": "af6430f7-4860-43fd-a729-a44c1b7e902f", "ref_doc_id": "af6430f7-4860-43fd-a729-a44c1b7e902f"}', 'node_id__282': '397c0dbe-6fc4-4e22-af59-317a0badedd3', 'embedding__282': '[-0.014973169192671776,-0.0011474371422082186,-0.00826896633952856,-0.04305797815322876,-0.027612682431936264,0.026209790259599686,-0.036475181579589 ... (32582 characters truncated) ... 0.026600981131196022,0.009166007861495018,-0.016605379059910774,-0.00907158199697733,0.002343773143365979,-0.010980323888361454,-0.02727544866502285]', 'text__282': 'Table\tof\tContents\n104\nThe\tcover\tpage\tfrom\tthe\tCompany’s\tAnnual\tReport\ton\tForm\t10-K\tfor\tthe\tyear\tended\tDecember\t31,\t2021,\tformat ... (4 characters truncated) ... tin\tiXBRL\t(included\tas\tExhibit\t101).\n\n†\t\t\t\tExecutive\tCompensation\tPlan\tor\tAgreement.\nForm\t10-K\tSummary\nNone.\n73', 'metadata___283': '{"page_label": "74", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_conten ... (1387 characters truncated) ... _id": "9053e49b-2b50-4430-b44b-c379e86bb797", "doc_id": "9053e49b-2b50-4430-b44b-c379e86bb797", "ref_doc_id": "9053e49b-2b50-4430-b44b-c379e86bb797"}', 'node_id__283': '9096865c-d22c-4edb-ad92-08cc70eda0fc', 'embedding__283': '[-0.027182336896657944,-0.014929641969501972,-0.021748533472418785,-0.018512222915887833,-0.017699817195534706,0.026569701731204987,-0.02863401547074 ... (32613 characters truncated) ... 012226058170199394,0.01698063686490059,-0.011540173552930355,-0.008370455354452133,0.002274073427543044,-0.0005044247955083847,-0.016288092359900475]', 'text__283': 'Table\tof\tContents\nSIGNATURES\nPursuant\tto\tthe\trequirements\tof\tSection\t13\tor\t15(d)\tof\tthe\tSecurities\tExchange\tAct\tof\t1934,\tthe\treg ... (362 characters truncated) ... \tbeen\tsigned\tbelow\tby\tthe\tfollowing\tpersons\ton\tbehalf\tof\tthe\tregistrant\tand\tin\tthe\ncapacities\tindicated\tas\tof\tFebruary\t3,\t2022.', 'metadata___284': '{"page_label": "74", "file_name": "a324b267-968e-4c84-80ee-59bf244bb342.pdf", "db_document_id": "a324b267-968e-4c84-80ee-59bf244bb342", "_node_conten ... (1387 characters truncated) ... _id": "9053e49b-2b50-4430-b44b-c379e86bb797", "doc_id": "9053e49b-2b50-4430-b44b-c379e86bb797", "ref_doc_id": "9053e49b-2b50-4430-b44b-c379e86bb797"}', 'node_id__284': '5948accc-ebe7-4bd2-8910-08b94806e265', 'embedding__284': '[-0.02335473708808422,0.0017766080563887954,-0.020171280950307846,-0.026833999902009964,-0.013255010358989239,0.018213313072919846,-0.023918179795145 ... (32518 characters truncated) ... 83037493750453,-0.009043269790709019,-0.0018540815217420459,-0.018283743411302567,-0.0033348817378282547,-0.018016109243035316,-0.014917168766260147]', 'text__284': 'Signature\nTitle\n/s/\tAndrew\tR.\tJassy\nAndrew\tR.\tJassy\nPresident\tand\tChief\tExecutive\tOfficer\t(Principal\tExecutive\tOfficer)\n/s/\tBrian\t ... (700 characters truncated) ... .\tRubinstein\nDirector\n/s/\tPatricia\tQ.\tStonesifer\nPatricia\tQ.\tStonesifer\nDirector\n/s/\tWendell\tP.\tWeeks\nWendell\tP.\tWeeks\nDirector\n74'}]
(Background on this error at: https://sqlalche.me/e/20/f405)

Hi,
First of all, thank you for creating such a great QA application.
I have a question (or rather a requirement): If I don't want to use it to analyze SEC data, but instead want to import and analyze my own PDF files to create my own knowledge based QA application, where can I make the simplest modifications? Thank you.

How would I be able to use Google Gemini instead of Open AI?

While running make run to launch the backend server, i am frequently seeing this error. I checked if other background processes and also change the port in .env but neither worked. any suggestions?

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/main.py", line 4, in
main(prog="alembic")
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 630, in main
CommandLine(prog=prog).main(argv=argv)
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 624, in main
self.run_cmd(cfg, options)
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/config.py", line 601, in run_cmd
fn(
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/command.py", line 398, in upgrade
script.run_env()
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/script/base.py", line 579, in run_env
util.load_python_file(self.dir, "env.py")
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
module = load_module_py(module_id, path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
spec.loader.exec_module(module) # type: ignore
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 940, in exec_module
File "", line 241, in _call_with_frames_removed
File "/workspaces/sec-insights/backend/alembic/env.py", line 9, in
from app.core.config import settings
File "/workspaces/sec-insights/backend/app/core/config.py", line 4, in
from pydantic import BaseSettings, AnyHttpUrl, EmailStr, validator
ModuleNotFoundError: No module named 'pydantic'
make: *** [Makefile:15: migrate] Error 1

make run_docker seemed to work but I ran into this error when running make migrate.

botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "http://localhost:4566/llama-app-backend-sec-insight"

(llama-app-backend-py3.10) rl13@rl13:~/RL-Work/Code/sec-insights/backend$ make migrate
echo "Running migrations."
Running migrations.
docker compose start db
[+] Running 1/1
⠿ Container backend-db-1 Started 0.4s
poetry run python -m alembic upgrade head
Traceback (most recent call last):
File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/main.py", line 4, in
main(prog="alembic")
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 630, in main
CommandLine(prog=prog).main(argv=argv)
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 624, in main
self.run_cmd(cfg, options)
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/config.py", line 601, in run_cmd
fn(
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/command.py", line 385, in upgrade
script.run_env()
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/script/base.py", line 578, in run_env
util.load_python_file(self.dir, "env.py")
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
module = load_module_py(module_id, path)
File "/home/rl13/.cache/pypoetry/virtualenvs/llama-app-backend-3I5GPRtC-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "/home/rl13/RL-Work/Code/sec-insights/backend/alembic/env.py", line 9, in
from app.core.config import settings
File "/home/rl13/RL-Work/Code/sec-insights/backend/./app/core/config.py", line 169, in
settings = Settings()
File "pydantic/env_settings.py", line 40, in pydantic.env_settings.BaseSettings.init
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 1 validation error for Settings
DATABASE_URL
field required (type=value_error.missing)
make: *** [Makefile:23: migrate] Error 1

Hello,

I'm trying to run the DELETE API under conversations:
getting the following foreign key reference error:
raise translated_error from error sqlalchemy.exc.IntegrityError: (sqlalchemy.dialects.postgresql.asyncpg.IntegrityError) <class 'asyncpg.exceptions.ForeignKeyViolationError'>: update or delete on table "conversation" violates foreign key constraint "message_conversation_id_fkey" on table "message" DETAIL: Key (id)=(79d87f70-4829-41aa-9043-1b61e7888a74) is still referenced from table "message". [SQL: DELETE FROM conversation WHERE conversation.id = $1::UUID] [parameters: ('79d87f70-4829-41aa-9043-1b61e7888a74',)] (Background on this error at: https://sqlalche.me/e/20/gkpj

Any idea how to resolve this?
I have tried to add cascade ON DELETE but alembic migrations are not updating the tables:

class Conversation(Base):
    messages = relationship("Message", back_populates="conversation", cascade="delete, merge, save-update")
    conversation_documents = relationship(
        "ConversationDocument", back_populates="conversation", cascade="delete, merge, save-update"
    )

Any help would be appreciated.

Hi, I'm trying to upload PDF file to the real S3 with this command:

make seed_db

and it seems to be successful:

Seeding storage with DB documents: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [01:49<00:00, 15.71s/it]
Done! 🏁                                                                                                                                                                                         
        - SEC PDF documents uploaded to the S3 assets bucket ✅                                                                                                                                  
        - Documents database table has been populated ✅                                                                                                                                         
        - Vector storage table has been seeded with embeddings ✅          

However it seems like after make run it's still serving PDF in my localstack. Did I miss anything?

I tried added a custom document and chat with it but it doesn't process. Am i missing something in my local setup?

Here is the snippet of what i tried.

(llama-app-backend-py3.11) ➜  backend git:(main) ✗ make chat
poetry run python -m scripts.chat_llama
(Chat🦙) pick_docs
(Pick📄) help

Documented commands (type help <topic>):
========================================
fetch  finish  help  quit  select  select_id

(Pick📄) select_id 127b9132-e258-4b8d-96cb-f32eccd1a3e9
Selected document ID 127b9132-e258-4b8d-96cb-f32eccd1a3e9
(Pick📄) finish
(Chat🦙) help

Documented commands (type help <topic>):
========================================
create  delete  detail  help  message  pick_docs  quit

(Chat🦙) detail
No active conversation. Use CREATE to start a new conversation.
(Chat🦙) create
HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /api/conversation/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x10178c7d0>: Failed to establish a new connection: [Errno 61] Connection refused'))
Quitting.
(llama-app-backend-py3.11) ➜  backend git:(main) ✗ make chat
poetry run python -m scripts.chat_llama
(Chat🦙) pick_docs
(Pick📄) select_id 127b9132-e258-4b8d-96cb-f32eccd1a3e9
Selected document ID 127b9132-e258-4b8d-96cb-f32eccd1a3e9
(Pick📄) 
Selected document ID 127b9132-e258-4b8d-96cb-f32eccd1a3e9
(Pick📄) 
Selected document ID 127b9132-e258-4b8d-96cb-f32eccd1a3e9
(Pick📄) finish
(Chat🦙) create
Created conversation with ID 99e25e90-5593-47ce-8bb3-184e2a7744bc
(Chat🦙) message what is the tile of the paper?


=== Message 0 ===
{'id': '860ba40d-450e-4063-ab25-ed9cf6346d5e', 'created_at': '2023-09-10T20:29:44.087949', 'updated_at': '2023-09-10T20:29:44.087949', 'conversation_id': '99e25e90-5593-47ce-8bb3-184e2a7744bc', 'content': '', 'role': 'assistant', 'status': 'ERROR', 'sub_processes': []}


====== Final Message ======

(Chat🦙) 

poetry run python -m alembic upgrade head
/root/.cache/pypoetry/virtualenvs/llama-app-backend--Qk0ygDj-py3.11/bin/python: No module named alembic.main; 'alembic' is a package and cannot be directly executed
make: *** [Makefile:15: migrate] Error 1

Setting up backend code in Pycharm to run locally and running into settings error, even after setting .env file....while running make migrate

make migrate
echo "Running migrations."
Running migrations.
docker compose create db
[+] Building 0.0s (0/0) docker:desktop-linux
[+] Creating 1/0
✔ Container backend-db-1 Running 0.0s
docker compose start db
poetry run python -m alembic upgrade head
Traceback (most recent call last):
File "anaconda3/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "anaconda3/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/main.py", line 4, in
main(prog="alembic")
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/config.py", line 630, in main
CommandLine(prog=prog).main(argv=argv)
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/config.py", line 624, in main
self.run_cmd(cfg, options)
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/config.py", line 601, in run_cmd
fn(
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/command.py", line 385, in upgrade
script.run_env()
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/script/base.py", line 578, in run_env
util.load_python_file(self.dir, "env.py")
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
module = load_module_py(module_id, path)
File "Library/Caches/pypoetry/virtualenvs/llama-app-backend-5r6C9Q_F-py3.10/lib/python3.10/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
spec.loader.exec_module(module) # type: ignore
File "", line 883, in exec_module
File "", line 241, in _call_with_frames_removed
File "Developer/react/sec-insights/backend/alembic/env.py", line 9, in
from app.core.config import settings
File "Developer/react/sec-insights/backend/./app/core/config.py", line 171, in
settings = Settings()
File "pydantic/env_settings.py", line 40, in pydantic.env_settings.BaseSettings.init
File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
pydantic.error_wrappers.ValidationError: 8 validation errors for Settings
OPENAI_API_KEY
field required (type=value_error.missing)
AWS_KEY
field required (type=value_error.missing)
AWS_SECRET
field required (type=value_error.missing)
POLYGON_IO_API_KEY
field required (type=value_error.missing)
DATABASE_URL
field required (type=value_error.missing)
S3_BUCKET_NAME
field required (type=value_error.missing)
S3_ASSET_BUCKET_NAME
field required (type=value_error.missing)
CDN_BASE_URL
field required (type=value_error.missing)
make: *** [migrate] Error 1

Hi thank you so much for your great work, but I got error Server: "react-select-4-live-region" Client: "react-select-3-live-region"

Hi, after following your tutorial the app is running correctly and I was trying to insert some docs via the upsert_document.py script. But I'm getting a lot of errors which indicates that the dependencies are are not resolved correctly (module Fire could not be found)

I'm using codespaces and installed the dependencies via poetry install
What am I missing here?

When replacing OpenAI() with AzureOpenAI() in backend/app/chat/engine.py, chat responses are empty. The frontend will display "Sorry, I either wasn't able to understand your question or I don't have an answer for it.".

I have confirmed that the AzureOpenAI parameters are correct, as I have gotten valid embeddings and chat completion responses back while debugging with the same objects passed to the chat_engine.

This is how I am constructing the chat_llm:

    chat_llm = AzureOpenAI(
        temperature=0,
        streaming=True,
        model=settings.AZURE_OPENAI_CHAT_MODEL_NAME,
        deployment_name=settings.AZURE_OPENAI_CHAT_DEPLOYMENT_NAME,
        api_key=settings.AZURE_OPENAI_API_KEY,
        azure_endpoint=settings.AZURE_OPENAI_API_BASE,
        api_version=settings.AZURE_OPENAI_API_VERSION,
        additional_kwargs={"api_key": settings.AZURE_OPENAI_API_KEY},
    )

I have added those Azure specific values to config.py and have confirmed the parameters are correct by getting valid completions and embeddings via the debugger. They are also set via set -a; source .env before running.
I've also explicitly set AZURE_OPENAI_ENDPOINT.

Has anyone else had success getting this project to work correctly with Azure Open AI?

Here is the error log:

(llama-app-backend-py3.11) bash-3.2$ make migrate
echo "Running migrations."
Running migrations.
docker compose start db
[+] Running 1/1
 ✔ Container backend-db-1  Started                                                                                             0.3s
poetry run python -m alembic upgrade head
The currently activated Python version 3.9.16 is not supported by the project (^3.10).
Trying to find and use a compatible version.
Using python3 (3.11.5)
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/__main__.py", line 4, in <module>
    main(prog="alembic")
  File "/Users.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/config.py", line 630, in main
    CommandLine(prog=prog).main(argv=argv)
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/config.py", line 624, in main
    self.run_cmd(cfg, options)
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/config.py", line 601, in run_cmd
    fn(
  File "/Users.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/command.py", line 385, in upgrade
    script.run_env()
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/.../Library/Caches/pypoetry/virtualenvs/llama-app-backend-cPGORASJ-py3.11/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/.../Desktop/jp_projects/ai_hackathons/sec-insights/backend/alembic/env.py", line 9, in <module>
    from app.core.config import settings
  File "/Users/.../Desktop/jp_projects/ai_hackathons/sec-insights/backend/app/core/config.py", line 169, in <module>
    settings = Settings()
               ^^^^^^^^^^
  File "pydantic/env_settings.py", line 40, in pydantic.env_settings.BaseSettings.__init__
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 7 validation errors for Settings
AWS_KEY
  field required (type=value_error.missing)
AWS_SECRET
  field required (type=value_error.missing)
POLYGON_IO_API_KEY
  field required (type=value_error.missing)
DATABASE_URL
  field required (type=value_error.missing)
S3_BUCKET_NAME
  field required (type=value_error.missing)
S3_ASSET_BUCKET_NAME
  field required (type=value_error.missing)
CDN_BASE_URL
  field required (type=value_error.missing)
make: *** [migrate] Error 1

Hey, I'd love to see an overview of how it would be possible to deploy the backend stack to AWS. I've seen #11 but I would like to deploy the entire backend to AWS (e.g. Beanstalk, ECS) and can't figure out what needs to be modified.

Got this error message when running the command docker compose up. It seems it's complaining the server connection? How to troubleshoot this issue? Thanks!

backend-llama-app-fastapi-1  | Running in AppEnvironment: local
backend-llama-app-fastapi-1  | 2023-09-09 20:17:43,336 [MainThread  ] [INFO ]  Set up logging with log level 10
backend-llama-app-fastapi-1  | 2023-09-09 20:17:43,336 [MainThread  ] [INFO ]  Skipping Sentry setup
backend-llama-app-fastapi-1  | 2023-09-09 20:17:43,336 [MainThread  ] [DEBUG]  Skipping migrations
backend-llama-app-fastapi-1  | INFO:     Will watch for changes in these directories: ['/app']
backend-llama-app-fastapi-1  | INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
backend-llama-app-fastapi-1  | INFO:     Started reloader process [1] using StatReload
backend-llama-app-fastapi-1  | INFO:     Started server process [20]
backend-llama-app-fastapi-1  | INFO:     Waiting for application startup.
backend-llama-app-fastapi-1  | ERROR:    Traceback (most recent call last):
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
backend-llama-app-fastapi-1  |     self._dbapi_connection = engine.raw_connection()
backend-llama-app-fastapi-1  |                              ^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3288, in raw_connection
backend-llama-app-fastapi-1  |     return self.pool.connect()
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect
backend-llama-app-fastapi-1  |     return _ConnectionFairy._checkout(self)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1267, in _checkout
backend-llama-app-fastapi-1  |     fairy = _ConnectionRecord.checkout(pool)
backend-llama-app-fastapi-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
backend-llama-app-fastapi-1  |     rec = pool._do_get()
backend-llama-app-fastapi-1  |           ^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 169, in _do_get
backend-llama-app-fastapi-1  |     with util.safe_reraise():
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
backend-llama-app-fastapi-1  |     raise exc_value.with_traceback(exc_tb)
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 167, in _do_get
backend-llama-app-fastapi-1  |     return self._create_connection()
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
backend-llama-app-fastapi-1  |     return _ConnectionRecord(self)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
backend-llama-app-fastapi-1  |     self.__connect()
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
backend-llama-app-fastapi-1  |     with util.safe_reraise():
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
backend-llama-app-fastapi-1  |     raise exc_value.with_traceback(exc_tb)
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
backend-llama-app-fastapi-1  |     self.dbapi_connection = connection = pool._invoke_creator(self)
backend-llama-app-fastapi-1  |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 637, in connect
backend-llama-app-fastapi-1  |     return dialect.connect(*cargs, **cparams)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 615, in connect
backend-llama-app-fastapi-1  |     return self.loaded_dbapi.connect(*cargs, **cparams)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect
backend-llama-app-fastapi-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  | psycopg2.OperationalError: could not connect to server: Connection refused
backend-llama-app-fastapi-1  | 	Is the server running on host "127.0.0.1" and accepting
backend-llama-app-fastapi-1  | 	TCP/IP connections on port 5432?
backend-llama-app-fastapi-1  |
backend-llama-app-fastapi-1  |
backend-llama-app-fastapi-1  | The above exception was the direct cause of the following exception:
backend-llama-app-fastapi-1  |
backend-llama-app-fastapi-1  | Traceback (most recent call last):
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/starlette/routing.py", line 677, in lifespan
backend-llama-app-fastapi-1  |     async with self.lifespan_context(app) as maybe_state:
backend-llama-app-fastapi-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 204, in __aenter__
backend-llama-app-fastapi-1  |     return await anext(self.gen)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/app/app/main.py", line 73, in lifespan
backend-llama-app-fastapi-1  |     if not check_current_head(cfg, engine):
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/app/app/main.py", line 26, in check_current_head
backend-llama-app-fastapi-1  |     with connectable.begin() as connection:
backend-llama-app-fastapi-1  |   File "/usr/local/lib/python3.11/contextlib.py", line 137, in __enter__
backend-llama-app-fastapi-1  |     return next(self.gen)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3228, in begin
backend-llama-app-fastapi-1  |     with self.connect() as conn:
backend-llama-app-fastapi-1  |          ^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3264, in connect
backend-llama-app-fastapi-1  |     return self._connection_cls(self)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
backend-llama-app-fastapi-1  |     Connection._handle_dbapi_exception_noconnection(
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2426, in _handle_dbapi_exception_noconnection
backend-llama-app-fastapi-1  |     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
backend-llama-app-fastapi-1  |     self._dbapi_connection = engine.raw_connection()
backend-llama-app-fastapi-1  |                              ^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 3288, in raw_connection
backend-llama-app-fastapi-1  |     return self.pool.connect()
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 452, in connect
backend-llama-app-fastapi-1  |     return _ConnectionFairy._checkout(self)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 1267, in _checkout
backend-llama-app-fastapi-1  |     fairy = _ConnectionRecord.checkout(pool)
backend-llama-app-fastapi-1  |             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 716, in checkout
backend-llama-app-fastapi-1  |     rec = pool._do_get()
backend-llama-app-fastapi-1  |           ^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 169, in _do_get
backend-llama-app-fastapi-1  |     with util.safe_reraise():
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
backend-llama-app-fastapi-1  |     raise exc_value.with_traceback(exc_tb)
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/impl.py", line 167, in _do_get
backend-llama-app-fastapi-1  |     return self._create_connection()
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 393, in _create_connection
backend-llama-app-fastapi-1  |     return _ConnectionRecord(self)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 678, in __init__
backend-llama-app-fastapi-1  |     self.__connect()
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 902, in __connect
backend-llama-app-fastapi-1  |     with util.safe_reraise():
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
backend-llama-app-fastapi-1  |     raise exc_value.with_traceback(exc_tb)
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/pool/base.py", line 898, in __connect
backend-llama-app-fastapi-1  |     self.dbapi_connection = connection = pool._invoke_creator(self)
backend-llama-app-fastapi-1  |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/create.py", line 637, in connect
backend-llama-app-fastapi-1  |     return dialect.connect(*cargs, **cparams)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 615, in connect
backend-llama-app-fastapi-1  |     return self.loaded_dbapi.connect(*cargs, **cparams)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  |   File "/root/.cache/pypoetry/virtualenvs/llama-app-backend-9TtSrW0h-py3.11/lib/python3.11/site-packages/psycopg2/__init__.py", line 122, in connect
backend-llama-app-fastapi-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
backend-llama-app-fastapi-1  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
backend-llama-app-fastapi-1  | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
backend-llama-app-fastapi-1  | 	Is the server running on host "127.0.0.1" and accepting
backend-llama-app-fastapi-1  | 	TCP/IP connections on port 5432?
backend-llama-app-fastapi-1  |
backend-llama-app-fastapi-1  | (Background on this error at: https://sqlalche.me/e/20/e3q8)
backend-llama-app-fastapi-1  |
backend-llama-app-fastapi-1  | ERROR:    Application startup failed. Exiting.

I noticed that the backed is using FastAPI and use python version LlamaIndex, can I use LlamaIndex.TS in NextJs to implement the same app?

After running make chat and create, I run into error: HTTPConnectionPool(host='localhost', port=8000): Max retries exceeded with url: /api/conversation/ (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f689d426a90>: Failed to establish a new connection: [Errno 111] Connection refused')). Any ideas?