Undocumented requirement for OPENAI_API_KEY about palm-kosmos-vision HOT 3 CLOSED

Thanks for raising this issue. It's important to note that the OPENAI_API_KEY is not a requirement for running the PaLM-Kosmos-Vision project, as evidenced by our live version here.

The error you're encountering suggests that the service_context and llm are not correctly set in LlamaIndex, which by default assumes OpenAI LLM unless specified otherwise. This could happen if there were changes in the default code.

To resolve this, please ensure that you specify the LLM you intend to use in your implementation. For more details on how to correctly implement LlamaIndex, please refer to their documentation.

Closing this issue as it seems to stem from a code modification on your end.

from palm-kosmos-vision.

I didn't alter the code at all.

from palm-kosmos-vision.

Upon further investigation I found that this issue is occurring when using latest version of llamaindex. 0.9 version onwards they changed how default embedding models is set.

so in create_chat_engine function, we need to change:

• service_context = ServiceContext.from_defaults(llm=llm)
  

to

• service_context = ServiceContext.from_defaults(llm=llm, embed_model="local")
  

This should fix the issue

from palm-kosmos-vision.