I'm following the second example but when init both chat memory and LLM takes too long (20 - 30s), can you explain why and give me some best practices when initializing it?

I am using AzureOpenAi, and the program uses TokenWindowChatMemory to control the token capacity: TokenWindowChatMemory.builder().capacityInTokens(4096), but when the number of tokens is almost 4096, the interface returns an error, it seems that OpenAiTokenizer did not calculate the token of functioncalling.

My temporary solution is to make the capacityInTokens parameter smaller, such as TokenWindowChatMemory.builder().capacityInTokens(3000).

This does not need to be solved right away, depending on your time, thank you!

I seem to have problem with PrompTemplates.

When i do the .apply(), new lines gets decoded to 


This makes GPT unable to answer my questions.

My prompttemplates gets resolved into this:

Prompt:
'How big is the apartment?'

Context:
'area in m2: 85

'

GPT responds that it doesnt know.

Of course i could sanitize my results before storing them in the vector database. But this seems unintentional.

(Note: this a fictional question and the response is 85 from gpt. In reality, when context is much longer (and contains many more 
, it gives me the "You havent specified the size".)

Is your feature request related to a problem? Please describe.
I find that langchain4j use tinylog instead of more popular combination solution of slf4j and one implementation. That may be ok, but also makes it difficult to integrate with existed logging system in java projects.

Describe the solution you'd like
Change tinylog to slf4j+logback or something else.

I've had created my own Milvus implementation but i havent shared it because i've kind of did it in a different way.
I see now that milvus has been added and maybe i should use the official one. (But i am in doubt, because its not mentioned in the changelog + i dont know if it can do what i wanna do)

The main difference is that in mine, i've added an additional id to my vectors, which is my document id.
So can save these objects seperately:

DocumentId: Book1
Text: The sky is blue

DocumentId: Book2
Text: The sky is red

Then i can do:

findRelevant("Book1", "What color is the sky", 1);

Then my milvus search will only search in all the "Book1" vectors.

I dont know if this is possible with current EmbeddingStore interface, but i havent figured a way to do it, so i made it myself.

Is this something possible to do or should i share my code?

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/E:/ProgramFile/maven_repository/ch/qos/logback/logback-classic/1.2.11/logback-classic-1.2.11.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/ProgramFile/maven_repository/org/slf4j/slf4j-simple/1.7.36/slf4j-simple-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/ProgramFile/maven_repository/org/slf4j/slf4j-reload4j/1.7.36/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/E:/ProgramFile/maven_repository/org/tinylog/slf4j-tinylog/2.6.2/slf4j-tinylog-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
Exception in thread "main" java.lang.NoSuchMethodError: 'java.util.Spliterator okhttp3.Headers.spliterator()'
at dev.ai4j.openai4j.RequestLoggingInterceptor.inOneLine(RequestLoggingInterceptor.java:49)
at dev.ai4j.openai4j.RequestLoggingInterceptor.log(RequestLoggingInterceptor.java:39)
at dev.ai4j.openai4j.RequestLoggingInterceptor.intercept(RequestLoggingInterceptor.java:29)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at dev.ai4j.openai4j.AuthorizationHeaderInjector.intercept(AuthorizationHeaderInjector.java:25)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at okhttp3.RealCall.getResponseWithInterceptorChain(RealCall.java:229)
at okhttp3.RealCall.execute(RealCall.java:81)
at retrofit2.OkHttpCall.execute(OkHttpCall.java:204)
at dev.ai4j.openai4j.SyncRequestExecutor.execute(SyncRequestExecutor.java:23)
at dev.ai4j.openai4j.RequestExecutor.execute(RequestExecutor.java:59)
at dev.langchain4j.model.openai.OpenAiEmbeddingModel.lambda$1(Unknown Source)
at dev.langchain4j.internal.RetryUtils.withRetry(Unknown Source)
at dev.langchain4j.model.openai.OpenAiEmbeddingModel.embedTexts(Unknown Source)
at dev.langchain4j.model.openai.OpenAiEmbeddingModel.embedAll(Unknown Source)
at com.wssoon.gpt.controller.ChatWithDocumentsExamples$If_You_Need_More_Control.main(ChatWithDocumentsExamples.java:105)

Hey, would anybody be interested in talking about this project at the upcoming Devoxx Belgium event (2-6 Oct) in Antwerp?
CFP is open @ https://dvbe23.cfp.dev
I apologise to use an issue for this but there are no contact details to be found, feel free to mail me via sja @ devoxx dot com

do we have a plan to support chroma ? thanks

When I want to implement database-based ChatMemory, I want to serialize List previousMessages to JSON and save it to the database.
When I use jackson for serialization, I get the error: No serializer found for class dev.langchain4j.data.message.UserMessage;

then I find Gson, which doesn't rely on Serializable, but when deserializing, I get an error: Abstract class can't be instantiated! Class name: dev.langchain4j.data.message.ChatMessage

I can't find another way, please help me, thanks!

Describe the solution you'd like
Can you please support more vectorstores besides Pincone? I want to use the Self-querying feature with some local stores.

Additionally could you verify the pinecone embedding storage is working ..ended up getting a bunch of errors when it tries to upload it.

Exception in thread "main" io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
at io.grpc.stub.ClientCalls.toStatusRuntimeException(ClientCalls.java:271)
at io.grpc.stub.ClientCalls.getUnchecked(ClientCalls.java:252)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:165)
at io.pinecone.proto.VectorServiceGrpc$VectorServiceBlockingStub.upsert(VectorServiceGrpc.java:505)
at dev.langchain4j.store.embedding.PineconeEmbeddingStoreImpl.addAllInternal(PineconeEmbeddingStoreImpl.java:120)

Or
when I try with source code after clean do get this issue even after placing the right jars.
matsim-org/matsim-code-examples#738

Can you support Milvus, please

can AIServices support streaming output?

Is there a best practice when searching for relevant embeddings in the vector db?

I see in the examples, its just searching for the question. But imagine a case like this:

Q: How tall is it?
A: What do you mean by 'it'?
Q: The eiffel tower
A: I am not sure how tall it is given the current context.

This happens because in the second question, i am simply searching for "the eiffel tower" in the vector database and not finding relevant context.
Would be cool if we could have a method, that would actually descibe what the question is, that we could search for in the vector db.

Would it be possible to have stream response when chat with documents?

Questions:
1 How to enable the debug parameter mode in langchain4j, just like verbose = True in langchain-py, you can know the specific execution status.
2 Is there a function that can call back the tokens consumed by the model calculation, so that it is easy to know the cost of each user execution.

Is it the function estimateTokenCount, which can be used to calculate the number of tokens consumed by the model?

3 If I want to define other custom tools, refer to ServiceWithToolsExample, the customerService tool is defined, but it is not executed.

Because I use openai’s domestic proxy domain name, OPENAI_URL is not https://api.openai.com, if I want to change the address of openai_url value, what should I do? In the code, this parameter is final。

When will the azure-openai-chat/embedding model be supported? It seems that LocalAIChatModel does not support it,If there are any other solutions, please let me know. Thank you.

Is your feature request related to a problem? Please describe.
It would be great if langchain4j supported some kind of guardrails functionality. See Guardrails AI for example (https://shreyar.github.io/guardrails/).

See https://github.com/FlagOpen/FlagEmbedding/tree/master/C_MTEB

Do you have WeChat group or QQ group?

Hey,
has anyone tested the operation of the library using Kotlin instead of Java?

I want to develop a chatbot that supports group chat and single chat, and needs to have a separate MemoryChat for each user. I hope to support this feature, thanks!

Do you have a roadmap? I would like to make some contributions.

Maybe i am doing something wrong. But heres two examples of whats going on.

Examples:

Init:

public interface PersonalHelperAgent {

	@SystemMessage({
			"You are a personal helper, help the user with whatever"
	})
	String chat(@UserId String userId, @UserMessage String userMessage);

}



PersonalHelperAgent chat;
chat = AiServices
				.builder(PersonalHelperAgent.class)
				.chatLanguageModel(chatModelv3)
				.chatMemorySupplier(() -> MessageWindowChatMemory.withMaxMessages(30)
				).build();

Example:

chat.chat(userId, "My name is John");
//response: Hello John!
chat.chat(userId, "Whats my name?");
//response: Your name is john.

When using PromptTemplate:

List<EmbeddingMatch<TextSegment>> relevant = embeddingStore.findRelevant(id, embed, maxResults, minSimilarity);

		PromptTemplate promptTemplate = PromptTemplate.from(
				"Answer the chat to the best of your ability.\n"
						+ "If you need to output numbers, create a table.:\n"
						+ "Chat:\n"
						+ "```{{question}}```\n"
						+ "\n"
						+ "Information:\n"
						+ "```{{information}}```");

		String information = relevant.stream()
				.map(match -> match.embedded().text())
				.collect(joining("\n\n"));

		Map<String, Object> variables = new HashMap<>();
		variables.put("question", "My name is john!");
		variables.put("information", information);

		Prompt prompt = promptTemplate.apply(variables);


chat.chat(userId, prompt.toUserMessage().text());
//response: Hello John!

//Same as before.. but.
variables.put("question", "What is my name?");
chat.chat(userId, prompt.toUserMessage().text());
//response: I dont have access to your personal data. how can i help you?

ToolExecutionRequest argumentsAsMap has a problem when parse argument to map, for example, default gson convert number to double, then cause java.lang.IllegalArgumentException: argument type mismatch when toolExecutor.execute beause my tool function parameter is integer

Please provide as much details as possible, this will help us to deliver a fix as soon as possible.
Thank you!

Describe the bug
ParagraphSplitter class
String[] paragraphs = text.split("\R\R");
There is no \R\R in Linux, this is a bug, please fix it.

Log and Stack trace
Please provide a log and a stack trace (with exception), if applicable.

To Reproduce
Please provide a relevant code snippets to reproduce this bug.

Expected behavior
A clear and concise description of what you expected to happen.

Please complete the following information:

• LangChain4j version: e.g. 0.20.0
• Java version: e.g. 11
• Spring Boot version (if applicable): e.g. 2.7.13

Additional context
Add any other context about the problem here.

Can AiServices supported multiple Retriever? maybe from multiple different Embedding Store

There is an exception during initialization caused by the intent of preparing user message for a method. The code is evaluating the hashCode method inherited from the Object class, but this method is not part of the interface.

In a scenario, I ask the AI, for example: "Help me take a leave tomorrow." and it will trigger a leave request page form with my name and the date pre-filled. Then, I can input other leave details like the reason on the form.
I want to use the function calling feature of OpenAI to achieve this functionality. I define a leaveRequest function with parameters for name and leave date, and then obtain the response parameter: function_call. I pass this function_call to the leave request page form to pre-fill the values.
I don't want langchain4j to automatically call the chatLanguageModel.sendMessages() method. Instead, I want to directly return the toolExecutionRequest so that I can carry the parameters and trigger the leave request page form.

I am looking for Azure OpenAI Service LLM provider support with langchain4j

Currently there is no limit for vectors per one upsert.

For Pinecone it's:

Max vector dimensionality is 20,000.

Max size for an upsert request is 2MB. Recommended upsert limit is 100 vectors per request.

I think it would be awesom if we could use ParagraphSplitter after HTMLExtractoren is finished. (ie. it needs to split sections better with two newlines so they count as paragraphs)

Is there a plan to add more agent toolkits? Eagerly waiting spark dataframe and spark sql features.

I figure out that langchain4j uses function-calling api work with tools,
but most llm except chatgpt does not support function-calling,
this way may not work for my request, do you have some advice?

In the single chat scenario, in order to reduce the use of tokens, I did not set chatmemory, and I wanted to use the tool, but I got an error：

This doesn't fit my scene 😣

is it possible to add hugging face embeddings as well with support for multiple transformers .

Additionally trying to take a look into the code section
Found that the import dev.ai4j.openai4j.*; package is not migrated to run the samples to test more

Hello author, I would like to ask when the model of chatGLM-6b will be integrated, thank you very much

would you like to support proxy for llm api? I really need to use this

I would like to connect my LangChain app to Azure OpenAI, is it possible?

Expose more channels to import Tool specification.
Maybe more fine-grained, such as method or function.

Exsited Static#Example

static class Calculator {

    @Tool("calculates the square root of the provided number")
    double squareRoot(@P("number to operate on") double number) {
        return Math.sqrt(number);
    }
}

@Test
void should_execute_tool_then_answer() {
    Calculator calculator = new Calculator();

    Assistant assistant = AiServices.builder(Assistant.class)
            .chatLanguageModel(chatLanguageModel)
            .tools(calculator)
            .build();

    String userMessage = "What is the square root of 485906798473894056 in scientific notation?";
    String answer = assistant.chat(userMessage);
}

// No need to declare redundant utility classes#Calculator
@Test
void should_execute_tool_then_answer() {
    Assistant assistant = AiServices.builder(Assistant.class)
          .chatLanguageModel(chatLanguageModel)
          // Assign the function_interface as tool_function directly.
          .toolFunction(Math::sqrt)
          // Only tool_description, use "Math::sqrt" as name
          .toolFunction(Math::sqrt, "calculates the square root of the provided number")
          // Full-define the tool_specification
          .toolFunction(Math::sqrt, ToolSpecification.builder()
                  .name("squareRoot")
                  .description("calculates the square root of the provided number")
                  .addParameter("arg0", NUMBER, JsonSchemaProperty.description("number to operate on"))
                  .build())
          .build();

    String userMessage = "What is the square root of 485906798473894056 in scientific notation?";

    String answer = assistant.chat(userMessage);
}

Instance#Example

class PersonMapper {
     Person selectById(Long id);
     Person selectxxx();
}

PersonMapper personMapper = ...

AiServices.builder(Assistant.class)
          .chatLanguageModel(chatLanguageModel)
          // Assign the function_interface as tool_function with tool_instance directly.
          .toolFunction(personMapper, PersonMapper::selectById)
          .build();

add support langchain ChainType stuff refinemap_reduce map_rerank four types

The method OpenAiStreamingChatModel.sendUserMessage only include temperature. how to support other parameters, eg : presencePenalty, maxTokens。

I am waiting for a redis integration with langchain4j.

Is your feature request related to a problem? Please describe.
Hello, this is the best java LLM framework I have ever used, I have been using and learning it lately.Thank you for your dedication.

But now I have a problem, MessageWindowChatMemory seems to be put in memory, every time my app is restarted, it will be lost. I tried serializing the MessageWindowChatMemory into redis for example, but this class doesn't have a default constructor, so it fails, and even if I add a default constructor to this class, the ChatMessage class and its children need to add a default constructor only then.

Describe the solution you'd like
If it is convenient, is it possible to add default constructors to these classes, or do you have any better ideas?

Describe alternatives you've considered
Is it possible to provide a method to store the chat history, allowing the user to overwrite it

Additional context
nothing

Is there a plan to support the llama2 model, or what are the best practices for extending the llama2 model on your own?

Is your feature request related to a problem? Please describe.
I want to provide context to a model that uses the prompt/completion pattern as follows:

{"prompt": "burger -->", "completion": " edible"}
{"prompt": "paper towels -->", "completion": " inedible"}
{"prompt": "vino -->", "completion": " edible"}
{"prompt": "bananas -->", "completion": " edible"}
{"prompt": "dog toy -->", "completion": " inedible"}

ref: https://docs.google.com/document/d/1rqj7dkuvl7Byd5KQPUJRxc19BJt8wo0yHNwK84KfU3Q/edit

Describe the solution you'd like
A dedicated class in the model.openai package would be great. Some instructions on how to do this or an example would help in the short term.

Describe alternatives you've considered
I've looked at all the classes in langchain4j and can't find anything related to this. Perhaps I've simply missed it. I've also searched a lot on Google.

This library has some support for this pattern:
https://github.com/TheoKanning/openai-java

It uses a FineTuneRequest class which points to a JSON Lines document uploaded to openai (note the above example is in that format). My fallback is to use this library together with langchain4j, but I would prefer to use langchain4j for everything if possible.

Additional context
Great job on this library. It is well written and quite easy to understand in most classes.