Giter VIP home page Giter VIP logo

java-client's Introduction

Qdrant ย  Java

Java library for the Qdrant vector search engine.

Javadoc Tests Apache 2.0 License Discord Roadmap 2024

Qdrant Java Client

Java client library with handy utility methods and overloads for interfacing with Qdrant.

๐Ÿ“ฅ Installation

Important

Requires Java 8 or above.

To install the library, add the following lines to your build config file.

Maven

<dependency>
  <groupId>io.qdrant</groupId>
  <artifactId>client</artifactId>
  <version>1.10.0</version>
</dependency>

SBT

libraryDependencies += "io.qdrant" % "client" % "1.10.0"

Gradle

implementation 'io.qdrant:client:1.10.0'

Note

Please make sure to include all necessary dependencies listed here in your project.

๐Ÿ“– Documentation

๐Ÿ”Œ Getting started

Creating a client

A client can be instantiated with

QdrantClient client = 
  new QdrantClient(QdrantGrpcClient.newBuilder("localhost").build());

which creates a client that will connect to Qdrant on https://localhost:6334.

Internally, the high level client uses a low level gRPC client to interact with Qdrant. Additional constructor overloads provide more control over how the gRPC client is configured. The following example configures a client to use TLS, validating the certificate using the root CA to verify the server's identity instead of the system's default, and also configures API key authentication:

ManagedChannel channel = Grpc.newChannelBuilder(
  "localhost:6334",
  TlsChannelCredentials.newBuilder()
    .trustManager(new File("ssl/ca.crt"))
    .build())
.build();

QdrantClient client = new QdrantClient(
  QdrantGrpcClient.newBuilder(channel)
    .withApiKey("<apikey>")
    .build());

The client implements AutoCloseable, though a client will typically be created once and used for the lifetime of the application. When a client is constructed by passing a ManagedChannel, the client does not shut down the channel on close by default. The client can be configured to shut down the channel on close with

ManagedChannel channel = Grpc.newChannelBuilder(
  "localhost:6334", 
  TlsChannelCredentials.create())
.build();

QdrantClient client = new QdrantClient(
  QdrantGrpcClient.newBuilder(channel, true)
    .withApiKey("<apikey>")
    .build());

All client methods return ListenableFuture<T>.

Working with collections

Once a client has been created, create a new collection

client.createCollectionAsync("my_collection",
  VectorParams.newBuilder()
    .setDistance(Distance.Cosine)
    .setSize(4)
    .build())
  .get();

Insert vectors into a collection

// import static convenience methods
import static io.qdrant.client.PointIdFactory.id;
import static io.qdrant.client.ValueFactory.value;
import static io.qdrant.client.VectorsFactory.vectors;

List<PointStruct> points =
    List.of(
        PointStruct.newBuilder()
            .setId(id(1))
            .setVectors(vectors(0.32f, 0.52f, 0.21f, 0.52f))
            .putAllPayload(
                Map.of(
                    "color", value("red"),
                    "rand_number", value(32)))
            .build(),
        PointStruct.newBuilder()
            .setId(id(2))
            .setVectors(vectors(0.42f, 0.52f, 0.67f, 0.632f))
            .putAllPayload(
                Map.of(
                    "color", value("black"),
                    "rand_number", value(53),
                    "extra_field", value(true)))
            .build());

UpdateResult updateResult = client.upsertAsync("my_collection", points).get();

Search for similar vectors

List<ScoredPoint> anush =
    client
        .searchAsync(
            SearchPoints.newBuilder()
                .setCollectionName("my_collection")
                .addAllVector(List.of(0.6235f, 0.123f, 0.532f, 0.123f))
                .setLimit(5)
                .build())
        .get();

Search for similar vectors with filtering condition

// import static convenience methods
import static io.qdrant.client.ConditionFactory.range;

List<ScoredPoint> points = client.searchAsync(SearchPoints.newBuilder()
  .setCollectionName("my_collection")
  .addAllVector(List.of(0.6235f, 0.123f, 0.532f, 0.123f))
  .setFilter(Filter.newBuilder()
    .addMust(range("rand_number", Range.newBuilder().setGte(3).build()))
    .build())
  .setLimit(5)
  .build()
).get();

โš–๏ธ LICENSE

Apache 2.0 ยฉ 2024

java-client's People

Contributors

anush008 avatar eddumelendez avatar generall avatar russcam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

java-client's Issues

Unable to create a Native image (GraalVM) using Qdrant Java libraries.

The Qdrant Java client libraries rely on the grpc-java project (https://github.com/grpc/grpc-java), which lacks support for native compilation due to a shaded Netty dependency.
This issue is widespread and affects multiple Java libraries.

An issue was filed in the grpc-java project about this, but eventually closed as not planned: grpc/grpc-java#10601.

It would be good to find a general solution for this.

GraalVM version used:

native-image --version
native-image 21.0.3 2024-04-16
GraalVM Runtime Environment Liberica-NIK-23.1.3-1 (build 21.0.3+10-LTS)
Substrate VM Liberica-NIK-23.1.3-1 (build 21.0.3+10-LTS, serial gc)

Errors while trying to compile a native image:

11:07:23.637 [ForkJoinPool-2-worker-2] ERROR io.grpc.netty.shaded.io.netty.handler.ssl.BouncyCastleAlpnSslUtils -- Unable to initialize BouncyCastleAlpnSslUtils.
java.lang.ClassNotFoundException: org.bouncycastle.jsse.BCSSLEngine
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.NativeImageClassLoader.loadClass(NativeImageClassLoader.java:637)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
	at java.base/java.lang.Class.forName0(Native Method)
	at java.base/java.lang.Class.forName(Class.java:421)
	at java.base/java.lang.Class.forName(Class.java:412)
	at io.grpc.netty.shaded.io.netty.handler.ssl.BouncyCastleAlpnSslUtils.<clinit>(BouncyCastleAlpnSslUtils.java:63)
	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)
	at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized(Unsafe.java:1160)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.ensureClassInitialized(ClassInitializationSupport.java:177)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ProvenSafeClassInitializationSupport.computeInitKindAndMaybeInitializeClass(ProvenSafeClassInitializationSupport.java:348)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ProvenSafeClassInitializationSupport.computeInitKindAndMaybeInitializeClass(ProvenSafeClassInitializationSupport.java:76)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.maybeInitializeAtBuildTime(ClassInitializationSupport.java:161)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.classinitialization.ClassInitializationSupport.maybeInitializeAtBuildTime(ClassInitializationSupport.java:150)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.SVMHost.isInitialized(SVMHost.java:318)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisType.isInitialized(AnalysisType.java:911)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.maybeEagerlyInitialize(BytecodeParser.java:4423)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1679)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.genInvokeStatic(BytecodeParser.java:1672)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBytecode(BytecodeParser.java:5416)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.iterateBytecodesForBlock(BytecodeParser.java:3426)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.iterateBytecodesForBlock(SharedGraphBuilderPhase.java:743)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.handleBytecodeBlock(BytecodeParser.java:3386)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.processBlock(BytecodeParser.java:3228)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.build(BytecodeParser.java:1137)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase$SharedBytecodeParser.build(SharedGraphBuilderPhase.java:162)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.BytecodeParser.buildRootMethod(BytecodeParser.java:1029)
	at jdk.internal.vm.compiler/org.graalvm.compiler.java.GraphBuilderPhase$Instance.run(GraphBuilderPhase.java:101)
	at org.graalvm.nativeimage.builder/com.oracle.svm.hosted.phases.SharedGraphBuilderPhase.run(SharedGraphBuilderPhase.java:116)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.run(Phase.java:49)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:434)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
	at jdk.internal.vm.compiler/org.graalvm.compiler.phases.Phase.apply(Phase.java:38)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.AnalysisParsedGraph.parseBytecode(AnalysisParsedGraph.java:146)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.parseGraph(AnalysisMethod.java:895)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsedHelper(AnalysisMethod.java:860)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.meta.AnalysisMethod.ensureGraphParsed(AnalysisMethod.java:843)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.parse(MethodTypeFlowBuilder.java:186)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlowBuilder.apply(MethodTypeFlowBuilder.java:621)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.createFlowsGraph(MethodTypeFlow.java:167)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.ensureFlowsGraphCreated(MethodTypeFlow.java:153)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.MethodTypeFlow.getOrCreateMethodFlowsGraphInfo(MethodTypeFlow.java:111)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.typestate.DefaultVirtualInvokeTypeFlow.onObservedUpdate(DefaultVirtualInvokeTypeFlow.java:114)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.flow.TypeFlow.lambda$addObserver$0(TypeFlow.java:475)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.executeCommand(CompletionExecutor.java:187)
	at org.graalvm.nativeimage.pointsto/com.oracle.graal.pointsto.util.CompletionExecutor.lambda$executeService$0(CompletionExecutor.java:171)
	at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1423)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
	at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
	at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
	at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
	at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)
[2/8] Performing analysis...  []                                                                       (324.2s @ 3.61GB)
   43,829 reachable types   (89.1% of   49,183 total)
   81,782 reachable fields  (65.2% of  125,390 total)
  218,772 reachable methods (61.0% of  358,401 total)
   12,695 types, 1,137 fields, and 10,037 methods registered for reflection
       12 native libraries: -framework Accelerate, -framework Carbon, -framework Cocoa, -framework CoreServices, -framework JavaRuntimeSupport, -framework Metal, -framework OpenGL, -framework QuartzCore, -framework Security, m, stdc++

Error: Classes that should be initialized at run time got initialized during image building:
 org.apache.commons.logging.LogFactory was unintentionally initialized at build time. To see why org.apache.commons.logging.LogFactory got initialized use --trace-class-initialization=org.apache.commons.logging.LogFactory
ch.qos.logback.core.util.StatusPrinter was unintentionally initialized at build time. To see why ch.qos.logback.core.util.StatusPrinter got initialized use --trace-class-initialization=ch.qos.logback.core.util.StatusPrinter
io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator'). To see why io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator
io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf'). To see why io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf
ch.qos.logback.core.util.Loader was unintentionally initialized at build time. To see why ch.qos.logback.core.util.Loader got initialized use --trace-class-initialization=ch.qos.logback.core.util.Loader
ch.qos.logback.core.status.StatusBase was unintentionally initialized at build time. To see why ch.qos.logback.core.status.StatusBase got initialized use --trace-class-initialization=ch.qos.logback.core.status.StatusBase
io.grpc.netty.shaded.io.netty.handler.ssl.PemValue the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted). To see why io.grpc.netty.shaded.io.netty.handler.ssl.PemValue got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.handler.ssl.PemValue
ch.qos.logback.classic.Level was unintentionally initialized at build time. To see why ch.qos.logback.classic.Level got initialized use --trace-class-initialization=ch.qos.logback.classic.Level
org.slf4j.LoggerFactory was unintentionally initialized at build time. To see why org.slf4j.LoggerFactory got initialized use --trace-class-initialization=org.slf4j.LoggerFactory
io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf
ch.qos.logback.core.CoreConstants was unintentionally initialized at build time. To see why ch.qos.logback.core.CoreConstants got initialized use --trace-class-initialization=ch.qos.logback.core.CoreConstants
ch.qos.logback.core.status.InfoStatus was unintentionally initialized at build time. To see why ch.qos.logback.core.status.InfoStatus got initialized use --trace-class-initialization=ch.qos.logback.core.status.InfoStatus
io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted). To see why io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey
io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator'). To see why io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator
org.apache.commons.logging.impl.SLF4JLogFactory was unintentionally initialized at build time. To see why org.apache.commons.logging.impl.SLF4JLogFactory got initialized use --trace-class-initialization=org.apache.commons.logging.impl.SLF4JLogFactory
io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf
io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf
io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-buffer/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil'). To see why io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil
ch.qos.logback.classic.Logger was unintentionally initialized at build time. To see why ch.qos.logback.classic.Logger got initialized use --trace-class-initialization=ch.qos.logback.classic.Logger
io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted the class was requested to be initialized at run time (from 'META-INF/native-image/io.grpc.netty.shaded.io.netty/netty-common/native-image.properties' in 'file:///Users/sagar/.m2/repository/io/grpc/grpc-netty-shaded/1.59.0/grpc-netty-shaded-1.59.0.jar' with 'io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted'). To see why io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted
io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf
io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf the class was requested to be initialized at run time (subtype of io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf). To see why io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf got initialized use --trace-class-initialization=io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf
To see how the classes got initialized, use --trace-class-initialization=org.apache.commons.logging.LogFactory,ch.qos.logback.core.util.StatusPrinter,io.grpc.netty.shaded.io.netty.buffer.ByteBufAllocator,io.grpc.netty.shaded.io.netty.buffer.UnpooledUnsafeDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.AbstractReferenceCountedByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledByteBuf,ch.qos.logback.core.util.Loader,ch.qos.logback.core.status.StatusBase,io.grpc.netty.shaded.io.netty.handler.ssl.PemValue,ch.qos.logback.classic.Level,org.slf4j.LoggerFactory,io.grpc.netty.shaded.io.netty.buffer.UnpooledHeapByteBuf,ch.qos.logback.core.CoreConstants,ch.qos.logback.core.status.InfoStatus,io.grpc.netty.shaded.io.netty.handler.ssl.PemPrivateKey,io.grpc.netty.shaded.io.netty.buffer.UnpooledDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledByteBufAllocator,org.apache.commons.logging.impl.SLF4JLogFactory,io.grpc.netty.shaded.io.netty.buffer.AbstractPooledDerivedByteBuf,io.grpc.netty.shaded.io.netty.buffer.UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeDirectByteBuf,io.grpc.netty.shaded.io.netty.buffer.ByteBufUtil,ch.qos.logback.classic.Logger,io.grpc.netty.shaded.io.netty.util.AbstractReferenceCounted,io.grpc.netty.shaded.io.netty.buffer.PooledSlicedByteBuf,io.grpc.netty.shaded.io.netty.buffer.PooledUnsafeDirectByteBuf
------------------------------------------------------------------------------------------------------------------------
                       82.6s (8.0% of total time) in 147 GCs | Peak RSS: 7.76GB | CPU load: 1.12
========================================================================================================================
Finished generating 'data-loader' in 5m 57s.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE

Vector value seem rounded

Hello,
I'm currently trying to add a QDrantEmbeddingStore to Lanchain4J (https://github.com/langchain4j) but I have a problem.
IT with testContainer keep failing because some Vector values seem rounded :
expected: Embedding { vector = [-0.086097434, 0.033505224, 0.017902793, ...
but was: Embedding { vector = [-0.08609743, 0.03350522, 0.017902792, ...

I don't know if it can be related to quantization, I did'nt enable quantization (but maybe it is enabled by default)
If anyone have a hint that would be appreciated.
Thanks

create collection fail when use java-client

Using the Qdrant Cloud Free tier

my code is ๏ผš
client.createCollectionAsync("MyCollecton", Collections.VectorParams.newBuilder() .setDistance(Collections.Distance.Cosine).setSize(1536).build()).get();

and the following exception was thrown after execution.


io.qdrant.client.QdrantClient : Create collection operation failed

io.grpc.StatusRuntimeException: UNIMPLEMENTED: HTTP status code 404
invalid content-type: text/plain; charset=utf-8
headers: Metadata(:status=404,content-type=text/plain; charset=utf-8,x-content-type-options=nosniff,date=Mon, 13 May 2024 10:30:18 GMT,content-length=19)
DATA-----------------------------
404 page not found

at io.grpc.Status.asRuntimeException(Status.java:537)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
... 1 common frames omitted

Is there something i missed?

Maven
io.grpc 1.59.1
io.qdrant 1.9.0
com.google.protobuf 3.24.0
guava 33.2.0-jre

Can not create Payload Index with datetime Type

Now I can not create Payload Index with datatime Type using javaclient, because of the following code.

public ListenableFuture<UpdateResult> createPayloadIndexAsync(
		String collectionName,
		String field,
		PayloadSchemaType schemaType,
		@Nullable PayloadIndexParams indexParams,
		@Nullable Boolean wait,
		@Nullable WriteOrderingType ordering,
		@Nullable Duration timeout
	) {
		CreateFieldIndexCollection.Builder requestBuilder = CreateFieldIndexCollection.newBuilder()
			.setCollectionName(collectionName)
			.setFieldName(field)
			.setWait(wait == null || wait);

		switch (schemaType) {
			case Keyword:
				requestBuilder.setFieldType(FieldType.FieldTypeKeyword);
				break;
			case Integer:
				requestBuilder.setFieldType(FieldType.FieldTypeInteger);
				break;
			case Float:
				requestBuilder.setFieldType(FieldType.FieldTypeFloat);
				break;
			case Geo:
				requestBuilder.setFieldType(FieldType.FieldTypeGeo);
				break;
			case Text:
				requestBuilder.setFieldType(FieldType.FieldTypeText);
				break;
			case Bool:
				requestBuilder.setFieldType(FieldType.FieldTypeBool);
				break;
			default:
				throw new IllegalArgumentException("Invalid schemaType: '" + schemaType + "'");
		}

Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS

Hi. I run my qdrant service by docker and without ssl. when operate with java client it throws NotSslRecordException exception but python client is ok. How can I use java client without ssl? (because my product service did not use ssl)

--------------------------this java code does not work-----------------------------
@test
public void testCreateCollection() {
try {
QdrantClient client = new QdrantClient(QdrantGrpcClient.newBuilder("localhost").build());
Collections.CollectionOperationResponse response = client.createCollectionAsync("my_collection",
Collections.VectorParams.newBuilder()
.setDistance(Collections.Distance.Cosine)
.setSize(4)
.build()).get();
} catch (Exception e) {
//here catch io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS
e.printStackTrace();
}
}

--------------------------this python code works-----------------------------
from qdrant_client import QdrantClient
from qdrant_client.http.models import Distance, VectorParams

client = QdrantClient(host="127.0.0.1", port=6333, grpc_port=6334, prefer_grpc=False)
client.create_collection(
collection_name="test_collection1",
vectors_config=VectorParams(size=4, distance=Distance.DOT),
)

Cannot set shardKey when deleting vectors

For some reason, I want to delete the vectors.

Here is the code.

 qdrantClient.deleteVectorsAsync(collectionName, vectors, ids).get()

I get below error message.

io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Wrong input: No shards found for shard key
	at io.grpc.Status.asRuntimeException(Status.java:533)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
2024-02-05 13:16:38.540 [] ERROR i.q.c.QdrantClient[2628] -- Delete vectors operation failed

But I didn't find how to set shardKey. Anyone know it?

Http client instead of grpc

I am currently working on a project that requires the use of an HTTP client, and I have already done the implementation using langchain4j. However, I noticed that httpclient is not currently included in the library, and I wanted to inquire if there are any plans to include it in the future.

If it is not planned to be included, I would be grateful if you could recommend an alternative HTTP client that would work well with qdrant. I appreciate any insight or guidance you can provide in this regard.

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

I add the pom and write a test case to search points. And replace io.grpc with version 1.60.1 .

         <dependency>
            <groupId>io.qdrant</groupId>
            <artifactId>client</artifactId>
            <version>1.7.0</version>
            <exclusions>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-core</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-stub</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-protobuf</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-services</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>io.grpc</groupId>
                    <artifactId>grpc-netty-shaded</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>com.google.guava</groupId>
                    <artifactId>guava</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>33.0.0-jre</version>
        </dependency>

But I got the error as below.

Handler dispatch failed: java.lang.NoSuchMethodError: 'com.google.protobuf.Internal$ProtobufList io.qdrant.client.grpc.Points$Vector.makeMutableCopy(com.google.protobuf.Internal$ProtobufList)'

Search code

            Points.SearchPoints.Builder searchBuilder = Points.SearchPoints.newBuilder();
            searchBuilder.setCollectionName(COLLECTION_NAME)
                    .addAllVector(vector.stream().map(Double::floatValue).toList())
                    .setShardKeySelector(Points.ShardKeySelector.newBuilder()
                            .addShardKeys(Collections.ShardKey.newBuilder()
                                    .setKeyword(SHARD_KEY)
                                    .build())
                            .build())
                    .setLimit(topK)
                    .setScoreThreshold((float) minSimilarity)
                    .setWithPayload(Points.WithPayloadSelector.newBuilder()
                            .setEnable(true)
                            .build());

Default demo does not work

I am following the example in the readme.md running against a local docker instance of Quadrant. My code throws an exception as soon as I fire a command, e.g. create a collection. I see the following message:


java.util.concurrent.ExecutionException: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]

	at com.google.common.util.concurrent.AbstractFuture.getDoneValue(AbstractFuture.java:566)
	at com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:547)
	at com.google.common.util.concurrent.FluentFuture$TrustedFuture.get(FluentFuture.java:88)
	at qdrant.CrudTest.createCollection(CrudTest.java:31)


...

Caused by: io.grpc.StatusRuntimeException: UNAVAILABLE: io exception
Channel Pipeline: [SslHandler#0, ProtocolNegotiators$ClientTlsHandler#0, WriteBufferingAndExceptionHandler#0, DefaultChannelPipeline$TailContext#0]
	at io.grpc.Status.asRuntimeException(Status.java:537)
	at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:538)
	at io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:489)
	at io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:453)
	at io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:486)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:574)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:72)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:742)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:723)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: io.grpc.netty.shaded.io.netty.handler.ssl.NotSslRecordException: not an SSL/TLS record: 000012040000000000000400100000000500004000000601000000
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decodeJdkCompatible(SslHandler.java:1308)
	at io.grpc.netty.shaded.io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1378)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468)
	at io.grpc.netty.shaded.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)

My code looks like

 QdrantGrpcClient lowLevelClient = QdrantGrpcClient
                .newBuilder("localhost")
                .build();
 client = new QdrantClient(lowLevelClient);

 Collections.VectorParams params = Collections.VectorParams
                .newBuilder()
                .setDistance(Collections.Distance.Cosine)
                .setSize(dim)
                .build();

Collections.CollectionOperationResponse response = client
                .createCollectionAsync(name, params)
                .get();

Support for recovery via snapshot

Unless I'm missing it somewhere, I don't see support for collection recovery via a snapshot. Is this support planned? Ideally it would support both reading the snapshot from a URI or uploading the snapshot via HTTP multi-part.

Allow custom strings as IDs

Hi,

it seems only UUID and Long are supported as Point IDs. I need however to bring my own Strings as ids. What is the best solution here?

Thanks

I had to add "implementation("com.google.protobuf:protobuf-java:3.24.0")"

I use kotlin. I had to add line implementation("com.google.protobuf:protobuf-java:3.24.0") to my build.gradle.kts to make qdrant java-client works.

Now I have this 2 lines:

implementation("io.qdrant:client:1.9.1")
implementation("com.google.protobuf:protobuf-java:3.24.0")

Secund think, I had to use exacly version 3.24.0, witch is pointed in https://github.com/qdrant/java-client/blob/master/build.gradle#L81

Maybe it because kotlin, but I think it is worth to say it in documentation.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.