Giter VIP home page Giter VIP logo

cosmosdb-server's Introduction

Vercel

Develop. Preview. Ship.

Documentation · Changelog · Templates · CLI


Vercel

Vercel’s Frontend Cloud provides the developer experience and infrastructure to build, scale, and secure a faster, more personalized Web.

Deploy

Get started by importing a project or using the Vercel CLI. Then, git push to deploy.

Documentation

For details on how to use Vercel, check out our documentation.

Contributing

This project uses pnpm to install dependencies and run scripts.

You can use the dev script to run local changes as if you were invoking Vercel CLI. For example, vercel deploy --cwd=/path/to/project could be run with local changes with pnpm dev deploy --cwd=/path/to/project.

See the Contributing Guidelines for more details.

Reference

cosmosdb-server's People

Contributors

andremaz avatar asleire avatar craigandrews avatar dependabot[bot] avatar dglsparsons avatar dsschneidermann avatar javivelasco avatar lonewarrior556 avatar lossyrob avatar mglagola avatar nkzawa avatar ofhouse avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cosmosdb-server's Issues

Cannot delete database without exception

It appears that the delete-database-endpoint returns nothing (empty response), which is unexpected (at least to the Node SDK). Test case to reproduce shown below.

--- a/test/index.ts
+++ b/test/index.ts
@@ -97,3 +97,9 @@ export const udf = withTestEnv(async client => {
     .fetchAll();
   assert.deepStrictEqual(resources, [true]);
 });
+
+export const deleteDatabase = withTestEnv(async client => {
+  await client.databases.create({ id: "test-database" });
+
+  await assert.doesNotReject(() => client.database("test-database").delete());
+});

When querying from a azure function using input binding, a query plan request is made that is not mapped

When creating a function app and a function with cosmosdb input binding and a sqlquery,
two post requests are made to "/dbs/:dbId/colls/:collId/docs", one of which has the "x-ms-cosmos-is-query-plan-request: True" header.
The Azure function expects a specific response from this request, after which the actual query request will be made.
The expected resposne looks like this:

{
    "partitionedQueryExecutionInfoVersion": "2",
    "queryInfo": {
        "distinctType": "None",
        "top": null,
        "offset": null,
        "limit": null,
        "orderBy": [],
        "orderByExpressions": [],
        "groupByExpressions": [],
        "groupByAliases": [],
        "aggregates": [],
        "groupByAliasToAggregateType": {},
        "rewrittenQuery": "",
        "hasSelectValue": false,
        "dCountInfo": null
    },
    "queryRanges": [
        {
            "min": "05C1A3C5D33F20083200",
            "max": "05C1A3C5D33F20083200",
            "isMinInclusive": true,
            "isMaxInclusive": true
        }
    ]
}

with attribute values depending on the specific query, for instance if "order by" is added, the attributes
orderBy and orderByExpressions will contain "Ascending/Descending" and the column names respectively

Invalid ResourceId's, cannot use with Java SDK

Hello, members of Zeit

With the official CosmosDB emulator not running on non-windows, this is more or less exactly what I'm looking for. I was however not able to use it together with the Java SDK.

It seems like the Node SDK is quite lenient and performs little to no validation on the resource ids*. Hence, generating RID's like you do probably work (I haven't actually tested your implementation with the Node SDK, but I assume it does given the readme etc).

The Java SDK however is a bit more strict and assigns much more meaning to them, as shown here. In addition to be accepted by the parser, they also need to successfully serialize back to the original string.

Since your implementation only generates a quite random ID (of 14 bytes?), the Java SDK will fail on inserting documents, as shown below.

java.lang.IllegalArgumentException: Invalid resource id PEtvD0--C30lC30lC30lC0--
	at com.microsoft.azure.documentdb.internal.ResourceId.parse(ResourceId.java:57)
	at com.microsoft.azure.documentdb.internal.routing.CollectionCache.resolveByRid(CollectionCache.java:116)
	at com.microsoft.azure.documentdb.internal.routing.CollectionCache.resolveCollection(CollectionCache.java:49)
	at com.microsoft.azure.documentdb.internal.SessionContainer.resolvePartitionKeyRange(SessionContainer.java:217)
	at com.microsoft.azure.documentdb.internal.SessionContainer.resolveSessionToken(SessionContainer.java:123)
	at com.microsoft.azure.documentdb.DocumentClient.applySessionToken(DocumentClient.java:3222)
	at com.microsoft.azure.documentdb.DocumentClient.doQuery(DocumentClient.java:3143)
	at com.microsoft.azure.documentdb.DocumentQueryClientInternal.doQuery(DocumentQueryClientInternal.java:47)
	at com.microsoft.azure.documentdb.internal.query.AbstractQueryExecutionContext.executeRequest(AbstractQueryExecutionContext.java:219)
	at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.executeOnce(DefaultQueryExecutionContext.java:159)
	at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.fillBuffer(DefaultQueryExecutionContext.java:99)
	at com.microsoft.azure.documentdb.internal.query.DefaultQueryExecutionContext.hasNext(DefaultQueryExecutionContext.java:71)
	at com.microsoft.azure.documentdb.internal.query.ProxyQueryExecutionContext.<init>(ProxyQueryExecutionContext.java:67)
	at com.microsoft.azure.documentdb.internal.query.QueryExecutionContextFactory.createQueryExecutionContext(QueryExecutionContextFactory.java:23)
	at com.microsoft.azure.documentdb.QueryIterable.createQueryExecutionContext(QueryIterable.java:70)
	at com.microsoft.azure.documentdb.QueryIterable.reset(QueryIterable.java:115)
	at com.microsoft.azure.documentdb.QueryIterable.<init>(QueryIterable.java:57)
	at com.microsoft.azure.documentdb.DocumentClient.queryDocuments(DocumentClient.java:1167)
	at com.microsoft.azure.documentdb.DocumentClient.queryDocuments(DocumentClient.java:1138)

* The source linked is of a quite old version. I can't seem to find the source for the most recent version of the Java SDK anywhere, reported here. This is obviously quite unfortunate, who knows what the SDK is doing these days.

Support for C# SDK v3

The C# SDK currently does not work with this CosmosDb server.
NuGet package: https://www.nuget.org/packages/Microsoft.Azure.Cosmos/

I discovered the following issues while attempting to make it work:

  • SDK can not be configured to ignore SSL validation errors. My workaround was using reflection to configure the SDK internals
  • Addresses endpoint as in #29 is not implemented. Workaround is configuring the C# SDK to use Gateway mode.
  • Whenever an error occurs (>399 status code), the C# SDK expects a case sensitive message property with non-null value in the response body. Otherwise it will throw a NullReferenceException.
  • "Meta" / root endpoint must return certain data, otherwise SDK will throw a NullReferenceException. Additionally the queryEngineConfiguration property is required for querying to work.
  • The C# parses and validates the _rid property. In particular, it requires RID for collections to have a specific bit set to 1 (first bit of 5th byte)
  • A property on partitionKeyRanges is invalid, it is maxInclusive but should be maxExclusive
  • C# SDK sends boolean headers with a "True" value instead of "true", causing the server to not handle them as true
  • C# SDK will query partitionKeys using an "Incremental Feed". See A-IM header. Essentially it expects an etag when retrieving partition keys, and it expects a 302 status code once all keys have been received. Current behavior causes an infinite loop in the C# SDK

After forking this repo and applying fixes for the above I am able to successfully use this server with the C# SDK.

I noticed you have a current draft PR with some fixes for the Java SDK, which will fix some of the issues here. I can make a PR with the remaining fixes if you'd like

Support for Java client

Seems java client is stricter than node one and doesn't work properly.
Maybe we want to add integration tests with the client and ensure it works.

#7 (comment)

Query to //addresses/?$resolveFor=dbs&$filter=protocol%20eq%20rntbd

The code Lib I am using makes a query to
//addresses/?$resolveFor=dbs&$filter=protocol%20eq%20rntbd when it launches..

it expects Json looking like this:

{
"Addresss":[
{
"isPrimary":true,
"physcialUri":"rntbd://foo.documents.azure.com:11018/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312777763206824p/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11316/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312759686602427s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11011/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312759686602428s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
},
{
"isPrimary":false,
"physcialUri":"rntbd://foo.documents.azure.com:11009/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312804907485886s/",
"protocol":"rntbd",
"partitionKeyRangeId":"M",
"partitionIndex":"0@0"
}
],
"_count":4
}

without the $filter=protocol%20eq%20rntbd it includes https ie

{"isPrimary":true,
  "physcialUri":"https://foo.documents.azure.com:11618/apps/16d6d56e-e73d-4069-98f5-2e677328a7d1/services/ce59e2ac-eac5-490f-9262-585b250f0399/partitions/2fe30e96-a74a-4596-a674-2c04a39c8603/replicas/132312777763206824p//",
  "protocol":"https",
  "partitionKeyRangeId":"M",
  "partitionIndex":"0@0"}

first of all wth is rntbd protocol, can it be mocked?

But the issue is that with out //addresses/blah returning anything, The Lib crashes parsing non existent json.
Can this be added to routes?

Throws when comparing documents

This is a SQL query I'm looking at:

SELECT * FROM c WHERE c.foo.bar = "xyz"

In this collection we have some documents with foo field and for some other's it's undefined.

Then when we ran this query, client give us an error and the server reports this:

TypeError: Cannot read property 'bar' of undefined
    at $h.paginate.$h.sort.$c.reduce.filter (eval at exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:8:21), <anonymous>:8:83)
    at Array.filter (<anonymous>)
    at eval (eval at exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:8:21), <anonymous>:6:15)
    at Object.exports.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/executor.js:13:12)
    at Query.exec (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-query/lib/index.js:29:34)
    at Documents.query (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-server/lib/account/items.js:53:55)
    at _read_items_1.default (/data/zeit/api/services/cosmos-local/node_modules/@zeit/cosmosdb-server/lib/handler/query-documents.js:24:33)
    at process._tickCallback (internal/process/next_tick.js:68:7)

Which key shoud i use?

Hi!

When i use test key as my dummy key I'm getting encode error, but when i use a valid key like emulator key the connection is refused.

I'm using python to test the connection.

Thanks in advance

Is this compatible with CosmosClient on an ASP.net Core application?

I am trying to use the cosmosdb-server for testing an ASP.net core application, but I always get an exception when trying to interact with the server:
System.Net.Http.HttpRequestException : The SSL connection could not be established, see inner exception. ---- System.Security.Authentication.AuthenticationException : The remote certificate is invalid according to the validation procedure.

Did I do something wrong?
I did not find a way to disable SSL certificate verification for CosmosClient.
I know, that the older DocumentClient has an option to pass a custom handler to ignore the SSL certificate, but there does not seem to be such an option for the CosmosClient.

Docker image

Any interest in publishing a docker image for this? I'm using this Dockerfile in an integration testing story via docker-compose and it appears to be working great

Upsert responding with "Resource with specified id or name already exists."

Hi,

we try to integrate your project within our CI/CD build pipeline. One test uses an Upsert command (2nd) to update an existing entry, which was previously added with a first upsert command (1st).

When the 2nd Upsert statement is issued, we get this reply:

{HTTPFailure}Status code: 409
{"code":"Conflict","message":"Resource with specified id or name already exists."}

As far as I understand, Upsert should NEVER return an error like "already exists" since the entry should just be overwritten or inserted. The test runs fine when running it agains a local installation of the Azure CosmosDB Emulator (https://docs.microsoft.com/de-de/azure/cosmos-db/local-emulator).

I guess this is a bug?

Feature request: add externalized configuration for certificates and logging

Hi guys. Thanks for this repo. I'm now using it succesfully to mock CosmosDb for the Azure IoT Device Simulation solution (https://github.com/Azure/device-simulation-dotnet/tree/DS-2.0.6).

I've set it up with docker-compose and have generated a self-signed CA certificate for the assigned hostname, and added it to the device simulation container, so when it connects using the Azure CSharp SDK, all SSL checks out and it just works(tm), aside from a few quirks that I had to look at (#46 and #48).

In doing this, I found a need for two main things:

  • Adding optional paths for an external cert and cert key to cosmosdb-server
  • Adding more default logging for the requests and responses

My solution was to take the index.js (after compilation) and provide a replacement instead, packaging it with the docker image.

I'm not a node wiz, by far, so my logging additions have just been hacked together from what I could find from examples online, here it is: https://gist.github.com/dsschneidermann/a9314afee146feaffbeada42829e2e73

I can also provide the Dockerfile, docker-compose.yml etc if anyone is interested.

Edit: I forgot, I'm also overriding the read-meta.js file in order to set the self-aware "databaseAccountEndpoint" address and switch the default consistency level to "Strong", as it's needed for my case. Gist: https://gist.github.com/dsschneidermann/952f9160d5ce08aa69908b0862dcb5c6

Error when a trigger is created

Hello,

I'm trying to create a trigger, but is returning a bad request.

Here it is my code

from pydocumentdb import document_client

DB_DATABASE = 'test_database2'
DB_COLLECTION = 'my_collection'
client = document_client.DocumentClient(
    'https://localhost:3000/', {'masterKey': ''})

print("deleting database")
try:
    db = next((data for data in client.ReadDatabases(
    ) if data['id'] == DB_DATABASE))
    client.DeleteDatabase(db['_self'])
except:
 pass

print("creating database")
db = client.CreateDatabase({'id': DB_DATABASE})

print("creating collection")
collection = client.CreateCollection(db['_self'], {'id': DB_COLLECTION})

path = 'dbs/{}/colls/{}'.format(DB_DATABASE, DB_COLLECTION)

# Create trigger
trigger_definition = {
    'id': '_type',
    'serverScript': """
        // Ensures that documents have a _type field.
        function validateNameExists() {
            var collection = getContext().getCollection();
            var request = getContext().getRequest();
            var docToCreate = request.getBody();

            // Reject documents that do not have a name property by throwing an exception.
            if (!docToCreate._type) {
                throw new Error('Document must include a "_type" property.');
            }
        }
    """,
    'triggerType': 'Pre',
    'triggerOperation': 'All'
}
try:
    client.CreateTrigger(collection['_self'], trigger_definition)
except Exception as e:
    print(e)

The request is created for
'https://localhost:3000//dbs/wgLLAG--C30lC30lC30lD0--/colls/V2u_QG--C30lC30lC30lC0--/triggers/'

With this header

<class 'dict'>: {'Cache-Control': 'no-cache', 'x-ms-version': '2016-07-11', 'User-Agent': 'Windows/10 Python/3.6.8 documentdb-python-sdk/2.0.0', 'x-ms-consistency-level': 'Session', 'x-ms-session-token': '', 'Content-Type': 'application/json', 'Accept': 'application/json'}

And with this body

<class 'dict'>: {'id': '_type', 'triggerType': 'Pre', 'triggerOperation': 'All', 'body': '\n        // Ensures that documents have a _type field.\n        function validateNameExists() {\n            var collection = getContext().getCollection();\n            var request = getContext().getRequest();\n            var docToCreate = request.getBody();\n\n            // Reject documents that do not have a name property by throwing an exception.\n            if (!docToCreate._type) {\n                throw new Error(\'Document must include a "_type" property.\');\n            }\n        }\n    '}```

Can't hit REST API

Hi,

I'm able to properly start the app (after installing with npm -g):

$ cosmosdb-server -p 3000
Ready to accept connections at [::]:3000

But I can't hit it:

$ curl localhost:3000
curl: (52) Empty reply from server

I'm on node v12.13.0 on RHEL7.

Do you know how I can proceed with this?

Bad Request when GET request includes document header x-ms-documentdb-isquery

This is from working on compatibility with the CosmosDb CSharp SDK

Error logged request:

{
   "method":"GET",
   "url":"/dbs/devicesimulation/colls/simulations/docs",
   "docdbHeaders":[
      "x-ms-documentdb-isquery",
      "True",
      "x-ms-documentdb-query-enablecrosspartition",
      "False",
      "x-ms-documentdb-query-iscontinuationexpected",
      "False"
   ],
   "response":{
      "statusCode":400,
      "statusMessage":"Bad Request",
      "headers":{
         "content-type":"application/json",
         "content-location":"https://cosmosdb-sim:3000/dbs/devicesimulation/colls/simulations/docs",
         "connection":"close",
         "x-ms-activity-id":"5879c77f-a5fe-432a-a4d2-4d9454658713",
         "x-ms-request-charge":"1"
      },
      "body":{
         "message":"no route"
      }
   },
   "errorMessage":null
}```

Expected result:
* Document is returned

Cause:
* Route lookup appends "_QUERY" when it sees that `x-ms-documentdb-isquery` is set.

Proposed fix by the associated PR:
* Restrict route lookup to only add _QUERY and _UPSERT for POST, where they are applicable

Support .NET client

Hello,

I started the server that way:

$ node lib/cli.js -p 8081
Ready to accept connections at [::]:8081

Then I installed the certificates so connection does not fail due to ssl error (If you are interesting I could write a PR for that, (the current cert.pem does not seems valid)

Then, when I try to connect to Cosmos DB using dotnet sdk, it's stuck

private const string ConnectionString = "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
var cosmosClient = new CosmosClientBuilder(ConnectionString)
	.Build();

So I ran it in with the debugger and notice those messages spamming in the debug console I can see it try to connect to another port

DocDBTrace Information: 0 : Marking endpoint https://localhost:3000/ unavailable for read
DocDBTrace Information: 0 : Current WriteEndpoints = (https://localhost:3000/) ReadEndpoints = (https://localhost:3000/)
DocDBTrace Information: 0 : Endpoint https://localhost:3000/ unavailable for Read added/updated to unavailableEndpoints with timestamp 12/20/2020 19:29:21
DocDBTrace Information: 0 : Resolving Master service address, forceMasterRefresh: False, currentMaster: 

This seems to be an hardcoded value from https://github.com/vercel/cosmosdb-server/blob/master/src/handler/read-meta.ts#L8

GROUP BY

Having trouble with GROUP BY in a query.

Query

const { resources } = await myContainer.items
    .query({
      query:
        'SELECT MAX(t.inc) AS inc FROM t WHERE t.userId=@userId GROUP BY t.orgId',
      parameters: [
        {
          name: '@userId',
          value: userId,
        },
      ],
    })
    .fetchAll();

Error

The above works just fine using a real Cosmos NoSQL server, but errors with this one with the following...

Error: Expected "!=", "%", "&", "*", "+", "-", "--", ".", "/", "<", "<<", "<=", "<>", "=", ">", ">=", ">>", ">>>", "?", "??", "AND", "BETWEEN", "IN", "OR", "ORDER", "[", "^", "|", "||", [ \t\n\r], or end of input but "G" found.

Versions

"@vercel/cosmosdb-server": "^0.14.0",
"@azure/cosmos": "^3.17.3",
Node: 18.12.1

Thanks

read-meta hardcoded to port 3000

The read-meta handler hardcodes port 3000, which is incorrect when running the server on another port. This causes the Python client to throw a connection refused exception after connecting to a server on another port:

from azure.cosmos import CosmosClient
cosmos = CosmosClient("https://localhost:32797", credential="fake")
 list(cosmos.list_databases())

This throws the exception:

 <urllib3.connection.HTTPSConnection object at 0x7fd48863dfd0>: Failed to establish a new connection: [Errno 111] Connection refused

due to the client trying to communicate with port 3000, because of the hard coded values.

This can be solved by passing server options to handlers, so that the port can be dynamically generated; this would be a lot of code changes so perhaps there's an easier way to avoid this?

Query triggers `Unexpected token '.'`

Looks like an internal error on the following query (went sent raw, untransformed):

SELECT
  COUNT(1)
FROM (
  SELECT
    c.name
  FROM c
  WHERE
    c.name = 'READY'
) AS c

How can i run it?

Hello!

I run npm install.
And now I'm trying to use cosmosdb-server -p 3000 but is giving command not found error

Am I doing something wrong?

Thanks!

Support for .NET Core EntityFramework client

I'm trying to use cosmosdb-server with the NuGet package Microsoft.EntityFrameworkCore.Cosmos version 3.1.0 and get a NullReferenceException during application startup when calling .Database.EnsureCreated():

System.NullReferenceException: Object reference not set to an instance of an object.
crit: Microsoft.AspNetCore.Hosting.Diagnostics[6]
      Application startup exception
System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.get_DefaultConsistencyLevel()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerCore cosmosContainerCore, Nullable`1 partitionKey, Stream streamPayload, Action`1 requestEnricher, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosClient.CreateDatabaseIfNotExistsAsync(String id, Nullable`1 throughput, RequestOptions requestOptions, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnceAsync(DbContext _, Object __, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnce(DbContext context, Object state)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExists()
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosDatabaseCreator.EnsureCreated()
   at Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade.EnsureCreated()
   at testclient.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, TestContext dbContext) in /home/martin/projects/cosmosdb-server/testclient/Startup.cs:line 34
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass13_0.<UseStartup>b__2(IApplicationBuilder app)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Azure.Cosmos.CosmosAccountServiceConfiguration.get_DefaultConsistencyLevel()
   at Microsoft.Azure.Cosmos.DocumentClient.GetInitializationTaskAsync(IStoreClientFactory storeClientFactory)
   at Microsoft.Azure.Cosmos.DocumentClient.EnsureValidClientAsync()
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.EnsureValidClientAsync(RequestMessage request)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(RequestMessage request, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler.SendAsync(Uri resourceUri, ResourceType resourceType, OperationType operationType, RequestOptions requestOptions, ContainerCore cosmosContainerCore, Nullable`1 partitionKey, Stream streamPayload, Action`1 requestEnricher, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.CosmosClient.CreateDatabaseIfNotExistsAsync(String id, Nullable`1 throughput, RequestOptions requestOptions, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnceAsync(DbContext _, Object __, CancellationToken cancellationToken)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExistsOnce(DbContext context, Object state)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.ExecuteImplementation[TState,TResult](Func`3 operation, Func`3 verifySucceeded, TState state)
   at Microsoft.EntityFrameworkCore.Storage.ExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosClientWrapper.CreateDatabaseIfNotExists()
   at Microsoft.EntityFrameworkCore.Cosmos.Storage.Internal.CosmosDatabaseCreator.EnsureCreated()
   at Microsoft.EntityFrameworkCore.Infrastructure.DatabaseFacade.EnsureCreated()
   at testclient.Startup.Configure(IApplicationBuilder app, IWebHostEnvironment env, TestContext dbContext) in /home/martin/projects/cosmosdb-server/testclient/Startup.cs:line 34
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.Invoke(Object instance, IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.ConfigureBuilder.<>c__DisplayClass4_0.<Build>b__0(IApplicationBuilder builder)
   at Microsoft.AspNetCore.Hosting.GenericWebHostBuilder.<>c__DisplayClass13_0.<UseStartup>b__2(IApplicationBuilder app)
   at Microsoft.AspNetCore.HostFilteringStartupFilter.<>c__DisplayClass0_0.<Configure>b__0(IApplicationBuilder app)
   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at testclient.Program.Main(String[] args) in /home/martin/projects/cosmosdb-server/testclient/Program.cs:line 16

It seems to expect some data about consistency levels being returned from Cosmos.

Minimal example can be created by:

  1. Creating a new ASP.NET Core project with dotnet 3.1: dotnet new web
  2. Creating empty DbContext:
public class TestContext : DbContext {
        public TestContext(DbContextOptions options) : base(options){}
}
  1. Configuring Cosmos DB in Startup and asking it to create a new DB.
public class Startup
    {
        public void ConfigureServices(IServiceCollection services) {
            services.AddDbContext<TestContext>(opt =>
            {
                opt.UseCosmos("https://localhost:8081", "ZHVtbXk=", "some-db");
            });
        }

        public void Configure(TestContext dbContext) {
            dbContext.Database.EnsureCreated();
        }
    }
  1. Running the example project: dotnet run

Is server do not support patch operation?

I've got no route error when functions with patch operation like below was triggered.


container
    .item('some_id')
    .patch([{
        op: 'add',
        path: '/some_paths',
        value: 'some_value'
    }])

Patch operation is this
https://learn.microsoft.com/en-us/azure/cosmos-db/partial-document-update

Full response was like this

{
      code: 400,
      body: { message: 'no route' },
      headers: {
        'content-type': 'application/json',
        'content-location': 'https://localhost:8081/dbs/next-auth/colls/users/docs/15b91394-2982-409f-8f71-024f4594872c',
        connection: 'close',
        'x-ms-activity-id': '8e4f9947-c8bc-4ba4-a624-42a169fbdb14',
        'x-ms-request-charge': '1',
        date: 'Wed, 01 Feb 2023 18:03:57 GMT',
        'content-length': '22',
        'x-ms-throttle-retry-count': 0,
        'x-ms-throttle-retry-wait-time-ms': 0
      },
      activityId: '8e4f9947-c8bc-4ba4-a624-42a169fbdb14'
    }

Project has no license file

I can't seem to find any detailed license information within this project. The package.json states that the license is MIT, but it would be good for confidence if this was corroborated by a LICENSE file in the project root. Would you mind adding one, or accepting one via a PR?

GetItemLinqQueryable ... ToFeedIterator invalid response from queryDocuments

Hi,

The route "/dbs/:dbId/colls/:collId/docs": queryDocuments, is not responding the expected body when using the following example with dotnet sdk

            var iterator = container.GetItemLinqQueryable<SomeObject>()
                .Where(x => ids.Contains(x.Id))
                .ToFeedIterator();

The responses received is (https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217)

{"Documents":[{"id":"1","someData":["a"],"_etag":"93f16095-8d53-4dc8-b8b7-11b9c29d0d60","_rid":"gAAGAIAAAAEBAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEBAAAAAAAAAA==/","_ts":1608932852},{"id":"2","someData":["b"],"_etag":"842c8917-e84b-4a92-99e8-4237daa2ff85","_rid":"gAAGAIAAAAECAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAECAAAAAAAAAA==/","_ts":1608932852},{"id":"3","someData":["c"],"_etag":"e8033da5-1761-48a7-b0dd-365fe1548a0b","_rid":"gAAGAIAAAAEDAAAAAAAAAA==","_self":"/dbs/gAAGAA==/colls/gAAGAIAAAAE=/docs/gAAGAIAAAAEDAAAAAAAAAA==/","_ts":1608932852}],"_count":3}

And here the response I get from azure

{"partitionedQueryExecutionInfoVersion":2,"queryInfo":{"distinctType":"None","top":null,"offset":null,"limit":null,"orderBy":[],"orderByExpressions":[],"groupByExpressions":[],"groupByAliases":[],"aggregates":[],"groupByAliasToAggregateType":{},"rewrittenQuery":"","hasSelectValue":true},"queryRanges":[{"min":"05C1A3C5D33F20083200","max":"05C1A3C5D33F20083200","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D12BA14B20083300","max":"05C1D12BA14B20083300","isMinInclusive":true,"isMaxInclusive":true},{"min":"05C1D3D3BB9304083400","max":"05C1D3D3BB9304083400","isMinInclusive":true,"isMaxInclusive":true}]}

Repro

Note: I had to generate my own CA then generate a certificate with CN=localhost then add the CA to /usr/local/share/ca-certificates/ and run sudo update-ca-certificates to pass the https check

using System;
using System.Collections.Generic;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Microsoft.Azure.Cosmos.Fluent;
using Microsoft.Azure.Cosmos.Linq;

namespace ReproBugCosmosDb
{
    class Program
    {
        public class SomeObject
        {
            public string Id { get; set; }
            public IList<string> SomeData { get; set; } = new List<string>();
        }

        private static CosmosClient _cosmosClient;
        public const string DevConnectionString = @"AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==";
        
        public const string DatabaseName = "SomeReproDatabase2";
        public const string ContainerId = "SomeContainerName";

        static async Task Main(string[] args)
        {
            _cosmosClient = new CosmosClientBuilder(DevConnectionString)
                .WithSerializerOptions(new CosmosSerializationOptions {PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase})
                .WithConnectionModeGateway()
                .Build();
            await ResetDatabaseAsync(DatabaseName);
            var database = _cosmosClient.GetDatabase(DatabaseName);

            await database.CreateContainerAsync(ContainerId, "/id");
            var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
            await container.UpsertItemAsync(new SomeObject {Id = "1", SomeData = new List<string> {"a"}}, new PartitionKey("1"));
            await container.UpsertItemAsync(new SomeObject {Id = "2", SomeData = new List<string> {"b"}}, new PartitionKey("2"));
            await container.UpsertItemAsync(new SomeObject {Id = "3", SomeData = new List<string> {"c"}}, new PartitionKey("3"));
            await container.UpsertItemAsync(new SomeObject {Id = "4", SomeData = new List<string> {"d"}}, new PartitionKey("4"));

            try
            {
                var result = await GetElementsByIds(new List<string> {"1", "2", "3"});
                if (!result.OrderBy(x => x).SequenceEqual(new[] {"a", "b", "c"}))
                    throw new Exception("Invalid result");
                Console.WriteLine("Success");
            }
            catch (CosmosException ex)
            {
                Console.WriteLine(ex.Diagnostics.ToString());
                throw;
            }
        }

        private static async Task ResetDatabaseAsync(string databaseName)
        {
            await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
            var database = _cosmosClient.GetDatabase(databaseName);
            await database.DeleteAsync();
            await _cosmosClient.CreateDatabaseIfNotExistsAsync(databaseName);
        }

        private static async Task<List<string>> GetElementsByIds(IList<string> ids)
        {
            var container = _cosmosClient.GetContainer(DatabaseName, ContainerId);
            var iterator = container.GetItemLinqQueryable<SomeObject>()
                .Where(x => ids.Contains(x.Id))
                .ToFeedIterator();
            var fetchedPhotoDocuments = new List<string>();

            while (iterator.HasMoreResults)
            {
                var item = await iterator.ReadNextAsync().ConfigureAwait(false);
                fetchedPhotoDocuments.AddRange(item.Resource.SelectMany(x => x.SomeData).ToList());
            }

            return fetchedPhotoDocuments.Distinct().ToList();
        }
    }
}

It throws here https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/v3Query/CosmosQueryClientCore.cs#L255

And data are loaded here
https://github.com/Azure/azure-cosmos-dotnet-v3/blob/master/Microsoft.Azure.Cosmos/src/Query/Core/Pipeline/CosmosQueryExecutionContextFactory.cs#L217

Exception

Unhandled exception. Microsoft.Azure.Cosmos.CosmosException : Response status code does not indicate success: InternalServerError (500); Substatus: 0; ActivityId: 00000000-0000-0000-0000-000000000000; Reason: (Value cannot be null. (Parameter 'providedRanges'));
 ---> System.ArgumentNullException: Value cannot be null. (Parameter 'providedRanges')
   at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
   --- End of inner exception stack trace ---
   at Microsoft.Azure.Cosmos.CosmosQueryClientCore.GetTargetPartitionKeyRangesAsync(String resourceLink, String collectionResourceId, List`1 providedRanges, Boolean forceRefresh)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.GetTargetPartitionKeyRangesAsync(CosmosQueryClient queryClient, String resourceLink, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, IReadOnlyDictionary`2 properties, FeedRangeInternal feedRangeInternal)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateFromPartitionedQuerExecutionInfoAsync(DocumentContainer documentContainer, PartitionedQueryExecutionInfo partitionedQueryExecutionInfo, ContainerQueryProperties containerQueryProperties, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.ExecutionContext.CosmosQueryExecutionContextFactory.TryCreateCoreContextAsync(DocumentContainer documentContainer, CosmosQueryContext cosmosQueryContext, InputParameters inputParameters, CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.AsyncLazy`1.GetValueAsync(CancellationToken cancellationToken)
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.LazyQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.NameCacheStaleRetryQueryPipelineStage.MoveNextAsync()
   at Microsoft.Azure.Cosmos.Query.Core.Pipeline.CatchAllQueryPipelineStage.MoveNextAsync()
--- Cosmos Diagnostics ---{"DiagnosticVersion":"2","Summary":{"StartUtc":"2020-12-25T21:59:18.9399687Z","RunningElapsedTimeInMs":28.6454,"UserAgent":"cosmos-netstandard-sdk/3.15.1|3.15.2|03|X64|Linux 5.4.0-58-generic 64-Ubu|.NET 5.0.1-servicing.20575.16|","TotalRequestCount":0,"FailedRequestCount":0,"Operation":"CosmosDiagnosticsContextCore"},"Context":[]}

Exception in read-document.ts getCollectionPartitionKeys when collection has no partitionKey saved

This is from working on compatibility with the CosmosDb CSharp SDK

Error logged request:

output for read-document.ts: getCollectionPartitionKeys(collection: ItemObject):
collection: {
  geospatialConfig: {},
  id: 'main',
  indexingPolicy: {
    automatic: true,
    indexingMode: 'consistent',
    includedPaths: [ [Object] ],
    excludedPaths: [],
    compositeIndexes: [],
    spatialIndexes: []
  },
  _etag: '96751c73-738f-46df-aa37-54aa6dc133b5',
  _rid: 'gAABAIAAAAM=',
  _self: '/dbs/gAABAA==/colls/gAABAIAAAAM=',
  _ts: 1604120454
}
TypeError: Cannot read property 'paths' of undefined
    at getCollectionPartitionKeys (/app/node_modules/@zeit/cosmosdb-server/lib/handler/read-document.js:15:36)
    at exports.default (/app/node_modules/@zeit/cosmosdb-server/lib/handler/read-document.js:61:32)
    at /app/node_modules/run-cosmosdb-sim/index.js:97:34
    at Server.<anonymous> (/app/node_modules/run-cosmosdb-sim/index.js:131:11)
    at Server.emit (events.js:314:20)
    at parserOnIncoming (_http_server.js:779:12)
    at HTTPParser.parserOnHeadersComplete (_http_common.js:122:17)
{
   "method":"GET",
   "url":"/dbs/devicesimulation/colls/main/docs/MasterNode",
   "docdbHeaders":[
      "x-ms-documentdb-partitionkey",
      "[]"
   ],
   "response":{
      "statusCode":500,
      "statusMessage":"Internal Server Error",
      "headers":{
         "content-type":"application/json",
         "content-location":"https://cosmosdb-sim:3000/dbs/devicesimulation/colls/main/docs/MasterNode",
         "connection":"close",
         "x-ms-activity-id":"7ac85c27-a88b-4099-bb94-e405df69f721",
         "x-ms-request-charge":"1"
      },
      "body":{
         "message":"Cannot read property 'paths' of undefined"
      }
   },
   "errorMessage":null
}

Expected result:

  • Document is returned

Cause:

  • Somehow the SDK or cosmosdb-server did not save an empty partition key for this document, and fails when retrieving it.

Proposed fix by this PR:

  • Add a conditional check on collection.partitionKey and return an empty list.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.