Giter VIP home page Giter VIP logo

surrealdb.net's Introduction

surrealdb.net

The official SurrealDB library for .NET.

⚠️ This driver is currently community maintained.

Getting started

Installation

dotnet add package SurrealDb.Net

How to use?

Supported protocols:

  • ✅ HTTP(S)
  • ✅ WS(S)
  • 🚧 and more to come...

Construct a new SurrealDB client

You can easily create a new SurrealDB client easily. All you have to do is define the endpoint to the SurrealDB instance.

var clientHttp = new SurrealDbClient("http://127.0.0.1:8000");
var clientHttps = new SurrealDbClient("https://cloud.surrealdb.com");
var clientWs = new SurrealDbClient("ws://127.0.0.1:8000/rpc");
var clientWss = new SurrealDbClient("wss://cloud.surrealdb.com/rpc");

// Signin & Use ns/db

Dependency injection

You can use Dependency Injection with the services.AddSurreal() function.

Default instance
var options = SurrealDbOptions
	.Create()
	.WithEndpoint("http://127.0.0.1:8000")
	.WithNamespace("test")
	.WithDatabase("test")
	.WithUsername("root")
	.WithPassword("root")
	.Build();

services.AddSurreal(options);

Then you will be able to use the ISurrealDbClient interface or SurrealDbClient class anywhere.

public class MyClass
{
	private readonly ISurrealDbClient _client;

	public MyClass(ISurrealDbClient client)
	{
		_client = client;
	}

	// ...
}

Note that the default lifetime of this service is Singleton. You can override this as follows:

services.AddSurreal(options, ServiceLifetime.Scoped);
Connection String

Consider the following appsettings.json file:

{
  "AllowedHosts": "*",
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "ConnectionStrings": {
    "SurrealDB": "Server=http://127.0.0.1:8000;Namespace=test;Database=test;Username=root;Password=root"
  }
}

You can use the Connection String instead of having to deal with a SurrealDbOptions.

services.AddSurreal(configuration.GetConnectionString("SurrealDB"));

It will automatically create a new SurrealDB using the Server endpoint and configure the client using the different values for namespace, database, username and password. Note that these values are optional but the endpoint is still required.

Multiple instances

Having a default instance for a project is enough most of the time, but there may be times when you'd like to target multiple SurrealDB instances, either at different addresses or at the same address but inside different NS/DBs. You can use multiple instances as long as you provide 1 interface per client, as in the following example.

interface IBackupSurrealDbClient : ISurrealDbClient { }
interface IMonitoringSurrealDbClient : ISurrealDbClient { }

services.AddSurreal(configuration.GetConnectionString("SurrealDB.Main"));
services.AddSurreal<IBackupSurrealDbClient>(configuration.GetConnectionString("SurrealDB.Backup"));
services.AddSurreal<IMonitoringSurrealDbClient>(configuration.GetConnectionString("SurrealDB.Monitoring"));

Here you will have 3 instances:

  • the default one, you can keep using ISurrealDbClient interface or SurrealDbClient class anywhere
  • a client for backup purpose, using the IBackupSurrealDbClient interface
  • a client for monitoring purpose, using the IMonitoringSurrealDbClient interface

Use the client

[ApiController]
[Route("[controller]")]
public class WeatherForecastController : ControllerBase
{
	private const string Table = "weatherForecast";

	private readonly ISurrealDbClient _surrealDbClient;

	public WeatherForecastController(ISurrealDbClient surrealDbClient)
	{
		_surrealDbClient = surrealDbClient;
	}

	[HttpGet]
	public Task<IEnumerable<WeatherForecast>> GetAll(CancellationToken cancellationToken)
	{
		return _surrealDbClient.Select<WeatherForecast>(Table, cancellationToken);
	}

	[HttpGet("{id}")]
	public async Task<IActionResult> Get(string id, CancellationToken cancellationToken)
	{
		var weatherForecast = await _surrealDbClient.Select<WeatherForecast>((Table, id), cancellationToken);

		if (weatherForecast is null)
			return NotFound();

		return Ok(weatherForecast);
	}

	[HttpPost]
	public Task<WeatherForecast> Create(CreateWeatherForecast data, CancellationToken cancellationToken)
	{
		var weatherForecast = new WeatherForecast
		{
			Date = data.Date,
			Country = data.Country,
			TemperatureC = data.TemperatureC,
			Summary = data.Summary
		};

		return _surrealDbClient.Create(Table, weatherForecast, cancellationToken);
	}

	[HttpPut]
	public Task<WeatherForecast> Update(WeatherForecast data, CancellationToken cancellationToken)
	{
		return _surrealDbClient.Upsert(data, cancellationToken);
	}

    [HttpPatch]
    public Task<IEnumerable<WeatherForecast>> PatchAll(
        JsonPatchDocument<WeatherForecast> patches,
        CancellationToken cancellationToken
    )
    {
        return _surrealDbClient.PatchAll(Table, patches, cancellationToken);
    }

    [HttpPatch("{id}")]
    public Task<WeatherForecast> Patch(
        string id,
        JsonPatchDocument<WeatherForecast> patches,
        CancellationToken cancellationToken
    )
    {
        return _surrealDbClient.Patch((Table, id), patches, cancellationToken);
    }

	[HttpDelete]
	public Task DeleteAll(CancellationToken cancellationToken)
	{
		return _surrealDbClient.Delete(Table, cancellationToken);
	}

	[HttpDelete("{id}")]
	public async Task<IActionResult> Delete(string id, CancellationToken cancellationToken)
	{
		bool success = await _surrealDbClient.Delete((Table, id), cancellationToken);

		if (!success)
			return NotFound();

		return Ok();
	}
}

How to contribute?

.NET release versions

The .NET release versions must follow these rules:

  • Should target at least the latest LTS (Long-Term Support) version
  • Should target at least the latest STS (Standard-Term Support) version

SurrealDb.Net targets .NET versions following the .NET Support Policy by Microsoft. Additionally, SurrealDb.Net targets .NET Standard 2.1 explicitly to continue support of the Mono runtime (Unity, Xamarin, etc...).

Note that the support for .NET standard 2.1 will be maintained until further notice.

Version Description Release Date End of Support
.NET Standard 2.1 June 27, 2016 N/A
.NET 6 LTS November 8, 2021 November 12, 2024
.NET 7 STS November 8, 2022 May 14, 2024
.NET 8 Current LTS November 14, 2023 November 10, 2026

Formatting

This project is using CSharpier, an opinionated code formatter.

Command line

You can install it on your machine via dotnet tool.

# Run this command at the root of the project
dotnet tool install csharpier

You can then use it as a cli:

dotnet csharpier .

The list of command-line options is available here: https://csharpier.com/docs/CLI

IDE integration

CSharpier supports multiple code editors, including Visual Studio, Jetbrains Rider, VSCode and Neovim. You will be able to run format on file save after configuring the settings in your IDE.

Testing

This project was written following testing best practices:

  • TDD, leveraging:
    • clean code/architecture
    • regression testing
    • adding new features and tests easily
  • a vast majority of tests are integration tests, ensuring compatibility with a concrete SurrealDB version
  • each integration test is using a separate SurrealDB namespace/database

Unit/Integration tests are written using xUnit and FluentAssertions.

You will need a local SurrealDB instance alongside the tests. Start one using the following command:

surreal start --log debug --user root --pass root memory --auth --allow-guests

Once ready, go to the root directory of the project and run the following command:

dotnet watch test --project SurrealDb.Net.Tests

Due to the asynchronous nature of Live Queries, they are tested against a separate project named SurrealDb.Net.LiveQuery.Tests. Where the default test project allow full parallelization, this project completely disable test parallelization. To execute tests on Live Queries, run the following command:

dotnet watch test --project SurrealDb.Net.LiveQuery.Tests

Note 1: Because Live Query tests are not run in parallel, it can take quite some time to run all tests.

Note 2: You can run the two test projects in parallel.

Benchmarking

This project also contains benchmarks in order to detect possible performance regressions.

You will need a local SurrealDB instance alongside the tests. Start one using the following command:

surreal start --user root --pass root memory --auth --allow-guests

Once ready, go to the root directory of the project and run the following command:

dotnet run -c Release --project SurrealDb.Net.Benchmarks --filter '*'

surrealdb.net's People

Contributors

kearfy avatar naisofly avatar odonno avatar simon-curtis avatar tobiemh avatar tomaszczyz avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

surrealdb.net's Issues

Make use of TestContainers

Use TestContainers so that we can always use the latest image of surrealdb during test execution.

  • Change CI/tests to run a surrealdb instance via TestContainers
  • Fallback to surrealdb cli when no container runtime installed
  • Update documentation (readme file)

Issues with WebSocket Connection

Hello,

I am doing some testing with .Net SurrealDB Client and I came across an issue that I do not have when using the HTTP client. When I do a .Select I get the error/exception "There was a problem with the database: Specify a namespace to use". Even though in the ServiceBuilder area in Program.cs and on the line before the Select I configure it to use a Namespace and Database "test". Whats weird is the same code works perfectly fine connecting over HTTP. It does connect fine over WebSocket but as soon as I do something I get the error. Below is the relevant code snippets. Let me know if you need anything else and thanks for the help.

Program.cs

var surrOptions = SurrealDbOptions.Create()
    .WithEndpoint("ws://192.168.122.49:8000/rpc") // Works fine with changing ws to http and removing the /rpc
    .WithNamespace("test")
    .WithDatabase("test")
    .WithUsername("root")
    .WithPassword("root")
    .Build();
builder.Services.AddSurreal(surrOptions);

Code/Method Snippet (Dependency Injected)

var health = await Surreal.Health();
if (!health)
    await Surreal.Connect();

Surreal.Configure("test", "test");

var counts = await Surreal.Select<CountType>("count_test");

if (counts.FirstOrDefault() != null)
    currentCount = counts.First();
else
    currentCount = new CountType();

private class CountType : Record
{
    public int Count { get; set; }
}

Problem Upserting records with composite (object) Ids

using the driver with WS engine:

SurrealDb.Net.Exceptions.SurrealDbException : There was a problem with the database: Found 'TestRecords:{"x":"xx","z":"zz"}' for the id field, but a specific record has been specified

public class TestRecord : Record {
    public string TestA { get; set; }
    public string TestB { get; set; }
    public string TestC { get; set; }
}
...
var testRecordResult = await surrealDbClient.Upsert(new TestRecord {
        Id = Thing.From("TestRecords", new { x = "xx", z = "zz" }),
        TestA = "Something",
        TestB = null,
        TestC = "SomethingElse",
    }, CancellationToken.None
);

yields the following problem:
SurrealDb.Net.Exceptions.SurrealDbException : There was a problem with the database: Found 'TestRecords:{"x":"xx","z":"zz"}' for the id field, but a specific record has been specified

I've noticed the same issue with the HTTP engine:

PUT http://localhost:8000/key/TestRecords/{"x":"xx","z":"zz"}

with body

{
    "id": "TestRecords:{\"x\":\"xx\",\"z\":\"zz\"}",
    "TestA":"xxxx",
    "TestB":null,
    "TestC":"xxxxx"
}

does not work

but with body

{
    "id": "TestRecords:{'x':'xx','z':'zz'}",
    "TestA":"xxxx",
    "TestB":null,
    "TestC":"xxxxx"
}

does work,

so this seems like a serialization issue when escaping strings of Ids that are objects?

Add Support For Polymorphic Deserialization

I encountered an exception trying to save an object instance with a property type that was abstract. A major improvement would be for this client library to allow saving / loading types that are derived from other types.

Problem with null values on schemafull tables

I have created a schemafull table with an optional field (option) and mapped a C# object to it. But when I try to insert new records using Upsert or Create, if the value is null, the process fails! It doesn't provide any useful information other that throwing a 'SurrealDb.Net.Exceptions.SurrealDbErrorResultException' , but it might be because surrealdb expects 'NONE' for empty fields and the .NET SDK tries to send 'NULL' instead,

Patch method

Applies JSON Patch changes to all records, or a specific record, in the database.

More info on JSON Patch

  • Add a new method Patch
    • With an override on table name
    • With an override on record id
  • If possible, make use of Microsoft.AspNetCore.JsonPatch package https://www.nuget.org/packages/microsoft.aspnetcore.jsonpatch/
  • Add tests
  • Update project SurrealDb.Examples.WeatherApi to add Patch method
  • Update documentation
    • Readme file
    • Official documentation website

v0.3.1: ConfigureJsonSerializerOptions not working properly

Hi!

I'm using the ConfigureJsonSerializerOptions parameter of SurrealDbClient constructor and, while in version v0.2.0 it was working properly, when I upgraded the nuget package to version v0.3.1 it stopped working.

As for example example, I have some configurations done at JsonSerializer and, when I'm calling GetValue from SurrealDbOkResult, the configurations are not working.

Record Ids of string type

To remove the need of a reference to surrealdb.net in the domain model an id of string type should be handled as well as the Thing type. Maybe mapping id to any properties that has the [Key] attribute. This would make it easier to adopt existing EF models.

Escaped table name in Thing

The record id of Thing can be escaped. This is already handled in the library. But the table name can also be escaped.

Update the parser/formatter to handle escaped table name on Thing, by reusing the existing logic for record id.

How to get the id back from the Create method?

I tried to create a record in the database using the Create method. If I dont put an Id property in the object it works fine, but then I don't get the id back from the Create method. If I put the id property, I receive an error because I can't send a null id or an empty string, but I would like SurrealDB to generate the id for me.

Any suggestions?

Suggestion to Standardize SDK Initialization Method Naming

I've been going through the documentation for the SDK drivers for our database, and I've noticed a discrepancy in the way we initialize a new host across different languages. Specifically, in the .NET SDK, we currently use SurrealDbHttpClient.New(host) for initialization. In contrast, other languages' documentation frequently employs the .init() convention.

To maintain consistency and predictability for developers familiar with other SDKs or expecting a more standard initialisation method, I'd like to suggest renaming the method from:

SurrealDbHttpClient.New(host)

to

SurrealDbHttpClient.Init(host)

F# client

  • Create a packageable F# library named SurrealDb.FSharp
  • Should follow the same convention as the C# client (class names, method names, etc...)
  • Make use of the F# paradigm (Result type, etc...)
  • [Optional] If possible, create a packageable C# library named SurrealDb.Core to avoid code duplication
  • Create some example projects SurrealDb.FSharp.Examples.Console and/or SurrealDb.FSharp.Examples.WeatherApi
  • [Optional] Add tests and benchmarks projects
  • Update documentation
    • Readme file
    • Official documentation website

Live queries

  • Create a method named LiveQuery that should create a new Live Query that returns an IAsyncEnumerable<T>
    • Method override using a Guid (id of the live query)
    • Method override using the table name and diff boolean value (to handle JSON Patch vs. full object)
  • Create a method named Kill that should kill a Live Query
  • Handle the WS implementation
  • Throw NotSupportedException for the HTTP implementation
  • Add tests
    • Add tests on IAsyncEnumerable returns
    • Add tests on Rx.NET Observable, using IAsyncEnumerable.ToObservable() extension method (Subscribe, Hot vs. Cold, etc...)
  • Update documentation
    • Readme file
    • Official documentation website

EF Core Database Provider

  • Create a packageable C# library named SurrealDb.EntityFrameworkCore.SurrealDb
  • Create a test project named SurrealDb.EntityFrameworkCore.Tests
  • Create an example project named SurrealDb.Examples.EntityFrameworkCore
  • Update documentation
    • Readme file
    • Official documentation website

Here is an interesting article on how to write an EF Core Provider: https://blog.oneunicorn.com/2016/11/11/so-you-want-to-write-an-ef-core-provider/ Not sure if this still work but it can be a good starting point.

AOT Strategy

Since dotnet is putting serious emphasis on AOT at the moment and SurrealDb is going to be known for it's performance. It might be worth thinking about the strategy for how we will deliver this. Including how we handle JSON deserialisation AOT and how that is going to affect the APIs

Add .NET Standard 2.0 support

Would it be possible for the driver to support .NET Standard 2.0? .NET Standard 2.1 is honestly not that useful compared to 2.0 because 2.0 is supported on .NET Framework 4.6.1+, and 2.1 is only supported on .NET Core and more recent version. According to the documentation by MS that version supports most of the stuff that's needed for libraries. What's not supported can be polyfilled/backported like AsyncInterfaces.

Performance investigation: WebSocket client cold start

According to the benchmark discovery, it seems that the WebSocket client has a cold start of 2 seconds: from creation to connection. It may take some time to connect to a live WebSocket server, but realistically it should not take that long.

Investigate to find the problem. It is possible that it comes from one or multiple reasons among the following:

  • Is it due to the WebSocket.Client NuGet library?
  • Is it due to the surrealdb WebSocket server then?
    • Need some understanding of the WS protocol and Rust implementation. Anyway, making samples using at least 2 other languages (like Python, JS, etc..) can help figure it out
  • It is due to the WebSocket C# default implementation? (internals of the Microsoft .NET source code)
    • A little bit skeptic on this, but a simple WsClient instance in a sample Console app can help to clarify this point
  • Is it only happening when the server is localhost?
    • Try to see the differences with 127.0.0.1 or even better with a remote surrealdb server
  • Is it OS related?
    • On Windows, it takes 2s, is it the same for Linux and macOS?
  • Other? If the other possible reasons are not valid, any idea?

Change Record to be an interface

Change the class Record to be an interface instead of a class. This will be a more versatile solution.
This may be a breaking change due to a name change (Record->IRecord) to follow naming conventions.

How to create objects that have other objects as properties?

Considering the following object:

public class ComponentDto : Record
{
    [JsonConverter(typeof(CustomThingConverter))]
    [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)]
    public new Thing? Id { get; set; }

    [JsonPropertyName("parent")]
    public DeviceDto? Parent { get; set; }

    [JsonPropertyName("sensor")]
    public SensorDto? Sensor { get; set; }

    [JsonPropertyName("status")]
    public string Status { get; set; }
}

Parent and Sensor are other objects in my application.

Considering that I wanna create a new ComponentDto and insert in the database, the following situation happens:

I instantiate the ComponentDto and I need to assign the Parent and the Sensor, but since SurrealDB relates things with the record type "table:id" and you do a FETCH to fetch the relations, I would need to assign a string "devices:id" to Parent and "sensors:id" to Sensor properties, but I can't do that since their types are not string.

My approach now will be to create the record with Parent and Sensor properties null and update the created component record with a surrealDbClient.Query() method where I can assign them to string and there is no type conflicts.

Is there any other approach that would be more suitable?

Proposal: Interpolated Parameterization

One of my personal favorite ways to deal with SQL is using interpolated strings to deal with parameters. The risk of SQL injection can be nullified by defining a special InterpolatedStringHandler. I've implemented this before in internal projects, and in a toy project of mine for postgres.

The tradeoff is that methods that accept parameters (query and live) would need to commit to this interpolated string handler only, not accepting regular strings with parameter objects anymore (so as to prevent accidental injection by previously interpolated strings that have since been interpolated into an actual string).

Ill put up some examples in a bit.

Discussion: Supported Releases

The current support windows (5 year LTS) extend support for .NET versions beyond Microsoft's support windows (3 year LTS). The commitment is admirable but ultimately more work than we need to take on imo. If a runtime version is no longer officially supported by Microsoft, why would we maintain support for that version.

Additionally, I would like to see a cautious plan to provide STS support as well, in their appropriate support windows (18 months). Not only would STS users reap benefits, but we would be more gradually moving to the next LTS as well.

String incorrectly interpreted as record in Schemafull table

I have a scemafull with a field defined as string and if I set it to a string which is in the form of a record (e.g. test:string) then it throws an exception. There is a workaround in surrealql, but I don't know how to do that in this SDK.

Minimal replication example:

using SurrealDb.Net;
using SurrealDb.Net.Models.Auth;

public class Test {
    public required string Name { get; set; }
}

var db = new SurrealDbClient("ws://127.0.0.1:8000/rpc");
await db.SignIn(new RootAuth { Username = "root", Password = "root" });
await db.Use("test", "test");

const string tableName = "test";

await db.RawQuery(@$"DEFINE TABLE {tableName} SCHEMAFULL;
DEFINE FIELD name ON TABLE {tableName} TYPE string;");

await db.Create<Test>(tableName, new Test(){Name = "Test normal data"});
await db.Create<Test>(tableName, new Test(){Name = "Test normal data2"});
await db.Create<Test>(tableName, new Test(){Name = "Test normal data3"});
await db.Create<Test>(tableName, new Test(){Name = "example:string"});

Where it throws Error: SurrealDb.Net.Exceptions.SurrealDbException: There was a problem with the database: Found example:string for field `name`, with record `test:djqxpvplqi7eorq6pp9t`, but expected a string

As a workaround, I can set the type to be any, but that is no longer representative of the data I am putting in it.

SurrealDbClient disconnects in WC endpoint

Dear all
Thank you for your hard work.

I have found a possible issue. It seems that the SurrealDBClient doesn't maintain the connection.
I have been using the latest 0.40 version with Core NET 8.0

This is the test code I have prepared. In the SurrealDB (latest version 1.3.1) is the table tweets with only one record.

DataBaseParams dbParams = new DataBaseParams()
{
    ipAddress = "192.168.0.1",
    port = 8000,
    username = "root",
    password = "root",
    database = "testdb",
    dataBaseEngine = DataBaseEngine.SurrealDB
};

var surrealDBClient = new SurrealDbClient(
                SurrealDbOptions
                    .Create()
                    .WithEndpoint($"ws://{dbParams.ipAddress}:{dbParams.port}/rpc")
                    .WithUsername(dbParams.username)
                    .WithPassword(dbParams.password)
                    .WithNamespace(dbParams.database)
                    .WithDatabase(dbParams.database)
                    .Build(),
                null,
                (_options) =>
                {
                    _options.IncludeFields = true;
                }
);

int iCount = 0;
do
{
    Console.WriteLine($"{DateTime.Now.ToString("yyyy-MM-ddTHH:mm:ssZ")} Query");
    SurrealDbResponse result = await surrealDBClient.RawQuery("SELECT * FROM tweets;");
    if (result.FirstOk != null)
    {
        Console.WriteLine($"{DateTime.Now.ToString("yyyy-MM-ddTHH:mm:ssZ")} Total Results: {result.FirstOk?.GetValue<List<Tweet>>()?.Count}");
    }
    else
    {
        Console.WriteLine(result.FirstError?.ToString());
    }
    Thread.Sleep(60 * 1000);
    iCount++;
} while (iCount < 100);

This code produces the following result

2024-03-27T12:06:49Z Query
2024-03-27T12:06:50Z Total Results: 1
2024-03-27T12:07:50Z Query

... It is stuck ...
I pressed CTRL+C

And the docker compose logs are

surrealdb-surreal-1  | 2024-03-27T11:06:51.372748Z DEBUG request: surreal::net::tracer: started processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" otel.name="GET /rpc" http.route="/rpc" http.request.id="ee0e2c68-f781-4c8b-94cf-e266570c2747" client.address="172.16.200.2"
surrealdb-surreal-1  | 2024-03-27T11:06:51.372912Z DEBUG request: surreal::net::tracer: finished processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" otel.name="GET /rpc" http.route="/rpc" http.request.id="ee0e2c68-f781-4c8b-94cf-e266570c2747" client.address="172.16.200.2" http.response.body.size="0" http.response.status_code=101 http.latency.ms=0
surrealdb-surreal-1  | 2024-03-27T11:06:51.373136Z TRACE surreal::rpc::connection: WebSocket ee0e2c68-f781-4c8b-94cf-e266570c2747 connected
surrealdb-surreal-1  | 2024-03-27T11:06:51.501393Z DEBUG rpc/call: surreal::rpc::connection: Process RPC request otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="use" otel.name="surrealdb.rpc/use" rpc.request_id="24c9bb56"
surrealdb-surreal-1  | 2024-03-27T11:06:51.501427Z DEBUG rpc/call: surreal::rpc::response: Process RPC response otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="use" otel.name="surrealdb.rpc/use" rpc.request_id="24c9bb56"
surrealdb-surreal-1  | 2024-03-27T11:06:51.566271Z DEBUG rpc/call: surreal::rpc::connection: Process RPC request otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="signin" otel.name="surrealdb.rpc/signin" rpc.request_id="5da0a4d0"
surrealdb-surreal-1  | 2024-03-27T11:06:51.607856Z TRACE rpc/call: surrealdb_core::iam::signin: Signing in as root otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="signin" otel.name="surrealdb.rpc/signin" rpc.request_id="5da0a4d0"
surrealdb-surreal-1  | 2024-03-27T11:06:51.607913Z DEBUG rpc/call: surreal::rpc::response: Process RPC response otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="signin" otel.name="surrealdb.rpc/signin" rpc.request_id="5da0a4d0"
surrealdb-surreal-1  | 2024-03-27T11:06:51.619769Z DEBUG rpc/call: surreal::rpc::connection: Process RPC request otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="1bf1a489"
surrealdb-surreal-1  | 2024-03-27T11:06:51.619912Z DEBUG rpc/call:execute:process:executor: surrealdb_core::dbs::executor: Executing: SELECT * FROM tweets otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="1bf1a489"
surrealdb-surreal-1  | 2024-03-27T11:06:51.620127Z TRACE rpc/call:execute:process:executor: surrealdb_core::dbs::iterator: Iterating: SELECT * FROM tweets otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="1bf1a489"
surrealdb-surreal-1  | 2024-03-27T11:06:52.296716Z DEBUG rpc/call: surreal::rpc::response: Process RPC response otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="1bf1a489"
surrealdb-surreal-1  | 2024-03-27T11:07:52.331155Z DEBUG rpc/call: surreal::rpc::connection: Process RPC request otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="fcd7f64"
surrealdb-surreal-1  | 2024-03-27T11:07:52.331286Z DEBUG rpc/call:execute:process:executor: surrealdb_core::dbs::executor: Executing: SELECT * FROM tweets otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="fcd7f64"
surrealdb-surreal-1  | 2024-03-27T11:07:52.331384Z TRACE rpc/call:execute:process:executor: surrealdb_core::dbs::iterator: Iterating: SELECT * FROM tweets otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="fcd7f64"
surrealdb-surreal-1  | 2024-03-27T11:07:53.036915Z DEBUG rpc/call: surreal::rpc::response: Process RPC response otel.kind="server" ws.id=ee0e2c68-f781-4c8b-94cf-e266570c2747 rpc.service="surrealdb" rpc.method="query" otel.name="surrealdb.rpc/query" rpc.request_id="fcd7f64"
surrealdb-surreal-1  | 2024-03-27T11:07:53.037459Z TRACE surreal::rpc::connection: WebSocket error: Error { inner: Protocol(ResetWithoutClosingHandshake) }
surrealdb-surreal-1  | 2024-03-27T11:07:53.037673Z TRACE surreal::rpc::connection: WebSocket ee0e2c68-f781-4c8b-94cf-e266570c2747 disconnected
surrealdb-surreal-1  | 2024-03-27T11:07:53.038084Z DEBUG request: surreal::net::tracer: started processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" otel.name="GET /rpc" http.route="/rpc" http.request.id="06cc2a07-65bd-42ea-92b8-5d31326093a3" client.address="172.16.200.2"
surrealdb-surreal-1  | 2024-03-27T11:07:53.038250Z DEBUG request: surreal::net::tracer: finished processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" otel.name="GET /rpc" http.route="/rpc" http.request.id="06cc2a07-65bd-42ea-92b8-5d31326093a3" client.address="172.16.200.2" http.response.body.size="0" http.response.status_code=101 http.latency.ms=0
surrealdb-surreal-1  | 2024-03-27T11:07:53.038505Z TRACE surreal::rpc::connection: WebSocket 06cc2a07-65bd-42ea-92b8-5d31326093a3 connected
surrealdb-surreal-1  | 2024-03-27T11:08:46.953012Z TRACE surreal::rpc::connection: WebSocket error: Error { inner: Io(Os { code: 104, kind: ConnectionReset, message: "Connection reset by peer" }) }
surrealdb-surreal-1  | 2024-03-27T11:08:46.953145Z TRACE surreal::rpc::connection: WebSocket 06cc2a07-65bd-42ea-92b8-5d31326093a3 disconnected

As seen, the first query works perfectly, but the following ones do not work or something happens with the SurreablDB and the message Iterating does not appear.

If I reduce the delay between Queries it work perfectly.
If I repeat the same experiment but with and HTTP enpoint it works perfectly.

Could you kindly take a look at it.

Bst regards.

Logging

  • [Question] Should we add some logging in the core library? In which functions?
  • Write a custom sink for Serilog. In a separate repository?
  • Support OpenTelemetry, if not already supported

Maybe @s-e-f has some ideas on this

Task.WaitAll(taskList.ToArray()) never fisnishes

Dear all
As always, thank you for your hard work

Maybe I'm doing something wrong, but I think I have found an issue.

Launching several tasks to Create several tweets in parallel and using Task.WaiAll() does not work correctly

I have been using the latest 0.40 version with Core NET 8.0
SurrealDB (latest version 1.3.1) is the table tweets with no records.

This is the test code I have prepared.


private static async Task Main(string[] args)
{
    DataBaseParams dbParams = new DataBaseParams()
    {
        ipAddress = "192.168.0.1",
        port = 8000,
        username = "root",
        password = "root",
        database = "testdb",
        dataBaseEngine = DataBaseEngine.SurrealDB
    };

    var surrealDBClient = new SurrealDbClient(
                    SurrealDbOptions
                        .Create()
                        .WithEndpoint($"ws://{dbParams.ipAddress}:{dbParams.port}/rpc")
                        .WithUsername(dbParams.username)
                        .WithPassword(dbParams.password)
                        .WithNamespace(dbParams.database)
                        .WithDatabase(dbParams.database)
                        .Build(),
                    null,
                    (_options) =>
                    {
                        _options.IncludeFields = true;
                    }
    );

    List<Tweet> tweets = new List<Tweet>();
    for (int iCount = 0; iCount < 5; iCount++)
    {
        tweets.Add(new Tweet()
        {
            id = $"ID{iCount}",
            username = $"user{iCount}",
        });
    }

    List<Task<Tweet>> list = new List<Task<Tweet>>();
    tweets.ForEach(tweet =>
    {
        var task = surrealDBClient.Create("tweets2", tweet);
        //var task = dummyTask();
        list.Add(task);
    });

    Task.WaitAll(list.ToArray());
}




Please notice there are 2 methods. One is the basic SurrealDB Create method

var task = surrealDBClient.Create("tweets2", tweet);

The other is a dummy method used to compare the execution.

var task = dummyTask();

private static async Task<Tweet> dummyTask()
{
   await Task.Delay(2000);
   return new Tweet()
   {
       id = "dummy",
       username = "pepito"
   };
}

When I use the code with the Create the execution is stuck on the Task.WaitAll() forever, but checking the DB with Surrealist I can see that the 5 records were inserted.

When I use the code with the DummyTask the execution continues and the program finishes.

I have not tested the code with HTTP instead of WS, but it could have the same behaviour.

IQueryable Select

Offer a way to query table records with better performance using the IQueryable<T> interface, somewhat paving the way for an EF Core implementation.

  • Return IQueryable<T> instead of IEnumerable<T> for Select method, or IOrderedQueryable<T> that offers sorting features, if possible
  • Create a custom query builder and use it build to a string query
  • Avoid usage of query builder if not needed (on full table projection)
  • Add tests
  • Create an example project to showcase some basic features (filtering, grouping, counting, ordering, projecting, etc...)
  • Update documentation with some examples/use cases

Some guides on how to implement IQueryable<T>/IOrderedQueryable<T>:

https://putridparrot.com/blog/creating-a-custom-linq-provider/

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.