Constraints

1. For actual REPL we need a resumable tokenizer, but it's fine to just catch
   an error, add more data and reparse whole data again
2. For command-line, we definitely want async parser because feeding huge dump
   to a command-line is a valid use case
3. Extends (2): It's expected that dump consists of separate, relatively small statements
   that are split with a semicolon
4. Partially extends (3): Actual query parser doesn't need to be
   streaming/async because we're going to build abstract syntax tree (AST) of the
   query anyway (which must fit memory)
5. Note also that it's hard to implement resumable parser as the last token can
   be partial if we're not on the end of file

Additional Rust constraints:

1. Any streaming/async parser can't borrow token from the buffer, so tokens
   need to have allocated String inside (performance and memory overhead)
2. Event blocking stream reader is much harder to write as you can't freely
   seek/lookahead into a stream

The Proposed Approach

Make a preparser that knows quotes, comments and semicolons.
That returns chunks split by a semicolon ;. It should be trivial to make
it resumable.

And then make classic tokenizer that borrows slices from the buffer, that
does parse all the tokens. This is the thing used by the parser.

Alternative

Use combine library. Which have all the needed tools for streaming parsers on top of blocking streams. And wrap around that idea. The downsides are:

1. Memory allocations overhead is still there
2. We're going to use combine for parser on top of combine for tokenizer, which
   makes code hard to reason about
3. Making it async is still a lot of dance

I'm trying to use edgedb-tokio together with edgedb-derive.

I'm facing the following error

unexpected type BaseScalar(BaseScalarTypeDescriptor { id: 00000000-0000-0000-0000-000000000100 }), expected std::str

and I'm not sure what I doing wrong in the following setup

#[derive(Queryable)]
struct User {
  id: Uuid,
  name: String
}
...
let conn = edgedb_tokio::create_client().await?;
conn.query_single("select User { id, name } filter .id = <uuid>$0;", &(&id.to_string(),)).await?;

.esdl

type User {
  required property name -> str;
}

What am I doing wrong here?

Currently, REPL is showing server traceback for all errors, need to hide this behind a flag like \verbose-errors. Perhaps, instances of InternalServerError should display server traceback by default to make bug reports more useful.

Calling transaction() on a client consumes it (https://docs.rs/edgedb-tokio/0.3.0/src/edgedb_tokio/client.rs.html#357-362). &self should be enough since Client represents a connection pool. See sqlx's transaction: https://docs.rs/sqlx-core/0.6.2/src/sqlx_core/pool/mod.rs.html#355

Is it possible to execute inserts with optional parameters?
ScalarArg is apparently not implemented for Option<T>.

db.query(
    " insert Obj { 
        optional := <optional str>$0, 
        ... 
     }", 
    &(Some(""))
)

Right now it seems incredibly cumbersome to insert something with optional properties, since you'd need to rewrite the SQL based on if the value is present.

Hello EdgeDB Team,

first of all many thanks for this great project!

With regard to using DDL it seems to me that not allowing DDL queries is hardcoded in the Rust client here:

How but how can I, e.g., create a type with the Rust client?

Query args is implemented for all tuples between 2 and 12 in length. This means that something like (Uuid) is not possible. QueryArgs is also not implemented for simply Uuid (or any scalar type) making it extremely unergonomic to have a single parameter in a query. Currently if you want to have a single parameter you are forced to supply Value which is not type safe as the user can supply any variant of the enumerable.

Also, derive(QueryArgs) does not appear to be implemented yet the documentation in the current release indicates to the user that it is derivable .

-- From the rust docs
"""A tuple of query arguments.

This trait is implemented for tuples of sizes up to twelve. You can derive it for a structure in this case it's treated as a named tuple (i.e. query should include named arguments rather than numeric ones)."""

I believe the best way forward is to implement QueryArgs for the QueryArg trait. This avoids users having to tell rustfmt to ignore warnings about unsed parentheses.

I would also recommend removing the documentation that QueryArgs is derivable until it actually is.

Since we have first pool API implemented in #88 we can have some plan for 0.1 which is nowhere near feature-complete, but can be used to play with API a little bit.

The plan is:

1. Experiment with executor trait. Notes:
   1. We can't make current query<Arg, Res>-like methods on dynamic value, because they aren't object-safe. The idea is to make less ergonomic methods (e.g. generic_query(dyn Arguments) -> dyn ResultDecoder) on the trait Executor itself and then ergonomic methods on dyn Executor. Or make some wrapper type. Not sure it will be ergonomic enough. If it will not work we can proceed with either non-object safe Executor trait or with API tied to a Pool object and this stage.
   2. We probably want &mut self on trait methods (they are &self on Pool). So that Transaction object can implement the trait too. This means actual pool object needs to be cloned for parallel tasks. Clones are cheap. And this will probably be the case anyway (to put pool object to the task state). And transaction will do the borrow, which is a good idea too.
2. Hide all fine-grained CLI API under unstable flag
3. Make sure that everything needed from underlying crates are re-exported from the edgedb_client.
4. Make sure that generated docs cover all essential use cases and all public things (warn(missing_docs))
5. Rename Pool -> Client

The limitations of this release are:

1. async-std-only
2. No transactions
3. No retries
4. No manually acquired connections, so CONFIG SET and START TRANSACTION and similar aren't possible too
5. Maybe some limitations on supported argument and result types

Previous discussion on #32

I have a Serializable struct that I want to insert into a query. How would I do that?

Before publishing client crate on crates.io we need:

1. Ensure that error reporting is adequate and errors can be handled apropriately. Probably get rid of anyhow::Error. If even if we leave anyhow, we should ensure that it's easy to find out if error is recoverable, if it's network error, etc.
2. Figure out if it makes sense to move query methods into a trait (async traits are kinda complex, though)
3. Review list of dependencies, currently there are too much of them
   1. Also async-std should not require unstable feature
   2. Some of them, like whoami should be moved under a feature gate
4. Review public exports, hide unneeded things and restructure modules if needed
   1. Move Sequence under a feature gate (e.g. unstable)
5. Query arguments refactoring
   1. QueryArgs trait
   2. QueryArg trait
   3. QueryArg implementation for all basic types
   4. QueryArgs implementation for tuple and container types
   5. derive(QueryArgs)
6. Create a Datetime struct to represent edgedb's native datetime type #42
7. Implement most of the RFC1004
   1. Reconnecting connection
   2. Connection pool
   3. Transactions
   4. Retriable transactions
   5. Transaction configuration/Retry options
   6. Capabilities

These tasks can be postponed:

1. Look into supporting multiple runtimes #49
2. Ensure that nothing is re-exported from edgedb-protocol, so that we can make breaking changes in latter without disturbing users
   1. Figure out what to do with Queryable trait
3. Take a look if we want to get rid of snafu for errors (used only internally)

Previous discussion in edgedb/edgedb-cli#112, #30

In the meantime, anyone can use git version at their own risk (API will probably break every now and then)

Currently, we only allow a single host/port pair to be passed to the connect() call. To make building HA setups around the binding(s) easier, we should allow passing a list of addresses to be tried in order. On its own, this would satisfy simple HA setups with static hostlists, either direct server addresses or load-balancing proxy lists.

For reference, PostgreSQL's libpq documentation on how to specify multiple hosts is here.

Their approach is hybrid: host and port are separate arguments and both may contain lists. If host and port are both lists, they must be of the same size, if host is a list and port is not, the specified port is applied to all hosts.

The upside of this approach is that it keeps things simple in the very common case of a single host/port, and the downside is that separate host and port lists are harder to specify and manage.

An alternative approach is to remove the port argument and instead allow passing host as either a string, a single (named) tuple, or a list of (named) tuples.

All of this applies to how the credentials file is structured, since we want to keep the resemblance with connect() call signatures.

Currently if I try this with the tutorial database I get an error:

SELECT User {
    name,
    email,
    last_post := (
        SELECT .<author[IS Media] {
            [IS Text].title,
        }
    ),
}
FILTER .email = '[email protected]';

The culprit seems to be specifically the polymorphic property "title" (if I fetch some common property of Media things work fine). If I don't use rust REPL this query works fine.

The preparser (#1) currently ends the statement on the first non-quoted semicolon. But at least migration queries have a lot of unquoted colons inside.

The REPL should show the server version and a hint to use \? for help, like the Python REPL does.

Currently you use SystemTime. But it has different range and precision from edgedb's type, so I think it makes sense to define a custom Datetime type which matches what edgedb can represent exactly.

Value would use this Datetime type, while Queryable would be implemented for both SystemTime and Datetime.

Background

The interactive mode prints results like this:

{"item1", "item2", "item3"}

For the non-interactive mode that isn't the best format, because it's custom and there is no tool to parse it.

Option 1

Use JSON as output.

Upsides:

1. Popular format
2. There is jq (and other tools)
3. Supports structured output well

Downsides:

1. For simple output like SELECT name := sys::Database.name can be ugly to use in the shell script
2. Some things like SELECT <json>field the output will be doubly-encoded

Option 1a

Output a single JSON output for the whole resultset:

[{"row_num": 1, "field1": "value1"}, {"row_num": 2, "field1": "value2"}]

Additional upsides:

1. Simple to parse when the resultset is small

Additional downsides:

1. Many tools can't parse JSON without loading the whole data into the memory
2. Harder value extraction using jq (edgedb | jq '.[] | .field')

Option 1b

Output each JSON row on a separate line:

{"row_num": 1, "field1": "value1"}
{"row_num": 2, "field1": "value2"}

Additional upsides:

1. Arbitrarily large values can be parsed that way.
2. Nicer output (we don't pretty-print JSONs)
3. Simple segmentation (i.e. by newline) for parallel processing (incl. by Hadoop/Spark, but shell tools too)
4. Simpler value extraction using jq (edgedb | jq .field)
5. It's relatively easy to reformat to tab-separated using jq (jq '@tsv "\([.field1, .field2])"' -r)

Tab-separated

Upsides:

1. Simple values can be read by all shell tools as is
2. JSON values can be represented without double escaping
3. "JSON Option 1a" can be emulated manually SELECT <json>... (tabs are escaped as \t in JSON)
4. More patterns are easily accessible, like SELECT (Type.field1, <json>Type { field2, field3})

Downsides:

1. Unclear how to represent structured output (probably as JSON)
2. Column names of the top level aren't there

Notes

1. We can implement all listed modes but the most useful should be selected by default
2. We should probably optimize for easier twiddling in the shell. It isn't hard to add a CLI flag when importing to Hadoop.
3. The (2) should not be taken as an excuse for non-streaming data. Because dealing with larger than memory data is a must for the database.

Dear EdgeDB team,

I am following edgedb already for a while and I really like the project and the ideas behind it. But am a bit hesitant to use it before the rust client is finished / stable / maybe beta ;) Therefore I would like to ask, what the current status is here and if you need any help / support to get this potentially going.

Best regards!

Tim

As this tweet says:

All client libraries are designed to gracefully reestablish connections as needed.

But this does not happen for edgedb-tokio. I see this error a lot when I leave my server running for too long. It is fixed as soon an I restart the server.

I have not deployed my project to production yet but I think this will cause serious problems there if there are no database requests for some time.

I have the following:

module default {
    type User {
        required link profile -> UserProfile {
            ON TARGET DELETE DELETE SOURCE;
            ON SOURCE DELETE DELETE TARGET;
        };
        required link auth -> UserAuth {
            ON TARGET DELETE DELETE SOURCE;
            ON SOURCE DELETE DELETE TARGET;
        };
    }

    type UserProfile {
        required property email ->  str {
            constraint exclusive;
        };
        required property username -> str {
            constraint exclusive;
        };
    }

    type UserAuth {
        property password_hash -> str;
    }
}

And I am trying to execute the following query:

SELECT {
  is_username_unique := EXISTS (SELECT UserProfile FILTER .username = <str>$0),
  is_email_unique := EXISTS (SELECT UserProfile FILTER .email = <str>$1)
}

It works fine in the edgedb UI.

Here is the rust code calling this query:

#[derive(Debug, Deserialize, Serialize, Queryable)]
struct T {
    is_username_unique: bool,
    is_email_unique: bool,
}
self.db_conn
    .query_required_single::<T, _>(CHECK_UNIQUENESS, &(username, email))
    .await
    .unwrap();

But I am getting this error:

I think this might be a bug but I don't really know where to start debugging. Any help will be appreciated.

Here is a workaround I am using for the time being until I figure it out:

 let a = serde_json::from_str::<T>(
            self.db_conn
                .query_required_single_json(CHECK_UNIQUENESS, &(username, email))
                .await
                .unwrap()
                .to_string()
                .as_str(),
        )
        .unwrap();

I'd expose 4 public modules from edgedb-protocol:

1. model - Types an application will need in its model
   • BigInt, Decimal
   • Duration, LocalDatetime, LocalDate, LocalTime
   • Re-export of uuid::Uuid
   • TBD: what to do about Datetime vs SystemTime and json
2. dynamic/value - Value, NamedTuple and related types which are needed to represent unknown edgedb data at runtime
3. serialization - traits for serialization, de-serialization (currently Queryable) and related types
4. messages - probably should split this one into a separate crate (or rather move the rest to a separate crate like edgedb-data)

What is the benefit of having RawCodec in addition to Queryable? It has a subset of the functionality, but for every type you'd want to implement RawCodec for, you'd also want to implement Queryable for. (It's also never used as trait, so if there are exceptions, you could just use a free function as helper)

I have the following EDGEQL query:

INSERT learning::Question {
  name := <str>$0,
  problem := <str>$1,
  slug := <str>$2,
  classes := (
    SELECT learning::Class
    FILTER .id in array_unpack(<array<uuid>>$3)
  ),
  authored_by := (
    SELECT users::Teacher
    FILTER .id in array_unpack(<array<uuid>>$4)
  )
}

And I am trying to insert the following data:

{ 
    "name": "hello world",
    "problem": "some text",
    "slug": "hello-world",
    "classes": [],
    "authored_by": []
}

It works when I try doing it via the UI.

Here is the complete error:

• EdgeDB Version: current master branch (maybe pre-1.0.6?)
• OS Version: Debian 10

Steps to Reproduce:

1. cargo init --lib <libname>
2. cd <libname>
3. echo 'edgeql-parser = { git = "https://github.com/edgedb/edgedb", version = "0.1.0" }' >> Cargo.toml
4. echo 'edgeql-rust = { git = "https://github.com/edgedb/edgedb", version = "0.1.0" }' >> Cargo.toml
5. cargo build

Output:

   Compiling edgeql-parser v0.1.0 (https://github.com/edgedb/edgedb#c6dc224f)
   Compiling edgedb-protocol v0.1.0 (https://github.com/edgedb/edgedb-rust#32dd51bd)
   Compiling edgeql-rust v0.1.0 (https://github.com/edgedb/edgedb#c6dc224f)
error[E0603]: struct `BigInt` is private
  --> /home/david/.cargo/git/checkouts/edgedb-e90570e8674397e2/c6dc224/edb/edgeql-rust/src/pynormalize.rs:11:30
   |
11 | use edgedb_protocol::value::{BigInt, Decimal};
   |                              ^^^^^^ private struct
   |
note: the struct `BigInt` is defined here
  --> /home/david/.cargo/git/checkouts/edgedb-rust-8bc8936b1acdc166/32dd51b/edgedb-protocol/src/value.rs:3:21
   |
3  | use crate::model::{ BigInt, Decimal, LocalDatetime, LocalDate, LocalTime, Duration, Uuid };
   |                     ^^^^^^

error[E0603]: struct `Decimal` is private
  --> /home/david/.cargo/git/checkouts/edgedb-e90570e8674397e2/c6dc224/edb/edgeql-rust/src/pynormalize.rs:11:38
   |
11 | use edgedb_protocol::value::{BigInt, Decimal};
   |                                      ^^^^^^^ private struct
   |
note: the struct `Decimal` is defined here
  --> /home/david/.cargo/git/checkouts/edgedb-rust-8bc8936b1acdc166/32dd51b/edgedb-protocol/src/value.rs:3:29
   |
3  | use crate::model::{ BigInt, Decimal, LocalDatetime, LocalDate, LocalTime, Duration, Uuid };
   |                             ^^^^^^^

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0603`.
error: could not compile `edgeql-rust`.

To learn more, run the command again with --verbose.

psql / libpq can read auth information from a .pgpass tabfile with the following format:

<host>:<port>:<database>:<user>:<password>

We don't do per-database auth, so our tab would omit the database field.

Compare the following:

queries> \d Media
abstract type default::Media {
    required single link __type__ -> schema::Type {
        readonly := true;
    };
    required single link author -> default::User;
    multi property hashtags -> std::str;
    required single property id -> std::uuid {
        readonly := true;
    };
    required single property post_date -> std::datetime {
        default := datetime_current();
    };
};
queries> DESCRIBE OBJECT Media AS TEXT;
{
  "abstract type default::Media {\n    required single link __type__ -> schema::Type {\n        readonly := true;\n    };\n    required single link author -> default::User;\n    multi property hashtags -> std::str;\n    required single property id -> std::uuid {\n        readonly := true;\n    };\n    required single property post_date -> std::datetime {\n        default := datetime_current();\n    };\n};",
}

Technically, these two should be displaying the same text, but because of string rendering quirks the output of the DESCRIBE is almost unreadable, which kinda defeats the purpose. I think that when rendering the output string of the DESCRIBE command we should not be escaping any whitespace characters.

The derived code contains some hardcoded local variables (buf, elements, ...) but it also creates a local variable for each field. I believe proc macros are not hygienic yet, so these could collide.

I see two possible solutions:

• don't generate local variables for fields (inline the expression into the struct construction)
• prefix the generated local variables, .e.g. field_foo instead of foo

Connection::execute and Connection::query are nice. Much better than echo ... | netcat

fn main() {
    let dsn = env::var("EDGEDB_DSN").unwrap();
    let mut builder = Builder::from_dsn(&dsn).unwrap(); 
    policy(&mut builder);
}
async fn policy(builder: &mut Builder) -> Result<Bytes, anyhow::Error> {
    task::block_on(async {
        let mut conn = builder.connect().await?;
        let res = conn.execute($some_statement).await?;
        let srv: Bytes =   res.as_ref()
                              .iter()
                              .map(|x| x) // placeholder
                              .cloned()
                              .collect::<Bytes>();
        println!("{:?}", std::str::from_utf8(&srv));
        Ok(srv)
    })
}

But there's a growing pain to implement Queryable on arbitrary Rust types. Any ideas on convenient ways to relieve this friction?

According to edgedb/edgedb#1768 , the date range was restricted to 0001 to 9999, but the Rust types can still represent the bigger range earlier versions of edgedb supported.

Hi,

I'm finally at a point with my program where I can send queries to EdgeDB, and I noticed that error messages are pretty unhelpful.

I'm using the eyre crate (similar to anyhow), and this is the current error I get:

Error: Error while storing the data in EdgeDB.

Caused by:
    EdgeQLSyntaxError: Unexpected ')'

Location:
    src/api/sync.rs:563:5

For some reason, the line and column number within the EdgeDB query source code isn't mentioned, which is pretty unhelpful (especially, because EdgeDB queries can be pretty big, because of the great compossibility of EdgeDB's query language).

If I change my main function to return Result<(), edgedb_tokio::Error>, the displayed error is as follows:

Error: Error(Inner { code: 67174656, messages: ["Unexpected ')'"], error: None, headers: {257: b"Traceback (most recent call last):\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/common/parsing.py\", line 420, in parse\n self.parser.token(token)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/parsing/lrparser.py\", line 57, in token\n self._act(token, tokenSpec) # type: ignore\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/parsing/lrparser.py\", line 81, in _act\n raise UnexpectedToken(\"Unexpected token: %r\" % sym)\nparsing.errors.UnexpectedToken: Unexpected token: <Token RPAREN \")\">\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 1738, in _compile\n return self._try_compile(ctx=ctx, source=source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 1764, in _try_compile\n statements = edgeql.parse_block(source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/edgeql/parser/__init__.py\", line 77, in parse_block\n return parser.parse(source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/common/parsing.py\", line 433, in parse\n raise self.get_exception(\nedb.errors.EdgeQLSyntaxError: Unexpected ')'\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/common/parsing.py\", line 420, in parse\n self.parser.token(token)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/parsing/lrparser.py\", line 57, in token\n self._act(token, tokenSpec) # type: ignore\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/parsing/lrparser.py\", line 81, in _act\n raise UnexpectedToken(\"Unexpected token: %r\" % sym)\nparsing.errors.UnexpectedToken: Unexpected token: <Token RPAREN \")\">\n\nThe above exception was the direct cause of the following exception:\n\nTraceback (most recent call last):\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler_pool/worker_proc.py\", line 55, in worker\n res = meth(*args)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler_pool/worker.py\", line 160, in compile\n units, cstate = COMPILER.compile(\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 2207, in compile\n unit_group = self._compile(ctx=ctx, source=source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 1747, in _compile\n raise denormalized_err\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 1745, in _compile\n self._try_compile(ctx=ctx, source=original)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/server/compiler/compiler.py\", line 1764, in _try_compile\n statements = edgeql.parse_block(source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/edgeql/parser/__init__.py\", line 77, in parse_block\n return parser.parse(source)\n File \"/home/user/.local/share/edgedb/portable/2.1/lib/python3.10/site-packages/edb/common/parsing.py\", line 433, in parse\n raise self.get_exception(\nedb.errors.EdgeQLSyntaxError: Unexpected ')'\n", 65530: b"1045", 65524: b"17", 65527: b"18", 65528: b"17", 65529: b"1044", 65525: b"16", 65522: b"1045", 65526: b"33", 65521: b"1044", 65523: b"33"} })

...which isn't great either.

If I unwrap the Result that contains the error, the displayed error is similar unhelpful.

Unfortunately, edgedb_errors::Error isn't documented (and seems more complicated than it should need to be), so I ended up adding the following code to figure out what is going on:

        .map_err(|e| {
            for (k, v) in e.headers() {
                let v = std::str::from_utf8(v.as_ref()).unwrap();
                println!("{k} -> {v}")
            }
            println!("kind_name => {}", e.kind_name());
            println!("kind_debug => {}", e.kind_debug());
            println!("initial_message => {:?}", e.initial_message());
            println!("hint => {:?}", e.hint());
            println!("details => {:?}", e.details());
            println!("position_start => {:?}", e.position_start());
            println!("position_end => {:?}", e.position_end());
            println!("line => {:?}", e.line());
            println!("column => {:?}", e.column());
            println!("code => {}", e.code());
            println!("server_traceback => {:?}", e.server_traceback());
            println!(
                "contexts => {:#?}",
                e.contexts().map(|x| format!("{x:?}")).collect::<Vec<_>>()
            );
            println!(
                "chain => {:#?}",
                e.chain().map(|x| format!("{x:?}")).collect::<Vec<_>>()
            );

            e
        })?;

Of that code, the following lines were useful:

            println!("line => {:?}", e.line());
            println!("column => {:?}", e.column());

It would be great, if the line and column number could be included with displayed errors.

In the meantime (in case someone else who needs this right now is stumbling upon this issue), I've created a small macro that adds the location of errors:

macro_rules! add_err_loc {
    () => {
        |err| {
            let line = line!();
            let column = column!();
            let msg = if let (Some(line2), Some(column2)) = (err.line(), err.column()) {
                format!("EdgeDB query error – at {line2}:{column2} of the query, and at {line}:{column} of the Rust src code.")
            } else {
                format!("EdgeDB query error – at {line}:{column} of the Rust src code.")
            };
            eyre::Report::new(err).wrap_err(msg)
        }
    };
}

...which can be used like this:

db.query::<String, _>("SELECT 'hello'", &())
    .await
    .map_err(add_err_loc!())?;

This also adds the Rust source code location, which seems to get lost when transactions are used, which is the reason why I've implemented this as a macro.

It's worrying me a bit that there is a Python traceback within the error (that it is there at all, and that the query parsing code fails with a traceback).

Are there any plans to replace that Python code with Rust?

Performance should be good enough with caching, but I'm pretty worried about correctness, which is why I think Rust would be a far better option for such code (code, which seems to handle potential user-supplied input, and therefore could lead to security vulnerabilities).

Rust also has fantastic libraries for displaying errors in source code, that would make it relatively easy to display errors similar to the Rust compiler:

• https://crates.io/crates/miette/
• https://crates.io/crates/ariadne/
• https://crates.io/crates/annotate-snippets/

Rust most likely also has better tools for type checking (for example RustTyC, which seems to make it easy to implement a Hindney-Milner-like type system)

The ability to clone a connection would be very useful. It would keep the options and remove all the unnecessary logic that multiple create_client() calls would run.

I cloned the repository and did some initial testing of adding #derive(Clone) to the Client struct. My very basic example worked, but the tests failed with poised lazy instance. If you need more details, just ask. (I could also open a PR.)

Thank you,
~BD103

I am trying to create a new client configuration that connects to my edgedb instance without validating the TLS certificate chain.

According to the documentation, I should pass an enum called TlsSecurity to the .tls_security method of the builder; however, this module does not seem to be exported by the library.

Example

let client_config = edgedb_tokio::Builder::uninitialized()
    .password("secret")
    .tls_security(edgedb_tokio::credentials::TlsSecurity::Insecure)
    .build()?;

Cargo.toml

[dependencies]
edgedb-tokio = { version = "0.3.0", features = ["unstable"] }

This happens for me in the current master branch (76777fe)

victor> SELECT <schema::Cardinality>'bad';
Error: error fetching element

Caused by:
    server message out of order: ErrorResponse(ErrorResponse { 
severity: Error, code: 83951616, message: "invalid input value for 
enum \'schema::Cardinality\': \"bad\"", attributes: {257: b"edb.errors.
InvalidValueError: invalid input value for enum 'schema::Cardinality': 
\"bad\"\n"} })

After this error the REPL just quits.

Seems that there's an accidentally committed merge file

edgedb-python is still updating . When edgedb-rust replace edgedb-python ? Is edgedb-rust just a toy or subject?

Motivation

We have few types in EdgeDB which don't have a rust type in rust standard library:

• std::LocalDateTime
• std::LocalDate
• std::LocalTime
• std::decimal
• std::bigint

There are crates that implement those types:

• chrono::NaiveDateTime
• chrono::NaiveDate
• chrono::NaiveTime
• num-bigint::BigInt
• bigdecimal::BigDecimal

But they are quite big to always depend on them (even chrono) and also there might be alternatives (e.g. there is decimal crate but that type is backed by u128, not unlimited precision).

Approach

For Dynamic Typing

1. The edgedb_protocol::value::Value type should contain an opaque type:

   pub struct Decimal { private_bytes: Bytes };
   pub enum Value {
       Decimal(Decimal)
   }
   

2. We add a feature flag for each such dependency:
   • with-num-bigint
   • with-bigdecimal
   • with-chrono
3. And for each opaque type, we implement conversion traits From and Into, TryFrom and TryInto where appropriate
4. Also, we add a feature flag all-types that enables at least one conversion method for every such type (to the most popular crate at the time of addition)

To activate a flag users needs to specify it in dependency declaration:

[dependencies]
edgedb-protocol = {version = "1.0", features=["with-bigdecimal", "with-chrono", "with-num-bigint"]}

Or just:

[dependencies]
edgedb-protocol = {version = "1.0", features=["all-types"]}

The latter we will use for edgedb-repl

For Static Typing

When we implement derive for the type, we would enable deserializing specified type by the same feature name bigdecimal, num-bigint and chrono.

For custom types, we should probably add analog of the #[serde(with="custom_module")] annotation.

Update 1: feature equal, crate name actually works; add all-types feature
Update 2: add a note about custom deserializer for static typing
Update 3: use conversion traits instead of named methods
Update 4: Use feature names distinct from crate name, because it's impossible to work with num_bigint without also num_trait; the prefix is always plural (_types) because even bigint has two types; also mention Try* traits.
Update 5: rename xx_types to with-xxx, the latter is what rust-postgres uses and it's better because it's possible to add versioned feature flags like with-chrono-v4

The patch should be quite similar to str codec or duration codec.

The basic idea is:

1. You add and empty codec structure in edgedb-protocol/src/codecs.rs
2. Then you implement trait Codec from it, that receives a buffer
3. And you register codec in scalar_codec function
4. Add needed error variants to edgedb-protocol/src/codecs.rs DecodeError
5. Add a test for decoding a type descriptor from binary data [*]
6. Add tests for decoding value itself from binary data

[*] We have unpacking code for all the descriptors but no tests yet

To get wire data, just run a repl and execute a query:

cargo run --bin edgedb
     
    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/edgedb`
tutorial> select 1
Descriptor: CommandDataDescription { headers: {}, result_cardinality: One, input_typedesc_id: 00000000-0000-0000-0000-0000000000ff, input_typedesc: b"\x04\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\xff\0\0", output_typedesc_id: 00000000-0000-0000-0000-000000000105, output_typedesc: b"\x02\0\0\0\0\0\0\0\0\0\0\0\0\0\0\x01\x05" }
Codec Int64
Data Data { data: [b"\0\0\0\0\0\0\0\x01"] }
Row Ok(Scalar(Int64(1)))

• CommandDataDescriptor.output_typedesc_id is the type descriptor data to put in the test
• Data.data is the data to put in the codecs' test

It seems, the Queryable trait isn't implemented for edgedb_protocol::value::Value.

Value would be quite valuable for debugging, logging, and so on.

I might miss something, but it looks like, that edgedb_tokio::transaction is not really usable in regular code.

The problem is, that the closure it accepts has to return edgedb_tokio::Error, which can't be constructed by users (as far as I can see).

A possible fixes could be to turn edgedb_tokio::Error into an enum with a type parameter E and an additional variant Custom(E), and make that variant constructable by external code.

Then transaction could accept a closure that returns an error type that implements Into<edgedb_tokio::Error>. Ortransaction could accept a type parameter E, and expect the error type edgedb_tokio::Error<E>.

In the meantime, are there any disadvantages by creating transactions manually? This seems to be the only usable option, at the moment.

1. REPL
   • Prompt bugs:
     • Cursor up doesn't go over visual line (kkawakam/rustyline#317, kkawakam/rustyline#324)
     • Enter in the middle of the line should not submit input even if that's fully valid
     • No continuation prompt (kkawakam/rustyline#311)
     • Syntax highlighting works inconsistently as you type
     • Multi-line entries in history file are split by lines (kkawakam/rustyline#334)
   • Detect disconnect and reconnect (also try: kkawakam/rustyline#235)
   • Display log messages
   • Formatting result sets
     • Basic
     • Prettify
     • JSON
       • Highlight/prettify JSON
     • Implicit properties
     • Implicit id on empty objects
     • Introspect types
     • Fix colors/styles to term256
   • Non-interactive mode
     • Basics
     • Tab-separated mode
     • Process last non-semicolon-delimited statement
   • Preparser should count parenthesis #5
   • Nice errors in long queries edgedb/edgedb#1148
   • Slash commands
     • \d, \describe
       • add syntax highlight to the output
     • \l, \list-databases
     • \lr, \list-roles
     • \lm, \list-modules
     • \lT, \list-scalar-types
       • basic
       • parameters
     • \lt, \list-object-types
     • \la, \list-aliases
     • \lc, \list-casts
     • \li, \list-indexes
     • \limit
     • ?
     • \c
     • \E, \last-error, \verbose-errors
     • \psql
     • \pgaddr
     • \e, \edit (edgedb/edgedb#1247)
     • \s, \history (edgedb/edgedb#1247)
     • Completion of slash command arguments (kkawakam/rustyline#229 (comment))
     • Word-wrap tables e.g. \ltS
   • Syntax highlighting
   • Connect using unix socket in --admin mode
   • Ellipsis for over the limit link fetches
   • Repl hangs on invalid functions
   • Save history
2. Commands
   • create/alter/drop role
   • create-superuser-role
   • configure
   • dump/restore
   • update command-line edgedb/edgedb#1039
3. Protocol implementation
   • Check what's left
   • Rename feature flags to with-* #6
   • dump/restore
   • LogMessage
   • ParameterStatus
   • Flush
   • use Flush instead of Sync during a single request processing
   • OptimisticExecute
   • Terminate
   • Bad protocol in handshake might need to be handled better
4. Packaging
   • Static, musl-based build
   • Actual distribution package
5. Extra features
   • create-database --init-schema-file=xxx.edgeql
   • allow $vars in REPL
     • non-string types support

Hi
Great project btw! Would like to see it working.
How to run it with Tokio runtime?
It will be nice if you adapt sqlx approach with supporting three different runtimes (https://github.com/launchbadge/sqlx/tree/master/sqlx-rt).

Previous discussion: #44 and #38

Options so far:

1. data_types is what is used in Python (or rather datatypes), but can be perceived to long for referencing too often
2. Keep it model (the name was was not discussed beforehand so should not be considered status quo)
3. data is shorter for data types
4. scalars -- but this might stop using it for more complex types if we need in future
5. Make it a separate crate, e.g. edgedb-types

Personally, I still like model, but data is also fine.

Would it be possible to support something similar to sqlx?

I think it could be really nice to have this feature when it comes to have a great developer experience 🙂

https://github.com/launchbadge/sqlx#sqlx-is-not-an-orm

This would offer greater flexibilty for serializing any array-like construct. For example, it would remove the need for intermediate allocations when transforming data before serializing it.

For example:

impl From<u32> for BigInt {
    fn from(v: u32) -> BigInt {
        return BigInt {
            negative: false,
            weight: 1,
            digits: vec![(v / 10000) as u16, (v % 10000) as u16],
        }.normalize();
    }
}

dividing a 32-bit value by 10_000 produces values up to 429_496, which is bigger than 2^16-1=65_535, overflowing the u16.

Fields in an Object can be optional/an empty set. Edgedb-derive generates code which assumes they're always present.