I believe it reaches the "trivia" id and crashes.

[2022-11-12 22:12:28] DEBUG surrealdb::iam Authenticated as super user
[2022-11-12 22:12:28] DEBUG surrealdb::dbs Executing: CREATE type::thing($table, $id) CONTENT $data
[2022-11-12 22:12:28] INFO  surreal::web 73.112.51.144:59429 POST /key/config/topics HTTP/1.1 200 "python-httpx/0.23.0" 3.223868ms[2022-11-12 22:12:28] DEBUG surrealdb::iam Authenticated as super user
thread 'tokio-runtime-worker' panicked at 'called `Option::unwrap()` on a `None` value', lib/src/sql/strand.rs:156:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted (core dumped)

import asyncio
from pprint import pprint
from surrealdb.clients.http import HTTPClient

# create a new client to connect to SurrealDB
client = HTTPClient(
    ip,
    namespace='test1',
    database='a',
    username='root',
    password='root',
)


defaults_example = {
    'color': 'red',
    'channel_id': 123456789,
    'topics': [
        'abcd',
        'defg',
        'hijk'
    ],
    'trivia': [
        {
            'Question': 'How many?',
            'Answer': '2',
        },
        {
            'Question': 'What is Obama\'s last name?',
            'Answer': '😐'
        }
    ]
}


async def config_test():
    for id, data in defaults_example.items():
        response = await client.create_one('config', id, data)
        pprint(response)


async def select_all():
    response = await client.select_all('config')
    pprint(response)


async def main():
    await config_test()
    await select_all()


asyncio.run(main())

A new version needs to be uploaded to PyPi that includes the fix in #26 .

Otherwise, authentication is impossible.

Workaround: install with pip install git+https://github.com/surrealdb/surrealdb.py.git

Is your feature request related to a problem?

The current HTTP client has no method merge to partially update record(s) by performing a query like

UPDATE person:tobbie MERGE {
    settings: {
        active: true
    }
};

Describe the solution

According to the official documentation from https://surrealdb.com/docs/integration/http#modify-one, it can be easily solved by adding the following code to class SurrealHTTP

async def merge(self, thing: str, data: Any) -> Dict[str, Any]:
        """Partially update records in a table, or a specific record, in the database.

        This function partially replaces the current document / record data with the
        specified data.

        This function will run the following query in the database:
        update $thing merge $data

        Args:
            thing: The table or record ID.
            data: The document / record data to update.

        Examples:
            Update all records in a table
                person = await db.merge('person')

            Update a record with a specific ID
                record = await db.merge('person:tobie', {
                    'name': 'Tobie',
                    'settings': {
                        'active': true,
                        'marketing': true,
                        },
                })
        """
        table, record_id = thing.split(":") if ":" in thing else (thing, None)
        response = await self._request(
            method="PATCH",
            uri=f"/key/{table}/{record_id}" if record_id else f"/key/{table}",
            data=json.dumps(data, ensure_ascii=False),
        )
        return response[0]['result']

and I've tested that it works correctly.

Alternative methods

no yet

SurrealDB version

surreal 1.0.0 for Linux on x86_64

surrealdb.py version

surrealdb.py 0.3.1 for Linux on x86_64 using Python 3.10

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

The HTTP client needs refactoring. There are several issues that I have found when running mypy.

Some of these errors include:

surrealdb/http.py:102: error: "SurrealHTTP" has no attribute "disconnect"; maybe "connect"?  [attr-defined]
surrealdb/http.py:147: error: Incompatible return value type (got "SurrealResponse", expected "str")  [return-value]
surrealdb/http.py:159: error: Incompatible return value type (got "SurrealResponse", expected "str")  [return-value]
surrealdb/http.py:184: error: Incompatible return value type (got "SurrealResponse", expected "List[Dict[str, Any]]")  [return-value]
surrealdb/http.py:213: error: Value of type "SurrealResponse" is not indexable  [index]
surrealdb/http.py:246: error: Value of type "SurrealResponse" is not indexable  [index]
surrealdb/http.py:281: error: Value of type "SurrealResponse" is not indexable  [index]
surrealdb/http.py:314: error: Value of type "SurrealResponse" is not indexable  [index]
surrealdb/http.py:336: error: No return value expected  [return-value]

As well as this there are several differences between the HTTP client and WS client. This can be confusing to users. For example:

• The __init__ method takes different arguments.
• Several methods are missing in the HTTP client.

Steps to reproduce

If mypy is installed in your environment:

poetry run mypy .

Expected behaviour

• No errors when running mypy.
• The methods and between clients are the same.

SurrealDB version

surreal 1.0.0-beta.8+20220930.c246533 for macos on x86_64

surrealdb.py version

surrealdb.py 0.30 for macOS on aarch64 using Python 3.9.1

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

Most methods return annotations are List[Dict[str, Any]].
This is correct when querying for (potentially) multiple items like Surreal.select("table"), without supplying a specific ID.
This is incorrect when querying with a specific ID, like Surreal.select("table:id"). Instead of a list of dicts, it directly returns a single dict.

Another thing missing in the typing of Optional. When select or delete get supplied a non-existent ID, the method returns None.

These issues only seem to be the case with Surreal (WS/RPC client), and not with SurrealHTTP, which does always return a list.

Steps to reproduce

async def example(db: Surreal) -> None:
    print(type(await db.create("table:id")))
    print(type(await db.update("table:id", {"field": "value"})))
    print(type(await db.select("table:id")))
    print(type(await db.delete("table:id")))
    print(await db.select("table:wrong_id"))
    print(await db.delete("table:wrong_id"))

results in:

<class 'dict'>
<class 'dict'>
<class 'dict'>
<class 'dict'>
None
None

While all methods are typed to return type of <class 'list'>

Expected behaviour

I can see a few options to resolve this, listed below with their pros and cons:

1. Change the typing of the affected methods, e.g. Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]
   + This is a non-breaking change, only affecting type annotations
2. Use type overloading, annotating either list or dict
   - This will only work if the id becomes a separate optional parameter - breaking change
   + More precise typing than Union
3. Only return list, as it is currently typed
   - This would not change the typing, but instead adjust the return value to the existing type - breaking change
   + This would make the Surreal and SurrealHTTP classes more aligned, returning the same types.

My opinion would be to implement idea number 1 now, as it is non-breaking. And keep the other ideas open for a breaking version bump (1.0?)

Please feel free to suggest other ideas as well.

And note, I'm more than happy to create the PR once we've decided on the desired solution.

SurrealDB version

1.3.1 for linux on x86_64

surrealdb.py version

0.3.2 (latest as of 2024-04-02)

Contact Details

Ideally in the issue itself. Otherwise: [email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Are there any plans to update the README? Some things that I think should be added include:

• Installation instructions
• A simple demo for how to use the driver
• A link to further documentation
• Maybe a contributing/code of conduct section?
• Maybe an image/surrealdb logo or something cool at the top (not necessary but kind of cool)

I would be happy to add any or all of the above to the README myself should the owners/maintainer like that.

Describe the bug

I have both python and rust installed ( i already use them in work ) , i tried to install the with-no rust runtime library but it fails to build and outputs an error message related to cross_beam channel compile error .

Compiling crossbeam-channel v0.5.8
Running rustc --crate-name crossbeam_channel --edition=2018 C:\Users\zaki2\.cargo\registry\src\github.com-1ecc6299db9ec823\crossbeam-channel-0.5.8\src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type lib --emit=dep-info,metadata,link -C opt-level=3 -C embed-bitcode=no --cfg "feature=\"crossbeam-utils\"" --cfg "feature=\"default\"" --cfg "feature=\"std\"" -C metadata=7d555cfdae4e1b1b -C extra-filename=-7d555cfdae4e1b1b --out-dir C:\Users\zaki2\AppData\Local\Temp\pip-req-build-ctjhnckw\target\release\deps -L dependency=C:\Users\zaki2\AppData\Local\Temp\pip-req-build-ctjhnckw\target\release\deps --extern cfg_if=C:\Users\zaki2\AppData\Local\Temp\pip-req-build-ctjhnckw\target\release\deps\libcfg_if-537c6f8aaba5b4b9.rmeta --extern crossbeam_utils=C:\Users\zaki2\AppData\Local\Temp\pip-req-build-ctjhnckw\target\release\deps\libcrossbeam_utils-2621beb9f2d66a0d.rmeta --cap-lints allow
error: future cannot be sent between threads safely
--> C:\Users\zaki2.cargo\git\checkouts\surrealdb-8c51c5db53af7153\ee3a1c2\lib\src\dbs\iterator.rs:427:2
|
427 | #[async_recursion]
| ^^^^^^^^^^^^^^^^^^ future created by async block is not Send
|
= help: within [async block@C:\Users\zaki2\.cargo\git\checkouts\surrealdb-8c51c5db53af7153\ee3a1c2\lib\src\dbs\iterator.rs:427:2: 427:20], the trait std::marker::Send is not implemented for *const async_executor::State
note: future is not Send as this value is used across an await
--> C:\Users\zaki2.cargo\registry\src\github.com-1ecc6299db9ec823\async-executor-1.5.2\src\lib.rs:936:17
|
889 | let mut old = with_waker(|waker| {
| ------- has type std::option::Option<async_executor::LocalQueue> which is not Send
...
936 | .await
| ^^^^^^ await occurs here, with mut old maybe used later
937 | }
| - mut old is later dropped here
= note: required for the cast from [async block@C:\Users\zaki2\.cargo\git\checkouts\surrealdb-8c51c5db53af7153\ee3a1c2\lib\src\dbs\iterator.rs:427:2: 427:20] to the object type dyn futures::Future<Output = std::result::Result<(), err::Error>> + std::marker::Send
= note: this error originates in the attribute macro async_recursion (in Nightly builds, run with -Z macro-backtrace for more info)

  error: could not compile `surrealdb` due to previous error

Steps to reproduce

using python 3.10 run pip install git+https://github.com/surrealdb/surrealdb.py@rust-no-runtime

Expected behaviour

i expect the library to just compile with no problems since python and rust already exist

SurrealDB version

surreal 1.0.0-beta.8+20220930.c246533 for windows on x86_64

surrealdb.py version

surrealdb-beta 0.0.3

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Hello,

I'm quite new on Surrealdb and trying to test the python driver. I saw the websocket modules but can't find any equivalent to the live query, is this supported yet ? Do you plan to do it / wants PR ?

The bug

Surreal.query("SLEEP") returns [{"result": "IAM error: Not enough permissions to perform this action", status: "ERR", time="116.125µs"}]

It might be not only SLEEP

Steps to reproduce

Python code:

from surrealdb import Surreal

async with Surreal(...) as db:
    print(await db.query("SLEEP 1s"))  # Will print an error

Using $ surreal sql shell:

> SLEEP 1s  -- Will be fine

Expected behaviour

Actually 'sleeping' for 1s

SurrealDB version

1.3.1 for macos on aarch64

surrealdb.py version

0.3.2 for macos on aarch64 using Python 3.12

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

When running the examples I get the following error:

Traceback (most recent call last):
  File "/home/main.py", line 89, in <module>
    asyncio.run(run_all())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/main.py", line 79, in run_all
    await create_all()
  File "/home/main.py", line 11, in create_all
    response = await client.create_all(table, data)
  File "/home/.venv/lib/python3.8/site-packages/surrealdb/clients/http.py", line 168, in create_all
    response = await self._request(
  File "/home/.venv/lib/python3.8/site-packages/surrealdb/clients/http.py", line 117, in _request
    surreal_data = surreal_json[0]
KeyError: 0

I have create a PR #26 to solve this issue. Not only that, it's also not handled as a authentication error. This is the content of surreal_json:

{'code': 403, 'details': 'Authentication failed', 'description': 'Your authentication details are invalid. Reauthenticate using valid authentication parameters.', 'information': 'There was a problem with authentication'}

Personally, I don't like just passing in the url like py client = HTTPClient("http://localhost:8000", namespace="test", database="test", username="root", password="root") because it is a bit confusing and inefficient. For example, if the url localhost is passed the requests made are actually significantly slower, speaking from past experience.
This is due to localhost being an extra local dns conf lookup.

To prevent this, perhaps there could be a config class for url and port. This way both https and http can be tried to ensure the correct protocol is used. This helps because if a user doesn't understand the difference in the protocols they won't receive raw errors.
In addition, reserved ports can be checked, localhost can be changed to 127.0.0.1 and protocols can be checked.

Describe the bug

i try to connect to surrealdb database through websocket, the connection works but the python app didn't receive the answer from database until websocket error (about 5 seconds)

Steps to reproduce

here is code python:

@router.post('/test/user')
async def get_user(request: Request):
    start_time = time.time()
    print(start_time)
       
    user = "lucile.coutouly"
    pas = "test"
   
    async with Surreal("ws://surrealdb:8000/rpc") as db:
        jeton = await db.signin({
            'NS': 'test',
            'DB': 'test',
            'SC': 'account',
            'user': user,
            'pass': 'nancay'
            })                
       
        print("---------------jeton")
   
    end_time = time.time()
    print(f"Time taken: {end_time - start_time} seconds")
    

and i get this from the two dockers:

archi-surrealdb-1  | 2023-08-04T13:04:19.029600Z  INFO request: surreal::net::tracer: started processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" user_agent.original="Python/3.11 websockets/10.4" otel.name="GET /rpc" http.route="/rpc" http.request.id="19ecf533-a9cf-4ddd-8e48-38d709f5d39b" client.address="172.18.0.4"
archi-surrealdb-1  | 2023-08-04T13:04:19.029758Z  INFO request: surreal::net::tracer: finished processing request otel.kind="server" http.request.method="GET" url.path="/rpc" network.protocol.name="http" network.protocol.version="1.1" user_agent.original="Python/3.11 websockets/10.4" otel.name="GET /rpc" http.route="/rpc" http.request.id="19ecf533-a9cf-4ddd-8e48-38d709f5d39b" client.address="172.18.0.4" http.response.body.size="0" http.response.status_code=101 http.latency.ms=0
archi-surrealdb-1  | 2023-08-04T13:04:19.030158Z TRACE surreal::net::rpc: WebSocket c8da133a-07c6-4a0e-a036-534ce62121cf connected
archi-surrealdb-1  | 2023-08-04T13:04:19.035981Z TRACE surreal::net::rpc: RPC Received: { id: '8cb81b92-b349-4e26-8aff-bc8204139c3e', method: 'signin', params: [{ DB: 'test', NS: 'test', SC: 'account', pass: 'nancay', user: 'lucile.coutouly' }] }
archi-surrealdb-1  | 2023-08-04T13:04:19.036283Z TRACE rpc signin:compute: surrealdb::dbs::iterator: Iterating: SELECT * FROM user WHERE user = $user AND crypto::argon2::compare(pass, $pass) websocket="c8da133a-07c6-4a0e-a036-534ce62121cf"
archi-surrealdb-1  | 2023-08-04T13:04:19.066350Z TRACE rpc response: surreal::rpc::res: Response sent response=Response { id: Some(Uuid(Uuid(8cb81b92-b349-4e26-8aff-bc8204139c3e))), result: Ok(Other(Strand(Strand("eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzUxMiJ9.eyJpYXQiOjE2OTExNTQyNTksIm5iZiI6MTY5MTE1NDI1OSwiZXhwIjoxNjkxMjQwNjU5LCJpc3MiOiJTdXJyZWFsREIiLCJOUyI6InRlc3QiLCJEQiI6InRlc3QiLCJTQyI6ImFjY291bnQiLCJJRCI6InVzZXI64p-obHVjaWxlLmNvdXRvdWx54p-pIn0.2kxuTcDpMN06qYkdkzQfCuCxOaF2wTNh_IfNi-HMp2C_OJ2LL3PeOztgP0ZGP8EPRjthLoNtzok_BbBgr25elQ")))) }
archi-surrealdb-1  | 2023-08-04T13:04:19.067930Z TRACE surreal::net::rpc: WebSocket c8da133a-07c6-4a0e-a036-534ce62121cf disconnected






archi-surrealdb-1  | 2023-08-04T13:04:24.031213Z TRACE surreal::net::rpc: WebSocket error: Error { inner: ConnectionClosed }
archi-fastapi-1    | 1691154259.024649
archi-fastapi-1    | ---------------jeton
archi-fastapi-1    | ---------------role
archi-fastapi-1    | Time taken: 5.007425546646118 seconds
archi-fastapi-1    | INFO:     172.18.0.7:60342 - "POST /auth/test/user HTTP/1.1" 200 OK





archi-surrealdb-1  | 2023-08-04T13:04:24.286890Z TRACE surreal::net::rpc: RPC Received: { method: 'ping' }
archi-surrealdb-1  | 2023-08-04T13:04:24.286943Z TRACE rpc response: surreal::rpc::res: Response sent response=Response { id: None, result: Ok(Other(None)) }

Expected behaviour

the response should be instantaneted.

SurrealDB version

nightly on docker

surrealdb.py version

last

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

Here is an example:

result = await db.query(f'''
  INSERT INTO list {{
    name: $name,
    desc: $desc,
  }}
''', {"name": "hello", "desc": None})

Where the desc column is type option.

I'd expect "None" in python to evaluate to the option empty value, but I get this error message:

[{'result': 'Found NULL for field `description`, with record `...`, but expected a option<string>', 'status': 'ERR', 'time': '234.712µs'}]

Steps to reproduce

See "describe bug".

Expected behaviour

See "describe bug".

SurrealDB version

surreal 1.0

surrealdb.py version

python 3.11.5 - surrealdb lib 0.3.1

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Add query params with safety measures against sqli into full query methods(such as the execute method).

Look at how PyMySQL got this done, it will need to be modified a bit because sdb may need some type declarations.

Description

I wanted to contribute to the project, starting with some minor fixes (e.g. typing), but I didn't find any instruction for a local dev setup. So I had to do my own trial and error:

1. Clone the repo (my fork)
2. pip install -r requirements.txt -r dev_requirements.txt.
3. pytest tests/ -> ModuleNotFoundError: No module named 'surrealdb.rust_surrealdb'
4. ./scripts/run_tests.sh -> ModuleNotFoundError: No module named 'docker'
5. pip install docker
6. ./scripts/run_tests.sh -> ModuleNotFoundError: No module named 'surrealdb.rust_surrealdb'
7. cargo build
8. pip install -e .
9. ./scripts/run_tests.sh -> Succes!

So nothing crazy was required to get it working, but it would be nice if the required steps were briefly documented in the CONTRIBUTING.MD, rather than that everyone has to go through this trial&error process.

Make contributing easier, and you'll get more contributions 😄

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

As per the discussion on #39, we can implement ruff for code formatting in addition to black. The proposed ruff configuration is as follows:

• isort, which is compatible with black
• pydocstyle, with a preference for Google style but open to suggestions
• flake8
• By using ruff, we won't need the .flake8 file. Additionally, ruff works well with pre-commit hooks.

We should also include mypy for type-checking in the project.

Please let me know your thoughts. Thank you 😄

The line surreal_data = surreal_json[0] in the http.py client introduces a behavior which cannot be handled by the end user. The majority of errors are KeyError errors and are completely unhanded (the else case is skipped).

There needs to be an additional exception in the following manner which also gives more info for debugging:

try:
    surreal_json: List[Dict[str, Any]] = jsonlib.loads(surreal_raw_data)
    surreal_data = surreal_json[0]
except JSONDecodeError:
    raise SurrealException(
        f"Invalid JSON response from SurrealDB: {surreal_raw_data}",
    )
except KeyError:
    raise SurrealException(
        "Query failed with status code "
        + f"{surreal_response.status_code}"
        + f"\ndetails: {surreal_json['details']}"
        + f"\ninformation: {surreal_json['information']}"
        + f"\ndescription: {surreal_json['description']}",
    )

What should the code line length be? pep-8 suggests 79 characters, black defaults to 88, what would you guys suggest?

Hey Guys

Im an DBA with a big love for python. I expected to find some code already here, since on the website it says "coming soon".
Are you already working on this in an other repo or is there really nothing so far? Would also like to contribute some of my time to the project.

Greets
Jerome

Hey everyone 👋🏻

SurrealDB will change the game. It would be great to have a well-tested, typed, and documented python library to use for future data science/ML projects. I would love to start contributing to this library. I was wondering what the roadmap is for 2023 and how I can get involved. I have been looking through the repository and wanted to highlight some areas we could improve. Let me know your thoughts! I am open to working on these areas. I believe having the below would give the repository a great baseline to allow for contributions to be more streamlined.

Sorry if some of these aspects of the repo have already been discussed! I couldn't find anything, so I thought it best to brain-dump everything here 😆

Also, could we open discussions for this repo? I am happy to discuss on this issue or in Discord - Thanks!

Code Formatting 💯

Some suggestions for code formatting:

• isort
• black
• flake8
• pydocstyle

Code Quality ✅

Some suggestions for code quality:

• pytest
• coverage

Documentation 📖

When it comes to documentation, I am not sure needs to align with the other SurrealDB libraries or not? Saying that we can agree on a documentation style. I would vote for Google Style and using MkDocs for the documentation itself.

GitHub Actions ⚗️

Some suggestions for GitHub Actions:

• Code formatting (black/isort).
• Code quality (pytest/coverage/pydocstyle/flake8).
• Run tests using Python 3.7, 3.8, 3.9, 3.10, 3.11
• Run tests on macOS, Linux, and Windows.
• CodeCov.
• SonarCloud.
• Branch protection rules.

Housekeeping 🧹

Some nice to haves to allow for easier contribution:

• Automated labels.
• PR templates.
• Issue templates.
• Document list of changes.
• Contribution guidelines.

I think there are 2 redundant functions:

In the current implementation the difference between create_all and create_one
is the provision of id.
But create_all implies that we can provide more than 1 entry.

In both CREATE methods I would expect the provision of id to be optional.
additionally can we provide examples/usage of inserting more than 1 entry?

Describe the bug

trying to poetry add surrealdb uses 0.3.1

Steps to reproduce

• poetry add surrealdb
• using 0.3.2 in the pyproject.toml will give

Because project depends on surrealdb (^0.3.2) which doesn't match any versions, version solving failed.

Expected behaviour

poetry add surrealdb adds 0.3.2 latest

SurrealDB version

1.0.0

surrealdb.py version

0.3.1

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Description

As you can see in below image at heading its saying db.merge and in the code snippet its saying db.change also for db.patch title its showing db.modify which confuses whats the real name to used those methods

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

Using the query() method, occasionally I will get a URL component 'query' too long error (httpx module)

Traceback

2024-02-06 19:15:44,231 ERROR:Traceback: Traceback (most recent call last):

await db.query(f"UPDATE example:example SET input=type::string($i),output=type::string($o)")

File "/usr/local/lib/python3.12/site-packages/surrealdb/http.py", line 188, in query

response = await self._request(method="POST", uri="/sql", data=sql, params=vars)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/surrealdb/http.py", line 119, in _request

surreal_response = await self._http.request(

^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 1546, in request

request = self.build_request(

^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_client.py", line 359, in build_request

return Request(

^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_models.py", line 329, in __init__

self.url = self.url.copy_merge_params(params=params)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_urls.py", line 366, in copy_merge_params

return self.copy_with(params=self.params.merge(params))

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_urls.py", line 354, in copy_with

return URL(self, **kwargs)

^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_urls.py", line 117, in __init__

self._uri_reference = url._uri_reference.copy_with(**kwargs)

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_urlparse.py", line 134, in copy_with

return urlparse("", **defaults)

^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/lib/python3.12/site-packages/httpx/_urlparse.py", line 200, in urlparse

raise InvalidURL(f"URL component '{key}' too long")

httpx.InvalidURL: URL component 'query' too long

Relevant httpx code:

for key, value in kwargs.items():
       if value is not None:
           if len(value) > MAX_URL_LENGTH:
               raise InvalidURL(f"URL component '{key}' too long")

Where MAX_URL_LENGTH = 65536

I know with certainty that the URL length (from my inputs) do not exceed 65535 characters.

Steps to reproduce

Use the query() method and input a long string via the config, for example:

await db.query("CREATE code SET input=type::string($i),output=type::string($o)", {
    "i":"some input code",
    "o":"stdout"})

With my own node.js SurrealHTTP class, I've managed to mitigate any errors like this with these axios parameters:

let config = {
            headers: {
                'Accept': 'application/json'
            },
            params: bindings,   // bind variables in the request params
            maxContentLength: Infinity,
            maxBodyLength: Infinity,
        };

Expected behaviour

The query should execute correctly, without issues.

SurrealDB version

1.1.1+20240116.b261047

surrealdb.py version

surrealdb.py 0.3.1 for windows running Python 3.12.1

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

The Surreal.patch method is currently broken due to the wrong method name being used in the request.
It is also wrongly typed, the expected input should be a list of dict, not a sole dict. And the return type should allow a single dict.

Steps to reproduce

Currently:

await db.update("table:id", {"field": "value"})

surrealdb.ws.SurrealPermissionException: Method not found

After changing the Request method in Surreal.patch from "modify" to "patch":

await db.update("table:id", {"field": "value"})

surrealdb.ws.SurrealPermissionException: There was a problem with the database: The JSON Patch contains invalid operations. Operations must be an array

Now using an array as documented here: https://surrealdb.com/docs/surrealdb/integration/websocket#patch

await db.patch("table:id", [
    {"op": "replace", "path": "/field", "value": "value"}
])

Success! But typing of both the data param, and the return type, is still wrong.

Expected behaviour

1. The request method name should be changed from "modify" to "patch", similar to #85
2. The data parameter should by typed to expect an Array, e.g. List[Dict[str, Any]]
3. The return type allows a single dict, same as #97

SurrealDB version

1.3.1 for linux on x86_64

surrealdb.py version

0.3.2 (latest as of 2024-04-02)

Contact Details

Ideally in the issue itself. Otherwise: [email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Hello everyone! 👋🏻

I have been using SurrealDB more frequently and also looking at the other client libraries in the Surreal family. Recently, I tried my hand at writing my own SurrealDB WebSocket client using websockets to handle the boilerplate, pydantic for validation, and closely following the syntax of the JavaScript library. You can find the code in my Gist 👨🏻‍💻

I was hoping that the community could take a look and suggest any improvements that could be implemented into the main Python library. This is my first time working with WebSockets in Python, so I would appreciate any feedback 😄

Packages used:

• websockets
• pydantic

You can set up a user table by following this tutorial. This Python implementation was greatly influenced by the JavaScript equivalent.

Thanks 🚀

Describe the bug

try to connect :
async with Surreal("ws://surrealdb:8000/rpc") as db:

        await db.signin({"user": "root", "pass": "root"})
        await db.use("test", "test")

get error
ERROR: Exception in ASGI application
devfram-fastapi-1 | Traceback (most recent call last):
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/uvicorn/protocols/http/h11_impl.py", line 428, in run_asgi
devfram-fastapi-1 | result = await app( # type: ignore[func-returns-value]
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in call
devfram-fastapi-1 | return await self.app(scope, receive, send)
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/fastapi/applications.py", line 289, in call
devfram-fastapi-1 | await super().call(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/applications.py", line 122, in call
devfram-fastapi-1 | await self.middleware_stack(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in call
devfram-fastapi-1 | raise exc
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in call
devfram-fastapi-1 | await self.app(scope, receive, _send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/cors.py", line 83, in call
devfram-fastapi-1 | await self.app(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in call
devfram-fastapi-1 | raise exc
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in call
devfram-fastapi-1 | await self.app(scope, receive, sender)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in call
devfram-fastapi-1 | raise e
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in call
devfram-fastapi-1 | await self.app(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 718, in call
devfram-fastapi-1 | await route.handle(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle
devfram-fastapi-1 | await self.app(scope, receive, send)
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/starlette/routing.py", line 66, in app
devfram-fastapi-1 | response = await func(request)
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 273, in app
devfram-fastapi-1 | raw_response = await run_endpoint_function(
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
devfram-fastapi-1 | return await dependant.call(**values)
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/app/main.py", line 208, in testdb
devfram-fastapi-1 | await db.signin({"user": "root", "pass": "root"})
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/surrealdb/ws.py", line 282, in signin
devfram-fastapi-1 | response = await self._send_receive(
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/surrealdb/ws.py", line 683, in _send_receive
devfram-fastapi-1 | return await self._recv()
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "/usr/local/lib/python3.11/site-packages/surrealdb/ws.py", line 711, in _recv
devfram-fastapi-1 | return ResponseSuccess(**response)
devfram-fastapi-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^
devfram-fastapi-1 | File "pydantic/main.py", line 341, in pydantic.main.BaseModel.init
devfram-fastapi-1 | pydantic.error_wrappers.ValidationError: 1 validation error for ResponseSuccess
devfram-fastapi-1 | id
devfram-fastapi-1 | str type expected (type=type_error.str)

Steps to reproduce

try to connect

Expected behaviour

connect

SurrealDB version

nightly

surrealdb.py version

last main

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Description

What are your opinions on the python versions that should be supported by this library?

Asking this question due to the usage of deprecated typehinting annotations such as typing.List and typing.Tuple - what are our plans to support/update these annotations?

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Is your feature request related to a problem?

Using the signin method is difficult on the Surreal websockets class. Different key and value pairs are needed for each sign in type. For example, root sign in requires a user and pass, whereas scope sign in requires a user, pass, database, namespace, scope, and any extra parameters. Using an untyped dictionary does not inform the developer what is required to sign in.

Describe the solution

To be more inline with the Rust client, we can create "credential" models using pydantic. Depending on the type of credentials the user needs, they will know exactly what parameters are needed.

For example:

from typing import Union

from pydantic import BaseModel, Field


class Root(BaseModel):
    """User credentials for the root user."""

    username: str = Field(
        ..., alias="user", title="Username", description="Username for the root user"
    )
    password: str = Field(
        ..., alias="pass", title="Password", description="Password for the root user"
    )

    class Config:
        """Config for the Root class."""

        allow_population_by_field_name = True


class Namespace(BaseModel):
    """User credentials for a namespace."""

    namespace: str = Field(
        alias="NS", title="Namespace", description="Namespace for the user"
    )
    username: str = Field(
        alias="user", title="Username", description="Username for the user"
    )
    password: str = Field(
        alias="pass", title="Password", description="Password for the user"
    )

    class Config:
        """Config for the Namespace class."""

        allow_population_by_field_name = True


class Database(BaseModel):
    """User credentials for a database."""

    namespace: str = Field(
        ..., alias="ns", title="Namespace", description="Namespace for the user"
    )
    database: str = Field(
        ..., alias="db", title="Database", description="Database for the user"
    )
    username: str = Field(
        ..., alias="user", title="Username", description="Username for the user"
    )
    password: str = Field(
        ..., alias="pass", title="Password", description="Password for the user"
    )

    class Config:
        """Config for the Database class."""

        allow_population_by_field_name = True


class Scope(BaseModel):
    """User credentials for a scope."""

    namespace: str = Field(
        ..., alias="ns", title="Namespace", description="Namespace for the user"
    )
    database: str = Field(
        ..., alias="db", title="Database", description="Database for the user"
    )
    scope: str = Field(..., alias="sc", title="Scope", description="Scope for the user")
    params: BaseModel = Field(
        ..., alias="params", title="Parameters", description="Parameters for the scope"
    )

    class Config:
        """Config for the Scope class."""

        arbitrary_types_allowed = True
        allow_population_by_field_name = True


Credentials = Union[Root, Namespace, Database, Scope]

The signin method would on the Surreal class would look something like:

    async def signin(self, credentials: Credentials) -> str:
        # noinspection PyShadowingNames
        """Signs this connection in to a specific authentication scope.

        Args:
            credentials: Variables used in a signin query.

        Returns:
            The token used for authenticating future requests.

        Examples:
            >>> from pydantic import BaseModel
            
            >>> from surrealdb.auth import Root

            >>> async with Surreal("ws://127.0.0.1:8000/rpc") as db:
            >>>     await db.signin(Root(username="root", password="root")),
        """
        response = await self._send_receive(
            Request(
                id=generate_uuid(),
                method="signin",
                params=(credentials.dict(by_alias=True),),
            ),
        )
        success: ResponseSuccess = _validate_response(
            response, SurrealAuthenticationException
        )
        token: str = success.result
        self.token = token
        return self.token

Using the method:

    async with Surreal("ws://127.0.0.1:8000/rpc") as db:
        await db.signin(Root(username="root", password="root"))

Alternative methods

Potentially could use a TypedDict to replicate this behaviour instead. However, pydantic will allow us to create stricter types overall.

SurrealDB version

surreal 1.0.0-beta.8+20220930.c246533 for macos on x86_64

surrealdb.py version

surrealdb.py 0.30 for macOS on aarch64 using Python 3.9.1

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Very interested in trying out SurrealDB using Python on ASGI (for me, Sanic). Feels like it could be a perfect match.

Any early release for this driver coming out soon? Would it be possible to get a preview?

Also, this is planned to be an asyncio based connector, correct? 😅

I feel like there's a decent number of people waiting for this and ready to help out, too, if needed.

Is your feature request related to a problem?

This would help users more easily get started with SurrealDB by importing data from in-memory formats.

Is there any chance of a from_arrow(), or from_polars() in the python client on the horizon? These are key resources from a datascience, mlops perspective. With the ability to object -> verb -> object, this makes a very nice potential for use as a knowledge graph, embedded with tables.
https://discordapp.com/channels/902568124350599239/1018618253695795261/1092798175796019281

Describe the solution

Not sure how to go about doing this but opening this feature request to discuss.

Alternative methods

SurrealDB version

1.0.0-beta.9+20230402.5eafebd for macos on aarch64

surrealdb.py version

latest

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

Describe the bug

When I try to enter a specific unicode string into surrealDB using the python client, I get a parse error.

import asyncio
from surrealdb import Surreal

async def test():
    """Example of how to use the SurrealDB client."""
    async with Surreal("ws://localhost:8000/rpc") as db:
        await db.signin({"user": "root", "pass": "root"})
        await db.use("test", "test")
        await db.create(
            "person",
            {
                "user": "제10",
            },
        )

if __name__ == "__main__":
    import asyncio

    asyncio.run(test())

Result:

  File "......database/test.py", line 9, in test
    await db.create(
  File "....../lib/python3.9/site-packages/surrealdb/ws.py", line 482, in create
    success: ResponseSuccess = _validate_response(
  File "....../lib/python3.9/site-packages/surrealdb/ws.py", line 161, in _validate_response
    raise exception(response.message)
surrealdb.ws.SurrealPermissionException: Parse error

Errors occur on strings like "제10", "가1234"

Steps to reproduce

[[package]]
name = "surrealdb"
version = "0.3.0"

Expected behaviour

Expecting a normal DB INSERT

SurrealDB version

1.0.0-beta.9+20230402.5eafebd for macos on aarch64

surrealdb.py version

0.3.0

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

As discussed on #39, it would be advantageous for the project to implement issue templates for the following categories: Bug Reports, Feature Requests, and Documentation.

I propose that the templates should be formatted similarly to the surrealdb repository and include additional information related to Python. This may include the version of Python, surrealdb.py, surrealdb, the operating system (e.g. Linux), and the installation source (pip or source).

Let me know your thoughts. Thank you. 😄

Is your feature request related to a problem?

I’m trying to use the surrealdb package and pydantic-settings package, but surrealdb’s pin to pydantic v1.x is causing conflicts.

Describe the solution

Bump surrealdb’s pydantic to v2.x or drop it.

Alternative methods

I could use surrealdb v0.2

SurrealDB version

1.0.0

surrealdb.py version

Trying to use 0.3.1

Contact Details

[email protected]

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct

I'm trying to add an edge between objects. As there is no direct API call, I'm trying to use a query:

query = f"""RELATE table:A->part_of->table:B SET time.written = time::now();"""
response = await client.execute(query)

Give a NameError: name 'time' is not defined
I'd expect that this is resolved in the database directly?

Is your feature request related to a problem?

There is no easy way to insert multiple records. This can made easier by creating a new method specifically for performing insert operations.

Describe the solution

A new method could be added to the SDK which could have the following signature:

db.insert(table: str, data: Union[Dict, List[Dict]], options: Optional[?])

This would allow you to insert a singular record or a list of records.

Alternative methods

Currently the method that we are using is to either generate a prepared string to do an insert statement:

db.query(```
INSERT INTO company {
	name: 'SurrealDB',
	founded: "2021-09-10",
	founders: [person:tobie, person:jaime],
	tags: ['big data', 'database']
};```)

or

db.query(```
INSERT INTO person [
   {id: "person:jaime", name: "Jaime", surname: "Morgan Hitchcock"},
   {id: "person:tobie", name: "Tobie", surname: "Morgan Hitchcock"},
];```)

SurrealDB version

1.0.0+20230913.54aedcd for macos on x86_64

surrealdb.py version

surrealdb.py 0.3.1 for macOS on x86_64 using Python 3.11.2

Contact Details

No response

Is there an existing issue for this?

• I have searched the existing issues

Code of Conduct

• I agree to follow this project's Code of Conduct