Comments (13)
Erhannis
commented on May 19, 2024
1
Great, that works. Thanks! I'll open more tickets if I find anything else
weird or wrong.
…On Mon, Feb 15, 2021, 2:10 PM collerek ***@***.***> wrote:
Yeah, it was changed in one of the old version to be in line with pydantic
where you have to specifically set field to optional or provide a default
value for the field to be not required. See that i forgot to update the
nullable part of docs, thanks for the tip, will change that.
As for the rules below is the section from the docs overview:
All fields are required unless one of the following is set:
- nullable - Creates a nullable column. Sets the default to None.
- default - Set a default value for the field.
- server_default - Set a default value for the field on server side
(like sqlalchemy's func.now()).
- primary key with autoincrement - When a column is set to primary key
and autoincrement is set on this column. Autoincrement is set by default on
int primary keys.
- pydantic_only - Field is available only as normal pydantic field,
not stored in the database.
So in your case you should just set nullable=True on js Field.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#95 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAXCNTPSMFO2HFNLPX4OI4DS7FWTHANCNFSM4XMBS25Q>
.
from ormar.
collerek
commented on May 19, 2024
I need more info on how you create the Things, since I cannot recreate:
import uuid
from typing import List
import databases
import pydantic
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
database = databases.Database(DATABASE_URL, force_rollback=True)
metadata = sqlalchemy.MetaData()
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class Thing(ormar.Model):
class Meta(BaseMeta):
tablename = "things"
id: uuid.UUID = ormar.UUID(primary_key=True, default=uuid.uuid4)
name: str = ormar.Text(default="")
js: pydantic.Json = ormar.JSON()
@app.get("/things", response_model=List[Thing])
async def read_things():
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample", response_model=List[Thing])
async def read_things():
await Thing(name="b", js=["asdf", "asdf", "bobby", "nigel"]).save()
await Thing(name="a", js="[\"lemon\", \"raspberry\", \"lime\", \"pumice\"]").save()
return await Thing.objects.order_by("name").all()
@app.post("/things", response_model=Thing)
async def create_things(thing: Thing):
thing = await thing.save()
return thing
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
def test_read_main():
client = TestClient(app)
with client as client:
response = client.get("/things_with_sample")
assert response.status_code == 200
# check if raw response not double encoded
assert '["lemon","raspberry","lime","pumice"]' in response.text
# parse json and check that we get lists not strings
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
# create a new one
response = client.post("/things", json={"js": ["test", "test2"], "name": "c"})
assert response.json().get("js") == ["test", "test2"]
# get all with new one
response = client.get("/things")
assert response.status_code == 200
assert '["test","test2"]' in response.text
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]from ormar.
Erhannis
commented on May 19, 2024
Huhhh. It looks like it depends on how you set the JSON field. Consider:
@app.post("/things/test")
async def testThing():
thing1 = Thing()
thing1.name="test1"
thing1.js=["js","set","after","constructor"]
await thing1.save()
await Thing(name="test2", js=["js", "set", "in", "constructor"]).save()
return "ok"
and a subsequent GET from /things yields
[
{
"id": "66142fbb-51fb-442b-a41b-78808f9b98e1",
"name": "test2",
"js": [
"js",
"set",
"in",
"constructor"
]
},
{
"id": "b6ea987b-da26-4e07-bc42-bfc0985a903c",
"name": "test1",
"js": "[\"js\", \"set\", \"after\", \"constructor\"]"
}
]
The after-constructor string behavior also shows up when I afterwards do e.g.
thing.js = ["updated"]
thing.update()
Glad to know it's intended to work how I expected it to work, btw.
from ormar.
collerek
commented on May 19, 2024
Still cannot reproduce :)
Even uploaded to github to check with python 3.6-3.9 included and all 3 backends (postgress, mysql and sqlite).
All tests pass.
import uuid
from typing import List
import databases
import pydantic
import pytest
import sqlalchemy
from fastapi import FastAPI
from starlette.testclient import TestClient
import ormar
from tests.settings import DATABASE_URL
app = FastAPI()
database = databases.Database(DATABASE_URL, force_rollback=True)
metadata = sqlalchemy.MetaData()
app.state.database = database
@app.on_event("startup")
async def startup() -> None:
database_ = app.state.database
if not database_.is_connected:
await database_.connect()
@app.on_event("shutdown")
async def shutdown() -> None:
database_ = app.state.database
if database_.is_connected:
await database_.disconnect()
class BaseMeta(ormar.ModelMeta):
metadata = metadata
database = database
class Thing(ormar.Model):
class Meta(BaseMeta):
tablename = "things"
id: uuid.UUID = ormar.UUID(primary_key=True, default=uuid.uuid4)
name: str = ormar.Text(default="")
js: pydantic.Json = ormar.JSON()
@app.get("/things", response_model=List[Thing])
async def read_things():
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample", response_model=List[Thing])
async def read_things_sample():
await Thing(name="b", js=["asdf", "asdf", "bobby", "nigel"]).save()
await Thing(name="a", js="[\"lemon\", \"raspberry\", \"lime\", \"pumice\"]").save()
return await Thing.objects.order_by("name").all()
@app.get("/things_with_sample_after_init", response_model=Thing)
async def read_things_init():
thing1 = Thing()
thing1.name = "d"
thing1.js = ["js", "set", "after", "constructor"]
await thing1.save()
return thing1
@app.put("/update_thing", response_model=Thing)
async def update_things(thing: Thing):
thing.js = ["js", "set", "after", "update"] # type: ignore
await thing.update()
return thing
@app.post("/things", response_model=Thing)
async def create_things(thing: Thing):
thing = await thing.save()
return thing
@pytest.fixture(autouse=True, scope="module")
def create_test_database():
engine = sqlalchemy.create_engine(DATABASE_URL)
metadata.create_all(engine)
yield
metadata.drop_all(engine)
def test_read_main():
client = TestClient(app)
with client as client:
response = client.get("/things_with_sample")
assert response.status_code == 200
# check if raw response not double encoded
assert '["lemon","raspberry","lime","pumice"]' in response.text
# parse json and check that we get lists not strings
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
# create a new one
response = client.post("/things", json={"js": ["test", "test2"], "name": "c"})
assert response.json().get("js") == ["test", "test2"]
# get all with new one
response = client.get("/things")
assert response.status_code == 200
assert '["test","test2"]' in response.text
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
response = client.get("/things_with_sample_after_init")
assert response.status_code == 200
resp = response.json()
assert resp.get("js") == ["js", "set", "after", "constructor"]
# test new with after constructor
response = client.get("/things")
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
assert resp[3].get("js") == ["js", "set", "after", "constructor"]
response = client.put("/update_thing", json=resp[3])
assert response.status_code == 200
resp = response.json()
assert resp.get("js") == ["js", "set", "after", "update"]
# test new with after constructor
response = client.get("/things")
resp = response.json()
assert resp[0].get("js") == ["lemon", "raspberry", "lime", "pumice"]
assert resp[1].get("js") == ["asdf", "asdf", "bobby", "nigel"]
assert resp[2].get("js") == ["test", "test2"]
assert resp[3].get("js") == ["js", "set", "after", "update"]from ormar.
Erhannis
commented on May 19, 2024
Looks like it's dependent on the response_model field? (I'd done some of my tests in a separate file.)
In the code you just sent, if write in the following two endpoints, I get different results.
@app.get("/things_typed", response_model=List[Thing])
async def read_things_untyped():
return await Thing.objects.order_by("name").all()
@app.get("/things_untyped")
async def read_things_typed():
return await Thing.objects.order_by("name").all()
Now, I'd be kindof ok with that behavior - at least there's a fix. But it sounds like maybe it wasn't intentional.
Also, the following yields different outputs, when that seems unlikely to be the expected result:
In [25]: t1 = Thing(name="t1",js=["thing1"])
In [26]: t1.json()
Out[26]: '{"id": "d2f9a2e6-0072-4f2c-8415-0aeb9cce44bb", "name": "t1", "js": ["thing1"]}'
In [27]: t2 = Thing(name="t2")
In [28]: t2.js = ["thing2"]
In [29]: t2.json()
Out[29]: '{"id": "110f1de6-a5a4-4852-86ef-41c89da8ccfd", "name": "t2", "js": "[\\"thing2\\"]"}'
(I've removed some I/O errors I keep getting in the console, which don't seem to correlate with what I type and don't seem to affect anything.)
from ormar.
Erhannis
commented on May 19, 2024
Further related strange behavior:
In [93]: t1 = Thing(name="t1",js=["thing1"])
In [94]: t1.json()
Out[94]: '{"id": "67a82813-d90c-45ff-b546-b4e38d7030d7", "name": "t1", "js": ["thing1"]}'
In [95]: await t1.save()
Out[95]: Thing({'id': UUID('67a82813-d90c-45ff-b546-b4e38d7030d7'), 'name': 't1', 'js': ['thing1']})
In [96]: t1.json()
Out[96]: '{"id": "67a82813-d90c-45ff-b546-b4e38d7030d7", "name": "t1", "js": "[\\"thing1\\"]"}'
In [97]: (await Thing.objects.get(id=t1.id)).json()
Out[97]: '{"id": "67a82813-d90c-45ff-b546-b4e38d7030d7", "name": "t1", "js": ["thing1"]}'
In [98]: await t1.update()
Out[98]: Thing({'id': UUID('67a82813-d90c-45ff-b546-b4e38d7030d7'), 'name': 't1', 'js': ['thing1']})
In [99]: (await Thing.objects.get(id=t1.id)).json()
Out[99]: '{"id": "67a82813-d90c-45ff-b546-b4e38d7030d7", "name": "t1", "js": "[\\"thing1\\"]"}'
Specifically, it looks like save() stringifies the json. This is not saved to the database at that time, but if you subsequently call update(), it is. Note further that in another test I just did, if I fetch t1 from the database immediately after saving, the json remains json, and in fact it may be safe until the next time you set the field. (For example, t1.js = t1.js stringified the json.)
I note that the database appears to store the two cases as "a string containing json" and "a string containing a string containing json".
from ormar.
Erhannis
commented on May 19, 2024
Re: db, like, if I do select js from things;, I get e.g. two rows:
["thing1"]
"[\"thing2\"]"
from ormar.
Erhannis
commented on May 19, 2024
Also, in testing just now I've discovered that you can set an invalid value on a choices-restricted field, save it, and then Thing.objects.all() throws a validation error. I can make a separate ticket for that, if you'd like.
from ormar.
Erhannis
commented on May 19, 2024
Clarification: you can set an invalid value after creation. Thing(enumField="invalid") fails, but t = Thing(enumField="valid") followed by t.enumField = "invalid" succeeds.
from ormar.
collerek
commented on May 19, 2024
Ok i will check that, thanks for reporting.
As for choices is it on json field or other one?
Cause I already saw that pydantic sets json field to any with None as default. In your code above you did not set the json column as nullable so you should not be able to instantiate a Thing without it.
If it's not the json column can you please open another ticket with a sample to reproduce?
Thanks for catching that! 🙂
from ormar.
collerek
commented on May 19, 2024
Ok, both should be fixed in 0.9.3.
Please update and check.
from ormar.
Erhannis
commented on May 19, 2024
Both seem fixed, thanks! Question, though - before, I could leave js off the call to the constructor, but now it tells me field required. I'd kindof have expected it to default to null, for a nullable field (the documentation seems to say it should default to nullable, not being a primary key), and js would not be required. Is it otherwise? Is each field required present in the constructor unless it has an explicit default? (I can make a separate ticket, if desired.)
from ormar.
collerek
commented on May 19, 2024
Yeah, it was changed in one of the old version to be in line with pydantic where you have to specifically set field to optional or provide a default value for the field to be not required. See that i forgot to update the nullable part of docs, thanks for the tip, will change that.
As for the rules below is the section from the docs overview:
All fields are required unless one of the following is set:
- nullable - Creates a nullable column. Sets the default to None.
- default - Set a default value for the field.
- server_default - Set a default value for the field on server side (like sqlalchemy's func.now()).
- primary key with autoincrement - When a column is set to primary key and autoincrement is set on this column. Autoincrement is set by default on int primary keys.
- pydantic_only - Field is available only as normal pydantic field, not stored in the database.
Also relation fields by default are nullable.
So in your case you should just set nullable=True on js Field.
from ormar.
Related Issues (20)
- use snake case for table names in the database
- Multiple reverse relations not working.
- Add `first_or_none`
- FastpAPI state has not database HOT 4
- How to configure alembic autogenerate with multiple model files HOT 5
- Error awaiting None
- Upgrade to SQLAlchemy 2.0 HOT 5
- Add SELECT DISTINCT feature to queryset
- Failing test: test_weakref_init
- Reverse relation not returned from get_pydantic
- Low performance on select_all() call HOT 1
- Nullable ForeignKey field does not get updated to null
- The future of ORMAR HOT 6
- Support specifying a table comment
- Please support pydantic-2.x HOT 3
- New release with the latest dependency upgrades HOT 2
- Unresolved attribute reference 'objects'
- Model field types and pyright
- v0.20.0 is released as stable instead of beta HOT 3
- Allow tables without primary key
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ormar.