Comments (5)
@CF-FHB-X version 0.2.0 has been pushed to pypi, the DATETIME
type has been added to the enumerations and a converter is registered with SQLite to allow for queries run from python to return a datetime
object (instead of text).
import pathlib
from datetime import timezone
from faker import Faker
from fudgeo.geopkg import GeoPackage, SpatialReferenceSystem, Field
from fudgeo.enumeration import GeometryType, SQLFieldType
from fudgeo.geometry import Point
from fudgeo.sql import ESRI_4326
from fudgeo import __version__
assert __version__ == '0.2.0'
fake = Faker()
# Create or connect to GeoPackage
path_gpkg = pathlib.Path.cwd().joinpath("fudgeo_tutorial.gpkg")
if path_gpkg.exists():
gpkg = GeoPackage(path_gpkg)
else:
gpkg = GeoPackage.create(path_gpkg)
srs = SpatialReferenceSystem("GCS_WGS_1984", "EPSG", 4326, ESRI_4326)
# Fake up some data
rows = []
for _ in range(100):
lat, lon, city, cc, tz = fake.location_on_land()
lon = float(lon)
lat = float(lat)
rows.append((fake.name(), city, cc, lon, lat, fake.date_object(),
fake.date_time(), fake.date_time(tzinfo=timezone.utc),
Point(x=lon, y=lat, srs_id=4326)))
# Define fields
fields = (
Field("full_name", SQLFieldType.text),
Field("city", SQLFieldType.text),
Field("country_code", SQLFieldType.text, 2),
Field("longitude", SQLFieldType.float),
Field("latitude", SQLFieldType.float),
Field("my_date", SQLFieldType.date),
# These next 2 are the ones that have issues:
Field("my_datetime", SQLFieldType.datetime),
Field("my_datetime_tz", SQLFieldType.datetime),
)
# Create feature class
table_name = "Datetime_Test"
fc = gpkg.create_feature_class(table_name, srs, fields=fields,
shape_type=GeometryType.point, overwrite=True)
# Write out the data
with gpkg.connection as conn:
conn.executemany(
f"""INSERT INTO {table_name} (
full_name, city, country_code,
longitude, latitude, my_date,
my_datetime, my_datetime_tz, SHAPE)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""", rows)
# Selecting the date and datetime columns gives date and datetime objects
with gpkg.connection as conn:
cursor = conn.execute(f"""SELECT my_date, my_datetime, my_datetime_tz FROM {table_name}""")
features = cursor.fetchall()
# table will now show in ArcGIS Desktop and ArcGIS Pro with
# Date type fo the date and datetime columns
from fudgeo.
@CF-FHB-X the ValueError
you are seeing is coming from internals of Python's SQLite implementation for the default converters / adapters for date
and datetime
. This is not an issue specific to fudgeo
.
The exception you see happens on retrieval of the datetime_tz
(i.e. with UTC offset / time zone aware) since the format of the internally stored string value (in SQLite) has trailing +## is not expected by the default adapters. In general the approach is to resolve datetime
s to UTC before inserting.
As for ArcGIS Desktop (10.8.1) and ArcGIS Pro (3.0.0), I see that they interpret the datetime and datetime_tz columns as Double
, which is fascinating guess of the data type since the most probably type is Text
(likely a bug / enhancement request to be reported to them).
Overcoming this issue in ArcGIS ecosystem you're onto something with the use of DATETIME
as the field type, that seems to make the data type for the column map to Date
in ArcGIS Desktop and ArcGIS Pro. Then to overcome the trickle down issue of no longer getting datetime
objects from python queries you can do this:
# Define fields
fields = (
Field("full_name", SQLFieldType.text),
Field("city", SQLFieldType.text),
Field("country_code", SQLFieldType.text, 2),
Field("longitude", SQLFieldType.float),
Field("latitude", SQLFieldType.float),
Field("date", SQLFieldType.date),
# These next 2 are the ones that have issues:
Field("datetime", "DATETIME"),
Field("datetime_tz", SQLFieldType.timestamp),
)
...
with gpkg.connection as conn:
cursor = conn.execute("""SELECT datetime "[timestamp]" FROM Datetime_Test""")
features = cursor.fetchall()
from fudgeo.
Is TIMESTAMP
actually even a supported data type for GeoPackages? I'm reading this spec: https://www.geopackage.org/spec/. It seems they recommend DATETIME
for datetimes.
from fudgeo.
So it does, thanks for the link.
from fudgeo.
Just awesome. Thanks a million! Will test it all on Monday.
Thanks again!
from fudgeo.
Related Issues (20)
- Reduce concatenation and byte array creation
- Do less when checking is_empty
- Change default for OGR contents to False
- Primary Key Field
- add geo interface
- Extension: Metadata
- Extension: Schema
- Move Envelope Code to enumeration
- Release v0.7.0
- GeoPackage Vector Tiles (PBF/MVT) HOT 1
- GeoPackage Raster Tiles HOT 2
- Solutions for Serving GPKG data
- Update for 3.12 Support
- Release v0.7.1
- GeoPackage v1.4.0
- Check Dependencies HOT 1
- Release v0.7.2
- Drop Support for Older Python Versions / Modernize Type Hinting
- Add a method for selection, returning a cursor
- Release v0.8.0
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fudgeo.