gacarrillor / appendfeaturestolayer Goto Github PK
View Code? Open in Web Editor NEWQGIS Processing plugin to add an algorithm for upserting features from a source vector layer to an existing target vector layer.
QGIS Processing plugin to add an algorithm for upserting features from a source vector layer to an existing target vector layer.
Hi,
Nice plugin. Updating an existing layer with data from another layer seems to be a normal usecase, but I need to create all kinds of queries when I want to do this via field calculator.
Do I understand correctly that this plugin updates ALL fields that have the same fieldname in both target en source layer?
If so, It would be very nice to have a selectbox where I can select which fields in the target layer to update. If my source layer has 20 fields of which only 2 contain new data and the rest are empty (or contain old data), they will overwrite the target layer, thus perhaps emptying fields in the target layer which should have been kept.
The workaround is to prep my source layer to only contain the fields with updated data.
I am trying to append layers between two geopackages - same structure - but it is causing a QGIS crash. Any thoughts?
Seems to run the append, then crashes. See stack trace below,
Cheers Grant
Python Stack Trace
Windows fatal exception: access violation
Thread 0x00000f98 (most recent call first):
File "C:\OSGeo4W/apps/qgis/./python\qgis\core\additions\edit.py", line 43 in __exit__
if not self.layer.commitChanges():
File "C:\Users/Grant/AppData/Roaming/QGIS/QGIS3\profiles\Grant/python/plugins\AppendFeaturesToLayer\processing\algs\AppendFeaturesToLayer.py", line 344 in processAlgorithm
target.endEditCommand()
Current thread 0x000050d8 (most recent call first):
File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\ProcessingPlugin.py", line 432 in executeAlgorithm
dlg.exec_()
File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\gui\ProcessingToolbox.py", line 232 in executeAlgorithm
self.executeWithGui.emit(alg.id(), self, self.in_place_mode, False)
Stack Trace
CPLODBCDriverInstaller::RemoveDriver :
QgsOgrProviderUtils::GDALOpenWrapper :
QgsPrecalculatedTextMetrics::count :
QgsOgrProviderUtils::wildcards :
QgsPrecalculatedTextMetrics::count :
QgsVectorLayerFeatureSource::QgsVectorLayerFeatureSource :
QgsVectorLayerProfileResults::visitFeaturesInRange :
QgsVectorLayer::createMapRenderer :
QgsMapRendererJob::prepareJobs :
QgsMapRendererParallelJob::startPrivate :
QgsMapCanvas::refreshMap :
QObject::qt_static_metacall :
QTimer::timerEvent :
QObject::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QEventDispatcherWin32Private::sendTimerEvent :
QEventDispatcherWin32::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QCoreApplicationPrivate::sendPostedEvents :
qt_plugin_query_metadata :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QDialog::exec :
PyInit_QtWidgets :
PyArg_ParseTuple_SizeT :
PyEval_EvalFrameDefault :
PyObject_GC_Del :
PyFunction_Vectorcall :
PyFloat_FromDouble :
PyVectorcall_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QMetaObject::activate :
PyInit_QtCore :
PyInit_QtCore :
PyType_GenericNew :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyFloat_FromDouble :
PyVectorcall_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractItemView::doubleClicked :
QTreeView::mouseDoubleClickEvent :
PyInit__gui :
QWidget::event :
QFrame::event :
QAbstractItemView::viewportEvent :
PyInit__gui :
QCoreApplicationPrivate::sendThroughObjectEventFilters :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :
QGIS Info
QGIS Version: 3.34.2-Prizren
QGIS code revision: 7d199797fc
Compiled against Qt: 5.15.3
Running against Qt: 5.15.3
Compiled against GDAL: 3.8.2
Running against GDAL: 3.8.2
System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.19045
The function ''Just APPEND all features, no matter of duplicates'' keeps giving me the error:
Layer ''layername'': OGR error creating feature -5018: Cannot call CreateFeature() with a set FID when a dataset is opened more than once.
How to solve this?
It would be useful to expand the ability to search for duplicates by:
Thank you for the plugin, it is very useful.
We all have encountered that many time dealing with point data like poi , admin points from multiple sources have duplicates. To append them with existing data we have two challenges geom and name could be slightly different.
So we need fuzzy and distance proximity at the same time.
Currently most people use postgis or python queries to find duplicates but i haven't come accross a tool which can achieve such results.
If needed i would provide you with python script to do the task but have no idea how it works in a plugin. Happy to help .
For ETL it can be useful to clear the destination before inserting.
Then we can do load, transform and dump the result.
For some processing i don't want to do upsert kind of insertion.
Just a plain clear the target and insert is also very usefull.
I have mapped all my source and target features in the Append Feature to Layer UI and generated the JSON file and saved it. QGIS is crasing when I tried to execute it in batch mode, however the single layer process is working fine.
AppendFeaturesToLayer 1.5.0, QGIS 3.32.1-Lima on Windows 10
Example:
qgis_process run "etl_load:appendfeaturestolayer" -- SOURCE_LAYER=/tmp/source.shp TARGET_LAYER=/tmp/target.shp ACTION_ON_DUPLICATE=0
I tried to add some features from a geopackage into a PostGIS layer. A txt field formatted as a json field is not imported in the PostGIS json field. If we tried to import using the copy and past function in Qgis, the import correctly works.
That is, leave attributes intact for duplicate features and only update the corresponding geometry.
This may be helpful to avoid triggers in DBs skipping attribute updates.
I have just found this plugin and it is really useful for inserting additional records in Geopackage layers, and probably other database types. THANKS!
I would like to suggest a feature to check if the record exists - I have a table that has an 'external key' / foreign key that corresponds with the data that I am trying to append and it would be great if the I could select a column in the two layers for the plugin to check 'if exists'. If the plugin could exclude the existing rows, and only append new rows that would be fantastic.
Dear Sir
When we use the "Executer comme processus de lot" ("Run as batch..." i suppose) and load our ".json" saved process, we have, for each step in process, the popup "Voulez-vous réinitialiser le champ de carte" ("Reset map field ?" i suppose) with yes/non buttons.
If we have many steps, it take time to click on it for each step
I think it would be great to have check-box option like "Same choice for all step" on the popup.
Have a good day
I'm using this plugin to add addresses to an existing field in my layer. The points were all loaded in weeks ago with the "address" field left blank, and I am updating the "address" field as we get new batches of data (about a quarter of the points each time). I nearly got the result I was looking for, but had a hiccup at the end. When I appended the source layer to the target layer and selected "If duplicate is found, UPDATE existing feature", the addresses populated in the correct fields. But when I closed the attribute table, all of the updated points had disappeared from the map. Trying to select one in the attribute table and 'zoom to selection' gave me a "no extent could be determined" error. The points still had their latitude and longitude coordinates in the attribute table, but would no longer show up anywhere on the map.
Updating the extents of geometry in the properties had no effect, and the CSV I used has no geometry (attribute only).
Hi Team,
QGIS is crashing while using this feature. I have around 35 mif files exported from AutoCAD, trying to append the data in mif files to the respective Geo packages. I have same number of Geo packages created in QGIS, have mapped the source and target layers and exported to JSON format. When I open it and run in batch mode QGIS crashes where as the same functionality is working fine if I run file by file and need help in the same. Please let me know for any other information.
Hello. It was simply to know if it was possible to save modifications toward a postgis table? (Postgresql Postgis serveur).
Thanks.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.