gacarrillor / appendfeaturestolayer Goto Github PK

Hi,
Nice plugin. Updating an existing layer with data from another layer seems to be a normal usecase, but I need to create all kinds of queries when I want to do this via field calculator.

Do I understand correctly that this plugin updates ALL fields that have the same fieldname in both target en source layer?
If so, It would be very nice to have a selectbox where I can select which fields in the target layer to update. If my source layer has 20 fields of which only 2 contain new data and the rest are empty (or contain old data), they will overwrite the target layer, thus perhaps emptying fields in the target layer which should have been kept.

The workaround is to prep my source layer to only contain the fields with updated data.

I am trying to append layers between two geopackages - same structure - but it is causing a QGIS crash. Any thoughts?

Seems to run the append, then crashes. See stack trace below,

Cheers Grant

Report Details

Python Stack Trace

Windows fatal exception: access violation

Thread 0x00000f98 (most recent call first):
  File "C:\OSGeo4W/apps/qgis/./python\qgis\core\additions\edit.py", line 43 in __exit__
    if not self.layer.commitChanges():
  File "C:\Users/Grant/AppData/Roaming/QGIS/QGIS3\profiles\Grant/python/plugins\AppendFeaturesToLayer\processing\algs\AppendFeaturesToLayer.py", line 344 in processAlgorithm
    target.endEditCommand()

Current thread 0x000050d8 (most recent call first):
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\ProcessingPlugin.py", line 432 in executeAlgorithm
    dlg.exec_()
  File "C:\OSGeo4W/apps/qgis/./python/plugins\processing\gui\ProcessingToolbox.py", line 232 in executeAlgorithm
    self.executeWithGui.emit(alg.id(), self, self.in_place_mode, False)

Stack Trace

CPLODBCDriverInstaller::RemoveDriver :
QgsOgrProviderUtils::GDALOpenWrapper :
QgsPrecalculatedTextMetrics::count :
QgsOgrProviderUtils::wildcards :
QgsPrecalculatedTextMetrics::count :
QgsVectorLayerFeatureSource::QgsVectorLayerFeatureSource :
QgsVectorLayerProfileResults::visitFeaturesInRange :
QgsVectorLayer::createMapRenderer :
QgsMapRendererJob::prepareJobs :
QgsMapRendererParallelJob::startPrivate :
QgsMapCanvas::refreshMap :
QObject::qt_static_metacall :
QTimer::timerEvent :
QObject::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QEventDispatcherWin32Private::sendTimerEvent :
QEventDispatcherWin32::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QCoreApplicationPrivate::sendPostedEvents :
qt_plugin_query_metadata :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QDialog::exec :
PyInit_QtWidgets :
PyArg_ParseTuple_SizeT :
PyEval_EvalFrameDefault :
PyObject_GC_Del :
PyFunction_Vectorcall :
PyFloat_FromDouble :
PyVectorcall_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QMetaObject::activate :
PyInit_QtCore :
PyInit_QtCore :
PyType_GenericNew :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyFloat_FromDouble :
PyVectorcall_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractItemView::doubleClicked :
QTreeView::mouseDoubleClickEvent :
PyInit__gui :
QWidget::event :
QFrame::event :
QAbstractItemView::viewportEvent :
PyInit__gui :
QCoreApplicationPrivate::sendThroughObjectEventFilters :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.34.2-Prizren
QGIS code revision: 7d199797fc
Compiled against Qt: 5.15.3
Running against Qt: 5.15.3
Compiled against GDAL: 3.8.2
Running against GDAL: 3.8.2

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.19045

The function ''Just APPEND all features, no matter of duplicates'' keeps giving me the error:

Layer ''layername'': OGR error creating feature -5018: Cannot call CreateFeature() with a set FID when a dataset is opened more than once.

How to solve this?

It would be useful to expand the ability to search for duplicates by:

• geometry;
• several fields at the same time (e.g. by geometry and creation date).

Thank you for the plugin, it is very useful.

We all have encountered that many time dealing with point data like poi , admin points from multiple sources have duplicates. To append them with existing data we have two challenges geom and name could be slightly different.

So we need fuzzy and distance proximity at the same time.

Currently most people use postgis or python queries to find duplicates but i haven't come accross a tool which can achieve such results.

If needed i would provide you with python script to do the task but have no idea how it works in a plugin. Happy to help .

For ETL it can be useful to clear the destination before inserting.
Then we can do load, transform and dump the result.

For some processing i don't want to do upsert kind of insertion.
Just a plain clear the target and insert is also very usefull.

I have mapped all my source and target features in the Append Feature to Layer UI and generated the JSON file and saved it. QGIS is crasing when I tried to execute it in batch mode, however the single layer process is working fine.
AppendFeaturesToLayer 1.5.0, QGIS 3.32.1-Lima on Windows 10

Example:

qgis_process run "etl_load:appendfeaturestolayer" -- SOURCE_LAYER=/tmp/source.shp TARGET_LAYER=/tmp/target.shp ACTION_ON_DUPLICATE=0

I tried to add some features from a geopackage into a PostGIS layer. A txt field formatted as a json field is not imported in the PostGIS json field. If we tried to import using the copy and past function in Qgis, the import correctly works.

That is, leave attributes intact for duplicate features and only update the corresponding geometry.

This may be helpful to avoid triggers in DBs skipping attribute updates.

I have just found this plugin and it is really useful for inserting additional records in Geopackage layers, and probably other database types. THANKS!

I would like to suggest a feature to check if the record exists - I have a table that has an 'external key' / foreign key that corresponds with the data that I am trying to append and it would be great if the I could select a column in the two layers for the plugin to check 'if exists'. If the plugin could exclude the existing rows, and only append new rows that would be fantastic.

Dear Sir

When we use the "Executer comme processus de lot" ("Run as batch..." i suppose) and load our ".json" saved process, we have, for each step in process, the popup "Voulez-vous réinitialiser le champ de carte" ("Reset map field ?" i suppose) with yes/non buttons.
If we have many steps, it take time to click on it for each step

I think it would be great to have check-box option like "Same choice for all step" on the popup.

Have a good day

I'm using this plugin to add addresses to an existing field in my layer. The points were all loaded in weeks ago with the "address" field left blank, and I am updating the "address" field as we get new batches of data (about a quarter of the points each time). I nearly got the result I was looking for, but had a hiccup at the end. When I appended the source layer to the target layer and selected "If duplicate is found, UPDATE existing feature", the addresses populated in the correct fields. But when I closed the attribute table, all of the updated points had disappeared from the map. Trying to select one in the attribute table and 'zoom to selection' gave me a "no extent could be determined" error. The points still had their latitude and longitude coordinates in the attribute table, but would no longer show up anywhere on the map.
Updating the extents of geometry in the properties had no effect, and the CSV I used has no geometry (attribute only).

Hi Team,

QGIS is crashing while using this feature. I have around 35 mif files exported from AutoCAD, trying to append the data in mif files to the respective Geo packages. I have same number of Geo packages created in QGIS, have mapped the source and target layers and exported to JSON format. When I open it and run in batch mode QGIS crashes where as the same functionality is working fine if I run file by file and need help in the same. Please let me know for any other information.

Hello. It was simply to know if it was possible to save modifications toward a postgis table? (Postgresql Postgis serveur).
Thanks.