[Bug]: Importing transactions from OFX deletes similar transactions from previous dates about actual HOT 14 CLOSED

I also experience similar issues.

When I import similar transactions with CSV it actual updates older ones with the newer date. For some this might occur as deleting, but for me the transactions note does not change.

Also, when you have locked transactions after reconcile, the same CSV is not updating old transactions but also not adding new ones.

from actual.

It breaks my OFX imports the same way for me too. If this behavior is by design, then the de-dupe logic is broken. Two transactions with
1/ different dates
2/ different note/memo
3/ same payees
4/ same amounts
should not be considered duplicates.

from actual.

where those older transactions imported via ofx?

from actual.

yes, i did not research this issue yet, but it happens when similar transactions are made on close dates

from actual.

Interesting. Would you mind providing reproduction steps (perhaps an existing budget file + an example OFX import)?

Transactions should not be deleted on import. Only create & update operations should be happening. So I'm curious to see how this could be reproduced.

from actual.

Hi @MatissJanis

I can confirm I experience a similar issue but with CSV imports where the original transaction was also imported by CSV. I find it happens when the transaction is for the same dollar value and description but different dates but have also seen it for a different date, different description and same dollar value. The one I notice the most is my loan payment and coffee shop purchases. If I had a loan payment for $500 on the 15/3/24 already imported and reconciled then when I import the next week which has a loan repayment on the 22/3/24 for the same amount the entry for the 15/3/24 disappears. I don't think it deletes but I think the date gets adjusted to the 22/3/24 and no transaction is imported meaning the balance is now off by $500 as you are missing a transaction. You can find out which ones have done this since I find they are not bolded like the other freshly imported transactions. Let me know if you need some sample csv's to replicate and I can pull some from my bank and set some fictitious descriptions.

from actual.

I can confirm that for me it is the same with ofx import, usually my same amount daily coffee transactions and lunch transactions are gone.

from actual.

Missing reproduction steps.

from actual.

Steps to reproduce.

1. Create an off budget account with a zero dollar starting balance.
2. Import week1.csv and confirm total balance is $465 and that the first transaction is on the 11/03/2024 and the last is on the 17/03/2024.
3. Import week2.csv

This should show that all transactions get overwritten/updated even the coffee shop purchases that have different descriptions.

week1.csv
week2.csv
Client version: v24.3.0
Server version: v24.3.0

from actual.

@britto89 that functionality is actually by design. The imports have de-dupe logic that attempts to match the new imported transactions to the old ones. And if they match - it updates their details. It matches by date and amount. Which in the week1/2 case doesn't work well because.. all the amounts are the same.

For your use-case my recommendation would be to perform imports with a months worth of data. And then to allow the de-dupe logic to take care of removing duplicates & creating new transactions. Instead of doing weekly exports.

However, I'm still missing reproduction steps for the root-issue reported here: transactions getting deleted. Please provide reproduction steps to delete transactions on import.

from actual.

Cant import anything, it has totally trashed my accounts.

from actual.

@britto89 that functionality is actually by design. The imports have de-dupe logic that attempts to match the new imported transactions to the old ones. And if they match - it updates their details. It matches by date and amount. Which in the week1/2 case doesn't work well because.. all the amounts are the same.

For your use-case my recommendation would be to perform imports with a months worth of data. And then to allow the de-dupe logic to take care of removing duplicates & creating new transactions. Instead of doing weekly exports.

However, I'm still missing reproduction steps for the root-issue reported here: transactions getting deleted. Please provide reproduction steps to delete transactions on import.

Well it might technically not be a delete, it probably just does not import similar transactions. I think all reproduction steps are already in this thread and it's getting clear what happens. I am importing monthly once and some transactions are so far always lost, usually the ones that are almost similar just like how other people described it.

We could also say that the de-duplication logic is rather harmful in the current implementation. Anyhow it should not de-duplicate anything as I import full months and there is in my case no overlap or anything that would need such a logic.

from actual.

@MatissJanis I think the needs info label can now be removed.

from actual.

👋 I understand how the existing de-duplication logic can be quite frustrating under some circumstances. The dedupe logic has been unchanged basically since the very inception of open-source Actual and while it's not absolutely perfect, IMO it does handle the majority of the scenarios that occur when doing file imports.

Here's a feature request I have opened: the ability to disable the dedupe logic. Which would give you folks a solution. PRs with the implementation are welcome!

The dedupe logic works quite well if larger time-ranges are used. So my recommendation for now (until someone builds the feature) would be to import larger time ranges. The specific time-range depends on your use case, but here's the logic you can follow: "previous time range * 2" (i.e. if you usually import a month - try importing the current month + previous in the same go; if you usually import a week - try importing this week + previous week).

I will close this bug report in favour of the feature ticket.

from actual.