maroux / firefly-ynab4-importer Goto Github PK
View Code? Open in Web Editor NEWImporter for moving from YNAB4 to Firefly-iii
License: MIT License
Importer for moving from YNAB4 to Firefly-iii
License: MIT License
Hey,
Thanks for building the tool! I am trying to import my YNAB4 info for the last six years into Firefly and have been running into a few issues and am currently stuck. Python isn't my forte so have been fixing things as best I could.
Not sure if you prefer separate issues for things that I was able to fix, so listing them out here:
Had to install Colorama manually:
It's a dependency in code but is not part of the pip package. So in my venv I had to install it manually.
Accounts with no starting balance break import:
For some of my account I don't have a starting balance so the import script crashes on with
Traceback (most recent call last):
File "firefly_ynab4_importer/main.py", line 1446, in <module>
cli()
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "firefly_ynab4_importer/main.py", line 1442, in import_transactions
importer.run(dry_run)
File "firefly_ynab4_importer/main.py", line 638, in run
self._process_accounts()
File "firefly_ynab4_importer/main.py", line 795, in _process_accounts
start_date, balance = starting_balances[acc]
KeyError: 'Tangerine MasterCard'
I got around this by changing line 792 to:
if acc in starting_balances:
start_date, balance = starting_balances[acc]
But now I'm stuck with this error:
Traceback (most recent call last):
File "firefly_ynab4_importer/main.py", line 1445, in <module>
cli()
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "firefly_ynab4_importer/main.py", line 1441, in import_transactions
importer.run(dry_run)
File "firefly_ynab4_importer/main.py", line 639, in run
self._process_transactions()
File "firefly_ynab4_importer/main.py", line 880, in _process_transactions
assert (
AssertionError: Unable to process transaction with unknown category: |Income:Available this month|
I can't figure out how that is supposed to be handled. Is this something in the config I need to change?
Thanks
Hi! I just wanted to start off by saying thank you for writing this! I had the fairly heavy task of importing almost six years of transactions from classic YNAB to Firefly, and despite the issues I'll be listing, this script simplified the task immensely!
That being said, I had some issues come up in the process. I was able to work through them all, but I wanted to post both how I solved them for other users, and also for the possibility of solutions in the code.
AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
2 config_file="config.toml",
3 register_file = "register.csv",
4 budget_file = "budget.csv",
5 dry_run = True,
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
27 filter_max_date=filter_max_date,
28 )
---> 29 importer.run(dry_run)
30
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
556 self._process_budgets()
557 self._process_accounts()
--> 558 self._process_transactions()
559 if not dry_run:
560 self._load_cache()
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _process_transactions(self)
808 budget = self._budget(tx)
809 if budget:
--> 810 assert budget in self.data.budgets, f"Unable to process transaction with unknown budget: |{budget}|"
811 # ignore hidden categories
812 if self.data.budgets[budget].active:
AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|
I fixed this by a) manually removing initial balance items from the register csv, and b) changing the lines with other transactions that were assigned to the category (mainly interest that was being charged the to pre-ynab debt balance) to be a misc. transaction.
{'message': 'The given data was invalid.', 'errors': {'auto_budget_amount': ['The amount is required.']}}
HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets
---------------------------------------------------------------------------
HTTPError Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
2 config_file="config.toml",
3 register_file="register.csv",
4 budget_file="budget.csv",
5 dry_run=False,
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
1356 filter_max_date=filter_max_date,
1357 )
-> 1358 importer.run(dry_run)
1359 # if __name__ == "__main__":
1360 # cli()
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
568 self._create_currencies()
569 self._create_categories()
--> 570 self._create_budgets()
571 self._create_budget_limits()
572 self._create_available_budgets()
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _create_budgets(self)
988 self.firefly_data.budgets[budget.name] = response.json()["data"]
989 else:
--> 990 response = self._session.post("/api/v1/budgets", json=data)
991 self.firefly_data.budgets[budget.name] = response.json()["data"]
992 except requests.HTTPError as e:
~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/sessions.py in post(self, url, data, json, **kwargs)
588 """
589
--> 590 return self.request('POST', url, data=data, json=json, **kwargs)
591
592 def put(self, url, data=None, **kwargs):
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in request(self, method, url, **kwargs)
511 print(response.json())
512 print(f"{kwargs=}")
--> 513 response.raise_for_status()
514 return response
515 def get_all_pages(self, url, params=None, **kwargs) -> dict:
~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
951
952 if http_error_msg:
--> 953 raise HTTPError(http_error_msg, response=self)
954
955 def close(self):
HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets
I manually created all the budget categories in firefly to get around this error.
Fixed by uncommenting that line.
This seems to occur on a subsequent execution when the previous attempt crashed. I fixed it by deleting the cache json before running (as mentioned in the issue)
I fixed this by changing them to their ascii equivilent in both the csv and and the expense accounts page in Firefly.
Just a few things I jotted down while working through this that I think would be nice-to-haves:
Hi,
Any ideas what could be causing this error mid import?
dacite.exceptions.WrongTypeError: wrong value type for field "revenue_accounts" - should be "typing.Dict[str, dict]" instead of value "{Redacted list of YNAB payees} of type "dict"
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.