Giter VIP home page Giter VIP logo

firefly-ynab4-importer's Issues

A few issues and a question

Hey,
Thanks for building the tool! I am trying to import my YNAB4 info for the last six years into Firefly and have been running into a few issues and am currently stuck. Python isn't my forte so have been fixing things as best I could.

Not sure if you prefer separate issues for things that I was able to fix, so listing them out here:

  1. Had to install Colorama manually:
    It's a dependency in code but is not part of the pip package. So in my venv I had to install it manually.

  2. Accounts with no starting balance break import:
    For some of my account I don't have a starting balance so the import script crashes on with

Traceback (most recent call last):
  File "firefly_ynab4_importer/main.py", line 1446, in <module>
    cli()
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "firefly_ynab4_importer/main.py", line 1442, in import_transactions
    importer.run(dry_run)
  File "firefly_ynab4_importer/main.py", line 638, in run
    self._process_accounts()
  File "firefly_ynab4_importer/main.py", line 795, in _process_accounts
    start_date, balance = starting_balances[acc]
KeyError: 'Tangerine MasterCard'

I got around this by changing line 792 to:

            if acc in starting_balances:
                start_date, balance = starting_balances[acc]

But now I'm stuck with this error:

Traceback (most recent call last):
  File "firefly_ynab4_importer/main.py", line 1445, in <module>
    cli()
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/raveesh/Projects/playground/ff-import/lib/python3.8/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "firefly_ynab4_importer/main.py", line 1441, in import_transactions
    importer.run(dry_run)
  File "firefly_ynab4_importer/main.py", line 639, in run
    self._process_transactions()
  File "firefly_ynab4_importer/main.py", line 880, in _process_transactions
    assert (
AssertionError: Unable to process transaction with unknown category: |Income:Available this month|

I can't figure out how that is supposed to be handled. Is this something in the config I need to change?

Thanks

Various errors and workarounds

Hi! I just wanted to start off by saying thank you for writing this! I had the fairly heavy task of importing almost six years of transactions from classic YNAB to Firefly, and despite the issues I'll be listing, this script simplified the task immensely!

That being said, I had some issues come up in the process. I was able to work through them all, but I wanted to post both how I solved them for other users, and also for the possibility of solutions in the code.

1. I was getting an error with the Pre-YNAB Debt Categories (possibly because I had them all hidden?)

AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|
---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
      2     config_file="config.toml",
      3     register_file = "register.csv",
      4     budget_file = "budget.csv",
      5     dry_run = True,

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
     27         filter_max_date=filter_max_date,
     28     )
---> 29     importer.run(dry_run)
     30 

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
    556         self._process_budgets()
    557         self._process_accounts()
--> 558         self._process_transactions()
    559         if not dry_run:
    560             self._load_cache()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _process_transactions(self)
    808                 budget = self._budget(tx)
    809                 if budget:
--> 810                     assert budget in self.data.budgets, f"Unable to process transaction with unknown budget: |{budget}|"
    811                     # ignore hidden categories
    812                     if self.data.budgets[budget].active:

AssertionError: Unable to process transaction with unknown budget: |Visa (Citibank) (hidden)|

I fixed this by a) manually removing initial balance items from the register csv, and b) changing the lines with other transactions that were assigned to the category (mainly interest that was being charged the to pre-ynab debt balance) to be a misc. transaction.

2. Creation of the budget categories didn't work (possibly because of a change to the API?)

{'message': 'The given data was invalid.', 'errors': {'auto_budget_amount': ['The amount is required.']}}
HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets
---------------------------------------------------------------------------
HTTPError                                 Traceback (most recent call last)
~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in <module>
----> 1 import_transactions(
      2     config_file="config.toml",
      3     register_file="register.csv",
      4     budget_file="budget.csv",
      5     dry_run=False,

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in import_transactions(config_file, register_file, budget_file, dry_run, firefly_url, firefly_access_token, filter_min_date, filter_max_date)
   1356         filter_max_date=filter_max_date,
   1357     )
-> 1358     importer.run(dry_run)
   1359 # if __name__ == "__main__":
   1360 # cli()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in run(self, dry_run)
    568             self._create_currencies()
    569             self._create_categories()
--> 570             self._create_budgets()
    571             self._create_budget_limits()
    572             self._create_available_budgets()

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in _create_budgets(self)
    988                         self.firefly_data.budgets[budget.name] = response.json()["data"]
    989                 else:
--> 990                     response = self._session.post("/api/v1/budgets", json=data)
    991                     self.firefly_data.budgets[budget.name] = response.json()["data"]
    992             except requests.HTTPError as e:

~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/sessions.py in post(self, url, data, json, **kwargs)
    588         """
    589 
--> 590         return self.request('POST', url, data=data, json=json, **kwargs)
    591 
    592     def put(self, url, data=None, **kwargs):

~/.repos/Firefly-YNAB4-Importer/firefly_ynab4_importer/main.py in request(self, method, url, **kwargs)
    511                 print(response.json())
    512                 print(f"{kwargs=}")
--> 513             response.raise_for_status()
    514         return response
    515     def get_all_pages(self, url, params=None, **kwargs) -> dict:

~/.local/share/virtualenvs/Firefly-YNAB4-Importer-br1XYdXy/lib/python3.8/site-packages/requests/models.py in raise_for_status(self)
    951 
    952         if http_error_msg:
--> 953             raise HTTPError(http_error_msg, response=self)
    954 
    955     def close(self):

HTTPError: 422 Client Error: Unprocessable Entity for url: <server redacted>/api/v1/budgets

I manually created all the budget categories in firefly to get around this error.

3. The function to create transactions is commented out by default (line 673 in main.py). Is this on purpose for some reason?

Fixed by uncommenting that line.

4. I had the same error reported in #2

This seems to occur on a subsequent execution when the previous attempt crashed. I fixed it by deleting the cache json before running (as mentioned in the issue)

5. I had some issues with special characters in the Payee column (e.g. é in café)

I fixed this by changing them to their ascii equivilent in both the csv and and the expense accounts page in Firefly.

Other Notes

Just a few things I jotted down while working through this that I think would be nice-to-haves:

  • The config template has some values that are different from the default. I found this challenging because I wondered which would be the "best" setting for most users. I think the default should be something that a new user can always try first, either with no config file or with the template.
  • Some sort of verbose mode that lists transactions and other info as its processing them would be very helpful to figure out where its breaking (I ended up needing to drop print statements into spots in the code to figure out the issues I had).

Import Errors

Hi,

Any ideas what could be causing this error mid import?

dacite.exceptions.WrongTypeError: wrong value type for field "revenue_accounts" - should be "typing.Dict[str, dict]" instead of value "{Redacted list of YNAB payees} of type "dict"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.