Where do you create or get the list of Categories and Subcategories (category.1), based on what? How to add my own categories?

Hi there,

I'm not a programmer and I have learned some basic coding skills over the years; I'm also a personal finance geek so this is just an amazing project for me !

That being said, I have already built my own google sheet template and I would like to use your code to retrieve the transactions and update the google sheet in the tab "transactions" (instead of spreading it into a tab per month).

Thanks !

Hi there,

I'm running into an error that I can't seem to fix :

"spreadsheetId": "1L2v2WbA0NWLzv8L9C76cGVxSSxPU7cxJnN7UsM4yY08",
"ranges": [
"Invalid Date!A:undefined"
]
}
✖ Error clearing ranges invalid date!a:undefined:

{
  error: 'Error: Invalid range[0]: Unable to parse range: Invalid Date!A:undefined'
}

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

Any idea where this is coming from ?

Instructions in README are a bit overly complicated for just getting started out. Would be cool to run something like:

npm run template

and have the proper API calls get routed through automatically.

All, thanks for your incredible patience here. Today is the day.

Version 2.x.x represents an almost complete rewrite from the ground-up, expect it to be rough around the edges until we complete a few rounds of beta testing. Thanks for helping make Mintable better.

What's New in v2.0.0

• Complete (~4000 lines) rewrite: Kept all the good stuff from v1.x.x, trimmed the fat where possible.
• Migration to Typescript: Easier to reason about correctness & establish rigid API constraints for integrations.
• Migration Script: Migrate your v1.x.x installation over in one command.
• Simplified Install Process/Documentation: Get up-and-running with Mintable in 3 commands.
• Global npm package: Call mintable as a command, anywhere on your machine without cloning anything.
• mintable CLI/setup wizard: Configure everything through an automated CLI-based setup wizard.
• Config Validation: Tells you the exact spot where your config is broken (if relevant).
• Better Logging: Almost every point of failure logs a human-readable error message.
• Debug Mode: Optionally log the results of every API request to debug strange behavior.
• Error Reporting: Click a single link which auto-fills an issue on GitHub with error/machine info.
• Balances Sheet: View the balances of all your accounts automatically on one sheet.
• Per-Account Configuration: Pull some accounts from Plaid, and others from a CSV (like Apple Card).
• CSV Exporter: Optionally export your results to a CSV file and use Excel/Numbers if you don't trust Google.
• Lots more

What's Not Working Yet

• #69 Sheets are not getting sorted correctly
• #68 Accounts table is broken if you have not setup any accounts yet
• #67 Calling updateConfig with the initialize argument nukes existing config if it exists
• #50 Export to CSV option (instead of Google Sheets)
• #49 Spreadsheet/CSV importer for banks not on Plaid
• #46 Add a test suite
• #57 Transaction Overrides
• #58 Transaction Filters
• Probably lots more

Testing Instructions

I've compiled a Testing Checklist below if you have the time, but if you don't want to start from scratch, I'd still appreciate any feedback.

Please report an Issue for anything not listed above which seems like a regression/bug and apply the Bug: v2.0.0 Beta label. If you encounter any issues with these instructions themselves or have any general feedback, please leave a comment below.

Testing Checklist

• Make a backup copy of your v1.x.x mintable.config.json.
• Install Mintable v2.x.x and migrate your v1.x.x config.
• Create a new blank Google Sheet.
• Change the integrations.google.documentId property in ~/mintable.jsonc to the blank one you just created.
• Update your transactions.
• Delete ~/mintable.jsonc.
• Use the setup wizard to configure Google Sheets and Plaid.
• Add a new account.
• Refresh the credentials for an existing account.
• Update your transactions.
• Make a backup copy of your production Google Sheet.
• Change the integrations.google.documentId property in ~/mintable.jsonc to the new backup sheet you just made.
• Update your transactions.

Lots of errors could be resolved with some nicer types, i.e. TypeScript.

The current onboarding experience is cumbersome and nothing really should be done through the terminal.

As part of #43, it came up that a unit test suite would be a good addition. In this thread, we could discuss what that might look like technology-wise.

Items to discuss:

1. Where it builds
   It looks like Travis-CI is what's required on branch and PR builds, so likely that should also run the test suite.

2. Tech stack
   I have the most experience with Mocha and Chai but, since this project uses React, something like Jest might be preferable to the other maintainers?

3. Unit / Integration / Functional Tests
   We could easily start off with just unit tests, adding in additional test suites later like component (integration) tests or functional test (e.g. Cypress).

Enable Regex-style matches and replacement for transaction info – i.e. name, location, category, etc.

I was editing the config file (trying to change the template location) and must of missed, or added, an extra comma, which caused it to be overwritten with the initial layout. This lost all the API keys for plaid and google sheets. Specifically it's the plaid ones that annoyed me since we only get a certain amount of linked accounts, but I know don't know that item id.

It might be an idea not to nuke existing config files on error, or store the API keys in a separate file after initial setup.

(I should also make backups, I know, but it's also preventable, and unexpected behaviour)

Hello,

In following the steps to migrate from v1 to v2, I had to update one of my bank accounts (got the error PlaidError: ITEM_LOGIN_REQUIRED after running mintable fetch.

1. I ran mintable account-setup
2. clicked update on the outdated bank account
3. successfully completed the handshake with plaid.
4. I then got back to the main screen (titled Plaid – Account Configuration) and clicked Link a new account.
5. It then redirects me back to plaid and shows me the success screen on step 3.

seeing this in the terminal

2020-06-24T22:18:29.688Z [ERROR] Encountered error during authentication.

 {
  display_message: '',
  error_code: 'item-no-error',
  error_message: '',
  error_type: ''
}

I cannot link a new account.

6. i reran mintable account-setup and I see a duplicate row now for the bank account i updated in step 1-3. Pressing link new account now works correctly.

Need a public way of doing tests (for branches/PRs/etc) with Plaid sandbox environment.

I have some WealthSimple accounts. Plaid doesn't support WealthSimple, afaik. However, WealthSimple have their own REST API: https://developers.wealthsimple.com/ - which looks like it does everything you'd need.

I'm happy to write a provider for WealthSimple. However, I'm not sure how to use it? I only see a single ACCOUNT_PROVIDER config setting? I have some accounts with banks that Plaid does support, as well as the WealthSimple accounts - so I'd need to set a provider on a per-account basis, maybe?

Is there currently a way to say "Get data for these accounts from this provider, and use this other provider for these accounts"?

I've been noticing a lot of duplicate transactions in the mintable spreadsheet. Doesn't seem to be a rhyme or reason as it happens with both pending and completed transactions. I understand the tabs are cleared and recreated so I assume these are coming from plaid - how can we validate?

Would be nice to be able to configure the properties that get extracted from each transaction, as well as additional reference header columns.

Hello,

I am trying to run mintable fetch and it creates only 1 tab Balances. It isnt created new tabs and putting transactions in it.
I see this output:

2020-06-24T22:41:31.730Z [INFO] Using default configuration file `/Users/happy/mintable.jsonc.`
2020-06-24T22:41:31.731Z [INFO] You can supply either --config-file or --config-variable to specify a different configuration.
2020-06-24T22:41:31.732Z [INFO] Successfully opened configuration file.
2020-06-24T22:41:31.732Z [INFO] Successfully parsed configuration.
2020-06-24T22:41:36.696Z [INFO] Successfully validated configuration.
2020-06-24T22:41:36.715Z [INFO] Fetching account ********* using plaid.
2020-06-24T22:41:36.735Z [INFO] Fetching account ********* using plaid.
2020-06-24T22:41:36.736Z [INFO] Fetching account ********* using plaid.
2020-06-24T22:41:37.107Z [INFO] Fetched 1 sub-accounts and 0 transactions.
2020-06-24T22:41:37.198Z [INFO] Fetched 1 sub-accounts and 19 transactions.
2020-06-24T22:41:37.408Z [INFO] Fetched 1 sub-accounts and 36 transactions.
2020-06-24T22:41:38.263Z [INFO] Fetched 1 sheets.
2020-06-24T22:41:38.671Z [INFO] Added sheet Balances.
2020-06-24T22:41:38.948Z [INFO] Cleared 1 range(s): 'Balances'!A1:G4.
2020-06-24T22:41:39.222Z [INFO] Updated 1 range(s): 'Balances'!A1:G4.
2020-06-24T22:41:39.488Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.517Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.551Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.572Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.727Z [INFO] Fetched 2 sheets.
(node:4635) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'properties' of undefined
    at GoogleIntegration.<anonymous> (/usr/local/lib/node_modules/mintable/lib/integrations/google/googleIntegration.js:86:115)
    at step (/usr/local/lib/node_modules/mintable/lib/integrations/google/googleIntegration.js:33:23)
    at Object.next (/usr/local/lib/node_modules/mintable/lib/integrations/google/googleIntegration.js:14:53)
    at fulfilled (/usr/local/lib/node_modules/mintable/lib/integrations/google/googleIntegration.js:5:58)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:4635) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1)
(node:4635) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
2020-06-24T22:41:39.762Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.815Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:39.894Z [INFO] Updated indices for 2 sheets.
2020-06-24T22:41:40.146Z [INFO] Fetched 2 sheets.
2020-06-24T22:41:40.517Z [INFO] Updated formatting for 2 sheets.

Just had a few transactions of these nature:

AUTOMATIC PAYMENT - THANK YOU

get categorized by Plaid as a Restaurant transaction. This is obviously not the case; we should provide a way to map certain regexes, i.e. autopay, e-payment, etc. to a certain category, like Credit Card Payment.

It would be nice to be able to return other account properties (e.g. 'subtype', 'mask') by adding them to TRANSACTION_COLUMNS

I have an account who's official_name is null, so it would be nice if I could have it fall back to name if official_name is null. This is tricky, though, because columns are configurable.

I think we missed the bill here and over complicated things. A terminal-based onboarding experience is probably better for most users here.

Is it possible to customize the host and port for the web interface. The port option does exist as shown below, but isn't used and 3000 is hardcoded everywhere else.

During the setup process, I kept getting a 401 error when trying to connect my Google Sheet. After further inspection, I noticed that the issue was due to whitespace from the copy/paste from the linked QuickStart page. When I removed the whitespace, everything worked. I'd be happy to provide a PR to improve this UX.

I was thinking about a few ways of modifying ConfigPropertyInputGroup. We could:

1. Always disallow leading/trailing whitespace. This could be unnecessarily limiting for some configuration properties where we'd want users to be able to. However, it would be pretty simple to implement.
2. Introduce an allowWhitespace passed as configuration. This could be more backward compatible and we could introduce it per-property (defaulting to false). For instance, I'm thinking we could configure it here, adding an allowWhitespace: false parameter to this config hash:
   

   mintable/src/pages/sheet-provider-setup.jsx

   Lines 16 to 19 in 52a8a2f

   	const configFileProperties = [
   	{ displayName: 'Client ID', propertyId: 'SHEETS_CLIENT_ID' },
   	{ displayName: 'Client Secret', propertyId: 'SHEETS_CLIENT_SECRET' }
   	]

Alternatively, we could sanitize the input on the backend. I was leaning away from this strategy because the issue I experienced is going to primarily happen when dealing with front-end user input, but that's a path forward as well.

What are your thoughts?

PS: Thank you for this project! I've only been using it for a few days and it has already been incredibly useful.

Hard to determine when a property is missing or extraneous.

Originally planned for #19 but PR is getting too large.

After we implement providers other than Google Sheets/Plaid, we'll want to provide a pluggable onboarding experience for them as well.

right now we’re doing something like this:

wrapPromise(
    doTheThing {
        ...
    }
)

and I think it overcomplicates things significantly. it’s one more layer of promises to reason about (bad!) and the output logs get duplicated and interleaved uncontrollably.

a more sane (and I think industry-standard way):

doTheThing {
    logging.info(“Starting the thing”)
    ...
    logging.info(“Done with the thing”)
    logging.debug(“Response or error content of the thing”)
}

As mentioned in #9, we need a more concrete interface to pass back and forth between the main index script and a provider.

I think adding TypeScript to the project would be a big win here.

Version 2.x.x represents an almost complete rewrite from the ground-up, expect it to be rough around the edges until we complete a few rounds of beta testing. Thanks for helping make Mintable better.

What's New in v2.0.0

• Complete (~4000 lines) rewrite: Kept all the good stuff from v1.x.x, trimmed the fat where possible.
• Migration to Typescript: Easier to reason about correctness & establish rigid API constraints for integrations.
• Migration Script: Migrate your v1.x.x installation over in one command.
• Simplified Install Process/Documentation: Get up-and-running with Mintable in 3 commands.
• Global npm package: Call mintable as a command, anywhere on your machine without cloning anything.
• mintable CLI/setup wizard: Configure everything through an automated CLI-based setup wizard.
• Config Validation: Tells you the exact spot where your config is broken (if relevant).
• Better Logging: Almost every point of failure logs a human-readable error message.
• Debug Mode: Optionally log the results of every API request to debug strange behavior.
• Error Reporting: Click a single link which auto-fills an issue on GitHub with error/machine info.
• Balances Sheet: View the balances of all your accounts automatically on one sheet.
• Per-Account Configuration: Pull some accounts from Plaid, and others from a CSV (like Apple Card).
• CSV Exporter: Optionally export your results to a CSV file and use Excel/Numbers if you don't trust Google.
• Lots more

What You've Helped Fix

• #67 Calling updateConfig with the initialize argument nukes existing config if it exists – Fixed in #74 / v2.0.0-beta.34
• #70 Clicking 'link a new account' is showing me the success screen of my last authentication – Fixed in #76 / v2.0.0-beta.34
• #73 mintable setup -> linking a new account successfully doesnt show the account – Fixed in #76 / v2.0.0-beta.34
• #71 mintable fetch will not create new sheet tabs – Fixed in #77 / v2.0.0-beta.34
• #49 Spreadsheet/CSV importer for banks not on Plaid – Fixed in #80 / v2.0.0-beta.34
• #68 Accounts table is broken if you have not setup any accounts yet – Fixed in #81 / v2.0.0-beta.36
• #81 Ability to delete accounts – Fixed in #81 / v2.0.0-beta.36
• #69 Sheets are not getting sorted correctly – Fixed in ac3fb91 / v2.0.0-beta.39
• GitHub Actions Integration – Fixed in v2.0.0-beta.40
• BitBar Integration – Fixed in v2.0.0-beta.40
• cron Integration – Fixed in v2.0.0-beta.40
• #50 Export to CSV option (instead of Google Sheets) – Fixed in #80 / v2.0.0-beta.41
• #57 Transaction Overrides – Added in #96 / v2.0.0-beta.44
• #58 Transaction Filters – Added in #96 / v2.0.0-beta.44

Testing Instructions

I've compiled a Testing Checklist below if you have the time, but if you don't want to start from scratch, I'd still appreciate any feedback.

Please report an Issue for anything not listed above which seems like a regression/bug and apply the Bug: v2.0.0 Beta label. If you encounter any issues with these instructions themselves or have any general feedback, please leave a comment below.

Testing Checklist

• Make a backup copy of your v1.x.x mintable.config.json.
• Install Mintable v2.x.x and migrate your v1.x.x config.
• Create a new blank Google Sheet.
• Optional If you set a template sheet in your v1.x.x config, create a tab with that title
• Change the integrations.google.documentId property in ~/mintable.jsonc to the blank one you just created.
• Update your transactions.
• Delete ~/mintable.jsonc.
• Use the setup wizard to configure Google Sheets and Plaid.
• Add a new account.
• Refresh the credentials for an existing account.
• Update your transactions.
• Make a backup copy of your production Google Sheet.
• Change the integrations.google.documentId property in ~/mintable.jsonc to the new backup sheet you just made.
• Update your transactions.

Rather than generic "bank" names like:

CHASE

we should try to fetch more helpful "account" names from Plaid, like

Chase Sapphire Reserve (xxxx-xxxx-xxxx-2345)

for easier grokking.

Bad. Users will be mad if they call mintable setup accidentally and lose their config.

Hi. Not sure what's happening but stuck in the initial setup.

Ok. Finally managed to pass the setup screen.
However I don't transactions populating the spreadsheet.

I can see data being pulled from plaid but nothing is showing in the sheet.

As mentioned on HN thread – there are some security/privacy concerns about using Google Sheets. Would be nice to have alternative handlers if you want to use Excel, for example.

As mentioned on HN thread – there are some security concerns about using Plaid.

As mentioned on Reddit thread – some users have banks which are not supported with Plaid.

2 sheets is largely inflexible, especially if you want lots of historical data from a cold start (say, for the last year).

Blocked on #12.

After using Mintable for a few months now the default sort is backwards – new sheets end up on the far right and it requires horizontal scrolling:

Steps to Reproduce:

1. cloned git repo, switched to 2.0 branch
2. followed instructions to install
3. migrated old config
4. ran mintable fetch

Error:

This script will help you add accounts to Plaid.
2020-06-24T01:33:43.473Z [INFO] Using default configuration file `/Users/willie/mintable.jsonc.`
2020-06-24T01:33:43.473Z [INFO] You can supply either --config-file or --config-variable to specify a different configuration.
2020-06-24T01:33:43.473Z [INFO] Successfully opened configuration file.
2020-06-24T01:33:43.474Z [INFO] Successfully parsed configuration.
File '/Users/willie/src/github.com/kevinschaich/mintable/lib/common/config.js' is a JavaScript file. Did you mean to enable the 'allowJs' option?
2020-06-24T01:33:45.940Z [ERROR] Failed to set up Plaid Account(s).

 TypeError: Cannot read property 'id' of null
    at Ajv._get$Id [as _getId] (/Users/willie/src/github.com/kevinschaich/mintable/node_modules/ajv/lib/ajv.js:392:14)
    at Ajv._addSchema (/Users/willie/src/github.com/kevinschaich/mintable/node_modules/ajv/lib/ajv.js:301:37)
    at Ajv.validate (/Users/willie/src/github.com/kevinschaich/mintable/node_modules/ajv/lib/ajv.js:95:26)
    at Object.exports.validateConfig (/Users/willie/src/github.com/kevinschaich/mintable/lib/common/config.js:98:27)
    at Object.exports.getConfig (/Users/willie/src/github.com/kevinschaich/mintable/lib/common/config.js:110:35)
    at /Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:54:35
    at step (/Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:33:23)
    at Object.next (/Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:14:53)
    at /Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:4:12)
    at Object.default (/Users/willie/src/github.com/kevinschaich/mintable/lib/integrations/plaid/add.js:47:44)
    at /Users/willie/src/github.com/kevinschaich/mintable/lib/scripts/cli.js:42:82
    at Array.forEach (<anonymous>)
    at Object.<anonymous> (/Users/willie/src/github.com/kevinschaich/mintable/lib/scripts/cli.js:42:19)
    at Module._compile (internal/modules/cjs/loader.js:1200:30) 

System Info:

arch: x64
platform: darwin
os: v19.5.0
mintable: v2.0.0
node: v14.4.0

Is there a config option to export all transactions to a single google sheet and update that sheet during future transaction retrieval?

Plaid's max transaction fetch size is 500.

Rather than fail if there's more than 500 transactions, we should figure out how to handle paging.

Need a handleErrorState function or similar so we're not just return false some places, empty arrays other places, undefined some places, etc.

Server should also have a sendErrorResponse and sendSucess function or similar.

As per Reddit, Amex (& likely others) don't support the Auth endpoint in Plaid. Lo and behold:

we're using this endpoint for both new accounts & updates. I suspect we can probably use the Balance endpoint here instead since we don't need account numbers.