matatika / tap-shopify Goto Github PK

View Code? Open in Web Editor NEW

3.0 2.0 12.0 388 KB

This Shopify tap produces JSON-formatted data following the Singer spec.

License: GNU Affero General Public License v3.0

Python 100.00%

meltano shopify singer

tap-shopify's Introduction

Singer Tap for Shopify

This Shopify tap produces JSON-formatted data following the Singer spec.

tap-shopify is a Singer tap for the Shopify REST API built with the Meltano Tap SDK for Singer Taps.

Configuration

How to get your access_token: Shopify Docs.

access_token String (required) - The access token to authenticate with the Shopify API.
store String (required) - Shopify store id, use the prefix of your admin url. e.g. https://[your store].myshopify.com/admin.
start_date String (optional) - The earliest record date to sync.
admin_url String (optional) - The full admin url for your Shopify store (overrides 'store' property).
is_plus_account Boolean (optional) - Enable Shopify plus account end points.

If you plan on using environment variables to declare these settings then you will be using:

TAP_SHOPIFY_ACCESS_TOKEN
TAP_SHOPIFY_STORE
TAP_SHOPIFY_START_DATE
TAP_SHOPIFY_ADMIN_URL
TAP_SHOPIFY_IS_PLUS_ACCOUNT

Supported Streams

Default Streams

Shopify Plus Streams

User

Roadmap

Add OAuth support w/ tests
Update schema format for 'date-time' fields and 'singer.decimal'

Installation

Use pip to install a release from GitHub

pip install git+https://github.com/Matatika/[email protected]

Configuration

Accepted Config Options

A full list of supported settings and capabilities for this tap is available by running:

tap-shopify --about

Source Authentication and Authorization

To read data from your store, you need a Shopify access token.

For a single store, Shopify recommends you create a Custom App

Log in to your Shopify store admin at https://.myshopify.com/admin
Click “Apps” in the menu
Click “Develop apps for your store” in the "Build custom apps for your unique needs" section
Click the “Create an app” button and give you app a name.
Next you need to "Configure Admin API scopes" with read access to all resources you require. e.g. read_locales,read_products,read_orders,read_locations,read_inventory,read_fulfillments,read_customers,
Finally, in "API credentials" tab, click "Install app" button. Copy the Admin API access token

Usage

You can easily run tap-shopify by itself or in a pipeline using Meltano.

Executing the Tap Directly

tap-shopify --version
tap-shopify --help
tap-shopify --config CONFIG --discover > ./catalog.json

Developer Resources

Initialize your Development Environment

pipx install poetry
poetry install

Create and Run Tests

Create tests within the tap_shopify/tests subfolder and then run:

poetry run pytest

You can also test the tap-shopify CLI interface directly using poetry run:

poetry run tap-shopify --help

Testing with Meltano

Note: This tap will work in any Singer environment and does not require Meltano. Examples here are for convenience and to streamline end-to-end orchestration scenarios.

Your project comes with a custom meltano.yml project file already created. Open the meltano.yml and follow any "TODO" items listed in the file.

Next, install Meltano (if you haven't already) and any additional plugins:

# Install meltano
pipx install meltano
# Initialize meltano within this directory
cd tap-shopify
meltano install

Now you can test and orchestrate using Meltano:

# Test invocation:
meltano invoke tap-shopify --version
# OR run a test `elt` pipeline:
meltano elt tap-shopify target-jsonl

Cloud hosting and SaaS

Our team would be happy to help www.matatika.com

License

AGPLv3 License

It is our intention that all subsequent changes to this software are made available to the community. Under the terms of this license, you must open source your platform code if you distribute this software-as-a-service. Applications that reside inside an organization’s network do not trigger the release of the source code.

tap-shopify's People

Contributors

Stargazers

Watchers

Forkers

danielpdwalker helloturbine gsumo bavaan-platform chintanpatelth dataproff luabase kingalban harisrab pnadolny13 farzeni prashiyn

tap-shopify's Issues

Failing to install tap on Mac

Running Pythong 3.11.4 via asdf on macos

Getting this error:

environment for 'extractors/tap-shopify'
Extractor 'tap-shopify' could not be installed: failed to install plugin 'tap-shopify'.

P.s.: This is my first Meltano project so might be doing something stupif, but I was able to install other extractors and loaders.

Here is the traceback:

meltano --log-level=debug add extractor tap-shopify                                      1372ms  Wed Sep 13 20:34:49 2023
2023-09-14T01:34:52.534520Z [debug    ] /etc/localtime found
2023-09-14T01:34:52.536639Z [debug    ] 1 found:
 {'/etc/localtime is a symlink to': 'America/Chicago'}
2023-09-14T01:34:52.539266Z [debug    ] Creating DB engine for project at '/Users/konung/Projects/StellaLife/Code/stellalife-elt-meltano/stellalife-elt' with DB URI 'sqlite://Users/konung/Projects/StellaLife/Code/stellalife-elt-meltano/stellalife-elt/.meltano/meltano.db'
2023-09-14T01:34:53.648126Z [debug    ] Found plugin parent            parent=tap-shopify plugin=tap-shopify source=<DefinitionSource.HUB: 1>
2023-09-14T01:34:53.697074Z [debug    ] Locked plugin definition       path=PosixPath('/Users/konung/Projects/StellaLife/Code/stellalife-elt-meltano/stellalife-elt/plugins/extractors/tap-shopify--matatika.lock')
Added extractor 'tap-shopify' to your Meltano project
Variant:	matatika (default)
Repository:	https://github.com/Matatika/tap-shopify
Documentation:	https://hub.meltano.com/extractors/tap-shopify--matatika

Installing extractor 'tap-shopify'...
2023-09-14T01:34:53.783222Z [debug    ] Packages for 'extractors/tap-shopify' have changed so performing a clean install.
2023-09-14T01:34:53.783618Z [debug    ] Removed old virtual environment for 'extractors/tap-shopify'
2023-09-14T01:34:53.783682Z [debug    ] Creating virtual environment for 'extractors/tap-shopify'
2023-09-14T01:34:54.271374Z [debug    ] Upgrading pip for 'extractors/tap-shopify'
2023-09-14T01:34:54.271534Z [debug    ] Upgrading with args '--upgrade pip' in existing virtual environment for 'extractors/tap-shopify'
2023-09-14T01:34:56.349951Z [debug    ] Installing with args 'git+https://github.com/Matatika/tap-shopify.git' into virtual environment for 'extractors/tap-shopify'
Extractor 'tap-shopify' could not be installed: failed to install plugin 'tap-shopify'.

2023-09-14T01:35:02.310653Z [debug    ] Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/__init__.py:102 │
│ in _run_cli                                                                                      │
│                                                                                                  │
│    99 │   """                                                                                    │
│   100 │   try:                                                                                   │
│   101 │   │   try:  # noqa: WPS225, WPS505                                                       │
│ ❱ 102 │   │   │   cli(obj={"project": None})                                                     │
│   103 │   │   except ProjectReadonly as err:                                                     │
│   104 │   │   │   raise CliError(                                                                │
│   105 │   │   │   │   f"The requested action could not be completed: {err}",                     │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/core.py:1157 in       │
│ __call__                                                                                         │
│                                                                                                  │
│   1154 │                                                                                         │
│   1155 │   def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any:                           │
│   1156 │   │   """Alias for :meth:`main`."""                                                     │
│ ❱ 1157 │   │   return self.main(*args, **kwargs)                                                 │
│   1158                                                                                           │
│   1159                                                                                           │
│   1160 class Command(BaseCommand):                                                               │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/cli.py:42 in    │
│ main                                                                                             │
│                                                                                                  │
│    39 │   │   │   args: Positional arguments for the Click group.                                │
│    40 │   │   │   kwargs: Keyword arguments for the Click group.                                 │
│    41 │   │   """                                                                                │
│ ❱  42 │   │   return super().main(*args, windows_expand_args=False, **kwargs)                    │
│    43                                                                                            │
│    44                                                                                            │
│    45 @click.group(                                                                              │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/core.py:1078 in main  │
│                                                                                                  │
│   1075 │   │   try:                                                                              │
│   1076 │   │   │   try:                                                                          │
│   1077 │   │   │   │   with self.make_context(prog_name, args, **extra) as ctx:                  │
│ ❱ 1078 │   │   │   │   │   rv = self.invoke(ctx)                                                 │
│   1079 │   │   │   │   │   if not standalone_mode:                                               │
│   1080 │   │   │   │   │   │   return rv                                                         │
│   1081 │   │   │   │   │   # it's not safe to `ctx.exit(rv)` here!                               │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/utils.py:612 in │
│ invoke                                                                                           │
│                                                                                                  │
│   609 │   │   enact_environment_behavior(self.environment_behavior, ctx)                         │
│   610 │   │   if ctx.obj.get("tracker"):                                                         │
│   611 │   │   │   ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx))            │
│ ❱ 612 │   │   super().invoke(ctx)                                                                │
│   613                                                                                            │
│   614                                                                                            │
│   615 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, DYMGroup):            │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/core.py:1688 in       │
│ invoke                                                                                           │
│                                                                                                  │
│   1685 │   │   │   │   super().invoke(ctx)                                                       │
│   1686 │   │   │   │   sub_ctx = cmd.make_context(cmd_name, args, parent=ctx)                    │
│   1687 │   │   │   │   with sub_ctx:                                                             │
│ ❱ 1688 │   │   │   │   │   return _process_result(sub_ctx.command.invoke(sub_ctx))               │
│   1689 │   │                                                                                     │
│   1690 │   │   # In chain mode we create the contexts step by step, but after the                │
│   1691 │   │   # base command has been invoked.  Because at that point we do not                 │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/utils.py:661 in │
│ invoke                                                                                           │
│                                                                                                  │
│   658 │   │   if ctx.obj.get("tracker"):                                                         │
│   659 │   │   │   ctx.obj["tracker"].add_contexts(CliContext.from_click_context(ctx))            │
│   660 │   │   │   ctx.obj["tracker"].track_command_event(CliEvent.started)                       │
│ ❱ 661 │   │   super().invoke(ctx)                                                                │
│   662                                                                                            │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/core.py:1434 in       │
│ invoke                                                                                           │
│                                                                                                  │
│   1431 │   │   │   echo(style(message, fg="red"), err=True)                                      │
│   1432 │   │                                                                                     │
│   1433 │   │   if self.callback is not None:                                                     │
│ ❱ 1434 │   │   │   return ctx.invoke(self.callback, **ctx.params)                                │
│   1435 │                                                                                         │
│   1436 │   def shell_complete(self, ctx: Context, incomplete: str) -> t.List["CompletionItem"]:  │
│   1437 │   │   """Return a list of completions for the incomplete value. Looks                   │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/core.py:783 in invoke │
│                                                                                                  │
│    780 │   │                                                                                     │
│    781 │   │   with augment_usage_errors(__self):                                                │
│    782 │   │   │   with ctx:                                                                     │
│ ❱  783 │   │   │   │   return __callback(*args, **kwargs)                                        │
│    784 │                                                                                         │
│    785 │   def forward(                                                                          │
│    786 │   │   __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any  # noqa: B902             │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/params.py:27 in │
│ decorate                                                                                         │
│                                                                                                  │
│   24 │   │   if database_uri:                                                                    │
│   25 │   │   │   ProjectSettingsService.config_override["database_uri"] = database_uri           │
│   26 │   │                                                                                       │
│ ❱ 27 │   │   return func(*args, **kwargs)                                                        │
│   28 │                                                                                           │
│   29 │   return functools.update_wrapper(decorate, func)                                         │
│   30                                                                                             │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/params.py:75 in │
│ decorate                                                                                         │
│                                                                                                  │
│   72 │   │   │   │   except MigrationError as err:                                               │
│   73 │   │   │   │   │   raise CliError(str(err)) from err                                       │
│   74 │   │   │                                                                                   │
│ ❱ 75 │   │   │   func(project, *args, **kwargs)                                                  │
│   76 │   │                                                                                       │
│   77 │   │   return functools.update_wrapper(decorate, func)                                     │
│   78                                                                                             │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/click/decorators.py:33 in   │
│ new_func                                                                                         │
│                                                                                                  │
│    30 │   """                                                                                    │
│    31 │                                                                                          │
│    32 │   def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R":                            │
│ ❱  33 │   │   return f(get_current_context(), *args, **kwargs)                                   │
│    34 │                                                                                          │
│    35 │   return update_wrapper(new_func, f)                                                     │
│    36                                                                                            │
│                                                                                                  │
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/add.py:163 in   │
│ add                                                                                              │
│                                                                                                  │
│   160 │   │                                                                                      │
│   161 │   │   if not success:                                                                    │
│   162 │   │   │   tracker.track_command_event(CliEvent.failed)                                   │
│ ❱ 163 │   │   │   raise CliError("Failed to install plugin(s)")                                  │
│   164 │                                                                                          │
│   165 │   _print_plugins(plugins)                                                                │
│   166 │   tracker.track_command_event(CliEvent.completed)                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CliError: Failed to install plugin(s)

The above exception was the direct cause of the following exception:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/konung/.local/pipx/venvs/meltano/lib/python3.11/site-packages/meltano/cli/__init__.py:112 │
│ in _run_cli                                                                                      │
│                                                                                                  │
│   109 │   │   except MeltanoError as err:                                                        │
│   110 │   │   │   handle_meltano_error(err)                                                      │
│   111 │   │   except Exception as err:                                                           │
│ ❱ 112 │   │   │   raise CliError(f"{troubleshooting_message}\n{err}") from err                   │
│   113 │   except CliError as cli_error:                                                          │
│   114 │   │   cli_error.print()                                                                  │
│   115 │   │   sys.exit(1)                                                                        │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
CliError: Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
join our friendly Slack community.

Failed to install plugin(s)

Schema errors for validation on required properties.

I seem to be running into a ton of errors with schema. Basically complaining about required values when they come in empty from shopify.

I ended up downloadint catalog and marking up a bunch of values that are expected as strings, integers, objects, etc as nullible .ie. "type": [ "object", "null" ], instead of just "type": [ "object"]

          Also lots of mismatches of object vs array type on things like "tax_lines" for example.

I can't imagine everybody is doing it manually. Am I missing something?

Thank you!

Support Python 3.11

This tap is constrained to Python < 3.11. Would be nice to support it.

tap-shopify/pyproject.toml

Line 13 in 8c52367

python = "<3.11,>=3.7.1"

Should the transactions stream be incremental?

It looks like the child transactions stream could probably be incremental based on the order updated_at date since the order will always get updated when a transaction is added and I think transactions are immutable. I dont see it explicitly documented anywhere but this comment suggests it https://community.shopify.com/c/customers-discounts-and-orders/does-creating-a-transaction-update-the-order-s-updated-at/m-p/1321740/highlight/true#M2521 and the API doesnt have a way to update so I think its a safe assumption.

@ReubenFrankel any idea if this is true?

jsonschema.exceptions.ValidationError on schema['properties']['billing_address']['properties']['address2']

Hi,

I'm trying to run this tap with meltano. I have a problem with abandoned_checkouts and customers collections on address2 property.

It looks like the tap expects the field to be a string but in my case it contains a null value.

here part of the error

jsonschema.exceptions.ValidationError: None is not of type 'string'
Failed validating 'type' in schema['properties']['billing_address']['properties']['address2']
{'type': 'string'}
On instance['billing_address']['address2']
None

Do you know if I can do something on my side to avoid this, like adding some transformations, or it is a problem that should be address directly in the tap?

Thanks

Tap needs to be update to Shopify API 2023-10 or allow configure api_version

HI
Tap seems to be hardcoded to 2022-01 which is no longer supported by Shopify

`order` schema: `jsonschema.exceptions.ValidationError: 'admin_graphql_api_id' is a required property`

From this Meltano Slack thread:

 2023-08-01T16:22:27.657124Z [info     ] jsonschema.exceptions.ValidationError: 'admin_graphql_api_id' is a required property cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657174Z [info     ]                                cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657221Z [info     ] Failed validating 'required' in schema: cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657269Z [info     ]     {'properties': {'id': {'type': 'integer'}, cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657316Z [info     ]                     'updated_at': {'format': 'date-time', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657372Z [info     ]                                    'type': ['null', 'string']}}, cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657419Z [info     ]      'required': ['admin_graphql_api_id', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657467Z [info     ]                   'body_html', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657513Z [info     ]                   'created_at', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657559Z [info     ]                   'handle',    cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657605Z [info     ]                   'id',        cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657651Z [info     ]                   'image',     cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657700Z [info     ]                   'images',    cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657745Z [info     ]                   'options',   cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657790Z [info     ]                   'product_type', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657836Z [info     ]                   'published_at', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657883Z [info     ]                   'published_scope', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657928Z [info     ]                   'status',    cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.657975Z [info     ]                   'tags',      cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658021Z [info     ]                   'template_suffix', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658066Z [info     ]                   'title',     cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658216Z [info     ]                   'updated_at', cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658271Z [info     ]                   'variants',  cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658332Z [info     ]                   'vendor'],   cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658380Z [info     ]      'type': 'object'}         cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658427Z [info     ]                                cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658475Z [info     ] On instance:                   cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr
 2023-08-01T16:22:27.658521Z [info     ]     {'id': 6885950652493, 'updated_at': '2023-08-01T08:59:18-07:00'} cmd_type=loader name=target-parquet run_id=66f5f6a0-2e97-4673-94e5-53e67bd6b233 state_id=2023-08-01T162225--tap-shopify--target-parquet stdio=stderr

Missing schema properties

`abandoned_checkouts`

2023-10-04T16:34:16.217414Z [info     ] 2023-10-04 11:34:16,217 | WARNING  | tap-shopify          | Properties ('note_attributes', 'source_identifier', 'source_url', 'customer.last_order_id', 'customer.verified_email', 'customer.multipass_identifier', 'customer.tax_exempt', 'customer.last_order_name', 'customer.currency', 'customer.phone', 'customer.accepts_marketing_updated_at', 'customer.marketing_opt_in_level', 'customer.tax_exemptions', 'customer.email_marketing_consent', 'customer.sms_marketing_consent', 'customer.admin_graphql_api_id', 'customer.default_address', 'source', 'name', 'billing_address.latitude', 'billing_address.longitude') were present in the 'abandoned_checkouts' stream but not found in catalog schema. Ignoring. cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify

2023-10-04T16:34:16.219713Z [info     ] 2023-10-04 11:34:16,219 | WARNING  | tap-shopify          | Properties ('note_attributes', 'source_identifier', 'source_url', 'customer.last_order_id', 'customer.verified_email', 'customer.multipass_identifier', 'customer.tax_exempt', 'customer.last_order_name', 'customer.currency', 'customer.phone', 'customer.accepts_marketing_updated_at', 'customer.marketing_opt_in_level', 'customer.tax_exemptions', 'customer.email_marketing_consent', 'customer.sms_marketing_consent', 'customer.admin_graphql_api_id', 'customer.default_address', 'source', 'name') were present in the 'abandoned_checkouts' stream but not found in catalog schema. Ignoring. cmd_type=elb consumer=False name=tap-shopify producer=True stdio=stderr string_id=tap-shopify

Originally posted by @konung in #14 (comment)

matatika / tap-shopify Goto Github PK

tap-shopify's Introduction

Singer Tap for Shopify

Configuration

Supported Streams

Default Streams

Shopify Plus Streams

Roadmap

Installation

Configuration

Accepted Config Options

Source Authentication and Authorization

Usage

Executing the Tap Directly

Developer Resources

Initialize your Development Environment

Create and Run Tests

Testing with Meltano

Cloud hosting and SaaS

License

tap-shopify's People

Contributors

Stargazers

Watchers

Forkers

tap-shopify's Issues

abandoned_checkouts

Recommend Projects

Recommend Topics

Recommend Org

`abandoned_checkouts`