singer-io / tap-ujet Goto Github PK

View Code? Open in Web Editor NEW

0.0 5.0 4.0 55 KB

License: GNU Affero General Public License v3.0

Python 98.09% Makefile 1.91%

tap-ujet's Introduction

tap-ujet

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

Pulls raw data from the UJET API
Extracts the following resources:
- Agents
- Agent Activity Logs
- Calls
- Chats
- Menus
- Menu Tree
- Teams
- Team Tree
- User Statuses
Outputs the schema for each resource
Incrementally pulls data based on the input state

Streams

agents

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/agents
Primary key fields: id
Foreign key fields: teams > id
Replication strategy: INCREMENTAL (query filtered)
- Bookmark: status_updated_at
- Bookmark query field: status_updated_at[from]
Transformations: none

agent_activity_logs

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/agent_activity_logs
Primary key fields: id
Foreign key fields: teams > id
Replication strategy: INCREMENTAL (query filtered)
- Bookmark: started_at
- Bookmark query field: started_at[from]
Transformations: none

calls

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/calls
Primary key fields: id
Foreign key fields: teams > id
Replication strategy: INCREMENTAL (query filtered)
- Bookmark: updated_at
- Bookmark query field: updated_at[from]
Transformations: none

chats

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/chats
Primary key fields: id
Foreign key fields: teams > id
Replication strategy: INCREMENTAL (query filtered)
- Bookmark: updated_at
- Bookmark query field: updated_at[from]
Transformations: none

menu_tree

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/menu/tree
Primary key fields: id
Foreign key fields: none
Replication strategy: FULL_TABLE
Transformations: de-nest all recursive 'children'

menus

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/menus
Primary key fields: id
Foreign key fields: none
Replication strategy: FULL_TABLE
Transformations: none

team_tree

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/team/tree
Primary key fields: id
Foreign key fields: none
Replication strategy: FULL_TABLE
Transformations: de-nest all recursive 'children'

teams

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/teams
Primary key fields: id
Foreign key fields: none
Replication strategy: FULL_TABLE
Transformations: none

user_statuses

Endpoint: https://{subdomain}.ujet.co/manager/api/v1/user_statuses
Primary key fields: id
Foreign key fields: none
Replication strategy: FULL_TABLE
Transformations: none

Quick Start

Install

Clone this repository, and then install using setup.py. We recommend using a virtualenv:

> virtualenv -p python3 venv
> source venv/bin/activate
> python setup.py install
OR
> cd .../tap-ujet
> pip install .

Dependent libraries The following dependent libraries were installed.

> pip install singer-python
> pip install singer-tools
> pip install target-stitch
> pip install target-json

Create your tap's config.json file. The server_subdomain is everything before .ujet.com. in the ujet URL. The account_name is everything between .ujet.com. and api in the ujet URL..

{
    "company_key": "YOUR_COMPANY_KEY",
    "company_secret": "YOUR_COMPANY_SECRET_CODE",
    "subdomain": "YOUR_COMPANY",
    "domain": "ujet",
    "start_date": "2019-01-01T00:00:00Z",
    "user_agent": "tap-ujet <api_user_email@your_company.com>"
}

Optionally, also create a state.json file. currently_syncing is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off.

{
    "currently_syncing": "calls",
    "bookmarks": {
        "agnets": "2019-09-27T22:34:39.000000Z",
        "agent_activity_logs": "2019-09-28T15:30:26.000000Z",
        "calls": "2019-09-28T18:23:53Z",
        "chats": "2019-09-27T22:40:30.000000Z"
    }
}

Run the Tap in Discovery Mode This creates a catalog.json for selecting objects/fields to integrate:
```
tap-ujet --config config.json --discover > catalog.json
```
See the Singer docs on discovery mode here.

Run the Tap in Sync Mode (with catalog) and write out to state file

For Sync mode:

> tap-ujet --config tap_config.json --catalog catalog.json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

To load to json files to verify outputs:

> tap-ujet --config tap_config.json --catalog catalog.json | target-json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

To pseudo-load to Stitch Import API with dry run:

> tap-ujet --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

Test the Tap

While code/bytecode/Stitchoping the ujet tap, the following utilities were run in accordance with Singer.io best practices: Pylint to improve code quality:

> pylint tap_ujet -d missing-docstring -d logging-format-interpolation -d too-many-locals -d too-many-arguments

Pylint test resulted in the following score:

TBD LATER
Your code has been rated at TBD/10

To check the tap and verify working:

> tap-ujet --config tap_config.json --catalog catalog.json | singer-check-tap > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

Check tap resulted in the following:

  Checking stdin for valid Singer-formatted data
  The output is valid.
  It contained 3704 messages for 9 streams.

        9 schema messages
     3640 record messages
       55 state messages

  Details by stream:
  +---------------------+---------+---------+
  | stream              | records | schemas |
  +---------------------+---------+---------+
  | agent_activity_logs | 3148    | 1       |
  | agents              | 28      | 1       |
  | menus               | 50      | 1       |
  | teams               | 5       | 1       |
  | team_tree           | 5       | 1       |
  | menu_tree           | 34      | 1       |
  | chats               | 0       | 1       |
  | calls               | 356     | 1       |
  | user_statuses       | 14      | 1       |
  +---------------------+---------+---------+

tap-ujet's People

Contributors

Watchers

Forkers

dscoleman bonomali jeffhuth-bytecode isabella232

tap-ujet's Issues

Errors during transform

When running this integration via Stitch, the following error started appearing in the logs:

2020-07-03 13:13:54,897Z   main - INFO Running tap-ujet version 0.0.3 and target-stitch version 3.0.3
2020-07-03 13:13:55,056Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2020-07-03 13:13:55,058Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2020-07-03 13:13:56,059Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [26] at offset None
2020-07-03 13:13:56,063Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-ujet --config /tmp/tap_discover_config.json --discover
2020-07-03 13:13:56,901Z    tap - INFO Starting discover
2020-07-03 13:13:56,902Z    tap - INFO discover schema for stream: menu_tree
2020-07-03 13:13:56,902Z    tap - INFO discover schema for stream: menus
2020-07-03 13:13:56,902Z    tap - INFO discover schema for stream: teams
2020-07-03 13:13:56,902Z    tap - INFO discover schema for stream: team_tree
2020-07-03 13:13:56,902Z    tap - INFO discover schema for stream: agent_activity_logs
2020-07-03 13:13:56,903Z    tap - INFO discover schema for stream: calls
2020-07-03 13:13:56,903Z    tap - INFO discover schema for stream: agents
2020-07-03 13:13:56,903Z    tap - INFO discover schema for stream: chats
2020-07-03 13:13:56,903Z    tap - INFO discover schema for stream: user_statuses
2020-07-03 13:13:56,903Z    tap - INFO Returning catalog: {'streams': [<singer.catalog.CatalogEntry object at 0x7f09579699e8>, <singer.catalog.CatalogEntry object at 0x7f0957969f28>, <singer.catalog.CatalogEntry object at 0x7f095796e160>, <singer.catalog.CatalogEntry object at 0x7f095796e320>, <singer.catalog.CatalogEntry object at 0x7f095796e7f0>, <singer.catalog.CatalogEntry object at 0x7f0957970668>, <singer.catalog.CatalogEntry object at 0x7f0957970c18>, <singer.catalog.CatalogEntry object at 0x7f09579737b8>, <singer.catalog.CatalogEntry object at 0x7f0957973908>]}
2020-07-03 13:13:56,914Z    tap - INFO Finished discover
2020-07-03 13:13:56,972Z   main - INFO Tap exited normally.
2020-07-03 13:13:56,974Z   main - INFO Saving list of discovered streams
2020-07-03 13:13:57,039Z   main - INFO Saving structure of stream menu_tree (tap_stream_id: menu_tree)
2020-07-03 13:13:57,060Z   main - INFO Saving structure of stream menus (tap_stream_id: menus)
2020-07-03 13:13:57,127Z   main - INFO Saving structure of stream teams (tap_stream_id: teams)
2020-07-03 13:13:57,153Z   main - INFO Saving structure of stream team_tree (tap_stream_id: team_tree)
2020-07-03 13:13:57,169Z   main - INFO Saving structure of stream agent_activity_logs (tap_stream_id: agent_activity_logs)
2020-07-03 13:13:57,236Z   main - INFO Saving structure of stream calls (tap_stream_id: calls)
2020-07-03 13:13:57,258Z   main - INFO Saving structure of stream agents (tap_stream_id: agents)
2020-07-03 13:13:57,279Z   main - INFO Saving structure of stream chats (tap_stream_id: chats)
2020-07-03 13:13:57,304Z   main - INFO Saving structure of stream user_statuses (tap_stream_id: user_statuses)
2020-07-03 13:13:57,322Z   main - INFO Writing catalog to file
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream menu_tree: name, position, parent_id, agents_count, deleted, id
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream teams: name, position, assignees, parent_id, agents_count, deleted, id
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream user_statuses: name, color, wfm_id, id
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream menus: name, position, agent_assignments, menu_type, parent_id, hidden, deleted, team_assignments, output_msg, id
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream calls: out_ticket_url, voip_provider, fail_reason, agent_info, voicemail_reason, lang, deflection, deflection_details, scheduled_at, participants, verified, support_number, end_user, offer_events, wait_duration, hold_duration, ends_at, offer_type, updated_at, outbound_number, selected_menu, menu_path, call_duration, connected_at, fail_details, rating, assigned_at, transfers, queued_at, recording_url, call_type, created_at, disconnected_by, out_ticket_id, photos, videos, answer_type, status, id, has_feedback
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream chats: out_ticket_url, fail_reason, assigned_at, lang, chat_duration, end_user, ends_at, longest_response_time, participants, verified, wait_duration, first_msg_sent_at, average_response_time, offer_events, message_count, offer_type, updated_at, selected_menu, last_msg_sent_at, chat_type, menu_path, transcript, rating, agent_info, transfers, created_at, out_ticket_id, photos, videos, status, id, has_feedback
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream team_tree: name, position, parent_id, agents_count, deleted, id
2020-07-03 13:13:57,355Z   main - INFO Selected fields for stream agents: location, last_login_time, agent_number, middle_name, last_name, channels, online, roles, email, call_count, id, crm_authenticated, teams, chat_count, created_at, wrap_up, first_name, status, avatar_url, status_updated_at
2020-07-03 13:13:57,443Z   main - INFO Current state: {"bookmarks": {"calls": "2020-07-02T18:14:00.000000Z", "agents": "2020-07-02T19:13:31.000000Z", "agent_activity_logs": "2020-04-02T06:57:42.000000Z"}, "currently_syncing": "calls"}
2020-07-03 13:13:57,444Z   main - INFO Starting tap: tap-env/bin/tap-ujet --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2020-07-03 13:13:57,453Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2020-07-03 13:13:58,217Z target - INFO Using batch_size_prefernces of {'full_table_streams': [], 'user_batch_size_preference': None, 'batch_size_preference': None}
2020-07-03 13:13:58,316Z    tap - INFO last/currently syncing stream: calls
2020-07-03 13:13:58,317Z    tap - INFO Skipping stream: agent_activity_logs
2020-07-03 13:13:58,317Z    tap - INFO selected_streams: ['calls', 'agents', 'chats', 'user_statuses', 'menu_tree', 'menus', 'teams', 'team_tree']
2020-07-03 13:13:58,317Z    tap - INFO START Syncing: calls
2020-07-03 13:13:58,317Z    tap - INFO Stream: calls, selected_fields: ['out_ticket_id', 'scheduled_at', 'call_type', 'agent_info', 'call_duration', 'fail_details', 'assigned_at', 'connected_at', 'outbound_number', 'menu_path', 'created_at', 'has_feedback', 'ends_at', 'recording_url', 'photos', 'lang', 'offer_events', 'wait_duration', 'updated_at', 'transfers', 'fail_reason', 'participants', 'selected_menu', 'videos', 'rating', 'voip_provider', 'status', 'disconnected_by', 'hold_duration', 'id', 'support_number', 'answer_type', 'queued_at', 'deflection_details', 'voicemail_reason', 'end_user', 'offer_type', 'out_ticket_url', 'deflection', 'verified']
2020-07-03 13:13:58,318Z    tap - INFO URL for Stream calls: https://gopuff.ujet.co/manager/api/v1/calls?page=1&updated_at[from]=2020-07-02T18:14:00.000000Z&per=100&sort_column=updated_at&sort_direction=ASC
2020-07-03 13:13:58,322Z   main - INFO State update: adding bookmarks.agent_activity_logs = "2020-04-02T06:57:42.000000Z"
2020-07-03 13:13:58,322Z   main - INFO State update: adding bookmarks.agents = "2020-07-02T19:13:31.000000Z"
2020-07-03 13:13:58,323Z   main - INFO State update: adding bookmarks.calls = "2020-07-02T18:14:00.000000Z"
2020-07-03 13:13:58,323Z   main - INFO State update: adding currently_syncing = "calls"
2020-07-03 13:13:59,880Z    tap - INFO HTTP request to "calls" endpoint took 1.560s, returned status code 200
2020-07-03 13:13:59,929Z    tap - INFO replicated 11 records from "calls" endpoint
2020-07-03 13:13:59,930Z    tap - CRITICAL Errors during transform
2020-07-03 13:13:59,931Z    tap - 	transfers.0.from_queue_priority_level: 0 does not match {'type': 'null'}
2020-07-03 13:13:59,931Z    tap - 	transfers.0: {'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16} does not match {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}
2020-07-03 13:13:59,931Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}
2020-07-03 13:13:59,931Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'type': 'null'}
2020-07-03 13:13:59,931Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}
2020-07-03 13:13:59,931Z    tap - 	: {'support_number': '+18554007833', 'participants': [{'fail_reason': 'nothing', 'user_id': None, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 609, 'connected_at': '2020-07-02T18:15:26.000Z', 'end_user_id': 173141, 'status': 'finished', 'type': 'end_user', 'hold_duration': 209, 'id': 641328}, {'fail_reason': 'nothing', 'user_id': 119, 'ended_at': '2020-07-02T18:19:53.000Z', 'call_id': 382550, 'call_duration': 210, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641333}, {'fail_reason': 'nothing', 'user_id': 62, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641339}], 'selected_menu': {'parent_id': 46, 'hidden': False, 'deleted': False, 'name': 'Status of a Refund N', 'position': 0, 'output_msg': 'You selected Status of a Refund N.', 'menu_type': 'ivr_menu', 'id': 47}, 'photos': [], 'agent_info': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user': {'out_contact_id': '5b4f0086a08d50677ed5d425', 'identifier': None, 'id': 173141}, 'offer_events': [{'casting_time': '2020-07-02T18:16:09.000Z', 'group': 'Group 1'}], 'queue_priority_level': 0, 'offer_type': 'cascade', 'ends_at': '2020-07-02T18:25:35.000Z', 'id': 382550, 'assigned_at': '2020-07-02T18:16:16.000Z', 'fail_reason': 'nothing', 'disconnected_by': 'disconnected_by_end_user', 'fail_details': None, 'voicemail_reason': 'not_voicemail', 'lang': 'en', 'verified': False, 'transfers': [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}], 'has_feedback': False, 'hold_duration': 209, 'recording_url': 'https://gopuff.api.kustomerapp.com/v1/messages/5efe247210206b001aa132ce/attachments/5efe26a9fcb8ab001923a33b?redirect=true', 'deflection_details': [], 'call_duration': 552, 'videos': [], 'outbound_number': None, 'voip_provider': 'voip_provider_twilio', 'scheduled_at': None, 'created_at': '2020-07-02T18:15:26.000Z', 'updated_at': '2020-07-02T18:25:35.000Z', 'deflection': 'no_deflection', 'queued_at': '2020-07-02T18:16:09.000Z', 'answer_type': 'manual', 'rating': None, 'out_ticket_url': 'https://gopuff.kustomerapp.com/app/customers/5b4f0086a08d50677ed5d425/event/5efe247237f41fe2e7727124', 'menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'status': 'finished', 'out_ticket_id': '5efe247237f41fe2e7727124', 'call_type': 'IvrCall', 'wait_duration': 14} does not match {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'support_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'offer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'participants': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'user_id': {'type': ['integer', 'null']}, 'ended_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'end_user_id': {'type': ['integer', 'null']}, 'call_id': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'type': {'type': ['string', 'null']}, 'hold_duration': {'type': ['integer', 'null']}, 'status': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'photos': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'photo_type': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'connected_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'videos': {'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'selected': True, 'type': 'array', 'inclusion': 'available'}, 'end_user': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'out_contact_id': {'type': ['string', 'null']}, 'identifier': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'offer_events': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'casting_time': {'format': 'date-time', 'type': ['string', 'null']}, 'group': {'type': ['string', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'disconnected_by': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'ends_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voicemail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'assigned_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_details': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'selected_menu': {'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'parent_id': {'type': ['integer', 'null']}, 'deleted': {'type': ['boolean', 'null']}, 'hidden': {'type': ['boolean', 'null']}, 'name': {'type': ['string', 'null']}, 'position': {'type': ['integer', 'null']}, 'output_msg': {'type': ['string', 'null']}, 'menu_type': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'id': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'automatic'}, 'lang': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'verified': {'selected': True, 'type': 'boolean', 'inclusion': 'available'}, 'queued_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'has_feedback': {'selected': True, 'type': ['boolean', 'null'], 'inclusion': 'available'}, 'hold_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'recording_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection_details': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'deflection': {'type': ['null', 'string']}, 'call_id': {'type': ['null', 'integer']}, 'id': {'type': ['null', 'integer']}, 'transfer_id': {'type': ['null', 'integer']}, 'from_menu_path': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'items_count': {'type': ['null', 'integer']}, 'materialized_path': {'type': ['null', 'string']}, 'name': {'type': ['null', 'string']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'call_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'transfers': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'outbound_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'scheduled_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'created_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'updated_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voip_provider': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'agent_info': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'answer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'rating': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'menu_path': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}, 'status': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_id': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'call_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'wait_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}}}
2020-07-03 13:13:59,931Z    tap - 
2020-07-03 13:13:59,931Z    tap - 
2020-07-03 13:13:59,931Z    tap - Errors during transform: [transfers.0.from_queue_priority_level: 0 does not match {'type': 'null'}, transfers.0: {'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16} does not match {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'type': 'null'}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, : {'support_number': '+18554007833', 'participants': [{'fail_reason': 'nothing', 'user_id': None, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 609, 'connected_at': '2020-07-02T18:15:26.000Z', 'end_user_id': 173141, 'status': 'finished', 'type': 'end_user', 'hold_duration': 209, 'id': 641328}, {'fail_reason': 'nothing', 'user_id': 119, 'ended_at': '2020-07-02T18:19:53.000Z', 'call_id': 382550, 'call_duration': 210, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641333}, {'fail_reason': 'nothing', 'user_id': 62, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641339}], 'selected_menu': {'parent_id': 46, 'hidden': False, 'deleted': False, 'name': 'Status of a Refund N', 'position': 0, 'output_msg': 'You selected Status of a Refund N.', 'menu_type': 'ivr_menu', 'id': 47}, 'photos': [], 'agent_info': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user': {'out_contact_id': '5b4f0086a08d50677ed5d425', 'identifier': None, 'id': 173141}, 'offer_events': [{'casting_time': '2020-07-02T18:16:09.000Z', 'group': 'Group 1'}], 'queue_priority_level': 0, 'offer_type': 'cascade', 'ends_at': '2020-07-02T18:25:35.000Z', 'id': 382550, 'assigned_at': '2020-07-02T18:16:16.000Z', 'fail_reason': 'nothing', 'disconnected_by': 'disconnected_by_end_user', 'fail_details': None, 'voicemail_reason': 'not_voicemail', 'lang': 'en', 'verified': False, 'transfers': [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}], 'has_feedback': False, 'hold_duration': 209, 'recording_url': 'https://gopuff.api.kustomerapp.com/v1/messages/5efe247210206b001aa132ce/attachments/5efe26a9fcb8ab001923a33b?redirect=true', 'deflection_details': [], 'call_duration': 552, 'videos': [], 'outbound_number': None, 'voip_provider': 'voip_provider_twilio', 'scheduled_at': None, 'created_at': '2020-07-02T18:15:26.000Z', 'updated_at': '2020-07-02T18:25:35.000Z', 'deflection': 'no_deflection', 'queued_at': '2020-07-02T18:16:09.000Z', 'answer_type': 'manual', 'rating': None, 'out_ticket_url': 'https://gopuff.kustomerapp.com/app/customers/5b4f0086a08d50677ed5d425/event/5efe247237f41fe2e7727124', 'menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'status': 'finished', 'out_ticket_id': '5efe247237f41fe2e7727124', 'call_type': 'IvrCall', 'wait_duration': 14} does not match {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'support_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'offer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'participants': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'user_id': {'type': ['integer', 'null']}, 'ended_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'end_user_id': {'type': ['integer', 'null']}, 'call_id': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'type': {'type': ['string', 'null']}, 'hold_duration': {'type': ['integer', 'null']}, 'status': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'photos': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'photo_type': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'connected_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'videos': {'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'selected': True, 'type': 'array', 'inclusion': 'available'}, 'end_user': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'out_contact_id': {'type': ['string', 'null']}, 'identifier': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'offer_events': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'casting_time': {'format': 'date-time', 'type': ['string', 'null']}, 'group': {'type': ['string', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'disconnected_by': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'ends_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voicemail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'assigned_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_details': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'selected_menu': {'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'parent_id': {'type': ['integer', 'null']}, 'deleted': {'type': ['boolean', 'null']}, 'hidden': {'type': ['boolean', 'null']}, 'name': {'type': ['string', 'null']}, 'position': {'type': ['integer', 'null']}, 'output_msg': {'type': ['string', 'null']}, 'menu_type': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'id': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'automatic'}, 'lang': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'verified': {'selected': True, 'type': 'boolean', 'inclusion': 'available'}, 'queued_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'has_feedback': {'selected': True, 'type': ['boolean', 'null'], 'inclusion': 'available'}, 'hold_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'recording_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection_details': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'deflection': {'type': ['null', 'string']}, 'call_id': {'type': ['null', 'integer']}, 'id': {'type': ['null', 'integer']}, 'transfer_id': {'type': ['null', 'integer']}, 'from_menu_path': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'items_count': {'type': ['null', 'integer']}, 'materialized_path': {'type': ['null', 'string']}, 'name': {'type': ['null', 'string']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'call_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'transfers': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'outbound_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'scheduled_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'created_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'updated_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voip_provider': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'agent_info': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'answer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'rating': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'menu_path': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}, 'status': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_id': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'call_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'wait_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}}}]
2020-07-03 13:13:59,931Z    tap - Traceback (most recent call last):
2020-07-03 13:13:59,931Z    tap -   File "tap-env/bin/tap-ujet", line 33, in <module>
2020-07-03 13:13:59,931Z    tap -     sys.exit(load_entry_point('tap-ujet==0.0.3', 'console_scripts', 'tap-ujet')())
2020-07-03 13:13:59,931Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2020-07-03 13:13:59,931Z    tap -     return fnc(*args, **kwargs)
2020-07-03 13:13:59,931Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ujet/__init__.py", line 53, in main
2020-07-03 13:13:59,931Z    tap -     state=state)
2020-07-03 13:13:59,932Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ujet/sync.py", line 298, in sync
2020-07-03 13:13:59,932Z    tap -     bookmark_type=endpoint_config.get('bookmark_type', None))
2020-07-03 13:13:59,932Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ujet/sync.py", line 201, in sync_endpoint
2020-07-03 13:13:59,932Z    tap -     parent_id=parent_id)
2020-07-03 13:13:59,932Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_ujet/sync.py", line 81, in process_records
2020-07-03 13:13:59,932Z    tap -     stream_metadata)
2020-07-03 13:13:59,932Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/transform.py", line 131, in transform
2020-07-03 13:13:59,932Z    tap -     raise SchemaMismatch(self.errors)
2020-07-03 13:13:59,932Z    tap - singer.transform.SchemaMismatch: Errors during transform
2020-07-03 13:13:59,932Z    tap - 	transfers.0.from_queue_priority_level: 0 does not match {'type': 'null'}
2020-07-03 13:13:59,932Z    tap - 	transfers.0: {'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16} does not match {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}
2020-07-03 13:13:59,932Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}
2020-07-03 13:13:59,932Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'type': 'null'}
2020-07-03 13:13:59,932Z    tap - 	transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}
2020-07-03 13:13:59,932Z    tap - 	: {'support_number': '+18554007833', 'participants': [{'fail_reason': 'nothing', 'user_id': None, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 609, 'connected_at': '2020-07-02T18:15:26.000Z', 'end_user_id': 173141, 'status': 'finished', 'type': 'end_user', 'hold_duration': 209, 'id': 641328}, {'fail_reason': 'nothing', 'user_id': 119, 'ended_at': '2020-07-02T18:19:53.000Z', 'call_id': 382550, 'call_duration': 210, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641333}, {'fail_reason': 'nothing', 'user_id': 62, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641339}], 'selected_menu': {'parent_id': 46, 'hidden': False, 'deleted': False, 'name': 'Status of a Refund N', 'position': 0, 'output_msg': 'You selected Status of a Refund N.', 'menu_type': 'ivr_menu', 'id': 47}, 'photos': [], 'agent_info': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user': {'out_contact_id': '5b4f0086a08d50677ed5d425', 'identifier': None, 'id': 173141}, 'offer_events': [{'casting_time': '2020-07-02T18:16:09.000Z', 'group': 'Group 1'}], 'queue_priority_level': 0, 'offer_type': 'cascade', 'ends_at': '2020-07-02T18:25:35.000Z', 'id': 382550, 'assigned_at': '2020-07-02T18:16:16.000Z', 'fail_reason': 'nothing', 'disconnected_by': 'disconnected_by_end_user', 'fail_details': None, 'voicemail_reason': 'not_voicemail', 'lang': 'en', 'verified': False, 'transfers': [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}], 'has_feedback': False, 'hold_duration': 209, 'recording_url': 'https://gopuff.api.kustomerapp.com/v1/messages/5efe247210206b001aa132ce/attachments/5efe26a9fcb8ab001923a33b?redirect=true', 'deflection_details': [], 'call_duration': 552, 'videos': [], 'outbound_number': None, 'voip_provider': 'voip_provider_twilio', 'scheduled_at': None, 'created_at': '2020-07-02T18:15:26.000Z', 'updated_at': '2020-07-02T18:25:35.000Z', 'deflection': 'no_deflection', 'queued_at': '2020-07-02T18:16:09.000Z', 'answer_type': 'manual', 'rating': None, 'out_ticket_url': 'https://gopuff.kustomerapp.com/app/customers/5b4f0086a08d50677ed5d425/event/5efe247237f41fe2e7727124', 'menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'status': 'finished', 'out_ticket_id': '5efe247237f41fe2e7727124', 'call_type': 'IvrCall', 'wait_duration': 14} does not match {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'support_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'offer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'participants': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'user_id': {'type': ['integer', 'null']}, 'ended_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'end_user_id': {'type': ['integer', 'null']}, 'call_id': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'type': {'type': ['string', 'null']}, 'hold_duration': {'type': ['integer', 'null']}, 'status': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'photos': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'photo_type': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'connected_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'videos': {'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'selected': True, 'type': 'array', 'inclusion': 'available'}, 'end_user': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'out_contact_id': {'type': ['string', 'null']}, 'identifier': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'offer_events': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'casting_time': {'format': 'date-time', 'type': ['string', 'null']}, 'group': {'type': ['string', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'disconnected_by': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'ends_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voicemail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'assigned_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_details': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'selected_menu': {'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'parent_id': {'type': ['integer', 'null']}, 'deleted': {'type': ['boolean', 'null']}, 'hidden': {'type': ['boolean', 'null']}, 'name': {'type': ['string', 'null']}, 'position': {'type': ['integer', 'null']}, 'output_msg': {'type': ['string', 'null']}, 'menu_type': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'id': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'automatic'}, 'lang': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'verified': {'selected': True, 'type': 'boolean', 'inclusion': 'available'}, 'queued_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'has_feedback': {'selected': True, 'type': ['boolean', 'null'], 'inclusion': 'available'}, 'hold_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'recording_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection_details': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'deflection': {'type': ['null', 'string']}, 'call_id': {'type': ['null', 'integer']}, 'id': {'type': ['null', 'integer']}, 'transfer_id': {'type': ['null', 'integer']}, 'from_menu_path': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'items_count': {'type': ['null', 'integer']}, 'materialized_path': {'type': ['null', 'string']}, 'name': {'type': ['null', 'string']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'call_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'transfers': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'outbound_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'scheduled_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'created_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'updated_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voip_provider': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'agent_info': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'answer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'rating': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'menu_path': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}, 'status': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_id': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'call_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'wait_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}}}
2020-07-03 13:13:59,932Z    tap - 
2020-07-03 13:13:59,933Z    tap - 
2020-07-03 13:13:59,933Z    tap - Errors during transform: [transfers.0.from_queue_priority_level: 0 does not match {'type': 'null'}, transfers.0: {'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16} does not match {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'type': 'null'}, transfers: [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}] does not match {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, : {'support_number': '+18554007833', 'participants': [{'fail_reason': 'nothing', 'user_id': None, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 609, 'connected_at': '2020-07-02T18:15:26.000Z', 'end_user_id': 173141, 'status': 'finished', 'type': 'end_user', 'hold_duration': 209, 'id': 641328}, {'fail_reason': 'nothing', 'user_id': 119, 'ended_at': '2020-07-02T18:19:53.000Z', 'call_id': 382550, 'call_duration': 210, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641333}, {'fail_reason': 'nothing', 'user_id': 62, 'ended_at': '2020-07-02T18:25:35.000Z', 'call_id': 382550, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'end_user_id': None, 'status': 'finished', 'type': 'agent', 'hold_duration': None, 'id': 641339}], 'selected_menu': {'parent_id': 46, 'hidden': False, 'deleted': False, 'name': 'Status of a Refund N', 'position': 0, 'output_msg': 'You selected Status of a Refund N.', 'menu_type': 'ivr_menu', 'id': 47}, 'photos': [], 'agent_info': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'connected_at': '2020-07-02T18:16:23.000Z', 'end_user': {'out_contact_id': '5b4f0086a08d50677ed5d425', 'identifier': None, 'id': 173141}, 'offer_events': [{'casting_time': '2020-07-02T18:16:09.000Z', 'group': 'Group 1'}], 'queue_priority_level': 0, 'offer_type': 'cascade', 'ends_at': '2020-07-02T18:25:35.000Z', 'id': 382550, 'assigned_at': '2020-07-02T18:16:16.000Z', 'fail_reason': 'nothing', 'disconnected_by': 'disconnected_by_end_user', 'fail_details': None, 'voicemail_reason': 'not_voicemail', 'lang': 'en', 'verified': False, 'transfers': [{'from_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Kara', 'last_name': 'Wright', 'name': 'Kara Wright', 'agent_number': None, 'id': 119}, 'to_queue_priority_level': 0, 'call_duration': 345, 'connected_at': '2020-07-02T18:19:50.000Z', 'deflection': 'no_deflection', 'to_menu_path': None, 'created_at': '2020-07-02T18:19:28.000Z', 'id': 1187, 'updated_at': '2020-07-02T18:25:35.000Z', 'from_queue_priority_level': 0, 'assigned_at': '2020-07-02T18:19:44.000Z', 'fail_reason': 'nothing', 'to_agent': {'avatar_url': 'https://ujet.s3.amazonaws.com/default-profile.png', 'first_name': 'Michael', 'last_name': 'Rono', 'name': 'Michael Rono', 'agent_number': None, 'id': 62}, 'status': 'transferred', 'from_menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'answer_type_path': 'manual_to_manual', 'wait_duration': 16}], 'has_feedback': False, 'hold_duration': 209, 'recording_url': 'https://gopuff.api.kustomerapp.com/v1/messages/5efe247210206b001aa132ce/attachments/5efe26a9fcb8ab001923a33b?redirect=true', 'deflection_details': [], 'call_duration': 552, 'videos': [], 'outbound_number': None, 'voip_provider': 'voip_provider_twilio', 'scheduled_at': None, 'created_at': '2020-07-02T18:15:26.000Z', 'updated_at': '2020-07-02T18:25:35.000Z', 'deflection': 'no_deflection', 'queued_at': '2020-07-02T18:16:09.000Z', 'answer_type': 'manual', 'rating': None, 'out_ticket_url': 'https://gopuff.kustomerapp.com/app/customers/5b4f0086a08d50677ed5d425/event/5efe247237f41fe2e7727124', 'menu_path': {'items_count': 2, 'materialized_path': '46/47', 'name': 'Question About Charges N/Status of a Refund N'}, 'status': 'finished', 'out_ticket_id': '5efe247237f41fe2e7727124', 'call_type': 'IvrCall', 'wait_duration': 14} does not match {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'support_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'offer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'participants': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'user_id': {'type': ['integer', 'null']}, 'ended_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'end_user_id': {'type': ['integer', 'null']}, 'call_id': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'type': {'type': ['string', 'null']}, 'hold_duration': {'type': ['integer', 'null']}, 'status': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'photos': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'photo_type': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'connected_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'videos': {'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'url': {'type': ['null', 'string']}, 'id': {'type': ['null', 'integer']}}}, 'selected': True, 'type': 'array', 'inclusion': 'available'}, 'end_user': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'out_contact_id': {'type': ['string', 'null']}, 'identifier': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'offer_events': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'casting_time': {'format': 'date-time', 'type': ['string', 'null']}, 'group': {'type': ['string', 'null']}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'disconnected_by': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'ends_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voicemail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'assigned_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_reason': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'fail_details': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'selected_menu': {'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'parent_id': {'type': ['integer', 'null']}, 'deleted': {'type': ['boolean', 'null']}, 'hidden': {'type': ['boolean', 'null']}, 'name': {'type': ['string', 'null']}, 'position': {'type': ['integer', 'null']}, 'output_msg': {'type': ['string', 'null']}, 'menu_type': {'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'id': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'automatic'}, 'lang': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'verified': {'selected': True, 'type': 'boolean', 'inclusion': 'available'}, 'queued_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'has_feedback': {'selected': True, 'type': ['boolean', 'null'], 'inclusion': 'available'}, 'hold_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'recording_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection_details': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'deflection': {'type': ['null', 'string']}, 'call_id': {'type': ['null', 'integer']}, 'id': {'type': ['null', 'integer']}, 'transfer_id': {'type': ['null', 'integer']}, 'from_menu_path': {'additionalProperties': False, 'type': ['null', 'object'], 'properties': {'items_count': {'type': ['null', 'integer']}, 'materialized_path': {'type': ['null', 'string']}, 'name': {'type': ['null', 'string']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'call_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}, 'transfers': {'selected': True, 'anyOf': [{'items': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'from_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'to_queue_priority_level': {'type': ['string', 'null']}, 'status': {'type': ['string', 'null']}, 'call_duration': {'type': ['integer', 'null']}, 'connected_at': {'format': 'date-time', 'type': ['string', 'null']}, 'deflection': {'type': ['string', 'null']}, 'to_menu_path': {'type': ['string', 'null']}, 'created_at': {'format': 'date-time', 'type': ['string', 'null']}, 'id': {'type': ['integer', 'null']}, 'updated_at': {'format': 'date-time', 'type': ['string', 'null']}, 'from_queue_priority_level': {'type': 'null'}, 'assigned_at': {'format': 'date-time', 'type': ['string', 'null']}, 'fail_reason': {'type': ['string', 'null']}, 'to_agent': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'wait_duration': {'type': ['integer', 'null']}, 'answer_type_path': {'type': ['string', 'null']}, 'from_menu_path': {'additionalProperties': False, 'type': ['object', 'null'], 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}}}, 'type': 'array'}, {'type': 'null'}], 'inclusion': 'available'}, 'outbound_number': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'deflection': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'scheduled_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'created_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'updated_at': {'format': 'date-time', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'voip_provider': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'agent_info': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'avatar_url': {'format': 'uri', 'type': ['string', 'null']}, 'first_name': {'type': ['string', 'null']}, 'last_name': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}, 'agent_number': {'type': ['integer', 'null']}, 'id': {'type': ['integer', 'null']}}}, 'answer_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'rating': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_url': {'format': 'uri', 'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'menu_path': {'additionalProperties': False, 'selected': True, 'type': ['object', 'null'], 'inclusion': 'available', 'properties': {'items_count': {'type': ['integer', 'null']}, 'materialized_path': {'type': ['string', 'null']}, 'name': {'type': ['string', 'null']}}}, 'status': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'out_ticket_id': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'call_type': {'selected': True, 'type': ['string', 'null'], 'inclusion': 'available'}, 'wait_duration': {'selected': True, 'type': ['integer', 'null'], 'inclusion': 'available'}}}]
2020-07-03 13:13:59,964Z target - INFO Serializing batch with 11 messages for table calls
2020-07-03 13:13:59,965Z target - INFO Sending batch of 33802 bytes to https://api.stitchdata.com/v2/import/batch
2020-07-03 13:14:00,127Z target - INFO Requests complete, stopping loop
2020-07-03 13:14:00,199Z   main - INFO Target exited normally with status 0
2020-07-03 13:14:01,137Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [24] at offset None
2020-07-03 13:14:01,138Z   main - INFO No tunnel subprocess to tear down
2020-07-03 13:14:01,138Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "Errors during transform". Target succeeded.

Bookmarks not changing query to endpoint

I'm using this integration via Stitch; it appears to be re-querying from the beginning of the integration's date rather than the bookmark. The logs continue on doing this until the 6 hour runtime is hit:

2020-05-29 15:08:00,947Z   main - INFO State update: adding bookmarks.agent_activity_logs = "2020-04-02T06:57:42.000000Z"
2020-05-29 15:08:00,947Z   main - INFO State update: adding bookmarks.agents = "2020-04-30T16:34:45.000000Z"
2020-05-29 15:08:00,947Z   main - INFO State update: adding bookmarks.calls = "2020-04-30T16:34:48.000000Z"
2020-05-29 15:08:00,947Z   main - INFO State update: adding currently_syncing = "menu_tree"
2020-05-29 15:08:00,957Z   main - INFO State update: changing currently_syncing from "menu_tree" to "calls"
2020-05-29 15:08:03,940Z    tap - INFO HTTP request to "calls" endpoint took 1.700s, returned status code 200
2020-05-29 15:08:04,245Z    tap - INFO replicated 0 records from "calls" endpoint
2020-05-29 15:08:04,245Z    tap - INFO Stream calls, batch processed 0 records, total processed records 0
2020-05-29 15:08:04,246Z    tap - INFO Write state for stream: calls, value: 2020-04-30T16:34:48.000000Z
2020-05-29 15:08:04,246Z    tap - INFO Synced Stream: calls, page: 1, records: 0 to 100 of 244812
2020-05-29 15:08:04,246Z    tap - INFO URL for Stream calls: https://gopuff.ujet.co/manager/api/v1/calls?page=2&per=100&sort_column=updated_at&sort_direction=ASC&updated_at%5Bfrom%5D=2019-01-01T00%3A00%3A00Z
2020-05-29 15:08:05,762Z    tap - INFO HTTP request to "calls" endpoint took 1.515s, returned status code 200
2020-05-29 15:08:06,053Z    tap - INFO replicated 0 records from "calls" endpoint
2020-05-29 15:08:06,053Z    tap - INFO Stream calls, batch processed 0 records, total processed records 0
2020-05-29 15:08:06,053Z    tap - INFO Write state for stream: calls, value: 2020-04-30T16:34:48.000000Z
2020-05-29 15:08:06,053Z    tap - INFO Synced Stream: calls, page: 2, records: 100 to 200 of 244811

Taking a quick look, I believe this is the code to blame:

        if page == 1:
            if bookmark_query_field:
                if bookmark_type == 'datetime':
                    params[bookmark_query_field] = start_date
                elif bookmark_type == 'integer':
                    params[bookmark_query_field] = last_integer
            if params != {}:
                querystring = '&'.join(['%s=%s' % (key, value) for (key, value) in params.items()])

Rather than the if/elif, shouldn't it just set params[bookmark_query_field] = max_bookmark_value as defined in the code just above it

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.