This is a Singer tap that produces JSON-formatted data following the Singer spec.
This tap:
-
Pulls raw data from the Snapchat Ads Marketing API
-
Extracts the following resources:
- organizations
- Targeting:
- Demographics
- Devices
- Geography
- Interests
- targeting_interests_scls: Snap Lifestyle Categories
- targeting_interests_dlxs: Oracle Datalogix DLXS
- targeting_interests_dlxc: Oracle Datalogix DLXC
- targeting_interests_dlxp: Oracle Datalogix DLXP
- targeting_interests_nln: Nielsen
- targeting_interests_plc: Placed Visitation Segments
- Locations
-
Outputs the schema for each resource
-
Incrementally pulls data based on the input state
-
Results are NOT sorted.
-
Most Endpoints (except
_stats_
reports andtargeting
endpoints)- Replication Strategy: INCREMENTAL, query all, filter results
- Bookmark:
updated_at
- Primary Key:
id
-
_stats_
reports:- Replication Strategy: INCREMENTAL, query
- Date Windows: Reports use from/to date windowing and incorporate swipe/view attribution (lookback) windows.
- Bookmarks:
end_time
- Primary Keys:
id
,start_time
-
targeting
endpoints:- Replication Strategy: FULL_TABLE
- Primary Key:
id
The Snapchat Ads Marketing API uses OAuth for API Authentication. Follow these instructions to: Obtain App Credentials.
- Login to Snapchat Business Manager.
- Select Business Settings > Business Details
- Click + OAuth App
- Accept / Save API Terms and Conditions
- Enter **App Name: **tap-snapchat-ads, Snap Redirect URI: YOUR_REDIRECT_URI, example: https://app.stitchdata.com/v2/integrations/platform.snapchat_ads/callback
- Copy your credentials,
client_id
andclient_secret
.
- Enter the following (adjusted) URL into your browser:
- URL: https://accounts.snapchat.com/login/oauth2/authorize?client_id=`YOUR_CLIENT_ID`&redirect_uri=`YOUR_REDIRECT_URI`
- Authenticate the API App with username, password, and connection approval (Continue, Continue).
- Code: Capture the ...&
code=xxxxxx…
in the redirect URL. Thiscode
is used in the next step to generate therefresh_token
.
- Run the following curl command with the parameters replaced to return your
refresh_token
. Therefresh_token
persists indefinitely and is used to generate anaccess_token
(short-lived) for authenticating each API Call.curl -0 -v -X POST https://accounts.snapchat.com/login/oauth2/access_token\ -H "Accept: application/json"\ -H "Content-Type: application/x-www-form-urlencoded"\ -d "client_id=YOUR_CLIENT_ID"\ -d "client_secret=YOUR_CLIENT_SECRET"\ -d "code=YOUR_CODE"\ -d "grant_type=authorization_code"\ -d "redirect_uri=YOUR_REDIRECT_URI"
From the response (JSON), record the refresh_token
.
-
Run the following curl command with the parameters replaced to return your access_token for authenticating each API Call.
curl -0 -v -X POST https://accounts.snapchat.com/login/oauth2/access_token\ -H "Accept: application/json"\ -H "Content-Type: application/x-www-form-urlencoded"\ -d "client_id=YOUR_CLIENT_ID"\ -d "client_secret=YOUR_CLIENT_SECRET"\ -d "grant_type=refresh_token"\ -d "refresh_token=YOUR_REFRESH_TOKEN"
From the response (JSON), record the access_token
.
- Create config.json and include the
client_id
,client_secret
, andrefresh_token
.{ "client_id": "YOUR_CLIENT_ID", "client_secret": "YOUR_CLIENT_SECRET", "refresh_token": "YOUR_REFRESH_TOKEN", "swipe_up_attribution_window": "28_DAY", "view_attribution_window": "7_DAY", "omit_empty": "true", "targeting_country_codes": "us, ca, mx", "start_date": "2020-01-01T00:00:00Z", "user_agent": "tap-snapchat-ads <[email protected]>" }
Additional config parameters
swipe_up_attribution_window
: Attribution window for swipe ups. 1_DAY, 7_DAY, 28_DAY (default).view_attribution_window
: Attribution window for views. 1_HOUR, 3_HOUR, 6_HOUR, 1_DAY (default), 7_DAY, 28_DAY.omit_empty
(true, false): Omits records with zero data for all metrics for a given date/entity. If there is data for any metric for a given date/entity, all metrics for that date/entity are returned.targeting_country_codes
: Comma-delimeted lists of lower-case 2-letter ISO Country Codes for Ads Targeting.
-
Install
Clone this repository, and then install using setup.py. We recommend using a virtualenv:
> virtualenv -p python3 venv > source venv/bin/activate > python setup.py install OR > cd .../tap-snapchat-ads > pip install .
-
Dependent libraries The following dependent libraries were installed.
> pip install target-json > pip install target-stitch > pip install singer-tools > pip install singer-python
-
Optionally, also create a
state.json
file.currently_syncing
is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off.{ "currently_syncing": "sitemaps", "bookmarks": { "ad_squad_stats_daily": { "end_time(parent_ad_squad_id:537a0933-87fb-4450-8f0c-9c890b451971)": "2020-06-24T07:00:00.000000Z", "end_time(parent_ad_squad_id:4a0ba80d-8c12-4467-9c91-d9888a28d65c)": "2020-06-24T07:00:00.000000Z", "..." }, "campaign_stats_hourly": { "end_time(parent_campaign_id:e76d71d4-1bb0-49d8-bf52-9c22fb35d24a)": "2020-06-24T15:00:00.000000Z", "end_time(parent_campaign_id:b4c59d32-06d3-40d9-b896-e1b6191d88a5)": "2020-06-24T15:00:00.000000Z", "..." }, "billing_centers": { "updated_at(parent_organization_id:d373b68c-a28e-4fbe-8ba2-38a85925d7be)": "2020-05-13T16:43:10.841000Z" }, "ad_squad_stats_hourly": { "end_time(parent_ad_squad_id:537a0933-87fb-4450-8f0c-9c890b451971)": "2020-06-24T15:00:00.000000Z", "end_time(parent_ad_squad_id:4a0ba80d-8c12-4467-9c91-d9888a28d65c)": "2020-06-24T15:00:00.000000Z", "..." }, "audience_segments": { "updated_at(parent_ad_account_id:a792a56f-66f0-407e-82ca-ccb30ced0ba7)": "2020-06-09T14:21:28.793000Z" }, "creatives": { "updated_at(parent_ad_account_id:a792a56f-66f0-407e-82ca-ccb30ced0ba7)": "2020-05-13T22:45:10.211000Z" }, "campaigns": { "updated_at(parent_ad_account_id:ef0ae960-145f-4b2f-a574-91ffc73f5bfe)": "2020-05-13T20:49:08.794000Z" }, "pixels": { "updated_at(parent_ad_account_id:353223b7-6d20-4686-899b-29176f7c439c)": "2020-05-14T02:18:37.604000Z" }, "ad_account_stats_hourly": { "end_time(parent_ad_account_id:a792a56f-66f0-407e-82ca-ccb30ced0ba7)": "2020-06-24T15:00:00.000000Z", "..." }, "ad_squads": { "updated_at(parent_ad_account_id:ff7ed2f7-5ca4-4a8b-9848-d09024e06b66)": "2020-05-14T00:04:37.878000Z" }, "ad_stats_hourly": { "end_time(parent_ad_id:1876617a-7f3a-4743-932a-f37e94481b52)": "2020-06-24T15:00:00.000000Z", "end_time(parent_ad_id:4ea60ebe-2224-45a2-8df7-96f82d392824)": "2020-06-24T15:00:00.000000Z", "..." }, "phone_numbers": { "updated_at(parent_ad_account_id:a792a56f-66f0-407e-82ca-ccb30ced0ba7)": "2020-04-01T00:00:00Z" }, "campaign_stats_daily": { "end_time(parent_campaign_id:e76d71d4-1bb0-49d8-bf52-9c22fb35d24a)": "2020-06-24T07:00:00.000000Z", "end_time(parent_campaign_id:b4c59d32-06d3-40d9-b896-e1b6191d88a5)": "2020-06-24T07:00:00.000000Z", "..." }, "ad_account_stats_daily": { "end_time(parent_ad_account_id:a792a56f-66f0-407e-82ca-ccb30ced0ba7)": "2020-06-24T07:00:00.000000Z", "..." } } }
-
Run the Tap in Discovery Mode This creates a catalog.json for selecting objects/fields to integrate:
tap-snapchat-ads --config config.json --discover > catalog.json
See the Singer docs on discovery mode here.
-
Run the Tap in Sync Mode (with catalog) and write out to state file
For Sync mode:
> tap-snapchat-ads --config tap_config.json --catalog catalog.json > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To load to json files to verify outputs:
> tap-snapchat-ads --config tap_config.json --catalog catalog.json | target-json > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
To pseudo-load to Stitch Import API with dry run:
> tap-snapchat-ads --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
-
Test the Tap
While developing the Snapchat Ads Marketing tap, the following utilities were run in accordance with Singer.io best practices: Pylint to improve code quality:
> pylint tap_snapchat_ads -d missing-docstring -d logging-format-interpolation -d too-many-locals -d too-many-arguments
Pylint test resulted in the following score:
Your code has been rated at 9.61/10.
To check the tap and verify working:
> tap-snapchat-ads --config tap_config.json --catalog catalog.json | singer-check-tap > state.json > tail -1 state.json > state.json.tmp && mv state.json.tmp state.json
Check tap resulted in the following:
Checking stdin for valid Singer-formatted data The output is valid. It contained 68128 messages for 46 streams. 46 schema messages 68128 record messages 653 state messages Details by stream: +---------------------------------+---------+---------+ | stream | records | schemas | +---------------------------------+---------+---------+ | ad_squads | 8 | 1 | | pixels | 1 | 1 | | funding_sources | 0 | 1 | | ads | 80 | 1 | | roles | 4 | 1 | | organizations | 1 | 1 | | billing_centers | 1 | 1 | | campaign_stats_hourly | 3754 | 1 | | ad_account_stats_daily | 156 | 1 | | ad_account_stats_hourly | 3743 | 1 | | ad_stats_daily | 237 | 1 | | phone_numbers | 0 | 1 | | product_sets | 2 | 1 | | ad_stats_hourly | 3822 | 1 | | audience_segments | 14 | 1 | | ad_squad_stats_hourly | 3750 | 1 | | campaigns | 12 | 1 | | product_catalogs | 2 | 1 | | ad_accounts | 1 | 1 | | ad_squad_stats_daily | 165 | 1 | | campaign_stats_daily | 169 | 1 | | media | 31 | 1 | | pixel_domain_stats | 1 | 1 | | members | 4 | 1 | | creatives | 11 | 1 | | targeting_interests_nln | 1343 | 1 | | targeting_postal_codes | 48335 | 1 | | targeting_genders | 3 | 1 | | targeting_interests_scls | 163 | 1 | | targeting_connection_types | 2 | 1 | | targeting_advanced_demographics | 76 | 1 | | targeting_interests_plc | 158 | 1 | | targeting_carriers | 495 | 1 | | targeting_location_categories | 159 | 1 | | targeting_device_makes | 843 | 1 | | targeting_languages | 21 | 1 | | targeting_interests_dlxp | 163 | 1 | | targeting_countries | 73 | 1 | | targeting_interests_dlxs | 187 | 1 | | targeting_metros | 357 | 1 | | targeting_ios_versions | 91 | 1 | | targeting_age_groups | 5 | 1 | | targeting_android_versions | 35 | 1 | | targeting_os_types | 2 | 1 | | targeting_regions | 97 | 1 | | targeting_interests_dlxc | 77 | 1 | +---------------------------------+---------+---------+
Copyright © 2020 Stitch