Giter VIP home page Giter VIP logo

tap-tiktok-ads's Introduction

tap_tiktok_ads

This is a Singer tap that produces JSON-formatted data from the TikTok Marketing API following the Singer spec.

This tap:

Authentication

The tap uses an access token provided by the TikTok Marketing API to make the calls to the API. The access token does not expire, but it'll become invalid if the advertiser cancels the authorization. An invalid access token cannot be renewed or refreshed. THis access token can be obtained by following the step-by-step instructions on the TikTok Marketing API Docs - Authorization and TikTok Marketing API Docs - Authentication.

Config File

The tap requires some fields to be completed in a config file in order to work. Currently the tap supports the fields:

  • start_date: Date from which the tap starts extracting data. RFC3339 format.
  • end_date: Date from which the tap ends extracting data. RFC3339 format. (Optional)
  • user_agent: User agent that makes the API call.
  • access_token: Access token for the TikTok Marketing API
  • accounts: A string containing comma-separated values of account ids.
  • request_timeout: The time for which request should wait to get response. It is an optional parameter and default value as 300 seconds.
  • sandbox (string, optional): Whether to communication with tiktok-ads's sandbox or business account for this application. If you're not sure leave out. Defaults to false.
{
  "start_date": "2019-01-01T00:00:00Z",
  "end_date": "2020-01-01T00:00:00Z",
  "user_agent": "tap-tiktok-ads <api_user_email@your_company.com>",
  "access_token": "YOUR_ACCESS_TOKEN",
  "accounts": "id1, id2, id3",
  "sandbox": "<true|false>",
  "request_timeout": 300
}

Install

Clone this repository, and then install using setup.py. We recommend using a virtualenv:

    > virtualenv -p python3 venv
    > source venv/bin/activate
    > python setup.py install
    OR
    > cd .../tap-tiktok-ads
    > pip install .

Dependent libraries

The following dependent libraries were installed.

    > pip install singer-python
    > pip install singer-tools
    > pip install target-stitch
    > pip install target-json

Setup

Create your tap's config.json file which should look like the following:

```json
{
    "start_date": "2019-01-01T00:00:00Z",
    "user_agent": "tap-tiktok-ads <api_user_email@your_company.com>",
    "access_token": "YOUR_ACCESS_TOKEN",
    "accounts": "id1, id2, id3",
    "request_timeout": 300
}
```

Optionally, also create a `state.json` file. `currently_syncing` is an optional attribute used for identifying the last object to be synced in case the job is interrupted mid-stream. The next run would begin where the last job left off.

```json
{
    "currently_syncing": "ad_insights",
    "bookmarks": {
        "accounts": "2020-04-06 10:52:54.000000Z",
        "campaigns": "2021-08-10T06:56:51.000000Z",
        "adgroups": "2021-08-10T06:56:51.000000Z",
        "ads": "2021-08-09T09:11:54.000000Z",
        "ad_insights": "2021-08-18T00:00:00.000000Z",
        "ad_insights_by_age_and_gender": "2021-08-18T00:00:00.000000Z",
        "ad_insights_by_country": "2021-08-18T00:00:00.000000Z",
        "ad_insights_by_platform": "2021-08-18T00:00:00.000000Z"
    }
}
```

Discovery Mode

Run the Tap in Discovery Mode. This creates a catalog.json for selecting objects/fields to integrate:

> tap-tiktok-ads --config config.json --discover > catalog.json

See the Singer docs on discovery mode here.

Sync Mode

Run the Tap in Sync Mode (with catalog) and write out to state file

For Sync mode:

> tap-tiktok-ads --config tap_config.json --catalog catalog.json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

To load to json files to verify outputs:

> tap-tiktok-ads --config tap_config.json --catalog catalog.json | target-json > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

To pseudo-load to Stitch Import API with dry run:

> tap-tiktok-ads --config tap_config.json --catalog catalog.json | target-stitch --config target_config.json --dry-run > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

Test the Tap

While developing the Linkedin Ads tap, the following utilities were run in accordance with Singer.io best practices: Pylint to improve code quality

> pylint tap_<tap_name> --disable 'broad-except,chained-comparison,empty-docstring,fixme,invalid-name,line-too-long,missing-class-docstring,missing-function-docstring,missing-module-docstring,no-else-raise,no-else-return,too-few-public-methods,too-many-argument
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)

To check the tap and verify working:

> tap-tiktok-ads --config tap_config.json --catalog catalog.json | singer-check-tap > state.json
> tail -1 state.json > state.json.tmp && mv state.json.tmp state.json

Check tap resulted in the following:

The output is valid.
It contained 12135 messages for 8 streams.

      8 schema messages
  11593 record messages
    534 state messages

Details by stream:
+-------------------------------+---------+---------+
| stream                        | records | schemas |
+-------------------------------+---------+---------+
| advertisers                   | 1       | 1       |
| campaigns                     | 23      | 1       |
| adgroups                      | 29      | 1       |
| ads                           | 67      | 1       |
| ad_insights                   | 955     | 1       |
| ad_insights_by_age_and_gender | 6908    | 1       |
| ad_insights_by_country        | 1415    | 1       |
| ad_insights_by_platform       | 2195    | 1       |
+-------------------------------+---------+---------+

Copyright © 2021 Stitch

tap-tiktok-ads's People

Contributors

hpatel41 avatar joaoromanogruvi avatar krispersonal avatar leslievandemark avatar sgandhi1311 avatar vi6hal avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

tap-tiktok-ads's Issues

Security Policy violation Branch Protection

This issue was automatically created by Allstar.

Security Policy Violation
PR Approvals not configured for branch master
Block force push not configured for branch master


This issue will auto resolve when the policy is in compliance.

Issue created by Allstar. See https://github.com/ossf/allstar/ for more information. For questions specific to the repository, please contact the owner or maintainer.

Missing advertiser_id in ad_insights_* records

Hi,
When I used this tap with target-stitch, sync failed because "advertiser_id" is one of key_properties but missing in ad_insights and ad_insights_* records. I modified the source code to add the field and sync job ran successfully.

Invalid metric fields: ['placement'].

It looks like some fields have been changed in the API -- I was receiving the above error for the field ['placement']. This may be unique to our acoount, maybe not? Forked the repo and removed 'placement' in streams.py and the ad_insights/adgroups.json files and now its running fine. Adding here for visibility

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.