diegolamanno / gatsby-source-greenhouse Goto Github PK

I'm using this package to pull in some job posts data but I'm getting the following error during the gatsby build...

Fetch Greenhouse data
Starting to fetch data from Greenhouse
Failed to fetch data from Greenhouse

The apiToken I have is only configured to access all endpoints related to jobs, i.e. jobs, job posts, job openings, etc. I can cURL -u with the apiToken and get the correct data so I'm sure that's working correctly. Any insight into what could be going wrong?

My gatsby-plugin.js file looks as follows:

require('dotenv').config({
  path: `.env.${process.env.NODE_ENV}`,
})

const path = require('path')

console.warn('process env', process.env.GREENHOUSE_API_KEY) // outputs the correct apiToken

module.exports = {
  plugins: [
    `gatsby-plugin-sass`,
    'gatsby-plugin-react-helmet',
    'gatsby-plugin-offline',
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: 'images',
        path: `${__dirname}/images`,
      },
    },
    'gatsby-plugin-sharp',
    'gatsby-transformer-sharp',
    {
      resolve: `gatsby-source-greenhouse`,
      options: {
        apiToken: process.env.GREENHOUSE_API_KEY,
        jobPosts: {
          live: true,
        },
      },
    },
  ],
}

By default the Harvest API defaults to 100 job listings per page: https://developers.greenhouse.io/harvest.html#get-list-job-posts

If you have more than 100 job listings gatsby-source-greenhouse doesn't fetch them all.

I tried to fix it by updating my config options in gatsby-config.js to:

{
   resolve: `gatsby-source-greenhouse`,
   options: {
      apiToken: `{token}`,
      jobPosts: {
         live: true,
         per_page: 300
      }
   },
},

But that doesn't seem to work. It looks like they might be getting ignored since even if I change them to live: false I still get the same jobs fetched.

If I manually overwrite the defaultPluginOptions in gatsby-source-greenhouse/gatsby-node.js the right number gets fetched:

const defaultPluginOptions = {
   jobPosts: {
      live: true,
      per_page: 500
   }
};

So, I think there's two problems:

1. The jobPosts config option gets ignored
2. Only the first page of jobs gets fetched

I'm trying to query all job posts across multiple boards. Specifically I have one job that has multiple posts with the same name, but allGreenhouseJobPost only includes one post of each job.

For example here's my query:

  allGreenhouseJobPost {
    edges {
      node {
        job_id
        external
        title
      }
    }
  }
}

This returns only one job, but it should return two jobs with the same job_id: one internal and one external.

The project got its first external contribution and that brought the concern of coding style guide. It's easier when you don't have to worry about that. Prettier is probably the best option

GitHub has identified the following security vulnerability related to axios:

Axios up to and including 0.18.0 allows attackers to cause a denial of service (application crash) by continuing to accepting content after maxContentLength is exceeded.

GitHub recommends the following remediation:

Upgrade axios to version 0.18.1 or later.

When making a lot of API calls to /v1/jobs, due to having many departments for example, or due to other plugins having many content updates and re-calling sourceNodes multiple times, the request eventually fails with the following error:

Failed to fetch jobs for department.

Upon further investigation I found out that the axios request inside getJobsForDepartment fails and throws a 429 Too Many Requests error, which in turn leads to process.exit(1) in the catch clause inside sourceNodes, which crashes the entire project.

My suggestion is to add a retry mechanism on failed requests, and add a backoff delay in the case of an HTTP 429 error.