Gatsby internal.contentDigest is a poor source for deciding whether content needs to be updated about gatsby-plugin-algolia HOT 6 CLOSED

Two things to mention:

1. Algolia updates an object 1:1, so there is no "partial Update" of an object.
2. All of these calculations only serve to reduce the burden and cost to Algolia.

In the previous versions you could define the fields which acts as a hash.
Because of this, we calculated a hash post-query in the transformer function, and named that field hash.
And because our paths where not part of the content itself (the field slug is not a full qualified path on a server), we where able also to set that as a configuration.

Our configuration looked like:

{
      appId: GATSBY_ALGOLIA_APP_ID,
      apiKey: ALGOLIA_WRITE_API_KEY,
      indexName: `${GATSBY_ALGOLIA_INDEX_NAME_PREFIX}`
      queries: algoliaQueries(languageTag),
      enablePartialUpdates: true, // default: false <<-- This got removed due to the change of requiring the internal.contentDigest field.
      matchFields: ['path', 'hash'], // default: ['modified'] <<-- There was the interesting magic
      skipIndexing: skipAlgoliaIndexing, // default: false <<-- This got removed, which is another issue but simple to solve
    }

And our transformer for a single object in the data array response was like:

function transformer(item) {
 const result = {
      ...item,
      objectID: `${item.contentful_id}_${item.node_locale}`,
      hash: createHashFromInputData(item),
      path: createPathFor(item),
    };
}

If you are

• aiming for a "one-size-fits-all" solution, you need to create a method that calculates hashes for each object tree or traverse all objects to calculate a hash over all contentDigest
• aiming for a simple solution but open for any kind of data structure, re-enable the matchFields option and explain in your docs what is happening and why.

IMHO a "one-size-fits-all" solution will never be good enough. So let the creators / developers be responsible on how to calculate the hash.

--- Update: My poorly translated point 2 in the section things to mention.

from gatsby-plugin-algolia.

Yes, that's an interesting point, there was an issue yesterday as well (#181, cc @drooJohnson) that was linked to contentDigest not working as needed.

I see two options here:

1. create a simple hash of the content of the record before indexing (takes some computing power, but likely isn't too bad)
2. allow override of the "digestField" so people can compute their own hash / combine existing fields (simple, but not sure if this would be too discoverable)

If you have any idea of how it should be most discoverable, I'm open to ideas.

from gatsby-plugin-algolia.

Makes sense, although I'd like to clarify that the load minimisation is for the customers, not Algolia. You're paying for load so it's in your interest, not Algolia's, to minimise it.

In that case the simplest solution is to change internal.contentDigest to an option to override, matchFields as that name used to exist, still defaulting to ['internal.contentDigest'].

If you're interested in making this happen fast, you can make a pull request, which I'm happy to approve :)

from gatsby-plugin-algolia.

Hi @Swissbite Any update on this, same issue we are also facing with contentDigest. The post helped to narrow down the issue. Here is another thought, function transformer(item) { contentDigest we can add current time/timestamp/random hash, instead of adding new field. just a thought till your PR updated to plugin

from gatsby-plugin-algolia.

You're right, the best way to ensure you're in charge is by consistently using the internal.contentDigest field (which also means the code doesn't need to be updated to deal with an array of fields instead of only one.

from gatsby-plugin-algolia.

I added this to the FAQ and documentation, so I think this issue is handled for now.

from gatsby-plugin-algolia.