Giter VIP home page Giter VIP logo

datasync-filesystem-sdk's Introduction

Contentstack

Contentstack is a headless CMS with an API-first approach. It is a CMS that developers can use to build powerful cross-platform applications in their favorite languages. Build your application frontend, and Contentstack will take care of the rest. Read More.

Contentstack DataSync Filesystem SDK

Contentstack DataSync provides Filesystem SDK to query applications that have locally stored contents in filesystem. Given below is the detailed guide and helpful resources to get started with Filesystem SDK.

Prerequisite

Configuration

Property Type Defaults Description
baseDir string ./_contents Optional Base directory of the folder where data is stored.
locale string 'en-us' Optional Default locale that'd be considered, if the .language() isn't specified in queries
referenceDepth number 2 Optional The default nested-reference-field depth that'd be considered when calling .includeReferences(). This can be overridden by passing a numerical argument to .includeReferences(4)
projections object {_content_type_uid: 0} Optional Keys that by default would be removed from results. Pass key: 0 to remove, key: 1 to override the existing..

Config Overview

Here's an overview of the SDK's configurable properties

{
  contentStore: {
    baseDir: './_contents',
    defaultSortingField: 'updated_at',
    locale: 'en-us',
    projections: {
      _content_type_uid: 0,
    },
    referenceDepth: 2,
  },
}

Sample SDK Query

Here's a sample SDK query to get started. Learn more on how to query using datasync-filesystem-sdk here.

import { Contentstack } from 'datasync-filesystem-sdk'
const Stack = Contentstack.Stack(config)

Stack.connect()
  .then(() => {
    return Stack.contentType('blog')
      .entries()
      .language('en-gb') // Optional. If not provided, defaults to en-us
      .include(['authors'])
      .includeCount()
      .includeContentType()
      .queryReferences({'authors.firstName': 'R.R. Martin'})
      .then((result) => {
        // Your result would be
        // {
        //   entries: [...], // All entries, who's first name is R.R. Martin
        //   content_type_uid: 'blog',
        //   locale: 'es-es',
        //   content_type: {...}, // Blog content type's schema
        //   count: 3, // Total count of blog content type
        // }
      })
  })
  .catch((error) => {
    // handle errors..
  })

Important: You need to call .connect(), to initiate SDK queries!

Once you have initialized the SDK, you can start querying on the filesystem

Querying

  • Notes

    • By default, 'content_type_uid' and 'locale' keys as part of the response.
    • If .language() is not provided, then the 1st language, provided in config.defaultLocale would be considered.
    • If querying for a single entry/asset (using .entry() OR .findOne()), the result will be an object i.e. { entry: {} }, if the entry or asset is not found, { entry: null } will be returned.
    • Querying multiple entries, would return { entries: [ {...} ] }.
    • By default, all entry responses would include their referred assets. If .excludeReferences() is called, no references (including assets) would not be returned in the response.
  • Query a single entry

// Sample 1. Returns the 1st entry that matches query filters
Stack.contentType('blog')
  .entry() // OR .asset()
  .find()
  .then((result) => {
    // Response
    // result = {
    //   entry: any | null,
    //   content_type_uid: string,
    //   locale: string,
    // }
  })
  .catch(reject)

// Sample 2. Returns the 1st entry that matches query filters
Stack.contentType('blogs')
  .entries() // for .assets() 
  .findOne()
  .then((result) => {
    // Response
    // result = {
    //   entry: any | null,
    //   content_type_uid: string,
    //   locale: string,
    // }
  })
  .catch(reject)
  • Querying a set of entries, assets or content types
Stack.contentType('blog')
  .entries() // for .assets() 
  .includeCount()
  .find()
  .then((result) => {
    // Response
    // result = {
    //   entry: any | null,
    //   content_type_uid: string,
    //   count: number,
    //   locale: string,
    // }
  })
  .catch(reject)

Advanced Queries

In order to learn more about advance queries please refer the API documentation, here.

Further Reading

Support and Feature requests

If you have any issues working with the library, please file an issue here at Github.

You can send us an e-mail at [email protected] if you have any support or feature requests.

Our support team is available 24/7 on the intercom. You can always get in touch and give us an opportunity to serve you better!

License

This repository is published under the MIT license.

datasync-filesystem-sdk's People

Contributors

aravindbuilt avatar vkalta avatar iyerrama29 avatar abhinav-from-contentstack avatar abhnv101 avatar antonyagustine avatar cs-raj avatar snyk-bot avatar netrajpatel avatar aman19k avatar contentstack-admin avatar shafeeqd959 avatar vkaltae9 avatar nandeesh-gajula avatar

Stargazers

Altaf Shaikh avatar

Watchers

Mayank avatar dhaval avatar Nikhil Jain avatar  avatar Ninad Hatkar avatar James Cloos avatar Pradeep avatar  avatar Sunil Nadar avatar Hitesh Baldaniya avatar kego avatar Roshan Gade avatar Rishiraj Upadhyay avatar Rohit Mishra avatar  avatar  avatar  avatar

datasync-filesystem-sdk's Issues

projections in configuration not working as described

According to the documentation

projections object {_content_type_uid: 0} Optional Keys that by default would be removed from results. Pass key: 0 to remove, key: 1 to override the existing..

it should be possible to pass 1, to active the field again. But it has no effect, because the check only checks if the property is present, it doesn't check the value

this.projections = Object.keys(this.contentStore.projections);
This gets all projections regardless if 0 or 1

result.docs.forEach((doc) => { this.projections.forEach((key) => { if (doc.hasOwnProperty(key)) { delete doc[key]; } }); });
This only checks the "key" regardless if the value is 0 or 1.

[BUG]: Add validations to .tag() method

Describe the bug

  • .tag() method accepts an empty array

To Reproduce

Query a content type
Create a query that uses to filter by tags
In the query, enter an empty array

Notice: The result is empty. In contentstack-javascript the result is all the entries. Thus, the behaviour is different in both

Query

Stack.contentType('blog')
  .entries()
  .tags([])
  .find()

Expected behavior
The query should be invalidated OR throw errors

For future references: If you wish to query for an empty 'Array' field, use $size operator inside query

// example
Stack.contentType('blog')
  .entries()
  .query({tags: {$size: 0}})

Regards,
Ramanathan K

[BUG] DataSync SDK throws an exception, when you query a content type that has not yet published references

We get the following exception, when we query a content type including references, and one (or more) references are not yet published.
TypeError: Cannot read properties of undefined (reading 'hasOwnProperty') at Stack.<anonymous> (/home/node/app/node_modules/datasync-filesystem-sdk/dist/stack.js:1208:49) at Generator.next (<anonymous>) at fulfilled (/home/node/app/node_modules/datasync-filesystem-sdk/dist/stack.js:11:58)

We are using
blogQuery.include([
"buttons",
"content",
"content.deal_items",
"content.deal_items.image16_9.channel",
"content.deal_items.image16_9.competition",
"content.deal_items.default_comparison",
"content.deal_items.secondary_button.reference",
"content.deal_items.filter_tags",
"content.rail.manually.content_link.reference",
"content.primary_button.reference",
"content.secondary_button.reference"
])

And the content type instance looks like this
`{
"title": "Deal Selection - Deal Cards",
"headline": "",
"subheadline": "",
"text": "",
"config": {
"variant": "dealCard",
"mobile": "stacked"
},
"deal_items": [
{
"uid": "blt3ea1b0c7d505e185",
"_content_type_uid": "deal_item"
},
{
"uid": "blt7cbdd8f42d317990",
"_content_type_uid": "deal_item"
},
{
"uid": "blt4019d2d57c5d735d",
"_content_type_uid": "deal_item"
},
{
"uid": "blt21af4abc87ccba18",
"_content_type_uid": "deal_item"
},
{
"uid": "blt7e20357fb7549664",
"_content_type_uid": "deal_item"
},
{
"uid": "bltacb034455a73cb53",
"_content_type_uid": "deal_item"
},
{
"uid": "blt587c8e2ad2466b21",
"_content_type_uid": "deal_item"
},
{
"uid": "blt08d2a4624e84e497",
"_content_type_uid": "deal_item"
}
],
"tags": [],
"locale": "de-de",
"_in_progress": false,
"_embedded_items": {
"text": []
},
"publish_details": [

],
"_rules": []
}`

One of the deal_item was not published and then we got this exception. It was solved by publishing the missing item, but the SDK should not throw an error in that case

Image property returning uid instead of null, if image was not published yet.

Scenario

  1. Create a new entry of a content type that includes an image entry
  2. Publish the new entry WITHOUT references, so the image itself is not published.

ContentStack API will return "null" for that image when querying that content type
"image": null
DataSync API returns the uid of the referenced image
"image": "blt5ec250ecf650b59b"

This breaks some logic because "null" or "undefined" checks fail because the value is for example "blt5ec250ecf650b59b" instead of "null".

Same happens when you "unpublish" the image without publishing the content type where its referenced.

Datasync runtime exception if nexted reference is not published

If a content type with a reference, which itself includes another reference, is published, but a nested reference is not. This causes a runtime exception

TypeError: Cannot read properties of undefined (reading 'hasOwnProperty')
at Stack. (/home/node/app/node_modules/datasync-filesystem-sdk/dist/stack.js:1556:61)
at Generator.next ()
at fulfilled (/home/node/app/node_modules/datasync-filesystem-sdk/dist/stack.js:11:58)"

Because the object that is checked here oldObjectPointerList[i].path[e]

if (oldObjectPointerList[i].path[e].hasOwnProperty('_content_type_uid') && Object.keys(oldObjectPointerList[i].path[e]).length === 2) {

is null if not all references are published. This happens if you use the blogQuery.includeReferences() which will include all nested references

Missing reference fields for nested reference fields when using .include()

There is a problem in the internal getReferencePath function

We have several content types where we have a "form" reference field, which itself has "fields" reference field.
When trying to fetch the fields entries its not working because the check is not working correctly for nested fields.
Simplified examples of the two content types
`{
"_content_type_uid": "pin_reset_page",
"locale": "de-de",
"unknown_customer": {
"form": [
{
"uid": "blt208e828db2bc1155",
"_content_type_uid": "form"
}
]
},
"updated_at": "2022-11-25T12:12:39.528Z",
"url": "/unified-mpp-pin-reset-page"
}

{
"_content_type_uid": "form",
"locale": "de-de",
"additional_text": "",
"fields": [{ "uid": "bltd0a76355d956c079", "_content_type_uid": "input_field" }],
"headline": "Jugendschutz-PIN vergessen?",
"tags": []

}
{
"_content_type_uid": "input_field",
"rules": [{ "uid": "blt58b06a7fe2b613b5", "_content_type_uid": "validation_rule" }],
"tags": [],
}
`

So when we call
blogQuery.includes(["unknown_customer.form", "unknown_customer.form.fields", " "unknown_customer.form.fields.rules"]);

We have the following values in the getReferencePath
currentInclude = [ 'unknown_customer', 'unknown_customer.form', 'unknown_customer.form.fields', 'unknown_customer.form.fields.rules' ] entryReferences = { 'unknown_customer.form': [ 'form' ] }
when we come to

const subStrArr = includePath.split('.');

the unknown_customer.form works find and and the "form" reference is added. But then the "unknown_customer.form.fields" gets handled the same way because the subStrArr is "unknown_customer" , same as before, where it needs to be "unknown_customer.form" because then the "fields" get added to the pending paths, same basically for the rules

This is a side effect of
https://github.com/contentstack/datasync-filesystem-sdk/pull/13/files#diff-1ebd2ec0fa0f7b0aac61f8683a96f14f9ba2f36cc068918c16b6c3aaf8418e0a

the part where the includePaths are checked

Just tested it with the old version, and that works.
But of course there was the other issue #10 that then breaks again

So we need a more sophisticated implementation here

I guess the problem is deeply nested references

@vkalta @netrajpatel @abhinav-from-contentstack

"include()" is not including some references if part of the include is named like another reference

Stack.contentType("account_defaultpage").entries().include(["back_link.reference","content.content_link.reference"]).find()
will not include the reference for content.content_link.reference.
If I remove "back_link.reference", it works.

I could narrow it down to getReferencePath in stack.ts
Because the if path is taken instead of the else.
if (path.length !== includePath.length) { subPath = subStr; pendingPath.push(includePath.slice(path.length + 1)); } else { subPath = includePath; }
Added related _contents content to reproduce this.
_contents.zip

When it comes to check "content_link.reference" it matches with "content" and therefore "link.references" is pushed instead of "content_link.reference" because there is also a reference called "content"

[BUG]: .entry('uid')/.asset('uid') doesn't work

Describe the bug

  • .entry(uid) doesn't work
  • .asset(uid) doesn't work

To Reproduce
Query a content type/asset
In the query, enter the entry's uid/asset's uid

Notice: The result is empty.

Query

Stack.contentType('blog')
  .entry({uid})

// OR

Stack.asset({uid})

Expected behavior
The query should return an object, if it exists

Workarounds
Use raw .query() for list-entries OR .entry()/.asset() without on singleton-content type

// example 1: On multiple content type
Stack.contentType('blog')
  .entries()
  .query({uid: {uid}})

// OR

Stack.assets()
  .query({uid: {uid}})
// example 2: On singleton content type
Stack.contentType('blog')
  .entry()

// OR

Stack.asset()

Regards,
Ramanathan K

References are lost due to incorrect merge during creation of schema list

Hello,

We encountered a problem while testing one of our components and traced it back to this file: GitHub link. The issue arises from the merging of lodash, which doesn't merge arrays as expected. You can find more details on the issue here.

Example interface:

{
    deals: {
        content: (rail | dealsSelection)[]
    };
    teaser: {
        content: (usp | teaserText)[]
    }
}

Below is the screenshot of schema created:
image

As you can see, both "deals" and "teaser" contain "content". When the schema is merged, it only retains the last "content", causing one of the references to be lost.

Our suggested fix, which worked for us while testing:
entryReferences = (0, lodash_1.mergeWith)(entryReferences, schema[this.types.references], (obj, src) => { if (lodash_1.isArray(obj)) { return Array.from(new Set(obj.concat(src))); } } );

Please let me know if this isn't clear. We can arrange a meeting to discuss the issue in more detail if needed. Thank you!

Regards,
Chaitra

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.