agency-kit / notion-cms Goto Github PK

We should provide an all inclusive system for establishing a project. A cli is perfect for this. Set up a config and you could even tell the CLI what keywords you want to capture in each page on the site then use that data to prompt NotionAI. You could also provide plugin hooks to do pre-processing and add content from generated sources/APIs before they even get sent to Notion. Then they can get edited from their manually.

CLI should be based on helper scripts that can also be used modularly. CLI commands should really provide one time use automation for scaffolding projects across Notion and local environment boundaries. Probably want to protect against the case where data already exists in the DB.

Parity with Ghost, and you may not always want to inject content as HTML.

This would give up to date times for which pages have been published since changes have been made.

Database results list presents the same set of information that is in each individual page response so reuse will cut back on unnecessary code and improve performance.

Currently only able to pull 50 blocks for page content.

Page properties in the default notion cms template (in Notion) include a select that is either published or draft (or archive but that should just be categorized as draft for the purpose of this issue).

We should only add pages tagged as published into the cms unless a draftMode flag is set in the user options then we pull everything in. This lets you preview the site with all content including drafts.

probably use something like this: https://github.com/commitizen/cz-cli or this https://github.com/changesets/changesets

You could define a template and use delimiters like {{ }} in a notion page, use a property like template: true and then other pages in the same hierarchy can have values that populate the template. This could provide a lot of value for building websites but also for email. Ghost CMS provides email so its likely you would want to manage both from Notion.

Then simply hook NotionCMS up to nodemailer, pull out the templates, populate and send!

In case Notion API fails on a given call, leave an option to queue and retry at least once.

A plugin exist to extend NotionCMS in some way. There are currently only two hooks where the plugins run but long term I think we should provide hooks at these points: post-db-call, pre-parse, post-parse, final.

Right now pre and post parsing just give you access to the blocks or html for an individual entry so we should add a hook post-db-call (working title) for pre processing the tree-level pre-content calls (in fact pre-content might be a good name) and a final or post-content hook for any final tree-level transforms. This opens up many more possibilities for what a plugin can do.

The main idea of plugins is that the core database that comes with NotionCMS is as minimal as possible. Whenever you add a plugin you can add data properties that you expect to see for each entry.

As such plugins should come with necessary template or schema for the notion database so that they are easy for users to consume. NotionCMS should provide plugin types and the plugin itself should update type expectations for the tree structure should it modify the tree in any way.

Here's the list of planned plugins:

• its possible templates should be a plugin instead of built in to core dunno
• navigation - aggregate marked pages, slugs and urls for building nice nav components.
• segmenter - just split content by horizontal rules into an array so your can do: cms.segment[0] in one component and cms.segments[1] in another.
• Notion component mapper
• images - pull all images in a page into a set structure, ability to perform some optimizations on them, caching, etc. aws hosted images from Notion expire after a couple of hours so aren't usable. You need to download the image from remote.
• internal links
• mermaid
• pull out all seo/head/metadata properties and make that an SEO plugin
• schema.org management
• i18n (probably use something like https://github.com/franciscop/translate as it would let the end user decide on translation engine)
• general purpose UI framework component mappings (MDX for notion). See vue-substrat-forms.ts for working example.
• simple forms - we can SSR using vue behind the scenes but all that matters is static form html for netlify works.
• NotionCMS lint. Set rules and warnings if your content contains certain patterns. eg I use (link) as a placeholder a lot but forget the link.
• front matter parsing - https://github.com/natemoo-re/ultramatter
• link preview builder- need to request and scrape title, metas etc server side.

Some import workflow improvements (see #30) :

• Ghost to NotionCMS
• MD local files to NotionCMS - using tryfabric/martian and NotionCMS cli.

Some loftier ones:

• post recommendations. Analyzes and builds an array of recommendations for consumption on current page.
• search - compile all pages into a graph and tokenize some keywords so that their index pops up and we provide a path to the page. For building a nice search component.
• graphNav - build a content graph (similar to search) that can plug into a force-directed graph visualization and use that to filter pages and navigate the site. Futuristic navigation.
• provide a serverless function + plugin that pushes form filled data back to a specific database in Notion.
• Semver at time of writing and current semver. Avoid doc-rot by using ncms plugin that writes to a column in the notion db telling you which documentation needs updated.
• a/b testing rig

Some plugin helpers we need to expose:

• type safe property extractor
• expose the tree crawler

via Notion API

Some issues:

• Tests require huge response objects that often go out of sync. Maintenance is hassle. there has got to be a better way.
  making the test tables much much smaller will help. We have tons of extraneous properties that are in the actual responses from Notion but aren't used or not needed to do a proper test.
• Some tests just don't exist and if its not tested its not working.

If I change draftMode to true, but the last cached version was built using no draft mode, I have to manually delete the cache or manually set the refresh timeout to a low number, run the program, then set the timeout back and run again. Could be much better experience.

Same if you add plugins - they don't actually run depending on the hook since certain hooks won't run if a cache is present. Maybe we should think about having all hooks run every time for plugin consistency?

For more complicated layouts we will need to perform multiple levels of calls to extract deeply nested blocks content if has_children is true. See https://developers.notion.com/reference/get-block-children

The simpler the final tree the better. I think we could provide an option to expose otherProps if the user wants but recommend using a plugin to extract the props and slot them into the right place in the tree instead. That way you can have a clean tree with properly extracted values. The alternative would be to use value-extraction up front to clean up the otherProps and possible just leave it in at that point.

consumption is intended for node runtime we don't need to minify the dist files.

Using glob patterns something like this */path/to/page or just the /path/to/page and the walker should recognize when we are satisfying the pattern and use that as the starting point for the query.

This is basically the only thing blocking from client side usage. Might also be nice to allow for setting a remote cache.

Curious what good client side use cases look like though.

This will pave the way for more complex caching objects like functions (template plugin will benefit from this).

See https://github.com/storybookjs/telejson and https://github.com/lxsmnsyc/seroval

To go along with the rejectSubPages helper. You may not want the full child tree but you may want to retain the paths to each child and the tree structure so you can use them for navigation.

This will let you define a plugin that can perform actions on multiple hooks and do multistep transformations.

Proposed API:

// static method
ncms.merge(
  new NotionCMS(/*unique db id, unique set of plugins */), 
  new NotionCMS(/*unique db id, unique set of plugins */)
).at('/path/to/append/node')

This would be for advanced use cases where you want to manage notion databases separately for whatever reason but still maintain a single CMS tree.

https://www.npmjs.com/package/marked-gfm-heading-id

the headerIds option is deprecated.

This used to happen but we had to strip out the mixedHtml marked html render extension because it was breaking deeply nested render (see #57). Maybe we can find a work around and reintroduce the mixed html text wrapping.

2 things need to be done - make sure we detect and build html for a table of contents block which will let us show the page-level toc but for the site wide toc (sidebar for example), provide whatever tree level helpers to make that easy to insert.

This will make multiple versions of the content easier to manage based on descriptive keys. For example:

content : {
   default: '<div>some content html</div>',
   plaintext: 'some content',
   markdown: '## some content',
   japanese: '...',
}

Could we design it so that if only a single key exists (default) that key is used to access the content? This would avoid breaking backwards compatibility and be a convenience factor as well.

rn the utilities are mostly tree level and I can imagine where you would want node level versions of tag collection utils etc.

Should be simple as adding the db hash (or some subset) to the cache file name.

Currently the findByKey helper will stop the first time it finds a key with the given route name so will never find others with the same name. We should extend the helper to add more specificity when querying pages.

We want to be able to define internal links in Notion but have them port to the actual site we are building. We should define a non-intrusive syntax for linking in Notion (should differentiate between when we want to maintain a link to a Notion page from the website) and transforming those paths to relative routes on the website.

This is because of a bad build that got published. We should also force a build to run before publishing every time!

Right now plugins let you do block -> block transformations or html -> html transformations but in order to do block -> html we need to take over the parser. Currently this is locked into place so my best idea for a fix is to implement a core plugin that houses the current parser as default but is over-rideable. This would be a special plugin and no other plugins will have access to the parser. I don't want to make it possible to provide custom parsers because I want to stick to the plugins-as-single-point-of-extension model.

Trickier than the other blocks because it falls outside of the normal child block flow. You have to do 2 rounds of API calls to get the full column content.

Column list -> children = column block(s) -> children = column content

See nartc/notion-stuff#14

This can be done using https://github.com/makenotion/notion-sdk-js#collectpaginatedapilistfn-firstpageargs

Currently the debug chain turns on all possible debugging tools - Notion API logs, NotionCMS console logs and (its turned off for now) fs output logs from the parser. If we instead used a debug object we could let end users specify which debugging features they want turned on.

Keep getting an error related to eval. Maybe we need to serialize functions optionally and have a eval: false default.

add an option to for children missing properties check their parents and inherit their notion properties. Should provide an options api for limiting which properties inheritance applies to.

Ex: I want to add head scripts and style links to every page on my site. I can use script and style properties and provide urls inside those props that point to the resources I want, but I will want to avoid having to define this for every page/subpage. If I can limit the properties that are inherited to script and style this is easily accomplished without also inheriting all props everywhere in the tree.

Could also provide mapping for paths to props.

Currently marked which is used under the hood to parse blocks -> markdown -> html will escape anything that isn't in their list of known html tags. We need to figure out how to provide patterns to match for custom tags.

something like:

new NotionCMS({
  // some options,
  output: {
    name: NotionText,
    date: NotionDate
    etc.
  }
})

When we cache results locally and ping the agency-kit polling server to see if the response has changed, we should record which specific pages have changed so that we can minimize calls from Notion CMS.

This will make the local build cycles much faster as number of pages in the CMS grows. If only one page changes, get just that page. Pairs well with incremental builds.

Better to try it and then default to using a fresh API call if cache fails for whatever reason.