mattpocock / graph-docs-cli Goto Github PK

View Code? Open in Web Editor NEW

94.0 3.0 1.0 689 KB

Revolutionise your docs using knowledge graphs built from Markdown

JavaScript 21.16% TypeScript 78.84%

graph-docs-cli's Introduction

This is an experimental package - I'm looking forward to your feedback on how to improve it.

Graph Docs CLI

Revolutionise your docs by building a knowledge graph from markdown.

Quickstart

Install graphviz
Install with yarn add -D graph-docs-cli
Create a folder called modules, and create two files: one.md and two.md
Inside two.md, add this markdown frontmatter:

---
deps: [one] # the filename of the module, without the .md extension
---

Run yarn graph-docs dev "./modules/*.md"
On localhost:3001, you'll see a graph of modules which will live-update based on changes you make to the deps.

What is this?

My name's Matt, and I'm a developer advocate at Stately. I was working on restructuring XState's documentation, and I noticed something.

The docs were not designed to be read in order. On each page, they referenced pieces of information that the viewer hadn't understood yet. There was advanced information mixed in to beginner concepts. Fundamental subjects were left too late. The docs were not organised around a desired learning path for the user.

I wanted to change this - and, as a developer, I wanted a data structure to help me. I chose a directed graph. I wanted to extract each piece of information from the docs into its own, small module - and have each module express its direct dependencies to other modules.

Modules

For instance - let's say you want to understand how to write an arrow function in JavaScript: () => {}. To understand that, you'll need to walk a path like this:

Arrow functions -> Normal functions
Normal functions -> Variables
Variables -> What is JavaScript?

Which, when put through graphviz, looks like this:

Let's imagine you also wanted to add const vs var vs let - you don't need to know about arrow functions for that, but you do need to know about variables:

What you end up with is a graph of optimal paths to learn any piece of information. This is extremely powerful for several reasons:

You can provide autogenerated paths based on what a user wants to learn (for instance a specific example, or integration)
Maintenance of docs becomes a lot easier. New modules can be slotted into the dependency graph easily.
Authoring modules becomes more declarative. It's a lot easier to write a module when you know exactly what pieces of information the user knows.

Circular dependencies

Modules of information are similar to modules of code - both can contain circular dependencies. In a docs page, this can result in two pieces of information where understanding one depends on understanding the other. This can result in a terrible learning experience. The learner has to read both modules fully, and only at the end (or perhaps on a re-read) will they understand the full picture.

In graph-docs, circular dependencies are not allowed. If you attempt it, you'll get a maximum call stack exceeded error in the CLI.

This constraint forces you to resolve the circular dependency the way you would in code - by either:

Extracting out the relevant bits to a new module
Colocating the modules into one big one (if they cannot be taught separately)

This is also what makes graph-docs different from something like Roam/Obsidian. These tools are fantastic for note-taking. But the fact that they allow circular dependencies makes them unoptimized for readers other than their author.

Groupings

When your graph gets to 100+ modules, it can be hard to see the wood through the trees. You'll naturally want to start grouping them together.

Most docs sites will need a kind of grouping anyway, for menu organisation etc.

To add groups, follow these steps:

Add a graph-docs.json, containing this JSON:

{
  "groups": [
    { "id": "basics", "color": "red" },
    { "id": "advanced", "color": "forestgreen" }
  ]
}

Add group: advanced to the frontmatter of one of the files, for instance:

---
group: advanced
---

You'll see the relevant node in the graphviz will turn green.

Integrations

Currently, there are no custom integrations with docs platforms as part of this package. We should add some!

For the XState docs, we've written a custom integration where we construct pages out of modules using a handlebars-style syntax:

# States

<!-- Links to the states.md module -->

{{ states }}

## Final States

<!-- Links to the final-states.md module -->

{{ final-states }}

(This isn't open-source yet, but will be when we release it.)

But we should offer this as part of graph-docs. Any thoughts appreciated in issues!

graph-docs-cli's People

Contributors

Stargazers

Watchers

Forkers

devdoshi

graph-docs-cli's Issues

git requirement?

just went through the quickstart and I needed to standup a git
repo with at least one commit in order to get the expected dev server running.

I'll paste the output of the failed run here.

~ $ cd dev/jstgit
~/dev/jstgit $ mkdir vizdocs
~/dev/jstgit $ cd !$
~/dev/jstgit $ cd vizdocs
~/dev/jstgit/vizdocs $ npm init -y
Wrote to /Users/lineus/dev/jstgit/vizdocs/package.json:

{
  "name": "vizdocs",
  "version": "0.0.1",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "kev <[email protected]> (https://lineus.dev/)",
  "license": "WTFPL"
}


~/dev/jstgit/vizdocs $ mkdir modules
~/dev/jstgit/vizdocs $ touch ./modules/{one,two}.md
~/dev/jstgit/vizdocs $ echo '---\ndeps: [one]\n---\n' > ./modules/two.md
~/dev/jstgit/vizdocs $ cat !$
~/dev/jstgit/vizdocs $ cat ./modules/two.md
---
deps: [one]
---

~/dev/jstgit/vizdocs $ yarn add -D graph-docs-cli
yarn add v1.22.18
info No lockfile found.
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
success Saved lockfile.
success Saved 55 new dependencies.
info Direct dependencies
└─ [email protected]
info All dependencies
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
├─ [email protected]
└─ [email protected]
✨  Done in 1.51s.
~/dev/jstgit/vizdocs $ yarn graph-docs dev "./modules/*.md"
yarn run v1.22.18
$ /Users/lineus/dev/jstgit/vizdocs/node_modules/.bin/graph-docs dev './modules/*.md'
warning: Not a git repository. Use --no-index to compare two paths outside a working tree
usage: git diff --no-index [<options>] <path> <path>

Diff output format options
    -p, --patch           generate patch
    -s, --no-patch        suppress diff output
    -u                    generate patch
    -U, --unified[=<n>]   generate diffs with <n> lines context
    -W, --function-context
                          generate diffs with <n> lines context
    --raw                 generate the diff in raw format
    --patch-with-raw      synonym for '-p --raw'
    --patch-with-stat     synonym for '-p --stat'
    --numstat             machine friendly --stat
    --shortstat           output only the last line of --stat
    -X, --dirstat[=<param1,param2>...]
                          output the distribution of relative amount of changes for each sub-directory
    --cumulative          synonym for --dirstat=cumulative
    --dirstat-by-file[=<param1,param2>...]
                          synonym for --dirstat=files,param1,param2...
    --check               warn if changes introduce conflict markers or whitespace errors
    --summary             condensed summary such as creations, renames and mode changes
    --name-only           show only names of changed files
    --name-status         show only names and status of changed files
    --stat[=<width>[,<name-width>[,<count>]]]
                          generate diffstat
    --stat-width <width>  generate diffstat with a given width
    --stat-name-width <width>
                          generate diffstat with a given name width
    --stat-graph-width <width>
                          generate diffstat with a given graph width
    --stat-count <count>  generate diffstat with limited lines
    --compact-summary     generate compact summary in diffstat
    --binary              output a binary diff that can be applied
    --full-index          show full pre- and post-image object names on the "index" lines
    --color[=<when>]      show colored diff
    --ws-error-highlight <kind>
                          highlight whitespace errors in the 'context', 'old' or 'new' lines in the diff
    -z                    do not munge pathnames and use NULs as output field terminators in --raw or --numstat
    --abbrev[=<n>]        use <n> digits to display object names
    --src-prefix <prefix>
                          show the given source prefix instead of "a/"
    --dst-prefix <prefix>
                          show the given destination prefix instead of "b/"
    --line-prefix <prefix>
                          prepend an additional prefix to every line of output
    --no-prefix           do not show any source or destination prefix
    --inter-hunk-context <n>
                          show context between diff hunks up to the specified number of lines
    --output-indicator-new <char>
                          specify the character to indicate a new line instead of '+'
    --output-indicator-old <char>
                          specify the character to indicate an old line instead of '-'
    --output-indicator-context <char>
                          specify the character to indicate a context instead of ' '

Diff rename options
    -B, --break-rewrites[=<n>[/<m>]]
                          break complete rewrite changes into pairs of delete and create
    -M, --find-renames[=<n>]
                          detect renames
    -D, --irreversible-delete
                          omit the preimage for deletes
    -C, --find-copies[=<n>]
                          detect copies
    --find-copies-harder  use unmodified files as source to find copies
    --no-renames          disable rename detection
    --rename-empty        use empty blobs as rename source
    --follow              continue listing the history of a file beyond renames
    -l <n>                prevent rename/copy detection if the number of rename/copy targets exceeds given limit

Diff algorithm options
    --minimal             produce the smallest possible diff
    -w, --ignore-all-space
                          ignore whitespace when comparing lines
    -b, --ignore-space-change
                          ignore changes in amount of whitespace
    --ignore-space-at-eol
                          ignore changes in whitespace at EOL
    --ignore-cr-at-eol    ignore carrier-return at the end of line
    --ignore-blank-lines  ignore changes whose lines are all blank
    -I, --ignore-matching-lines <regex>
                          ignore changes whose all lines match <regex>
    --indent-heuristic    heuristic to shift diff hunk boundaries for easy reading
    --patience            generate diff using the "patience diff" algorithm
    --histogram           generate diff using the "histogram diff" algorithm
    --diff-algorithm <algorithm>
                          choose a diff algorithm
    --anchored <text>     generate diff using the "anchored diff" algorithm
    --word-diff[=<mode>]  show word diff, using <mode> to delimit changed words
    --word-diff-regex <regex>
                          use <regex> to decide what a word is
    --color-words[=<regex>]
                          equivalent to --word-diff=color --word-diff-regex=<regex>
    --color-moved[=<mode>]
                          moved lines of code are colored differently
    --color-moved-ws <mode>
                          how white spaces are ignored in --color-moved

Other diff options
    --relative[=<prefix>]
                          when run from subdir, exclude changes outside and show relative paths
    -a, --text            treat all files as text
    -R                    swap two inputs, reverse the diff
    --exit-code           exit with 1 if there were differences, 0 otherwise
    --quiet               disable all output of the program
    --ext-diff            allow an external diff helper to be executed
    --textconv            run external text conversion filters when comparing binary files
    --ignore-submodules[=<when>]
                          ignore changes to submodules in the diff generation
    --submodule[=<format>]
                          specify how differences in submodules are shown
    --ita-invisible-in-index
                          hide 'git add -N' entries from the index
    --ita-visible-in-index
                          treat 'git add -N' entries as real in the index
    -S <string>           look for differences that change the number of occurrences of the specified string
    -G <regex>            look for differences that change the number of occurrences of the specified regex
    --pickaxe-all         show all changes in the changeset with -S or -G
    --pickaxe-regex       treat <string> in -S as extended POSIX regular expression
    -O <file>             control the order in which files appear in the output
    --rotate-to <path>    show the change in the specified path first
    --skip-to <path>      skip the output to the specified path
    --find-object <object-id>
                          look for differences that change the number of occurrences of the specified object
    --diff-filter [(A|C|D|M|R|T|U|X|B)...[*]]
                          select files by diff type
    --output <file>       Output to a specific file

node:internal/errors:844
  const err = new Error(message);
              ^

Error: Command failed: git diff --name-only HEAD
warning: Not a git repository. Use --no-index to compare two paths outside a working tree
usage: git diff --no-index [<options>] <path> <path>

Diff output format options
    -p, --patch           generate patch
    -s, --no-patch        suppress diff output
    -u                    generate patch
    -U, --unified[=<n>]   generate diffs with <n> lines context
    -W, --function-context
                          generate diffs with <n> lines context
    --raw                 generate the diff in raw format
    --patch-with-raw      synonym for '-p --raw'
    --patch-with-stat     synonym for '-p --stat'
    --numstat             machine friendly --stat
    --shortstat           output only the last line of --stat
    -X, --dirstat[=<param1,param2>...]
                          output the distribution of relative amount of changes for each sub-directory
    --cumulative          synonym for --dirstat=cumulative
    --dirstat-by-file[=<param1,param2>...]
                          synonym for --dirstat=files,param1,param2...
    --check               warn if changes introduce conflict markers or whitespace errors
    --summary             condensed summary such as creations, renames and mode changes
    --name-only           show only names of changed files
    --name-status         show only names and status of changed files
    --stat[=<width>[,<name-width>[,<count>]]]
                          generate diffstat
    --stat-width <width>  generate diffstat with a given width
    --stat-name-width <width>
                          generate diffstat with a given name width
    --stat-graph-width <width>
                          generate diffstat with a given graph width
    --stat-count <count>  generate diffstat with limited lines
    --compact-summary     generate compact summary in diffstat
    --binary              output a binary diff that can be applied
    --full-index          show full pre- and post-image object names on the "index" lines
    --color[=<when>]      show colored diff
    --ws-error-highlight <kind>
                          highlight whitespace errors in the 'context', 'old' or 'new' lines in the diff
    -z                    do not munge pathnames and use NULs as output field terminators in --raw or --numstat
    --abbrev[=<n>]        use <n> digits to display object names
    --src-prefix <prefix>
                          show the given source prefix instead of "a/"
    --dst-prefix <prefix>
                          show the given destination prefix instead of "b/"
    --line-prefix <prefix>
                          prepend an additional prefix to every line of output
    --no-prefix           do not show any source or destination prefix
    --inter-hunk-context <n>
                          show context between diff hunks up to the specified number of lines
    --output-indicator-new <char>
                          specify the character to indicate a new line instead of '+'
    --output-indicator-old <char>
                          specify the character to indicate an old line instead of '-'
    --output-indicator-context <char>
                          specify the character to indicate a context instead of ' '

Diff rename options
    -B, --break-rewrites[=<n>[/<m>]]
                          break complete rewrite changes into pairs of delete and create
    -M, --find-renames[=<n>]
                          detect renames
    -D, --irreversible-delete
                          omit the preimage for deletes
    -C, --find-copies[=<n>]
                          detect copies
    --find-copies-harder  use unmodified files as source to find copies
    --no-renames          disable rename detection
    --rename-empty        use empty blobs as rename source
    --follow              continue listing the history of a file beyond renames
    -l <n>                prevent rename/copy detection if the number of rename/copy targets exceeds given limit

Diff algorithm options
    --minimal             produce the smallest possible diff
    -w, --ignore-all-space
                          ignore whitespace when comparing lines
    -b, --ignore-space-change
                          ignore changes in amount of whitespace
    --ignore-space-at-eol
                          ignore changes in whitespace at EOL
    --ignore-cr-at-eol    ignore carrier-return at the end of line
    --ignore-blank-lines  ignore changes whose lines are all blank
    -I, --ignore-matching-lines <regex>
                          ignore changes whose all lines match <regex>
    --indent-heuristic    heuristic to shift diff hunk boundaries for easy reading
    --patience            generate diff using the "patience diff" algorithm
    --histogram           generate diff using the "histogram diff" algorithm
    --diff-algorithm <algorithm>
                          choose a diff algorithm
    --anchored <text>     generate diff using the "anchored diff" algorithm
    --word-diff[=<mode>]  show word diff, using <mode> to delimit changed words
    --word-diff-regex <regex>
                          use <regex> to decide what a word is
    --color-words[=<regex>]
                          equivalent to --word-diff=color --word-diff-regex=<regex>
    --color-moved[=<mode>]
                          moved lines of code are colored differently
    --color-moved-ws <mode>
                          how white spaces are ignored in --color-moved

Other diff options
    --relative[=<prefix>]
                          when run from subdir, exclude changes outside and show relative paths
    -a, --text            treat all files as text
    -R                    swap two inputs, reverse the diff
    --exit-code           exit with 1 if there were differences, 0 otherwise
    --quiet               disable all output of the program
    --ext-diff            allow an external diff helper to be executed
    --textconv            run external text conversion filters when comparing binary files
    --ignore-submodules[=<when>]
                          ignore changes to submodules in the diff generation
    --submodule[=<format>]
                          specify how differences in submodules are shown
    --ita-invisible-in-index
                          hide 'git add -N' entries from the index
    --ita-visible-in-index
                          treat 'git add -N' entries as real in the index
    -S <string>           look for differences that change the number of occurrences of the specified string
    -G <regex>            look for differences that change the number of occurrences of the specified regex
    --pickaxe-all         show all changes in the changeset with -S or -G
    --pickaxe-regex       treat <string> in -S as extended POSIX regular expression
    -O <file>             control the order in which files appear in the output
    --rotate-to <path>    show the change in the specified path first
    --skip-to <path>      skip the output to the specified path
    --find-object <object-id>
                          look for differences that change the number of occurrences of the specified object
    --diff-filter [(A|C|D|M|R|T|U|X|B)...[*]]
                          select files by diff type
    --output <file>       Output to a specific file


    at checkExecSyncError (node:child_process:828:11)
    at execSync (node:child_process:899:15)
    at getDotfile (/Users/lineus/dev/jstgit/vizdocs/node_modules/graph-docs-cli/bin/dotfile.js:42:61)
    at async startServer (/Users/lineus/dev/jstgit/vizdocs/node_modules/graph-docs-cli/bin/startServer.js:36:19) {
  status: 129,
  signal: null,
  output: [
    null,
    Buffer(0) [Uint8Array] [],
    Buffer(7268) [Uint8Array] [
      119,  97, 114, 110, 105, 110, 103,  58,  32,  78, 111, 116,
       32,  97,  32, 103, 105, 116,  32, 114, 101, 112, 111, 115,
      105, 116, 111, 114, 121,  46,  32,  85, 115, 101,  32,  45,
       45, 110, 111,  45, 105, 110, 100, 101, 120,  32, 116, 111,
       32,  99, 111, 109, 112,  97, 114, 101,  32, 116, 119, 111,
       32, 112,  97, 116, 104, 115,  32, 111, 117, 116, 115, 105,
      100, 101,  32,  97,  32, 119, 111, 114, 107, 105, 110, 103,
       32, 116, 114, 101, 101,  10, 117, 115,  97, 103, 101,  58,
       32, 103, 105, 116,
      ... 7168 more items
    ]
  ],
  pid: 22096,
  stdout: Buffer(0) [Uint8Array] [],
  stderr: Buffer(7268) [Uint8Array] [
    119,  97, 114, 110, 105, 110, 103,  58,  32,  78, 111, 116,
     32,  97,  32, 103, 105, 116,  32, 114, 101, 112, 111, 115,
    105, 116, 111, 114, 121,  46,  32,  85, 115, 101,  32,  45,
     45, 110, 111,  45, 105, 110, 100, 101, 120,  32, 116, 111,
     32,  99, 111, 109, 112,  97, 114, 101,  32, 116, 119, 111,
     32, 112,  97, 116, 104, 115,  32, 111, 117, 116, 115, 105,
    100, 101,  32,  97,  32, 119, 111, 114, 107, 105, 110, 103,
     32, 116, 114, 101, 101,  10, 117, 115,  97, 103, 101,  58,
     32, 103, 105, 116,
    ... 7168 more items
  ]
}

Node.js v17.6.0
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
~/dev/jstgit/vizdocs $

Open-sourcing XState's Docs?

Hello! First off, thanks for working on this project and sharing it -- it's a really great idea!

In the "Integrations" section of the readme, you mention plans to open-source the way XState's docs turn modules into pages with a custom syntax (I'm guessing that's what https://graph-docs.vercel.app/ is?). Since it looks like you're no longer with Stately, do you have any idea as to when or if this well ever be released?

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.