tchayen / markdown-links Goto Github PK

Would be nice if you could export the generated link graph like edeno/d3-save-svg does.

Hi,

if I reference a file using the long/default markdown syntax (eg. [My Link Text](./FileName.md) to reference FileName.md), the file will be discovered and shown correctly in the graph.

However, if I reference a file with a space in the filename, it's common practice to encode the space with %20, like [My Link Text](./File%20Name.md) for for File Name.md. However, those links are not discovered by markdown-links and are therefore not shown in the graph.

Any possibility to fix this?

PS: Thanks for the nice extension! :-)

I have a project directory that includes a node_modules directory with a couple thousand markdown files. While I believe that a few thousand markdown files should be a doable task, in the short term, is there perhaps a way I could tell markdown-links to ignore some folders? If I move this folder out of my directory tree and restart, it's able to handle the remaining 400 or so files admirably.

In fact, the whole Foam stack, including this repo and the vscode-markdown-notes extension, can then successfully handle my ~450,000-file Dropbox (with only those 400 markdown files (although, for some reason, only 285 files are reported in the corner of the graph view), which Zettlr can't yet do. (Although, some tasks like finding backlinks, are a bit slow.)

Since two days ago I get an error when commanding Markdown Links: Show Graph:

Command 'Markdown Links: Show Graph' resulted in an error ([UriError]: Scheme contains illegal characters.)

I use the "markdown-links.fileIdRegexp": "(?<=^#).+$", setting which I got from here. My folder has ~1.500 files

I have an existing set of notes where I have my page titles defined in frontmatter (the notes are used in gatsby-theme-brain), like this:

---
title: Drafts
aliases: ["drafts"]
---

Would you be open to a PR that checks the frontmatter for a title, but falls back to the first h1 if not found?

Here's a screen recording of the issue. You can see that "test-page" gets duplicated and the graph gets really bizarre. I have to close and re-open the graph for it to work again. (zoom in to get rid of the odd grid that is showing)

I think it would be nice to have a box containing all the related backlinks for a file. It would work just the same as the other backlinks extensions that use the [[wiki-links]], but then with normal hyperlinks. The current extensions in Visual Studio Code only support backlinks for this Roam-like [[wiki-links]] approach.

When using the obsidian appearance, if there are more than 8 child nodes under the node, some of the child nodes will be located on the line between other nodes.
This problem does not occur with the default style.

When there are 9 child nodes

When there are many child nodes

This is syntax used by this extension which seems to be a more developed/advanced interlinked markdown extension than many others. I'm not sure if this syntax is being used anywhere else but it seems like a good way of linking notes this way while still being able to provide a title.

Currently this extension is not availbale for VSCodium. To correct that, you just need to follow this guide.

Hello,
@tchayen Thanks a lot for this project ! I come from Foam (issue foambubble/foam#2).

I have some main Markdown documents that in the end are the main nodes of the graph. It can be a widely used idea or a definition.
I would like to highlight these nodes in the graph to quickly see how the graph is organised and where are the main nodes.

A feature i'd like to see is color / icons on graph node :
When my node is a main node, i want to highlight it (maybe clicking on the node and changing the color).

The extension is in desperate need of automated testing. I will soon start implementing them.

Some ideas that come to my mind:

• how given side effect (addition of a file, removal of a file, alteration of a link, alteration of a title) influences rendered graph,
• what entries to nodes and edges arrays are caused by parsing of given file,
• Mini integration tests: what graph is generated by given directory.
• Snapshot tests: what SVG is generated by given graph.

Problem

Currently, we use vscode.workspace.findFiles('**/*.md') to find all fiels that we are about to parse (parse.ts)

This finds files in the whole workspace which might not be desirable. My particular use case is that I've got two folders in my workspace:

• permanent notes
• reference notes

The permanent notes are about a particular topic and the reference notes are notes taken from a book or an article. permanent notes are almost always linking to reference notes (i.e. saying that I learnt that particular information in book XYZ).

The current parsing mechanism results in showing huge clusters of notes all linked by reference note (a book or an article) whilst I'm only interested in the connections between permanent notes.

Suggested solution

We could allow users to configure the findFiles expression. So instead of

parse.ts

const files = await vscode.workspace.findFiles(
    `**/*{${(getFileTypesSetting() as string[]).map((f) => `.${f}`).join(",")}}`
  );

we would use the configured value and only if that's missing, we would default to the current expression:

const searchExpression = getConfiguration("searchExpression");
const defaultExpression = `**/*{${(getFileTypesSetting() as string[]).map((f) => `.${f}`).join(",")}}`;
const files = await vscode.workspace.findFiles(searchExpression || defaultExpression);

This would be the most performant solution, but we could as well introduce some allow/deny list that would be used when we iterate through the findFiles() result:

for (const file of files) {
    const hiddenFile = path.basename(file.path).startsWith(".");
    if (!hiddenFile) {
      promises.push(fileCallback(graph, file.path));
    }
  }

How does this sound @tchayen ?

Hi, I reported the Open Graph command not working in the Foam discord the other day with an error message saying there was an illegal character causing the error.

I found the link causing it, it was scraped from a web page with Notion (I exported all my Notion data into my own workspace)

This worked on the previous version of Foam but no the latest (0.3.0)

Here's the link, the angle brackets were causing it but it was weirdly nested:

[create a free Cloudinary account](<[https://cloudinary.com/users/register/free](https://cloudinary.com/users/register/free%5C)>).

Like the title says it's an edge case, just an FYI

Thanks for the awesome Extension 🙏

When viewing files from my Foam workspace using Gistpad, I am ubable to view the graph.
It is showing,

Is there any workaround for this?

I often utilize rmarkdown formatted documents. Would it be possible to include these as links in the graph view, perhaps through a settings parameter?

The following setting gets markdown preview to work along with backlinks in vscode-markdown-notes, but doesn't seem to work for the graph view in the markdown-links extension:

  "files.associations": {
    "*.rmd": "markdown"
  },

Include other Files & URLs linked to

Example:

[MyPDF](../MyPDF.pdf)
Should appear as Node with the name MyPDF in the Graph

[MyLink](http://abc.my.link.com)
Should appear as Node with name MyLink in the Graph

Loving your work on this extension!

It would be cool if a new document could get added to the graph automatically.

Consider a use case:

1. I am writing a document while the graph is open
2. I want to create a new document, called LInk
3. I type [Link](link.md)inthe current document and create the file in the directory (possibly via an automation)
4. I give the new file a # Heading

Expected outcome: New document "Link: shows up in the graph.

This is definitely a nice to have, just logging it here.

I'd really love the option of directional links - it'd help my notes a lot. Any chance of it being brought in? I appreciate that the README says explicitly that direction is not given, is this is choice based on usage, or a note taking "schema"?

Triple square bracket links represent "folgezettel" links. It is supported by Neuron: https://neuron.zettel.page/linking.html

Some vscode extensions, like https://github.com/svsool/vscode-memo#memo, have no problem supporting both variations. However, this extension works only with [[..]]. It entirely ignores [[[..]]] links.

If this little feature gets added, then neuron users would be able to use this great extension!

On existing plugin, I always need to click on the node to switch to a different editor to view/edit the content of the file.
It would be convenient if I could see the details of all files in only one view.

So，I need a graph like a mindmap editor:

• Show other parts of markdown (not just # H1 heading)，such as：
  • Headers: ## H2, ### H3, ...
  • Text (Bold, Italic, ...)
  • Ordered Lists
  • Unordered Lists
  • Images
  • Links (Markdown, wiki, ...)
  • Code
  • ...
• Can globally control the level of display
• Collapse/Expand a node's subitem by hand
• hover on a node will show the content
• focus mode
  When the node of a file is selected(using ctrl+click or Double-click),
  it will re-layout to make it in the center and can hide/show unrelated nodes
• edit content
  Edit content directly in the graph maybe difficult，use VSCode's editor temporarily

eg.

[Horizontal genetic transfer](horizontal-genetic-transfer.md) 

What is an adjacency list?

An adjacency list is a data structure used to represent a (typically directed) graph. There are a few different ways you can model it, but I am thinking of something like this:

interface Node {
  id: string;
  // This is an array of IDs of nodes that this node links to
  links: string[];
  // Depending on what kind of traversal you need to do, can also have backlinks
  backlinks: string[];
}

// The string keys are the IDs of the nodes, so it is a map of ID -> Node
type AdjacencyList = Record<string, Node>;

The nice thing about this data structure is it allows for O(1) lookup of a node and its neighbors.

Why bother?

I believe it would make working with the graph data easier. I have mostly been thinking about the idea of a "partial graph" where you only show the currently focused node and its immediate neighbors. For that, you need to do a breadth-first traversal from the current node. With the current data model (arrays of nodes and arrays of edges), you have to do a lot of searching to find all the necessary nodes. I'm not totally sure on the time complexity there but I think its something like O(n^2) where n is the number of nodes.

I have a rough start of this implementation going locally but it needs some work. I mostly wanted to open this issue to discuss it and see if this is a direction worth exploring.

Implementing this also requires a function to convert the data from an adjacency list into the nodes/edges arrays that d3 expects. We can do a basic graph traversal for this.

Hi, I started trying out foam, but I wasn't able to get the graph working correctly. It even won't work with the markdown-links examples.

Did I miss something?

Version: 1.50.1 (user setup)
Commit: d2e414d9e4239a252d1ab117bd7067f125afd80a
Date: 2020-10-13T15:06:15.712Z
Electron: 9.2.1
Chrome: 83.0.4103.122
Node.js: 12.14.1
V8: 8.3.110.13-electron.0
OS: Windows_NT x64 10.0.19041

Markdown links v0.8.0

Apparently proper support for removing files was lost in action.

This one will currently be blocked as first I want to focus on writing tests.

I hope there is another display mode, with the current document as the center, the left side represents those files that reference the current file, and the right represents those files referenced by the current file.

Like this:mindmap

VS Code correctly links [[foo bar]] to foo-bar.md but this isn't currently detected by the graph or properly rendered in the produced HTML. Similarly, Foo correctly links to foo.md, but this link isn't detected either.

For regular markdown links [a_link](a_file.md), I think it is necessary to add backlink support

To show which nodes are the 'hubs' of the graph, it might be nice to be able to display nodes with lots of edges as larger, or make node size a function of connection count (maybe this falls under "Configurable parameters of graph simulation." in the roadmap?)

It'll be very convenient to open a not by double-clicking (or cmd+click) on the graph node.

I've just started using Foam, and links work fine (and are shown) if the .md resides in the workspace directory, however I need to add other directories using the "Add Folder to workspace" command. When I do this and add a reference a link between a .md in the original workspace and one in the newly added workspace it isn't shown in the graph

I don’t like to have all files I want to add in the graph have the first heading the same as the title, especially longer documents, or documents I would want to publish as or send by copying and pasting it (Ex. Emails). It would be great if an option was available to select between using the file names for generating the graph and first level heading for generating the graph

I'd love to be able to have some links to an idea which doesn't need a document to be created for it to appear on the graph. This would allow quick thoughts and links to be jotted and the effect would show a graph of overlap of ideas rather than documents. This would enable shorthand brainstorming.

I propose that if even if a link [[new_car]] is created, it would appear on the graph even if a file does not exist for it. If it is clicked, in either the graph or in the document, then a file is created.

I am opening a common issue to discuss various systems for linking and discovering titles that I am aiming to support at some point in the extension.

I would love to have some discussion here first to find out what people expect and what are the chances that those expectations can live together.

Linking

As a reminder, there are only two ways to link files in Markdown: links and references. All the other methods like [[Linking by title]], linking by some arbitrary ID ([[20200715001]], <g04d0sa2>) are not part of the CommonMark standard and will be present in Markdown Links only as configurable options.

Possible options

• [[Wiki link]] maps to kebab case filename of the same string #21 (Some kind of a title links to some-kind-of-a-title.md if such file exists).
• [[Wiki link]] maps to title (see Title discovery below).
• [[Wiki link|Alias]] (as the two above).
• Zettlr-style links (already supported by default, will be moved out to an option).
• Neuron-style links (<iD0f4f1l3>). Deprecated.
• [[[Folgezettel link]]] used in Neuron and described here.
• Foam-like references ([Reference]: file-name instead of [Reference]: file-name.md)

Title discovery

I have used an assumption that title of the file will be the first heading of rank 1, but of course it is not true for everyone.

What comes to my mind is settle on several ways to discover a title, give them some default precedence and make it configurable.

An example default can be:

1. Frontmatter title
2. Rank 1 header if there is only one
3. Filename if globally unique
4. Filename with the enclosing directory

What's next?

Please comment the issue with your idea about linking if you have one. Please share your feedback about the the things I have mentioned above. As soon as we find a way to support all alternatives (or come to conclusion that it is out of scope) we can start implementing support for those.

I have an absolute path links, that works fine in VS Code, but the plugin does not detect them as links.

Example like this:

<!-- first.md -->

# First

Hello world!

<!-- some/long/path/second.md -->

# Second

[here](/first.md)

We can replace root as workspaceFolder.

P.s. I want first to discuss this feature, and if it is good than can implement. :)

After installing the extension in VScode, the command show graph is not available. Am I doing it wrong?

Version: 1.46.1
Commit: cd9ea6488829f560dc949a8b2fb789f3cdc05f5d
Date: 2020-06-17T21:17:14.222Z
Electron: 7.3.1
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Darwin x64 19.5.0

In reference to foambubble/foam#5

@tchayen @jevakallio

The ideas I had in mind for the implementation are below.

I would like to specify folders that Markdown-Links searches in the settings.

Once the folders are selected, by default, the graph shows the relationship between all files in the identified folders. When a file is open, an option in the graph should be available to:

1 Show links from all folders
2 Show links from current folder only
3 Show links from current folder and its subfolders
4 Memorize graph
5 Show memorized graph

The graph will keep changing dynamically in options 2 and 3 to reflect the folder of the active file and focus on the active file.

It would be great to link to a particular section within another md file! This way you can be very precise about the link and makes navigating a lot better

Hello,
Is there a way to autofocus the graph on the selected markdown?
I have a very big graph and it is hard to find the blue dot, an autofocus option would be really nice

Steps to reproduce (assuming starting with the value "active"):

1. Open a markdown file
2. ctrl+ship+p -> Show Graph
3. Observe the graph shown in the current column
4. Close the graph tab
5. Change the setting to "two"
6. ctrl+ship+p -> Show Graph
7. Observe the graph still shown in the current column
8. Restart VS Code
9. ctrl+ship+p -> Show Graph
10. Observe the graph correctly shown in column two

please have message point to the error to be fixed; or, escaped.

Most likely I used a 'poor' 'self note' method, causing this.

Hey there, thanks for this extension! It's been super useful in helping me visualise a pretty large link network.

I have noticed that between every Show Graph, the resulting graph from markdown links is always completely different. This isn't usually a problem, it's interesting to see your data in different ways, but the "default" graph type can often overlay lines in a way that some other edges or vertices are completely obscured. It's also hard to grab screenshots for someone when you have to re-run show-graph, then zoom in on the area you want, and repeat until it displays it in a representative way.

This is probably a hard feature, but do you think that perhaps the default graph algorithm can benefit from some kind of determinism that goes through some rebalancing like in the d3.js force testing ground? Perhaps with exposed force parameters etc.

Do you think this is a possible thing to do statically?

Hello everyone,

I'm using markdown-links as a part of foam to organize my thesis work.

When writing summaries of articles, if I use the article title it becomes difficult to read the graph. I could use something else but organization wise it will just lead to entropy.
I currently have only one node like it but it is easy to see it won't scale.

My suggestion, to keep the parser simple (using the first, single # sign as title) is to allow cutting or wrapping the title.
I believe this to be a simple change and I do not mind implementing it.

In the line above, we can just add substring(0, limit).concat("...") where limit is defined by the user. In the case that title != null of course.

Like the gif above, when I use the ID-based links feature, what I expected is navigate to the target file, but a new file named with the ID is created, is it a bug? or it is the designed behavior?

thanks :)

• Issue Type: Bug
• Extension Name: markdown-links
• Extension Version: 0.7.0
• OS Version: Windows_NT x64 10.0.19042
• VSCode version: 1.47.3

⚠️ We have written the needed data into your clipboard. Please paste! ⚠️

{
	"messages": [],
	"activationTimes": {
		"codeLoadingTime": 12,
		"activateCallTime": 0,
		"activateResolvedTime": 1,
		"activationReason": {
			"startup": false,
			"extensionId": {
				"value": "tchayen.markdown-links",
				"_lower": "tchayen.markdown-links"
			},
			"activationEvent": "onLanguage:markdown"
		}
	},
	"runtimeErrors": [
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		},
		{
			"name": "Error",
			"message": "Webview is disposed"
		}
	]
}

In the library I am using D3 force for force-directed graph drawing which is a way of generating graphs that look good enough in most cases and is easy enough to implement (or to come up with in the first place).

Unfortunately the more I use it, the more problems with it I encounter:

1. Graph is non-deterministic. Each time, due to randomness in its behaviour, entirely different graph might be generated. This one should be possible to solve in current approach (however initial tries failed).
2. Lack of non-invasive additions. Each time a node is added, the whole graph is regenerated (which bumps into problem 1). Ideally, it should stay exactly the same and only run a portion of simulation ticks to give the node 'correct' (in terms of distance to other nodes) place. This one should be possible to solve in current approach.

I think it would be good to scale the size of the node in relation to the size, to give it more importance. Maybe set the scaling factor as an option?

Hello,

This is a great project and helps me visualize the links between various Markdown files in a directory. I've recently been using a different extension called Notes that allows me to look at my Markdown notes without switching workspaces by creating and displaying a separate "Notes" directory. This means that I could be coding using a workspace while also adding Markdown content in that directory.

Markdown Links just seems to use the current working directory of the current workspace. Is it possible to add an option to change the working directory so that Markdown notes can be visualized in this directory?

Thank you!

When having a lot of markdown files with lots of links, things can get messy and the overview the graph adds declines.

More often than not, you're interested in the relations of the current file.
It would be cool to be able to filter node and only show nodes that are connected to the current file by N edges (where N is configurable in the graph view).

That way you can remove most clutter in the graph by setting N to e.g. 1.