Giter VIP home page Giter VIP logo

obsidian-regex-pipeline's Introduction

Regex Pipeline

(Sharing rulesets in Discussions is welcomed)

Regex Pipeline is an Obsidian plugin that allows users to setup custom regex rules to automatically format notes, this is especially useful in scenerios like building personal knowledge database, because you often clip webpage from same sources.

Usage

Mr. Partan provided a nice writeup about how to use the plugin, I recommend it if you are not familiar with regex or software usage. (September 2021, v1.0.9)

First, enable the plugin, a file named index.txt should be created at .obsidian/regex-rulesets/. Due to how Obsidian protects your disks, you have to specify what ruleset files are there to be read, that's why we need a index file.

Starting from version 1.0.8, you can add rulesets through the + button in the menu, but you still have to go to .obsidian/regex-rulesets/ and modify the files you want to edit/remove, because it's hard to provide good editing experience that rivals common editors like VSCode.

Starting from version 1.1.0, you can apply rulesets via the right-click menu. The available option count can be adjusted in the settings.

Starting from version 1.2.0, quick rulesets (mentioned right above) can be invoked through Obsidian's command system when "Quick Commands" are set to above 0 in the settings.

Writing Rulesets

A ruleset contains one or more rule, the format looks like:

:: Any "SEARCH" becomes "REPLACE"
"SEARCH"->"REPLACE"

✅ These work:

"SEARCH"->"REPLACE"
"SEARCH"
->"REPLACE"
"SEARCH"
->
"REPLACE"
"SEARCH"->
"REPLACE"

❌ These do NOT work (Empty line inbetween not allowed; Nothing except new line is allowed right before and after the ->)

"SEARCH"

->
"REPLACE"
"SEARCH"->

"REPLACE"
"SEARCH"
->

"REPLACE"

Multi-line replacement string:

"SEARCH"->"REP
LACE"
:: Any "SEARCH" becomes "REP
:: LACE"

Regex Flags

By default, gm (multiline) flag is appended to the SEARCH regex, you can overwrite this by providing your own flags, for example, use gmu flag in this way:

"SEARCH"gmu->"REPLACE"

Noted that gm flags are bascially neccessary for this plugin to be useful, you seldom wants to replace only 1 occurances or operate on a note only contains 1 line.

Replace With Nothing

"SEARCH"->""
:: Any "SEARCH" becomes ""

Basically this is how we remove matched content.

Indexing

Rulesets must be saved in .obsidian/regex-rulesets/, and must be included in the index.txt, one file per line. The order of the lines determines the displaying order of the rulesets in the menu.

Applying Rulesets

Press the sidebar button of this plugin to show the rulesets menu, select your ruleset, and it'll be applied.

The menu is available as a command so you can also bind it to a shortcut.

Note: The plugin support applying rules to selection, if anything is selected, only selection is modified!

Examples

NumberToAlphabet .obsidian/regex-rulesets/number-to-alphabet.txt

"1"->"A"
"2"->"B"
"3"->"C"
"4"->"D"
"5"->"E"
"6"->"F"
"7"->"G"
"8"->"H"
"9"->"I"

Table_c2 This ruleset help you transform selected content into a table of 2 columns! Every 2 non-empty line will form a row.

.obsidian/regex-rulesets/Table_c2

"^(.+)$\n\n^(.+)$"->"| $1 | $2 |"

Linebreak with br tag This ruleset replace all newline with <br>.

"^(.+)\s+?\n(?=^.+)"->"$1<br>"

Take a look in samples folder for more examples, including a very complex one like the above gif!

Recommendations

FAQ

My ruleset file doesn't work,The notification says there's 0 replacement, but I'm sure the format is correct.

It's possible that your ruleset file is in non-UTF8 encoding, this happens with some editor applications, please refer to #12.

obsidian-regex-pipeline's People

Contributors

geekpro101 avatar no3371 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

obsidian-regex-pipeline's Issues

Scroll restart when applying a regex

When you need to apply a regular expression to a note, which is a bit long, the page scrolling restarts, returning to the beginning of the note. This is quite annoying. But I want to thank you for making this plugin so good.

Refusing to process one example in docs

This is a fantastic plugin - thank you so much for writing it

I have one curious anomaly

All my own regexes work fine, and all the other examples work fine, but one example in the docs throws up an error. It is this one.
Whatever I do it says "which contains 0 regex replacements" and does nothing at all. I wondered whether one of the characters might be a unicode character that is not what it seems, so retyped it from scratch, but no matter what I do this attempt to pre-pend something to lines (from the docs) does not work

"^(.+)$"->">$1"

Why I cannot delete existing regex rules?

I have turned on the setting——save rules in vault. And the regex-rulesets folder has only an empty file index.txt.
But there are still 2 old regex rules which I created before I turned on the setting, and I cannot delete them.
image

I have tried restarting my Obsidian or my computer, but they all failed.

Rulesets can't be included in commands

There are only two entries in commands

  1. RegEx Pipeline: Undifined (not working)
  2. RegEx Pipeline: Apply Ruleset

I would like to add RegEx Rulesets to create Macros using another plugin.
But it can't be implemented without inputs in commands.

Regex detected however not applied

Hello!

I am having trouble with embedding YT links. Could you please explain why this regex works

"https:\/\/www\.youtube\.com\/watch\?v="->"https://www.youtube.com/embed/"
"&ab_channel=[A-Za-z0-9.]+"->""

but this one does not

"https:\/\/www\.youtube\.com\/watch\?v=([A-Za-z0-9]+)&ab_channel=[A-Za-z0-9.]+"
->
"https://www.youtube.com/embed/$1"

The latter says that regex has been found, however, no replacement is actually performed. Thank you!

Extension of Rulesets

Rulesets are defined in .txt format, but without extension. For saving, please add the extension ".txt" because otherwise you have to rename it manually on MacOS to edit it.

Add scrolling to list of rulesets

Love your plugin. I use it all the time. However, I now have a fairly long list of rulesets, and the ruleset selection window within Obsidian does not scroll. Would it be possible to add scrolling? Cheers.

Settings

  • Context menu option count
  • Toggle: store plugin data inside or outside of .obsidian

Feature Request: Extract Regexp Result

Hi,
thank you for bring us such a nice tool.
obsidian-regex-pipeline is the only tool that can deal with the content within a note using regexp.

I use the note to store the logs. Sometime, I want to find the text with certain pattern within the log. However, rather than replace those matching text with some new text, I want to copy them into the clipboard.

Now, there is an idea: is it possible to extract each of the matching text(or strings) and append them one by one into the clipboard rather than replace them one by one with the given text in the template?

I think this will be very useful.

Looking forward to your reply.

regex pipeline doesn't working correctly

Issue description

In obsidian regex pipeline plug-in, overwriting regex flags manually doesn't works properly. giving multiple flags will either show applied 0 regex replacement or will show applied 1 regex replacement without actually replacing anything. for example when using 'line break with br tag' regex ruleset, it replaces all line breaks with br tag including line breaks from inside of code block. and there is no option to find a match and find sub matches from that match and replace those sub matches ('SEARCH' --> 'SEARCH' --> 'REPLACE'), nor a option to disable regex from replacing inside of code blocks.

Steps to reproduce the issue

  1. try this regex pattern "."gms->"1" and you will see it doesn't changes anything at all. and even just giving single s flag will not work as expected as it doesn't even replaces the newline characters. but after removing all the flags it will work correctly. and character escape (like \n \t) doesn't even work inside the 'REPLACE', thus having no choice, had to use back reference

What's the expected result?

  • after matching the whole code block, i used lookaround with capturing group to capture all the br tag inside the code block. but inside of lookaround making quantifier lazy with ? quantifier somehow breaks the regex pattern, but works correctly in the regex101 site. thus can't replaces all the br tags with new line character

What's the actual result?

  • if regex worked correctly it would had replaced all the br tag from inside the code block back to the new line character

Additional details / screenshot

regex problem

Feature suggestion: Flag to disable replacements unless some text is selected

I use Plugin commands in the chain of automated commands (a macro) by QuickAdd Obsidian plugin.
Sometimes, when the macro fails to select the line for which the replacements should be limited, the Regex-Pipeline plugin introduces replacements in the whole document, which, unless timely recognized, becomes a major "pain-in-the-butt" to fix.

For example, the following workflow allows creating a callout with term-definition lists, like the following:

> > [!INFO]- Used abbreviations 
> > - term_1: definition_1
> > - term_2: definition_2
> > - term_3: definition_3

with the following QuickAdd workflow:
See QuickAdd macro definition image

and the following QuickAdd template:

> > [!INFO]- Used abbreviations 
> > - {{VALUE: Abbreviations}}

and the following input for Abbreviations: "term_1, defition_1; term_2, definition_2; term_3, definition_3."

and the following Regex-Pipeline ruleset:

:: Replace ; with a new line and a dash for list item ("-") inside a nested callout
";\s*"->"
> > - "

:: Replace the first in newly created lines comma (",") with the colon (":") 
:: (separator between term and its definition)
"^([^,]+),\s*"->"$1: "


:: Remove final dot/period 
"\.\s*$"->""

I recognize that part of the problem is lack of mechanisms in QuickAdd to select the whole text inserted with "{{VALUE}}".
As a workaround, I insert text as a single line. But when there is an empty line in the end of input, the QuickAdd will only select the last empty line, and the Regex-Pipeline plugin will understand this as a need to make replacements in the whole document, which is not the intention.

I want to have a configuration option to limit all operations to only the selected text and just signal in the console if nothing is selected.

Can rulesets be exposed as commands?

I would love to be able to add running a regex ruleset to a macro that I have, but there is no way for me to invoke the plugin in a way that would specify the ruleset (already). You don't need to add icons or anything, but exposing configured rulesets as commands would give a lot of flexibility to use rulesets with other plugins (macros) or directly from slash commands..

Diacritics are not matched

I am trying to insert a mention "@" in front of a first name in a string inside square brackets, but I deal with a lot of Portuguese folks, and when the name has a diacritic, such as an accent or a tilde, I can't get any of these strings to match the first name:

"(\s\[)(\w*\s)"->"$1@$2"

"(\s[\w*,\s)(\w*\])"->"$1@$2"

"(\s\[)(\p{L}*\s)"->"$1@$2"

"(\s\[\w*,\s)(\p{L}*\])"->"$1@$2"

Any ideas?

Stop scrolling to beginning after applying regex

Thank you for a great plugin.
Is there any way to apply the regex without it scrolling back to the beginning of the text afterwards.
I have to paste a lot of data and then apply rules to this data. After every regex I have to scroll down the page again, back to where I was working.
Ideally I could apply a regex and my cursor would remain where it was.
Even better, would be the ability to highlight a portion of text and select regex rule to apply from right click menu

Thanks again

Cannot apply ruleset to Kanban view

If I want to apply a ruleset to a kanban board made with the Kanban plugin, I get an error saying "No active Markdown file!". I have to go into the Markdown view, apply the ruleset, and then switch to the kanban via again.

Would it be possible to directly apply rulesets to kanban boards?

Alternate location for saving rules

I can see that there's an option for saving rules to the vault. I'm wondering whether there's a way to save it somewhere else?
I know saving it in the plugin folder can cause sync issues, but having it in the vault is also not ideal.

The obsidian icon folder plugin was also experiencing the same issue and a recent update now allows us to set our own path and they recommend .obsidian/icons. (see more here: FlorianWoelki/obsidian-iconize#52)

Can we not save the rules to .obsidian/regex-rules?

Replace doesn't work except for examples

It works like a charm for example regex, but I haven't got it to work with my own regexp, even though I tested it at regexp101.com, and another plugin Regex Search and Replace also works as expected.

Either of these doesn't work and always had 0 replacement:
"[[" gm -> "any" X
"[[" gm -> "any" x

below attempts didn't solve the issue either:

  • copy the expression to an existing example ruleset
  • add new ruleset from obsidian
  • restart obsidian

appreciate any idea, a good and useful plugin.
Mac OS
Obsidian 0.15.9, Blue Topaz theme

New line character in the replacement side

Could you please provide a sample that has inserts a new line (\n) character on the replacement side. For example if I want to split a timestamp 01:30 to

01
30

how do I do it?

I tried the following, but didn't work:

  "(\d{2}:\d{2})"->"$1\n$2"
  "(\d{2}:\d{2})"->"$1\r\n$2"
  "(\d{2}:\d{2})"->"$1`\n`$2"

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.