Enabling the extension combines consecutive whitespace into single spaces about unprompted HOT 3 CLOSED

Hi,

Regarding your examples, the [get test] block is not interpreted as a shortcode because Unprompted views it as being nested within a non-existent shortcode (1girl). Since [1girl] is not a valid shortcode, Unprompted assumes you want the entire block printed as a literal.

What you can do in this case is simply use a backtick ` to escape the outermost shortcode:

[set test].5[/set]

`[1girl : seawater : [get test] ]

Result:

[1girl : seawater : 0.5 ]

I will be the first to admit that this is not the most intuitive form of escaping code, but it is a limitation of the shortcode library I'm using (for now anyway.) It does work though, and it might allow our extensions to play well with each other. Feel free to run some tests on your end.

Let me comment on a few of the other points you brought up:

Something that caught my attention is that in generated images metadata, all the [get] tags are substituted, and all the [set] tags are removed. For my workflow, this makes it a little impractical as I would like to keep the variables in the metadata of generated images, to be able to preserve the in-place parameters of the prompt and keep it maintainable even if the image comes from the PNG Info tab.

I'm fairly sure that A1111's API makes it possible to write new metadata to the PNG info. We could either chuck the original, unprocessed prompt verbatim into a field, or have individual fields for every variable. Thoughts?

As I am unaware of most of the inner workings of Unprompted, I'm curious about one thing: is whitespace stripping really the right solution to ensure substitutions do not happen at word boundaries? I wonder how practical/appropriate would it be to preserve whitespace by default, and instead use something like a backslash \ before a newline to strip it (like it is possible to do in Python, to split a line into multiple lines at locations that would otherwise make the parser unhappy) or like C's ## token pasting operator in macros.

One of my design goals with Unprompted is making sure it's accessible to non-programmers who want to think about syntax as infrequently as possible. :) edit for clarity - This is partially because Unprompted moonlights as an all-purpose text generator, not just a Stable Diffusion extension (where the simplicity of writing multi-line templates is arguably less crucial.)

You could, of course, accomplish what Unprompted does by writing your templates in raw Python, where you need to be explicit about formatting, making sure to wrap every string output in print() or separating your lines with backslashes.

I'm trying to avoid as much of that as I can without infringing on the programmer's will, or in this case, compatibility with other extensions. It's a balancing act.

So far these are pretty much all of the solutions and feedback that come to mind. I'll keep looking for other possibilities, please let me know if I missed something or if you think of another possible approach.

Here's one more approach worth considering: it's possible to change your Unprompted config on a per-template basis using the [config] shortcode. So maybe in your case, you would include something like this at the top of the file:

[config]{"syntax":{"sanitize_before":{"\\":"\\n"},"sanitize_after":{"\\n":"\n"}}}[/config]

Along with setting strip_consecutive_whitespace to false, when I implement that setting.

This at least helps "future-proof" your template a little, instead of depending on a config-user.json file being set up a certain way.

Anyway, hopefully I was able to provide some useful info here. I'm sort of firing in the dark until I get a chance to try Prompt Fusion myself, so I apologize if any of my suggestions were off the mark.

Looking forward to your response.

from unprompted.

Hi ljleb,

Thank you for reaching out!

I think you've raised two separate issues here:

1. Unprompted's newline behavior is controlled by the syntax.sanitize_before and syntax.sanitize_after settings in config.json. By default, these settings will swap newlines into spaces because it allows for intuitive scripting in many cases. Consider the following example:

photo of a
[choose]
dog
lion
tiger
parrot
alex jones
[/choose]

In the current setup, you don't have to worry about whether or not you added an extra space to the end of the first line. This can save a lot of time when debugging lengthy templates.

2. The stripping of consecutive whitespace is an additional cleanup step that also simplifies debugging. I would be happy to introduce a setting to disable this behavior, but if you can think of another solution that would make our extensions compatible with each other, I'm all ears.

When I find the time, I'd like to try out Prompt Fusion--it looks very cool--and maybe I can get a better understanding of your syntax.

Perhaps I could introduce a [literal] shortcode that bypasses Unprompted's normal rules, which would allow the content to work with Prompt Fusion? Just spitballing here.

from unprompted.

Hey ThereforeGames, first of all thanks for the quick reply. Apologies if my initial comment seems a bit grumpy 😅.

I tried updating the syntax.sanitize_before and syntax.sanitize_after dicts in config.json, but unfortunately, as you said, there seems to be an additional sanitizing step that strips all consecutive whitespace chars into single spaces, independently of the content of these properties. For this reason, updating the config didn't solve the issue.

One solution I'm considering for this is removing variables on Prompt Fusion's side altogether. I think this might be a good solution as, IIUC -- even though its main purpose seems to be closer to prompt randomization than prompt control -- Unprompted can be used to refactor a prompt into a more manageable/readable string using variable definitions. For example instead of a man wearing a blue t-shirt and blue jeans, you could write [set color]blue[/set] a man wearing a [get color] t-shirt and [get color] jeans.

However, while exploring this solution, I tried something like this (using Prompt Fusion interpolation syntax):

[set test].5[/set]

[1girl : seawater : , [get test] ]

And it does not seem to replace [get test] with .5 for some reason. I also tried using normal prompt editing syntax:

[set test].5[/set]

[1girl : seawater : [get test] ]

Same result, the substitution does not happen. Am I doing something wrong? If we can get something like this working for most cases using Unprompted's variable syntax, then I'll gladly remove variables completely on Prompt Fusion's side.

If variables are not intended to be used like this in Unprompted, then I'm not sure removing Prompt Fusion's variable syntax is a good idea. An alternative in this case could be to introduce a different syntax for PF maybe. I would like to suggest introducing a config value on PF's side to determine how to parse variable definitions; however, config values for syntax does not seem like a good idea to me, as it makes it harder to reproduce old prompts without knowing the exact config used at generation time.

I don't think it's a good solution to add knowledge of other extensions like PF to Unprompted (for example, to add back newlines to PF's variable definitions... if PF is active... and then we need to keep the PF variable definition parser up to date on both extensions... yep, looks like a mess haha).

Something that caught my attention is that in generated images metadata, all the [get] tags are substituted, and all the [set] tags are removed. For my workflow, this makes it a little impractical as I would like to keep the variables in the metadata of generated images, to be able to preserve the in-place parameters of the prompt and keep it maintainable even if the image comes from the PNG Info tab.

One problem I see with the literal tag described above, is that [literal] will become part of the prompt tokens (and then it will be tokenized and passed to the model along with everything else) when disabling Unprompted, or when using prompts containing only this tag in environments that don't have Unprompted installed. In my case, this means that, if I want to reproduce an old image, I have to manually enable or disable Unprompted depending on whether Unprompted was enabled when the old image was generated. This is of course if the [literal] tag is to remain in generated images metadata; if all tags are removed from the metadata prompts, then it should be possible to disable Unprompted when reproducing an old image, as no Unprompted tag syntax should appear in the metadata prompt. Although, this behaviour makes this solution less appealing IMO as, as I said above, I'd like to be able to keep variable definitions in image metadata, to preserve in-place parameters and ease of edition when using the "send to txt2img" feature in the PNG Info tab.

As I am unaware of most of the inner workings of Unprompted, I'm curious about one thing: is whitespace stripping really the right solution to ensure substitutions do not happen at word boundaries? I wonder how practical/appropriate would it be to preserve whitespace by default, and instead use something like a backslash \ before a newline to strip it (like it is possible to do in Python, to split a line into multiple lines at locations that would otherwise make the parser unhappy) or like C's ## token pasting operator in macros.

So far these are pretty much all of the solutions and feedback that come to mind. I'll keep looking for other possibilities, please let me know if I missed something or if you think of another possible approach.

All of that being said, in the case we find a solution to make PF's variable syntax work with Unprompted enabled, whitespace stripping still poses a problem for my workflow, as I use newlines extensively in my prompts to make it easier to read and organize subsections. Using newlines, I also find it much easier to relocate subsections of the prompt (i.e. by selecting a region delimited by newlines, ctrl+x, and then ctrl+v on a newline character). Having the newlines removed in the metadata makes it slightly impractical in my workflow, as I have to add maybe 7 to 20 newlines every time I send an old picture from PNG Info to i.e. txt2img.

from unprompted.