As per the title. This would be completely optional, and of course not all filters would even HAVE settings, but I think its really nice:

{
    "run_with": "python",
    "location": "./credit_creator.py",
    "name": "Credit Creator",
    "settings": {
        "identifier": "sirlich"
    }
}

Gets called like:
python -u credit_creator.py '{"identifier":"sirlich"}'

More details in #5 and #7.

This filter-type will allow the user to specify an arbitrary URL (or do we confine to github?), that will install the folder into .regolith/cache, with some name-spacing.

Relies on the use of regolith install. Please see #5

Consider moving existing readme into running.md or something.

We need to explain what Regolith is, whats completed (broadly), and whats still left to do (broadly). We can provide links to the example project (say its private), and then also to the public filters.

The problem is that the os.Rename doesn't work when the target file already exists. The Setup function creates empty build folder (but it shouldn't). The build folder should be deleted only if the creation of .tmp is finished.

os.Rename line:

As Regolith becomes more complex, its clear that the data folder needs to be made required, or at least default.

I suggest packs/data but does anyone have another preferred path?

Should be created with regolith init and also set into the default config.json

venv is a tool for creating python dependency "virtual environments". If we are installing dependencies via pip, we should make sure thats happening in a virtual environment, per filter.

Related to #22.

We need to provide a traceback of course, if requested, but many of our errors are currently producing a tracback, even when they really shouldn't.

For example, running regolith run without regolith init will cause a very yucky error, that is difficult to interpret. In these cases, we should write a clear, concise error, with instructions on what to do next:

This will allow properties set inside of the config.json to be read and understood by the downloaded filter, when running.

Code for this currently exists for the settings key, but we should expand this solution for all keys inside of filters.

Right now, there is an opinionated ordering:

1. script
2. settings (merged)
3. external arguments (in remote filter)
4. internal arguments (set inside of the config.json locally)

This is "fine", but we may need extra power sometimes.

We can create a new field called "argumentOrder" like this:

"argumentOrder": "${script} ${settings} ${localArguments} ${remoteArguments}"

If argument order exists, the command should be built by find-and-replacing the values into the argument order.

When should it trigger? On any file save?

Currently, standard filters are directly linked (hard-coded) to use the regolith-filters github repo. We should also allow standard filters to point to Bedrock OSS repositories, so large or ambitious filters can have their own repo, whilst still being considered standard.

https://github.com/bridge-core/project-config-standard

Example config:

{
    "type": "minecraftBedrock",
    "name": "Test",
    "namespace": "sirlich",
    "author": "sirlich",
    "targetVersion": "1.16.0",
    "description": "",
    "regolith": {
        "profiles": {
            "dev": {
                "export": {
                    "target": "development"
                },
                "filters": [
                    {
                        "run_with": "python",
                        "location": "./filters/kra_convert.py"
                    },
                    {
                        "run_with": "python",
                        "location": "./filters/flat_features.py"
                    },
                    {
                        "run_with": "python",
                        "location": "./filters/languages.py"
                    },
                    {
                        "run_with": "python",
                        "location": "./filters/debug_molang/debug_molang.py",
                        "settings": {
                            "places": 5
                        }
                    }
                ],
                "unsafe": false
            }
        }
    },
    "packs": {
        "behaviorPack": "./BP",
        "resourcePack": "./RP"
    }
}

I implemented the "world" export target (for issue #14) but we haven't decide how to handle some cases. Current implementation is very simple.

The "world" export target requires exactly one of the two properties "worldPath" or "worldName". Providing both of them will result in an error.

Problems

1. In current implementation "worldPath" can lead anywhere. It just creates the some additional folders in that path and exports files there. The folders are "<worldPath>/behavior_packs/<project_name>_bp" and "<worldPath>/resource_packs/<project_name>_rp". I'm not sure if that's ok. Maybe the export path should be relative to .../com.mojang/minecraftWorlds folder?
2. The exporter doesn't create the behavior_packs.json and resource_packs.json in the world. Maybe it should?
3. If there is behavior_packs.json and/or resource_packs.json file in the world and it has a reference to an existing BP/RP should the exporter export files to the path of that reference? What if the file points at a pack inside development_behavior_packs/development_resource_packs.

Currently its missing some files

Color scheme

Logo

Short name?

Can build upon the base in minecraft_utils.go.

I guess we want these targets:

• exact (filepath)
• dev packs (requires a definition for the name of the RP and the BP, after exporting)
• world (same req as above)
• none (just leave in build)

It would be nice to have "data" path for regolith which should be copied to the working directory with RP and BP.

• The folder will be copied inot .tmp/data during runtime.
• The profile will have a new dataPath property with a path relative to the root of the project that points at the data folder

Should be done with GH pages, as a docs folder

Write now we move it into tmp. We need to move it BACK into the main data location, to allow filters to save data there.

This issue will be used as a thread to discuss the possible changes to venvPath setting.

Current implementation

Regolith config.json file has currently 4 depth levels.

1. Root - defined by the Bedrock-OSS/project-config-standard
2. "regolith" - the root of the regolith configuration
3. "profiles" - a list of profiles. A profile is a single task that you can trigger with regolith run command
4. "filters" - a filter is a single action within the profile (it can be a python, js or shell script)

Currently the venvPath is a property of a profile. Adding that configuration to the file will cause creation of a single python environment for all filters in the profile. Skipping that property will (the default behavior) creates separate venv for every filter in .regolith/cache/<filter path>/venv.

venvPath can be either relative or absolute path. If user choose to put relative path in there it will create the virtual environment in a path relative to .regolith/venvs/.

Problems

1. Where in the config.json should we put the venvPath property

1.1. It should be the property for entire project (inside "regolith")
1.2 It should be the property of a profile (regolith -> profiles -> [profile name]). This is the current solution.
1.3 It should be the proprety of a filter (regolith -> profiles -> -> filters -> [filter name]).

2. If the path is relative, what should be the root

2.1. .regolith/venvs/<user path> - This is the current solution
2.2. .regolith/cache/<user path> - The problem of this solution is that we already use .regolith/cache/<filter name> for caching filters. If someone choose to name the venv the same as the filter it could create some collisions.
2.3 .regolith/cache/venvs/<user path> - with this solution i suggest moving the filter cache from .regolith/cache/<filter name> to .regolith/cache/filters/<filter name>.

3. What should be the default behavior

3.1. Separate venv for every filter - This is the current solution
3.2. Separate venv for every profile
3.3. Everything should use single venv

4. Should we allow absolute paths

4.1. Yes
4.2. No

Discuss?

What does it look like to download and use regolith for the random user?

Can they rely on regolith gopath thing, or do they need to include an .exe in every project.

What can we do to make this process awesome and easy?

If setup function fails to create one of the directories for some reason the program will continue because there nothing checks for the errors. This ultimately leads to errors further in the code where there is an assumption that all directories have been created.

Right now, the build folder is created for all export targets, which doesn't make much sense.

• Rename none to local
• Remove build directory for all export targets that are not local
• Improve efficiency of export by using a MOVE not a COPY (doesn't apply to local)

This means setting up a proper testing suite, which will test Regolith. We can trigger this as part of CI.

What do we want to do here?

Could not install dependency github.com/Bedrock-OSS/regolith-filters//texture_list: error downloading 'https://github.com/Bedrock-OSS/regolith-filters.git': git must be available and on the PATH

We need to fully define the different security levels. Currently the possible run-types are: (in order of safety)

• Standard Library
• Local Filters
• Online Filters

Local is dangerous cause somebody can say "please put this here, and then run X". Its safer than online filters though, because the user at least has to handle the source, and could peek inside.

Online filters could also be compromised, which is why we cache by default, and require a regolith install --force (or another command -syntax isn't 100% decided yet) to flush them + redownload.

We need to make the user agree to some kind of "I understand the risks" clause. Of course, we cannot just store this inside of the regolith.json, because then somebody could simply have it pre-checked (like the MC server EULA).

We should instead look into storing the "regolith safety level" somewhere else.

We could store the git SHA of the installed commit, and then do a quick check to see if anything needs to be updated?

Not sure if better to auto-run when compiling, or add a check command?

The default rn is that filters will easily get out-dated, since even running regolith install will skip satisfied dependencies. You have to manually delete (or did we add command to clear out the tmp directory?) before reinstalling.

This will be used for storing online-filters in the .regolith/cache (do we want a different folder name?).

The start of this can be found in remote_filters.go. I found a library (go-getter) that looks like it should do the job.

As Stirante pointed out, we probably want to use some kind of name-spacing of folders:
.regolith/cache/github.com/bedrock-OSS/filters/texture_list/....

The data flow:

• An 'online' filter specifies a URL where the filter is located
• If the filter exists in the cache, use it
• If the filter does NOT exist in the cache, throw a warning, and say: please run 'regolith install', or 'regolith run -- install.
• Installing should run through every profile, and install everything by unpacking the folder into the correct namespace.

Some systems may not have python installed, or may have the wrong version. We should install a stand-alone python version if this missmatch exists.

Should be a folder in .regolith where stuff is copied when moved/deleted.

This will make regolith slower, but safer. Should be configured in config.json

When running regolith, it should check whether there is a newer version of regolith available. We can limit it only to regolith install because if filter definition changes, it would make easier for the user to try updating regolith.

Per the https://github.com/bridge-core/project-config-standard

Our current implementation is pretty strict in what can be run. Right now, only Python.

The reason we have unique run-targets for each language instead of just running arbitrary shell command is because its safer, and because it allows us to add additional logic as-needed, for example -u for python to enforce the buffer to flush.

I'm not fixing that because I don't understand the intention behind that code and maybe I'm missing something important but possibly there is an error in the logic of this function:

• CASE 1: checked[filter.RunWith] exists -> ok == true AND c == true -> ok || !c == true
• CASE 2: checked[filter.RunWith] is missing-> ok == false AND c == false -> ok || !c == true
ok || !c is always true.

Should provide:

• Ability to run regolith (?)
• json-schema for regolith.json

Logs from running regolith.exe in clean regolith-project

PS D:\Golang\regolith-project\regolith-project> regolith.exe run
[INFO]  Running profile: dev
[DEBUG] Cleaning .regolith/tmp and build folders
[INFO]  Running filter 'Delete Blocks'
{error 26 0  chdir D:\Golang\regolith-project\regolith-project\.regolith\tmp: The system cannot find the path specified.}
bedrock-oss.github.com/regolith/src.RunSubProcess
        D:/Golang/regolith-project/regolith-github-repo/src/filters.go:109
bedrock-oss.github.com/regolith/src.RunPythonFilter
        D:/Golang/regolith-project/regolith-github-repo/src/filters.go:97
bedrock-oss.github.com/regolith/src.RunFilter
        D:/Golang/regolith-project/regolith-github-repo/src/filters.go:88
bedrock-oss.github.com/regolith/src.RunProfile
        D:/Golang/regolith-project/regolith-github-repo/src/filters.go:68
main.main.func2
        D:/Golang/regolith-project/regolith-github-repo/main.go:57
github.com/urfave/cli/v2.(*Command).Run
        C:/Users/Laptop/go/pkg/mod/github.com/urfave/cli/[email protected]/command.go:163
github.com/urfave/cli/v2.(*App).RunContext
        C:/Users/Laptop/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:313
github.com/urfave/cli/v2.(*App).Run
        C:/Users/Laptop/go/pkg/mod/github.com/urfave/cli/[email protected]/app.go:224
main.main
        D:/Golang/regolith-project/regolith-github-repo/main.go:85
runtime.main
        D:/programs/Go/src/runtime/proc.go:225

The flow has been decided as this:

Filter echo_npc will have a folder called data in their repo, at same level as filters.json.

When installed, the folder will be moved into data/echo_npc/*

In other words, the data folder inside a remote filter can be used to setup the default config values for the filter.

Right now, regolith install will not checked the installed filters for additional online dependencies. It should do so.

Example: regolith run dev --output-type development.

Open questions:

• Can you do this: regolith run --output-type development. (no target seleted)
• What about exact which needs a path argument?

Currently, we debug everything. We should have some log-levels defined, and we can select what level we want in regolith.json.

Maybe Error, Warning, Log, Verbose, VeryVerbose?

The flow is explained in #5.

This command should loop through every filter, in every profile, and install the required dependencies.

The logger is probably not initialized when running with no args.

Optional key in a profile: venv_path sets an explicit path to use for that profiles venv, instead of a per-filter venv.

Most online filters need to be specified via URL, but the standard library can be accessed via name.

Write the logic that allows you to specify a short-name like "texture_list", and have it automatically resolve into the correct github URL, to be treated like a normal online-filter.

We can make a few assumptions here, like the repo name, the organization name, and the fact that all filters must be placed at the top level of the project (or do we want to organize filters different? Open question.)