I'd like to be able to enforce consistent single vs double quotes (whichever, idc). But I'd also like for existing quotes to not be removed.
Especially for version numbers we pre-emptively quote some entries to avoid accidentally writing something like python: 3.10 (which is actually 3.1, not 3.10 !) when upgrading from python 3.9, for example.

Error:

TomlSort.__init__() got an unexpected keyword argument 'only_sort_tables'

The latest toml-sort module uses the new CommentConfiguration, SortConfiguration and FormattingConfiguration as parameters. See related PR here.

My local YAML formatter places an initial indent after the parent keyword in a .pre-commit-config.yaml file. The pretty-format-yaml hook formats it differently. I was wondering what the reason was for the discrepancy.

Local Formatter

repos:
  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
    rev: v2.5.0
    hooks:
      - id: pretty-format-yaml
        args: [--autofix]

Hook Formatter

repos:
- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.5.0
  hooks:
  - id: pretty-format-yaml
    args: [--autofix]

.pre-commit-config.yaml:

default_language_version:
  python: python3

repos:
- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks.git
  rev: v2.4.0
  hooks:
  - id: pretty-format-java
    args: [--autofix]
  - id: pretty-format-yaml
    args: [--autofix, --indent, '2']

Because of OpenJDK 16 feature: https://openjdk.org/jeps/396

I get the following error:

Run command: ('java', '--add-exports', 'jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED', '--add-exports', 'jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED', '--add-exports', 'jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED', '--add-exports', 'jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED', '--add-exports', 'jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED', '-jar', 'C:\\Users\\user\\.cache\\pre-commit\\google-java-formatter1.15.0.jar', 

And the solution would be to either add the following argument to the java command:

java --illegal-access=permit

Or this list of parameters:

--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.model=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.parser=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.processing=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED
--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED
--add-opens=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED
--add-opens=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED

How can I do it in this pre-commit hook?

I just spent an hour wondering while an INI file fails the hook, until I manually loaded it using the ConfigParser (which is what the hook does, too). So, I think to make failures a bit more useful change

to

 except Error as error:
     print("Input File {0} is not a valid INI file: {1}".format(ini_file, str(error))) 
     return 1

or some such? Thanks!

Hi @macisamuele

Appreciate your great work! I wonder if there is a way to format arrays in .toml using pretty-format-toml.

[array]
array1 = [1, 2, 3, 4,

5, 6, 7, 8, 9, 10]
array2 = [[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10],

[1, 2, 3, 4, 5, 6, 7, 8, 9, 10]]

Hello
I use language-formatters-pre-commit-hooks - pretty-format-ini to check the ini files of an older program. This program is for windows and uses cp1250 encoding. When checking, the following message appears:

Traceback (most recent call last):
File "C:\Program Files\Python\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\Python\Python39\lib\runpy.py", line 87, in run_code
exec(code, run_globals)
File "C:\Program Files\Python\Python39.cache\pre-commit\reponx58ayq8\py_env-python3.9\Scripts\pretty-format-ini.EXE_main.py",
line 7, in
File "C:\Program Files\Python\Python39.cache\pre-commit\reponx58ayq8\py_env-python3.9\lib\site-packages\language_formatters_pre_co
mmit_hooks\pretty_format_ini.py", line 26, in pretty_format_ini
string_content = input_file.read()
File "C:\Program Files\Python\Python39\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xed in position 16: invalid continuation byte

Apparently there is a problem with the code page of the file. When I convert the file to UTF8 the check runs without any problems. Is there any way to define the code page for pretty-format-ini?

I am attaching two files one in utf8 and one for cp1250. Is it possible to possibly work around this?

Thank you in advance for the answer

test-cp1250.txt
test-utf8.txt

The toml-sort library sometimes deletes comments. Specifically, toml-sort has the notion of an "orphan" comment where there are empty newlines between the comment and the subsequent section.

I usually trust pre-commit hooks to not do anything destructive, so this feels like a fairly serious issue. Thus I asked in pappasam/toml-sort#59 if functionality could be added which might prevent this.

When running the KTLint pre-commit hook I see the error:

OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpz7bctuzw' -> '/home/dave/.cache/pre-commit/ktlint0.40.0.jar'

I guess because on my system /tmp is not on the same btrfs sub-volume as /home.

The source of the problem is the use of os.rename in language_formatters_pre_commit_hooks/utils.py. Replacing this with shutil.move should fix this error.

My pre-commit yaml is:

  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
    rev: v2.1.0
    hooks:
    - id: pretty-format-kotlin
      args: [--autofix]

Full error message:

KTLint...................................................................Failed
- hook id: pretty-format-kotlin
- exit code: 1

[cwd=/home/dave/home/calorimeter] Run command: ('java', '-version')
[return_code=0] | openjdk version "11.0.9.1-internal" 2020-11-04
OpenJDK Runtime Environment (build 11.0.9.1-internal+0-adhoc..src)
OpenJDK 64-Bit Server VM (build 11.0.9.1-internal+0-adhoc..src, mixed mode)

Downloading https://github.com/pinterest/ktlint/releases/download/0.40.0/ktlint
Traceback (most recent call last):
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pretty_format_kotlin.py", line 25, in __download_kotlin_formatter_jar
    return download_url(get_url(version), "ktlint{version}.jar".format(version=version))
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/utils.py", line 75, in download_url
    os.rename(tmp_file_name, final_file)
OSError: [Errno 18] Invalid cross-device link: '/tmp/tmpz7bctuzw' -> '/home/dave/.cache/pre-commit/ktlint0.40.0.jar'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/bin/pretty-format-kotlin", line 8, in <module>
    sys.exit(pretty_format_kotlin())
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pre_conditions.py", line 55, in wrapper
    return f(*args, **kwargs)
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pretty_format_kotlin.py", line 55, in pretty_format_kotlin
    ktlint_jar = __download_kotlin_formatter_jar(
  File "/home/dave/.cache/pre-commit/repo2681bua4/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pretty_format_kotlin.py", line 27, in __download_kotlin_formatter_jar
    raise RuntimeError(
RuntimeError: Failed to download https://github.com/pinterest/ktlint/releases/download/0.40.0/ktlint. Probably the requested version, 0.40.0, is not valid or you have some network issue.

I have realized that pretty-format-ini removes comment lines. Is this on purpose?

I tried running it on my project and it made my toml file very difficult to read with it's long exclude pattern.: facebookresearch/habitat-sim@91f8ea5

Enforcing a max line length would do wonders for readability in this case or at least not collapsing multiline strings like that.

Alternative not destroying multiline strings woudl also be helpful.

Looks like it's easy to fix with https://docs.python.org/3/library/shutil.html#shutil.move per https://stackoverflow.com/questions/42392600/oserror-errno-18-invalid-cross-device-link

Downloading https://github.com/google/google-java-format/releases/download/v1.10.0/google-java-format-1.10.0-all-deps.jar
Traceback (most recent call last):
  File "/home/draftcode/.cache/pre-commit/repop2cp8yu_/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pretty_format_java.py", line 37, in __download_google_java_formatter_jar
    return download_url(url_to_download, "google-java-formatter{version}.jar".format(version=version))
  File "/home/draftcode/.cache/pre-commit/repop2cp8yu_/py_env-python3.9/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/utils.py", line 75, in download_url
    os.rename(tmp_file_name, final_file)
OSError: [Errno 18] Invalid cross-device link: '/tmp/tmp_yncpea2' -> '/home/draftcode/.cache/pre-commit/google-java-formatter1.10.0.jar'

The issue is caused by Ktlint 0.37.0:
pinterest/ktlint#761

Here an example of a failed build:
Yelp/swagger-gradle-codegen#133

Consider this YAML file:

repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
  rev: v4.0.1
  hooks:
  - id: trailing-whitespace
  - id: end-of-file-fixer
  - id: check-yaml
  - id: debug-statements
- repo: https://github.com/PyCQA/flake8
  rev: 3.9.2
  hooks:
  - id: flake8
    args: [--max-line-length=100, "--per-file-ignores=python/myrepo/*.py:F401"]

The last line is formatted to

    args: [--max-line-length=100, --per-file-ignores=python/myrepo/*.py:F401]

(the "..." are removed). This is however illegal. Indeed pre-commit-hooks complains about this:

while scanning a plain scalar
  in ".pre-commit-config.yaml", line 13, column 35
found unexpected ':'
  in ".pre-commit-config.yaml", line 13, column 73

Reference: pre-commit/pre-commit-hooks#650

The use of files for backward compatibility seems to defeat the added flexibility by types.

While there are, currently, 2 extensions and 5 files mapped to ini type, pinning files to *.ini extension will result in only 1 extension (ini itself) being matched.

'*.ini'
'*.txsprofile'

'.coveragerc'
'.gitconfig'
'.hgrc'
'.pypirc'
'setup.cfg'

Would it not be better to remove the files key and have people that do need backward compatibility add them in their .pre-commit-hooks.yaml. (Same applies to other hooks)

Thanks for your consideration!

In case a long string is present on the file it might get weirdly processed. #1 was one occasion of this.

While checking #3 I've noticed that the behaviour of string splitting might happen also for values in mappings.

Easy way of reproducing

from io import StringIO
from ruamel.yaml import YAML
yaml_content  = "k: {}".format('a ' * 100)

fixed_content = StringIO()
YAML().dump(YAML().load(yaml_content), fixed_content)

print("Initial content:")
print(yaml_content)
print("Fixed content:")
print(fixed_content.getvalue())

Initial content:
k: a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a
Fixed content:
k: a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a
  a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a a
  a a a a a a a a a a a a a a a a a a a a

This is probably caused by some mis-configuration of ruamel.yaml

I have a file:

epsd:
    [
            0.00000000e+00, 2.00000000e-04, 6.97510220e-01, 6.97710220e-01,
            1.14101113e+00, 1.14121113e+00, 1.30379685e+00, 1.30399685e+00,
            1.59313535e+00, 1.59333535e+00, 1.67477980e+00, 1.67497980e+00,
            1.74995852e+00, 1.75015852e+00, 1.92651521e+00, 1.92671521e+00,
            2.22921949e+00, 2.22941949e+00, 2.25524963e+00, 2.25544963e+00]

sigd: [
            0.00000000e+00, 2.00000000e-04, 6.97510220e-01, 6.37087406e-01,
            1.08038734e+00, 1.05259037e+00, 1.21517607e+00, 1.17774795e+00,
            1.46688286e+00, 1.41467867e+00, 1.49612181e+00, 5.44543851e-01,
            6.19522524e-01, 3.86097256e-01, 5.62453891e-01, 3.26369082e-01,
            6.28873246e-01, 5.81974365e-01, 6.07804478e-01, 5.72717686e-01]

that is formatted:

epsd: [0.00000000e+09, 2.00000000e-04, 6.97510220e-01, 6.97710220e-01, 1.14101113e+00, 1.14121113e+00, 1.30379685e+00, 1.30399685e+00, 1.59313535e+00, 1.59333535e+00, 1.67477980e+00, 1.67497980e+00, 1.74995852e+00, 1.75015852e+00, 1.92651521e+00, 1.92671521e+00, 2.22921949e+00, 2.22941949e+00, 2.25524963e+00, 2.25544963e+00]

sigd: [0.00000000e+09, 2.00000000e-04, 6.97510220e-01, 6.37087406e-01, 1.08038734e+00, 1.05259037e+00, 1.21517607e+00, 1.17774795e+00, 1.46688286e+00, 1.41467867e+00, 1.49612181e+00, 5.44543851e-01, 6.19522524e-01, 3.86097256e-01, 5.62453891e-01, 3.26369082e-01, 6.28873246e-01, 5.81974365e-01, 6.07804478e-01, 5.72717686e-01]

I'm using:

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.1.0
  hooks:
  - id: pretty-format-yaml
    args: [--preserve-quotes, --autofix, --indent, '2']

Hi,
it is possible to get a new tag after #97 is merged?

I know that pre-commit supports using SHA commits as rev, but I'd feel more "safe" to have a versioned tag to refer to.

Thank you for you project and dedication.

error：
PermissionError: [WinError 32] 另一个程序正在使用此文件，进程无法访问。
means file not close

in utils：

    with tempfile.NamedTemporaryFile(delete=False) as tmp_file:  # Not delete because we're renaming it
        shutil.copyfileobj(r.raw, tmp_file)
        tmp_file.flush()
        os.fsync(tmp_file.fileno())
        tmp_file.close()  # add this is ok
        os.rename(tmp_file.name, final_file)

Currently ktlint is called with log level debug which leads to a lot of log messages in case the hook is failing. It is sometimes hard to spot the actual rule violation. What about allowing to configure the log level to e.g. INFO?

We run pre-commit YAML formatting on our Ansible playbooks with the following configuration:

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
    rev: v2.9.0
    hooks:
      - id: pretty-format-yaml
        args: [--autofix, --preserve-quotes, --indent, '2', --offset, '2']

By default, all sequences are indented, including root-level sequences. So this YAML file:

- a: 1
- b: 2

Is converted to this with two leading whitespaces:

  - a: 1
  - b: 2

This was addressed in this Stackoverflow question with a custom transform function, would be nice to have something similar here as well.

Hey together, I'm using this tool for my Kotlin projects and would like to migrate to Java 17 soon.
At the moment there is an assertion that ktlint can only be run with Java <16, but I think this is actually obsolete. I was able to run ktlint with Java 17 and even Java 19.

Would you be up to remove this Java constraint in the pre-commit hook?

This issue has been opened as follow-up from #3 (comment)

@macisamuele issue to tracking when you will release a v2.8.0 (to notify me and perhaps others to upgrade whenever you do).

I'm currently using a rev: 54f7f7c5cb9501110b928f2a98faa8024142f8d7 in order to get #147 which is not on 2.7.0.

pre-commit autoupdate keeps changing that (newer) rev to the (older) rev: v2.7.0 due to pre-commit/pre-commit#2811.

Reasoning

Some of the companies have strict rules in terms of downloading binaries from external storages. In this case jar file is downloaded from external storage which is prohibited in many companies (as many companies use own artifact storage such as Artifactory or GHES).

Proposition

Add a possibility to define path to the jar file, either local one or web.

Benefits

• Increase security as it is not needed to download jar from external storage
• In case of local path - increasing speed as it is not needed to download jar everytime

Thanks for fixing up this formatter! I'm putting the fix from #154 into effect, and found out why .pre-commit-config.yaml was excluded from formatting.

The formatter is converting certain strings to boolean values. For instance, the input

args: [--wrap, 'no', number]

produces the output

args: [--wrap, no, number]

The quotes were there to prevent 'no' from being parsed as a boolean. This causes an error after formatting certain .pre-commit-config.yaml files that call the mdformat pre-commit hook with those args. Attempts to further use that .pre-commit-config.yaml file result in Expected string got bool.

I believe that this would be corrected by using round_trip_load and round_trip_dump on the YAML data here. Those calls are designed to properly handle a round trip from YAML to an internal format and back to YAML, which is this formatter is doing.

The problem and solution is explained by the author of reumel.yaml in this Stack Overflow answer.

Could trailing commas for TOML be configurable? I'd rather keep them (in fact, enforce them). Trailing commas lead to better git diff.

The ktlint output parsing used to feed files to fix into ktlint includes some non-file output from ktlint that is then feed into the autofix CLI call:

$ /home/johannes/.cache/pre-commit/repojigxkeve/py_env-python3.10/bin/python /home/johannes/.cache/pre-commit/repojigxkeve/py_env-python3.10/bin/pretty-format-kotlin --ktlint-version=0.49.0 --autofix path/TheFile.kt
[cwd=/home/johannes/src/somewhere] Run command: ('java', '-version')
[return_code=0] | Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel
openjdk version "11.0.19" 2023-04-18
OpenJDK Runtime Environment (build 11.0.19+0-suse-1.1-x8664)
OpenJDK 64-Bit Server VM (build 11.0.19+0-suse-1.1-x8664, mixed mode)

[cwd=/home/johannes/src/somewhere] Run command: ('java', '--add-opens', 'java.base/java.lang=ALL-UNNAMED', '-jar', '/home/johannes/.cache/pre-commit/ktlint0.49.0.jar', '--log-level', 'debug', '--relative', '--', 'path/TheFile.kt')
[return_code=1] | Picked up _JAVA_OPTIONS: -Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel
10:10:47.235 [main] DEBUG com.pinterest.ktlint.cli.internal.KtlintServiceLoader - Discovered RuleSetProviderV3 with id 'standard' in ktlint JAR
10:10:47.352 [main] DEBUG com.pinterest.ktlint.cli.internal.KtlintServiceLoader - Discovered ReporterProviderV2 with id 'baseline' in ktlint JAR
10:10:47.352 [main] DEBUG com.pinterest.ktlint.cli.internal.KtlintServiceLoader - Discovered ReporterProviderV2 with id 'plain' in ktlint JAR

...

10:10:47.740 [pool-1-thread-1] DEBUG com.pinterest.ktlint.rule.engine.api.KtLintRuleEngine - Finished with linting file 'TheFile.kt'
path/TheFile.kt:48:1: Unexpected indentation (10) (should be 8) (standard:indent)

Summary error count (descending) by rule:
  standard:indent: 1
10:10:47.743 [main] DEBUG com.pinterest.ktlint.cli.internal.KtlintCommandLine - Finished processing in 515ms / 1 file(s) scanned / 1 error(s) found
10:10:47.744 [main] DEBUG com.pinterest.ktlint.cli.internal.KtlintCommandLine - Exit ktlint with exit code: 1

Running ktlint format on {'', 'path/TheFile.kt', 'Summary error count (descending) by rule', '10', 'Picked up _JAVA_OPTIONS', '\t', '           - standard', '  standard'}
[cwd=/home/johannes/src/somewhere] Run command: ('java', '--add-opens', 'java.base/java.lang=ALL-UNNAMED', '-jar', '/home/johannes/.cache/pre-commit/ktlint0.49.0.jar', '--log-level', 'debug', '--relative', '--format', '--', '', 'path/TheFile.kt', 'Summary error count (descending) by rule', '10', 'Picked up _JAVA_OPTIONS', '\t', '           - standard', '  standard')

The following shows, that some other output on stdout/stderr got picked up:

Running ktlint format on {'', 'path/TheFile.kt', 'Summary error count (descending) by rule', '10', 'Picked up _JAVA_OPTIONS', '\t', ' - standard', ' standard'}

Hi,
Looks like pretty format yaml hook formatting multiline strings in YAML sequence to single line.
For example I have this:

sequence:
  - very long line which need to be mulitine
    second part of multine string

Pretty format YAML converts this example to this:

sequence:
  - very long line which need to be mulitine second part of multine string

The problem

Hello, when trying to use this hook I'm getting the following error:

Go Formatter.............................................................Failed
- hook id: pretty-format-golang
- exit code: 1
Traceback (most recent call last):
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/bin/pretty-format-golang", line 5, in <module>
    from language_formatters_pre_commit_hooks.pretty_format_golang import pretty_format_golang
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pretty_format_golang.py", line 6, in <module>
    from language_formatters_pre_commit_hooks.pre_conditions import golang_required
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/pre_conditions.py", line 9, in <module>
    from language_formatters_pre_commit_hooks.utils import run_command
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/lib/python3.9/site-packages/language_formatters_pre_commit_hooks/utils.py", line 10, in <module>
    import requests
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/lib/python3.9/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/Users/myuser/.cache/pre-commit/repofaj4nkze/py_env-python3/lib/python3.9/site-packages/urllib3/__init__.py", line 38, in <module>
    raise ImportError(
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with LibreSSL 2.8.3. See: https://github.com/urllib3/urllib3/issues/2168
golangci-lint............................................................Passed

The error seems to be thrown by the virtual env of the repo, and not my local Python version, but I could be misunderstanding that.

Tried but didn't work

So far I tried:

• Updating urllib3
• Installing OpenSSL
• Reinstalling Python 3.9
• Deleting everything in /Users/myuser/.cache/pre-commit/*

My setup/environment

• MacBook Pro 13-inch, M1, 2020
• Apple M1 chip, 16 GB RAM
• go version go1.19.1 darwin/arm64
• git version 2.39.2 (Apple Git-143)
• pre-commit 2.20.0

.pre-commit-config.yaml

repos:
  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
    rev: v2.3.0
    hooks:
      - id: pretty-format-golang
        args:
          - --autofix
  - repo: https://github.com/golangci/golangci-lint
    rev: v1.44.2
    hooks:
      - id: golangci-lint
        entry: golangci-lint run
        args:
          - --max-issues-per-linter=0
          - --max-same-issues=0
          - --config=.code_quality/.golangci.yml
          - --new-from-rev=HEAD~1 # comment this arg if you want to run pre-commit run --all-files

I also tried with the version 2.8.0 of this repo (and deleted the cache of pre-commit) and nothing changed, same error.

Multiple yaml content in one file, it fails to validate

---
apiVersion: test
kind: test
---
apiVersion: test2
kind: test2
---

Currently, when running pretty-format-java hook with Java version 16, it fails due to more restrictive default settings (see https://openjdk.java.net/jeps/396).

Can you please support this by adding the required flags to the java command (see https://github.com/google/google-java-format#jdk-16)?

Considering the following valid YAML (this is the result I would like to achieve after auto-fix by pretty-format-yaml):

services:
  svc:
    image: "..."
    volumes:
      - "/tmp:/tmp"

Using

--indent=4
--offset=2

causes the auto-fix of the file to be over-indented as follows:

services:
    svc:  # all nested mappings over-indented
        image: "..."
        volumes:
          - "/tmp:/tmp"  # indent as expected here

while

--indent=2
--offset=0

results in:

services:
  svc:  # nesting indents preserved
    image: "..."
    volumes:
    - "/tmp:/tmp"   # de-indented

It seems that the following should be valid:

--indent=2
--offset=2

but it is blocked by the following check:

docker file:

FROM python:3.7-alpine

# Install JRE-11 as we will run pre-commit hooks that depends an Java 11+
RUN apk add --no-cache openjdk11-jre

ENV PRE_COMMIT_HOME /pre-commit-docker-cache
ENV PRE_COMMIT_LANGUAGE_FORMATTERS_VERSION ${version of the library to install}

RUN set -x \
    && pip install --no-cache-dir language-formatters-pre-commit-hooks==${PRE_COMMIT_LANGUAGE_FORMATTERS_VERSION} \

    # Run pre-commit-hook to ensure that jars are downloaded and stored in the docker image
    # Run the hooks that you're planning to run within docker.
    # This reduces premission issues as well has makes all the run fast as the lazy-dependencies are pre-fetched
    && pretty-format-java  \

    # Update permissions as hooks will be run as your host-system user (your username) but the image is built as root
    && chmod a+r ${PRE_COMMIT_HOME}/*

error message：

python setup.py egg_info did not run successfully.
    #6 40.57   │ exit code: 1
    #6 40.57   ╰─> [3 lines of output]
    #6 40.57       sys.argv ['/tmp/pip-install-mnbi530b/ruamel-yaml-clib_14aea1e590fc4b8987de685981b34bbf/setup.py', 'egg_info', '--egg-base', '/tmp/pip-pip-egg-info-1pm_sbt2']
    #6 40.57       test compiling /tmp/tmp_ruamel_0s6b_6o1/test_ruamel_yaml.c -> test_ruamel_yaml compile error: /tmp/tmp_ruamel_0s6b_6o1/test_ruamel_yaml.c
    #6 40.57       Exception: command 'gcc' failed with exit status 1
    #6 40.57       [end of output]

I know the latest version of google-java-format 1.10.0 is used in this hook.

But when we explicitly specify the version as args in pretty-format-java hook its failing to download.

  - repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
    rev: v2.0.0
    hooks:
      - id: pretty-format-java
        args: [--autofix,--google-java-formatter-version=1.10.0]

Terminal Output:

RuntimeError: Failed to download https://github.com/google/google-java-format/releases/download/google-java-format-1.10.0/google-java-format-1.10.0-all-deps.jar. Probably the requested version, 1.10.0, is not valid or you have some network issue.

I see you are using the below url format in the code
https://github.com/google/google-java-format/releases/download/google-java-format-{version}/google-java-format-{version}-all-deps.jar

But from google java release of 1.10.0 has this below url
https://github.com/google/google-java-format/releases/download/v1.10.0/google-java-format-1.10.0-all-deps.jar

The --add-exports command line option was added in Java 9.
When running on Java 1.8, you don't need to add these exports. Java fails to start.

Unfortunately, a lot of enterprise legacy code still uses JDK 1.8, This version is going to be supported till the end of 2030.

As a workaround I've added an alias to my ~/.bashrc, and I run gitc instead of git commit:

alias gitc='PATH=~/.sdkman/candidates/java/17/bin:$PATH git commit'

Hello!

I see the "offset" parameter defined here for the yaml files. However, when I try to use it in v2.6.0, I get the following message:

Pretty format YAML.......................................................Failed
- hook id: pretty-format-yaml
- exit code: 2

usage: pretty-format-yaml
       [-h]
       [--autofix]
       [--indent INDENT]
       [--preserve-quotes]
       [--line-width LINE_WIDTH]
       [filenames [filenames ...]]
pretty-format-yaml: error: unrecognized arguments: --offset

My pre-commit-config:

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.6.0
  hooks:
  - id: pretty-format-yaml
    args: [--autofix, --indent, '2', --offset, '2']

Am I doing something wrong?

Where can I find documentation about the options I can pass to the linters? (eg. autofix)

thanks

So I have for example this file:

val1: adsf
val2: [
  1,
  2,
  # comment
  3,
  4,
]

but when the formatter runs, I get this output:

val1: adsf
val2: [1, 2, 3, 4]

Is it possible to not compact lists into a one liner, and to keep comments? maybe we can have a setting that keeps new lines, which might prevent this from collapsing? This also happens with this yaml file:

# comments are allowed in yaml
{
  val1: adsf,
  # some comment
  val2: [1, 2, 3, 4]
}

which is converted to this:

# comments are allowed in yaml
{val1: adsf, val2: [1, 2, 3, 4]}

This is my config:

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.0.0
  hooks:
  - id: pretty-format-yaml
    args: [--autofix, --indent, '1']

In the ktlint python script, ktlint is run with --log-level=debug.

This slows down the pre-commit check and I personally find it to be noisy. Would you entertain the idea of setting the default to info instead of debug or making it possible to configure/disable the logging?

I am noticing that for when run pretty-format-toml for toml with contents like below

# full config list in https://docs.streamlit.io/library/advanced-features/configuration
# can see current config via command "streamlit config show"

[runner]
# disabling streamlit magics 
# (See https://docs.streamlit.io/library/api-reference/write-magic/magic)
# prefer to call streamlit commands explicitly
magicEnabled = false


[browser]
# disabling sending usage stats to Streamlit
gatherUsageStats = false

will correct to

# full config list in https://docs.streamlit.io/library/advanced-features/configuration
# can see current config via command "streamlit config show"

[browser]
gatherUsageStats = false

[runner]
magicEnabled = false

removing comments from before, assuming this is not expected behavior (i have other files with comments that not having this issue)

Thanks for providing these nice hooks!

I was wondering, whether you would be willing to add an argument to the pretty-format-ini hook, that would allow to have spaces, rather than tabs for indents.

The reason for this is that many tools for checking and fixing config files would insert spaces (eg setup-cfg-fmt, tox-ini-fmt), ending up in a tab→space→tab (or space→tab→space) loop.

Thanks for your consideration,
lcnittl

While the default pre-commit hooks has a pretty-format JSON hook, it is has some serious customizability/prettifying issues that seem to be out of scope: pre-commit/pre-commit-hooks#521 and there is blocking my adoption of using that. Perhaps adding a more feature-ful, customizable JSON prettifier to this repository would be appropriate.

@CAM-Gerlach any interest? This blocks my auto-formatting of JSONs.

Since JSONs can't have comments we don't have to worry about #26 at the very least. Perhaps there is an easy fix in JSON.dumps that I am unaware regarding spitting out short-arrays.

For things like cargo.toml, I'd prefer to keep the packages section at the top of the file.

And in many cases, it can be helpful to organize data in a toml file for reader understanding.

On Windows, with AFAIK all current python versions,
the default installation does not assume utf-8 file encoding.

This leads to decoding errors with perfectly fine YAML files
(PYTHONUTF8=1 was manually set before on this system):

bsh ❯ PYTHONUTF8=0 pre-commit run -a
Trim Trailing Whitespace.................................................Passed
Fix End of Files.........................................................Passed
Check Yaml...............................................................Passed
Check Toml...............................................................Passed
Check JSON...........................................(no files to check)Skipped
Check for case conflicts.................................................Passed
Check for added large files..............................................Passed
fix UTF-8 byte order marker..............................................Passed
Mixed line ending........................................................Passed
Check docstring is first.................................................Passed
Check vcs permalinks.....................................................Passed
black....................................................................Passed
Pretty format YAML.......................................................Failed
- hook id: pretty-format-yaml
- exit code: 1

Traceback (most recent call last):
  File "d:\devel\python\python39\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "d:\devel\python\python39\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\Users\XXX\.cache\pre-commit\repo1kdjaww0\py_env-python3.9\Scripts\pretty-format-yaml.EXE\__main__.py", line 7, in <module>
  File "C:\Users\XXX\.cache\pre-commit\repo1kdjaww0\py_env-python3.9\lib\site-packages\language_formatters_pre_commit_hooks\pretty_format_yaml.py", line 71, in pretty_format_yaml
    string_content = "".join(input_file.readlines())
  File "d:\devel\python\python39\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 3321: character maps to <undefined>

pyupgrade................................................................Passed
Reorder python imports...................................................Passed
.gitlab-ci.yml linter....................................................Passed
markdownlint.............................................................Passed
autoflake................................................................Passed
nbstripout...............................................................Passed

XXXX@currywurst in …/nimbus-evaluation on  main [$] is 📦 v0.1.0 via 🐍 pyenv Mem: 18GiB/32GiB Sw: 12GiB/33GiB took 6s
bsh ❯ uname -a
MINGW64_NT-10.0-19043 currywurst 3.1.7-340.x86_64 2021-10-12 16:29 UTC x86_64 Msys

with, shortened:

- repo: https://github.com/macisamuele/language-formatters-pre-commit-hooks
  rev: v2.2.0
  hooks:
  - id: pretty-format-yaml
    args:
    - --autofix
    - --indent
    - '2'
    - --preserve-quotes

Please note: yaml/pyyaml#123

Filed in relation to dependabot/dependabot-core#3263, since it's not clear who's at fault.

To reproduce:

1. Put a key/value pair like time: "15:00" in a YAML file
2. Run pretty-format-yaml --autofix on the file

Result:

The key/value pair is changed to time: 15:00.

It looks like this clashes with the sexagesimal (base 60) definition of YAML integers, effectively turning the original string into an integer value.

For example:

[cwd=/Users/foo] Run command: java -jar /Users/foo/.cache/pre-commit/google-java-format-1.7-all-deps.jar --replace src/main/java/com/Foo.java
[return_code=0] |
[cwd=/Users/foo] Run command: java -version
[return_code=0] | openjdk version "1.8.0_252"
OpenJDK Runtime Environment Corretto-8.252.09.1 (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM Corretto-8.252.09.1 (build 25.252-b09, mixed mode)

for some reason this library makes loading some yaml files incredibly slow -- for example this one https://github.com/pypsa-meets-earth/pypsa-earth/blob/aab4d18542466e3ffd89cf5af19f5be99aabdf6a/config.default.yaml

I also reported this upstream but figured it might be useful to ban the broken versions here as well

It seems that since the ini formatter is broken since including v2.0.0.

This file for example:

[   flake8  ]



max-line-length    =         88

extend-ignore          =
                                         E203

passes, not being reformatted.
In v1.6.1 this file was reformatted to:

[   flake8  ]
max-line-length = 88
extend-ignore =
	E203

Cheers, lcnittl

this should not use AOSP as a default