I would be really interested in using this tool with C# (.cs) code, any chance you can add it to the list of supported languages?
It has a treesitter parser: https://github.com/tree-sitter/tree-sitter-c-sharp
A C# identifier is not allowed to start with $ so it'd work great (makes me wonder... how do you handle that in JS, where $ is a valid identifier?)

I would also love if ast-grep could work with TS/JS code that is inside Vue SFC .vue files, but that might be more difficult as you'd need a .vue SFC parser to extract the <script> block(s) and figure out their lang attribute...

When i type a command like this:

sg --pattern '$A || $B' --lang ts .////////

The found file output like this:

.////////compiler\factory\utilitiesPublic.ts

What happened?

Any tips would be appreciated, thanks!

I followed the quick start guide but got the following:

$ cargo install ast-grep
...
$ cd /tmp
$ git clone [email protected]:microsoft/TypeScript.git --depth 1
...
$ cd TypeScript
$ sg -p '$PROP && $PROP()' -l ts .
./tests/cases/conformance/types/namedTypes/optionalMethods.ts
17|    let g1 = x.g && x.g();
44|    let g1 = x.g && x.g();
46|    let h1 = x.h && x.h();
munmap_chunk(): invalid pointer
[1]    25462 IOT instruction (core dumped)  sg -p '$PROP && $PROP()' -l ts .

Searching in src/ works fine:

$ sg -p '$PROP && $PROP()' -l ts src
src/services/services.ts
1541|    documentRegistry: DocumentRegistry = createDocumentRegistry(host.useCaseSensitiveFileNames && host.useCaseSensitiveFileNames(), host.getCurrentDirectory()),
2190|        const customTransformers = host.getCustomTransformers && host.getCustomTransformers();
src/services/stringCompletions.ts
596|    const ignoreCase = !(host.useCaseSensitiveFileNames && host.useCaseSensitiveFileNames());
659|    const ignoreCase = !(host.useCaseSensitiveFileNames && host.useCaseSensitiveFileNames());
src/services/shims.ts
1395|                this.documentRegistry = createDocumentRegistry(host.useCaseSensitiveFileNames && host.useCaseSensitiveFileNames(), host.getCurrentDirectory());
src/tsserver/nodeServer.ts
949|                    (os.homedir && os.homedir()) ||
974|        const homePath = (os.homedir && os.homedir()) ||
src/compiler/moduleSpecifiers.ts
936|    const globalTypingsCacheLocation = host.getGlobalTypingsCacheLocation && host.getGlobalTypingsCacheLocation();
src/harness/harnessIO.ts
174|        tryEnableSourceMapsForHost: () => ts.sys.tryEnableSourceMapsForHost && ts.sys.tryEnableSourceMapsForHost(),
175|        getMemoryUsage: () => ts.sys.getMemoryUsage && ts.sys.getMemoryUsage(),

Info

$ cargo install ast-grep  # no `sg --version` flag?
    Updating crates.io index
     Ignored package `ast-grep v0.1.11` is already installed, use --force to override
$ rustc --version
rustc 1.65.0 (897e37553 2022-11-02)
$ neofetch
            .-/+oossssoo+/-.               schneider@lenovo
        `:+ssssssssssssssssss+:`           ----------------
      -+ssssssssssssssssssyyssss+-         OS: Ubuntu 22.10 x86_64
    .ossssssssssssssssssdMMMNysssso.       Host: 80R0 Lenovo G50-80
   /ssssssssssshdmmNNmmyNMMMMhssssss/      Kernel: 5.19.0-26-generic
  +ssssssssshmydMMMMMMMNddddyssssssss+     Uptime: 58 mins
 /sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Packages: 2940 (dpkg), 14 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Shell: zsh 5.9
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Resolution: 1440x900
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   DE: GNOME 43.0
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   WM: Mutter
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   WM Theme: Adwaita
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Yaru-dark [GTK2/3]
 /sssssssshNMMMyhhyyyyhdNMMMNhssssssss/    Icons: Yaru-dark [GTK2/3]
  +sssssssssdmydMMMMMMMMddddyssssssss+     Terminal: tmux
   /ssssssssssshdmNNNNmyNMMMMhssssss/      CPU: Intel i7-5500U (4) @ 3.000GHz
    .ossssssssssssssssssdMMMNysssso.       GPU: Intel HD Graphics 5500
      -+sssssssssssssssssyyyssss+-         Memory: 3603MiB / 7864MiB

Good afternoon, this is such an interesting project with huge potential!

I want to write a rule which finds functions without a return type. For example, this code would trigger an error:

const foo = () => {
	return 1;
}

While this code would not:

const foo = (): int => {
	return 1;
}

The TS playground for the latter function looks like this:

My first thought was to make a rule which matches arrow_function (easy..) but when it comes time to find a missing value, I hit a wall.

Any suggestions or direction? This was my latest rule:

id: empty-return-type
message: Must supply a return type to a function
severity: warning
language: TypeScript
rule:
  all:
    - not: # doesn't work - matches all ast nodes except a type annotation. 
        kind: type_annotation
    - inside:
        any:
          - kind: arrow_function
          # TODO Insert more function types here

Please take care and thank you for reading.

I have the following Rust code:

patterns = match config.include.clone() {
    Some(patterns) => patterns,
    None => Vec::from([cwd
        .join("**/*.toml")
        .normalize()
        .to_string_lossy()
        .into_owned()]),
};

And I want to replace with this:

patterns = config.include.clone().unwrap_or_else(|| {
    Vec::from([cwd
        .join("**/*.toml")
        .normalize()
        .to_string_lossy()
        .into_owned()])
})

I tried to do it with this command:

sg -p '$A match { Some($X) => $X, None => $N, }' -r '$A.unwrap_or_else($N)' --lang rust

But it can't match pattern. How can I solve it? Also, maybe it would be better if README has examples with this complex types of AST nodes.

I'm trying to simply search for all expressions that match a pattern and summarize them in a downstream piped command. It'd be nice to not have to parse ANSI codes to do that. 😕 Great tool otherwise!

• bump version
  • napi package json
  • cargo toml
  • cargo lock
  • cli package json
• change log

Hi, @HerringtonDarkholme I'm digging into the source code.

When i run sg --help

USAGE:
    sg.exe [OPTIONS] --pattern <PATTERN> --lang <LANG> [PATH]

ARGS:
    <PATH>    The path whose descendent files are to be explored [default: .]

OPTIONS:
        --debug-query          Print query pattern's tree-sitter AST
    -h, --help                 Print help information
        --hidden               Include hidden files in search
    -i, --interactive
    -l, --lang <LANG>          The language of the pattern query
    -p, --pattern <PATTERN>    AST pattern to match
    -r, --rewrite <REWRITE>    String to replace the matched AST node

The -i, --interactive line is missing description.

If you don't mind, I can try adding a description like this:
Start an interactive session

Currently we only have wasm imported, but none of them actually works with variable

https://medium.com/source-and-buggy/data-driven-performance-optimization-with-rust-and-miri-70cb6dde0d35

low priority

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency @napi-rs/cli to v2.18.2
• chore(deps): update dependency @types/node to v20.12.7
• chore(deps): update dependency typescript to v5.4.5
• chore(deps): update rust crate anyhow to 1.0.82
• chore(deps): update rust crate napi-build to 2.1.3
• chore(deps): update rust crate thiserror to 1.0.59
• fix(deps): update dependency @swc/core to v1.4.16
• fix(deps): update rust crate clap_complete to 4.5.2
• fix(deps): update rust crate napi to 2.16.4
• fix(deps): update rust crate napi-derive to 2.16.3
• fix(deps): update rust crate pyo3 to 0.21.2
• fix(deps): update rust crate serde_json to 1.0.116
• fix(deps): update rust crate toml_edit to 0.22.12
• chore(deps): update robinraju/release-downloader action to v1.10
• fix(deps): update rust crate tree-sitter-elixir to 0.2.0
• fix(deps): update rust crate tree-sitter-java to 0.21.0
• fix(deps): update rust crate tree-sitter-json to 0.21.0
• 🔐 Create all rate-limited PRs at once 🔐

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• fix(deps): update dependency tree-sitter to v0.21.1
• fix(deps): update rust crate tree-sitter-bash to 0.21.0
• fix(deps): update rust crate tree-sitter-c to 0.21.0
• fix(deps): update rust crate tree-sitter-cpp to 0.22.0
• fix(deps): update rust crate tree-sitter-go to 0.21.0
• fix(deps): update rust crate tree-sitter-javascript to 0.21.0
• fix(deps): update rust crate tree-sitter-lua to 0.1.0
• fix(deps): update rust crate tree-sitter-native to 0.22.5
• fix(deps): update rust crate tree-sitter-rust to 0.21.0
• fix(deps): update rust crate tree-sitter-scala to 0.21.0
• Click on this checkbox to rebase all open PRs at once

Ignored or Blocked

These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.

• fix(deps): update rust crate serde_yaml to 0.9.34
• fix(deps): update rust crate tree-sitter-python to 0.21.0
• chore(deps): update dependency ubuntu to v22
• chore(deps): update github artifact actions to v4 (major) (actions/download-artifact, actions/upload-artifact)

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

I observed it is lacking when I submitted #109. It would be useful to have one.

I wanted to package ast-grep for Homebrew and Nixpkgs, but I can't without a license.

I look forward to trying this someday when I can install it! If this is working for anyone else, I'll come back with more details. For now, I have to run so it's just a friendly heads up. I'm on a MBP (intel) Monterey 12.6, fish shell, node 16.17.0, npm 8.15.0.

Thanks! Good luck!

@sxzz may be interested in the issue

Like parents from jQuery.

  ancestors(filter?: string): Array<SgNode>

We could add more variants of SgNode, like SgNode<CallExpression>, SgNode<StringLiterial>

@babel/types is a good example for reference.

The cli command name sg is already taken by a shadow tools command

NAME
sg - execute command as different group ID

Cf. https://github.com/shadow-maint/shadow

It is installed by default in /usr/bin/sg at least on arch-linux.

Weirdly npm install -g @ast-grep/cli didn’t even install the cli in ~/.local/bin for me (I have prefix=~/.local configured for npm). Maybe that is also due to the name collision.

I actually like the pattern of .g for grep commands. Pitty that ag is already taken.
How about cg for code-grep?

We can use Color::Never to test text only.

When I run sg scan anywhere, it imediately fails with the following error:

thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', crates/cli/src/config.rs:8:48

The rust backtrace does not produce anything useful.

for example, matching script/style in Vue/svelte/html. Or js/rb in ejs/erb.

for reusing common languages in playground or as library.

This includes

• matched meta variables
• labels

Sourcemap is almost always needed for rewriting in code transformation tool. Supporting it in ast-grep core is reasonable.

Currently, I am writing automation for my Vue project and due to AST headache, I am using REGEX to replace the content.

I found ast-grep and it looks promising ❤️

One thing missing in playground compared to CLI mode is rewriting. Can you please add it so someone can just paste and try it before running on CLI?

P.S. It will be great if you add more examples to docs

Awesome project! I'm interested in potentially using ast-grep as the foundation for Ruff's plugin system (some discussion in astral-sh/ruff#283).

What I'm envisioning: users express custom lint rules via the ast-grep YAML syntax; Ruff picks up those rules, parses + executes them via ast-grep, and includes the surfaced messages + fixes in its own output abstractions.

Before I dig in, I wanted to see if (1) this kind of use-case is within-scope for ast-grep, (2) if it's possible now or depends on future work, and (3) if you had any pointers on how to get started (I know the API docs are a work-in-progress so any tips would be appreciated).

How can I change the output colors sg gives me? The current ones seem pretty unusable for me,

I think the issue here is that I'm using a light font color, which doesn't contrast well with the pink/light green highlights. Maybe related to #2?

I am using TS project to modify my source code.

Can you please add docs on TS API?

Thanks :)

Currently all matching is done in main thread.

At the moment we use stdout lock for printing matches. This is slow and the actual matching is single threaded.
So using mspc is performance-wise equivalent to current implementation.

This can help organize a lot of common code in scan/run command.

trait Worker {
  type Item;
  fn walker() -> Walker;
  fn producer() -> Item;
  fn consumer(i: Item) -> Result<()>;
}