Specifically, I was testing:

s := "pg_dump db -U usr -F c > bck.dump"
argv, _ := shellwords.Parse(s)
for _, v := range argv {
  println(v)
}

Which returned:

pg_dump
db
-U
usr
-F
c

I noticed in the source code (https://github.com/mattn/go-shellwords/blob/master/shellwords.go#L246) that we're not accepting ">" as a token amongst other characters like "|". Since these characters are very important in certain contexts, I think it might make sense to add a flag which will recognize them as tokens (by flag I mean something similar to p.ParseBacktick = true).

In Windows, backslashes are used at the OS path separator. When I run this code, I get an incorrect result for the value of cmd:

func main() {
	sh := `upx c:\github.com\jftuga\test\test.exe`
	cmd, err := shellwords.Parse(sh)
	if err != nil {
		fmt.Println("err:", err)
		return
	}
	fmt.Println(cmd)
}

This outputs:

[upx c:github.comjftugatesttest.exe]

when it should output:

[upx c:\github.com\jftuga\test\test.exe]

Can this be fixed, please?

I just wanted to say thank you, I've made a very small utility because I'm lazy and your package is a central part of if!

Would you consider adding support for $(command) besides supporting backticks for executing subcommands?

Hi, i suggest the ability to replace the os.Getenv function with a generic function - to be more specific adding another field to parser (https://github.com/mattn/go-shellwords/blob/master/shellwords.go#L34):

type Parser struct {
	ParseEnv      bool
        Getenv        func(string) string
	ParseBacktick bool
	Position      int
}

if it is nil, os.Getenv is used (this is for backward compatibility). If it weren't for backward compatibility, I would suggest it replaces the ParseEnv variable entirely.

Would be happy to send out a PR if this makes sense.

When ParseEnv is true, variables like
line := "ssh 127.0.0.1 "echo $JAVA_HOME""
is parsed (e.g. shellwords.Parse(line)) into one element slice:
["ssh", "127.0.0.1", "echo", "/usr/local/jdk1.8.0_171"]
But the expected result should be a 3-element slice:
["ssh", "127.0.0.1", "echo /usr/local/jdk1.8.0_171"]

line := "ansible 127.0.0.1 -m shell -a "ps -ef | grep console""
is parsed (e.g. shellwords.Parse(line)) into one element slice:
["ansible", "127.0.0.1", " -m", "shell", "-a", "ps", "-ef"]
But the expected result should be a 6-element slice:
["ansible", "127.0.0.1", " -m", "shell", "-a", "ps -ef | grep console"]

Hi,

It looks like the parsing of $() is not supported. Any plans on adding this?

E.g.
"foo $(bar baz)" --> ["foo", "$(bar baz)"]

Thank you!

go-shellwords fails to parse sh -c "printf 'Hello\tworld\n'"
it get parsed as ["sh", "-c", "printf 'Hellotworldn'"]

I spent quite a bit of time trying to understand a weird problem with Hashicorp Vault Agent. The post-template-rendering command execution failed. The error was unhelpful:

failed parsing command: fork/exec : no such file or directory:

"

After much befuddlement, it eventually turned out that this only happened when the command contained backticks, and was due to SHELL not being set in the environment, leading to a failure in shellRun in util_posix.

It seems like it would be reasonable to default to /bin/sh here if the variable is not set.

I would expect that

Parse(`foo "" bar ''`)

would give ["foo", "", "bar", ""] with the empty strings appearing as results, but this is not the case, you would only get ["foo","bar"]. Is this intended behavior?

Bad Case

I tried to parse curl http://httpbin.org/get?a=1&b=2 to shell word list as the following code

package main

import (
	"fmt"
	"log"

	"github.com/mattn/go-shellwords"
)

func main() {
	sw, err := shellwords.Parse("curl http://httpbin.org/get?a=1&b=2")
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(sw)
}

• expect: [curl http://httpbin.org/get?a=1&b=2]
• actal: [curl http://httpbin.org/get?a=1]

version

go: 1.16.5
go-shellwords: v1.0.12

My code uses | pipe character, the @753a2322a99f87c0eff284980e77f53041555bc6 commit now captures this character and discards the remaining line.

Could you add an option to disable this? And the other characters now being captured by the library.

When ParseEnv is true, shell variables like
export CMD='program -f abc.txt'
or
export CMD="program -f abc.txt"
is parsed (e.g. shellwords.Parse("${CMD}")) into one element slice:
["program -f abc.txt"]
But the expected result should be a 3-element slice:
["program" "-f" "abc.txt"]

Hello, thank you for this great library.
I've been using go-shellwords in my applications and it greatly helps me.

I've noticed go-shellwords's quote handling is different from general shells when ParseEnv is true.

Is it possible to change that, or is this intentional behavior?

shell

$ cat a.sh
echo 0:$1
echo 1:$2
echo 2:$3

$ bash a.sh a '' c
0:a
1:
2:c

go-shellwords

parser := shellwords.Parser{ParseEnv: true}
result, _ := parser.Parse("a '' c")
for i, v := range result {
  fmt.Printf("%d:%s\n", i, v)
}
// Result:
// 0:a
// 1:c

It's not uncommon, in certain scenarios, to use parens inside args.

An example can be seen in the Traefik docs here (click CLI and look at the example), where it's expected that, to set a specific value, the arg --providers.docker.constraints=Label('a.label.name','foo') be used.

With the current implementation of go-shellwords, parsing a string including an arg like that throws an error, even though this is a valid shell-word string.

Hi @mattn , I found a thing while trying to use go-shellwords.

go test -v 
=== RUN   ExampleShellwords
--- FAIL: ExampleShellwords (0.02s)
got:
["-flag=val0" "val1" "-flag=val2"]
want:
["-flag=val0" "-flag=val1" "-flag=val2"]
FAIL
exit status 1
FAIL	testsw	0.041s

cat testsw_test.go 
package testsw

import (
	"fmt"
	"github.com/mattn/go-shellwords"
)

func ExampleShellwords() {
	parser := shellwords.NewParser()
	parser.ParseBacktick = true
	args, err := parser.Parse("-flag=val0 -flag=`cat testdata/flag-val.txt` -flag=val2")
	if err != nil {
		panic(err)
	}
	fmt.Printf("%q\n", args)
	// Output:
	// ["-flag=val0" "-flag=val1" "-flag=val2"]
}

cat go.mod 
module testsw

go 1.12

require github.com/mattn/go-shellwords v1.0.5

cat testdata/flag-val.txt 
val1

I appreciate this project is old and may not be maintained anymore, but before forking it I thought I'd file my particular feature request in the event that it is still maintained.

It would be great if the Parser had an option to ParseSmartQuotes, which would:

• Treat “ and ” as ".
• Treat ‘ and ’ as '.

That's it. Thanks!

$ cat shellparse.go
package main
import ("github.com/mattn/go-shellwords"; "fmt")

func main() {
	args, _ := shellwords.Parse("a > b")
	fmt.Println(args)
}

yields in

$ go run shellparse.go 
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/mattn/go-shellwords.(*Parser).Parse(0xc420051ef8, 0x4f6e67, 0x5, 0x0, 0x0, 0xc420051f01, 0xc42000e230, 0xc420051f10)
        /home/alex/.go/src/github.com/mattn/go-shellwords/shellwords.go:143 +0xdff
github.com/mattn/go-shellwords.Parse(0x4f6e67, 0x5, 0x403d8c, 0xc42001c0b8, 0x0, 0xc420051f68, 0x4bb063)
        /home/alex/.go/src/github.com/mattn/go-shellwords/shellwords.go:177 +0x72
main.main()
        /home/alex/lib/software/example/go/src/my/hello/shellparse.go:5 +0x3a
exit status 2

$ go version
go version go1.10.2 linux/amd64

Quoted whitespace-only arguments are ignored.

Please see this diff for added testcases:

index 2c47b06..f8c9741 100644
--- a/shellwords_test.go
+++ b/shellwords_test.go
@@ -23,6 +23,14 @@ var testcases = []struct {
        {`var "--bar baz"`, []string{`var`, `--bar baz`}},
        {`var --"bar baz"`, []string{`var`, `--bar baz`}},
        {`var  --"bar baz"`, []string{`var`, `--bar baz`}},
+
+       {`a "b"`, []string{`a`, `b`}},
+       {`a " b "`, []string{`a`, ` b `}},
+       {`a "   "`, []string{`a`, `   `}}, // FAILS !
+
+       {`a 'b'`, []string{`a`, `b`}},
+       {`a ' b '`, []string{`a`, ` b `}},
+       {`a '   '`, []string{`a`, `   `}}, // FAILS !
 }
 
 func TestSimple(t *testing.T) {

Hello, I'm trying to parse the following command:

command := "RUN /bin/sh -c echo hi --password "pdhjcrkCHJfSchjd3f837"/ # buildkit"
args, err := shellwords.Parse(command)

and I'm getting
"RUN", "/bin/sh", "-c", "echo", "hi", "--password", "pdhjcrkCHJfSchjd3f837/", "#", "buildkit"
so one of the arguments is "pdhjcrkCHJfSchjd3f837/" and it's wrong.

Do you have an idea what can I do in order to avoid it?

Thank you :)

Hi, I noticed shellwords.Parse does not handle backslash-escaped whitespaces at the end of the argument.

words, err := shellwords.Parse("trailing escaped space\\ ")
// invalid command line string

This is due to strings.TrimSpace at the beginning of the function.

When i do 'go get github.com/mattn/go-shellwords' on my vps which is Debian Jessie 8 32 bit i get the error eerr.Stderr undefined (type *exec.ExitError has no field or method Stderr)

Docker is using this as vendor package. In docker we can check for checking copyright for all the vendor packages and expects any files in .*LICENSE.*' '.*COPYRIGHT.*' '.*COPYING.*' pattern, currently while validating this package it is printing the warning message.

Please create explicit licence file to avoid any warnings while validation, so create a LICENCE file in parent directory with the MIT licence referred in the README file.

Following commands throws errors:

• $(echo xxx | xargs)
• $(sleep 1 && sleep 1)
• $(sleep 1; sleep 1)

Here is full runnable code: https://github.com/curusarn/go-shellwords-issue/blob/master/main.go

Output:

Commandline: $(sleep 1; sleep 1)
2019/08/10 22:19:46 invalid command line string
exit status 1