cristianoliveira / ergo Goto Github PK

Today we use as default the TLD .dev but would be nice to change it when needed or for test. So we should have a flag on run command in order to change it for a custom "domain".

ergo run -domain=.test

Things it must have:

• Domain validation. it must validate the domains, for instance foo is not but .foo is valid.
• It must reflect on localhost:2000/proxy.pac
• A simple flag name. feel free to come up with your idea.
• Update readme with it.
• A test would be nice, but if you think it is hard to test we can help you.

As reported by @ebastos , the install script is falling because of some incompatible "bashisms" I think this script should be as POSIX compliant as possible to work in most shells out there. So I am opening this issue for a refactor in this script.

The error in ubuntu:

$ cat /etc/issue.net 
Ubuntu 17.04
$ ls -ld /bin/sh
lrwxrwxrwx 1 root root 4 Jun  6 17:53 /bin/sh -> dash

Which causes the installer to fail:

$ curl -s https://raw.githubusercontent.com/cristianoliveira/ergo/master/install.sh | sh
sh: 5: declare: not found
sh: 6: Syntax error: "(" unexpected
A possible way to get the install script more POSIX would be to get rid of some bashisms. We can do this using case instead of the current associative array.

Another way would be just using x86_64 instead of linux-amd64 and make it simpler.

I am having the following error sporadically:

Try contacting your system administrator.
ERR_ICANN_NAME_COLLISION

I searched around to understand it and reached this thread.
https://superuser.com/questions/919278/err-icann-name-collision-when-trying-to-use-localhost-dev-in-chrome

It seems it occurs because .dev domain has become a registered ICANN Top Level Domain. So some domains that really exist and are registered can collide.

So I would like to open a discussion for changing the .dev to another TLD which is not registered yet.

My suggestions would be .local but I am not convinced about this option yet

What do you think?

When running the tests it tries to find an ergo binary inside of bin folder.
In order to ensure that it will be there, we should include it into Makefile:

test: build
	@go test ./... -v

and then use make test for testing.

PLUS: Would be nice to have a section on README (Testing) about how to test properly the project. Something like this:
https://github.com/cristianoliveira/funzzy#tests

refers to #24

Hey, I just came across this via an answer on stackoverflow. I'd love to be able to use this, but I have a few local servers across subdomains, and I do need them to be subdomains, I can't just use different domains for them because they share cookies.

Thanks for the project, and I'm a big Ergo Proxy fan too :)

I was thinking now that it's starting to have more options/commands/configs if ergo need some kind of documentation. Is it already necessary?

It would be nice to have a validation every time ergo runs in order to avoid badly formatted files and undefined errors.

That is simple. Just record your screen basically doing the same steps that we have on currently demo. After you will need to parse this video into an Animated GIF. Please make sure that the quality is good enough. Then override the demo.gif with your new one.

Feel free to ask any questions regarding this process.

Enable use of verbose flag. It should log each request that pass through the proxy. It will help in debugging ergo and the apps running on it. :)

in the folder start simple http server

➜ python3 -m  http.server 8080 

then configure ergo per documentation

➜ echo "test http://localhost:8080" > .ergo
➜ ergo list
Ergo Proxy current list:
 - http://test.dev -> http://localhost:8080 

➜ ergo run                    
Ergo Proxy listening on port 2000 for domains .dev

Using httpie

➜ http http://test.dev/ 

http: error: ConnectionError: HTTPConnectionPool(host='test.dev', port=80): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f519d5af490>: Failed to establish a new connection: [Errno 111] Connection refused',)) while doing GET request to URL: http://test.dev/

However when I go through python's process, it works

➜ http http://localhost:8080/main.js
HTTP/1.0 200 OK
Content-Length: 189
Content-type: application/javascript
Date: Sat, 27 Jan 2018 19:56:30 GMT
Last-Modified: Sat, 27 Jan 2018 03:34:43 GMT
Server: SimpleHTTP/0.6 Python/3.6.4

import Vue from "vue";
import store from "./vuex/store";
import App from "./components/App.vue";

new Vue({
  store, // inject store to all children
  el: "body",
  components: { App }
});

➜ ergo -v       
v0.2.2

➜ uname -a
Linux dmitry.io 4.14.14-300.fc27.x86_64 #1 SMP Fri Jan 19 13:19:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

For now, I use ergo list and open the apps using the printed links. It may be convenient an endpoint to see on the browser

I just noticed that we could youse the 'foo.localhost' and it will point to 127.0.0.1 but the proxy must bind at port 80. We should use this as an alternative way to create the subdomains :)

Running ergo run using verbose flag is not working. Mainly because the subcomand run doesn't accept the verbose flag even though the usage says.

It's a good opportunity to sort the way we currently manage the flags its becoming complex because it has more subcommands than at the beginning.

This is the problematic line
https://github.com/cristianoliveira/ergo/blob/master/main.go#L96

Would be nice to have tests for it.

Right now there is a repetitive code that copies the original content of the config file at the beginning of the test and then rewrites it at exit so that we can have the same values at each test.
see

• https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L113
• https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L599
• https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L672
• https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L672

A file copy of the original config should be done and that should be used instead. This could be put in a separate method to avoid code repetition.

While testing on windows I noticed that when running ergo using a bad file it raises the follow error:

ergo run
panic: runtime error: index out of range

goroutine 1 [running]:
github.com/cristianoliveira/ergo/proxy.LoadServices(0x12d8403, 0x7, 0x0, 0x0, 0x0)
        /Users/cristianoliveira/go/src/github.com/cristianoliveira/ergo/proxy/config.go:61 +0x43e
main.main()
        /Users/cristianoliveira/other/ergo/main.go:58 +0x2f2

The file .ergo content:

foo

run:

ergo -v

output:

# A simple blank line

Upgrade log message to this format:

[datetime] "<HTTP Verb> <Route>" <StatusCode>

# Example
[28/Aug/2017 14:16:51] "GET http://feedcast.dev./favicon.ico" 404

Today we have only support for http:// and it is not possible to map for instance Redis or Mongo:

• mongodb://localhost: 27017
• redis://localhost:6379

It would be nice if we can map them

Today we have 30% for go lint on goreport. We can do better.

https://goreportcard.com/report/github.com/cristianoliveira/ergo

Golint is a linter for Go source code.

        ergo/commands/setup.go
        Line 12: warning: exported function Setup should have comment or be unexported (golint)
        Line 14: warning: don't use underscores in Go names; var proxy_url should be proxyURL (golint)

        ergo/proxy/config.go
        Line 10: warning: exported type Service should have comment or be unexported (golint)
        Line 15: warning: exported type Config should have comment or be unexported (golint)
        Line 23: warning: exported method Config.GetService should have comment or be unexported (golint)
        Line 35: warning: exported function NewConfig should have comment or be unexported (golint)
        Line 44: warning: exported function LoadServices should have comment or be unexported (golint)

        ergo/commands/url.go
        Line 8: warning: exported function Url should have comment or be unexported (golint)

        ergo/proxy/proxy.go
        Line 37: warning: exported function NewErgoProxy should have comment or be unexported (golint)
        Line 78: warning: exported function ServeProxy should have comment or be unexported (golint)

        ergo/main.go
        Line 11: warning: exported var VERSION should have comment or be unexported (golint)
        Line 13: warning: exported const USAGE should have comment or be unexported (golint)

        ergo/tests/ergo_run_test.go
        Line 22: warning: don't use underscores in Go names; var apps_outoput should be appsOutoput (golint)
        Line 47: warning: don't use underscores in Go names; var apps_outoput should be appsOutoput (golint)
        Line 72: warning: don't use underscores in Go names; var apps_output should be appsOutput (golint)

        ergo/commands/list.go
        Line 8: warning: exported function List should have comment or be unexported (golint)

        ergo/commands/list_names.go
        Line 8: warning: exported function ListNames should have comment or be unexported (golint)

        ergo/commands/run.go
        Line 9: warning: exported function Run should have comment or be unexported (golint)

We need to add some integration tests cause part of the code remains untouched by the current tests like the /commands.

The osx setup fails when the system is not on the wifi network.

In #44 the tests for OSX just skip if the setup fails.
This happens here:
https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L362
and here:
https://github.com/cristianoliveira/ergo/blob/master/tests/ergo_run_test.go#L508

There is also a mention in the comments there.
They should fail if setup fails just like Linux and Windows tests do.

We want to avoid run integration tests (tests inside tests folder) when not passed the flag -integration to go test command. go test .../. -v -integration.

Please take a look on how to implement this here:
https://siongui.github.io/2017/04/28/command-line-argument-in-golang-test/

This refers to #24

While trying to understand the code, I tried to run the tests. There we are looking for ergo binary using relative path in ergo_run_test.go. Can we simply replace it ergo, since we might already have it installed in out $PATH?

The issue #54 brought a good idea. In order to make easy installing ergo for Windows I think we can have a script like install.sh that we have for linux.

This line:
https://github.com/cristianoliveira/ergo/blob/master/proxy/config.go#L89

Mutext use should be changed to avoid race conditions. It easier to change the config struct to not use an instance of a mutex manager.
So instead of

type Config struct {
	mutex      *sync.Mutex

We use:

type Config struct {
	mutex      sync.Mutex

And rely on zero-value feature for "instantiating" it

Considering Go is easily supported by multiple Operating Systems and platforms, the install script should be flexible enough to allow future expansion beyond Linux x64.

I have already tested the system on FreeBSD (after small modifications).
The idea is that it runs ok on FreeBSD and perhaps it would run the same on every unix.
To make the system easier to use I believe that we should call the setup switch for linux and freebsd: gnome instead of linux-gnome and separately freebsd-gnome.
Both Linux and FreeBSD are used with the Gnome desktop, thus both of them use the same commands (gsettings set and get) to setup the proxy.
By doing so, we can also unite the tests for linux and freebsd.
Being an radical change I decided it is better to ask for opinions.
Should I continue with the port?

This is just a suggestion.
I was thinking that maybe it is better to separate the unit tests from the integration tests (which seem to be more end-to-end tests - see what a Google blog says). This is in the Makefile of course.
This way we can test the code with the unit tests without compiling the binary and then run the integration/end-to-end tests only after the build.
I say that these could be end-to-end tests especially when we add the tests for the setup and for the run . As the setup and run part especially should test if the proxy is being used, and also if an outside page will be accessible while running the proxy.
The new flow would be:

1. Run the unit tests
2. Run the build
3. Run the end-to-end tests
4. Create the releases (if it is the case).
   This would solve the problem with the tests depending on the executable.

Some times when I run ergo and there is another process running on 2000 port it doesn't run. It should ask me If I want to kill the other instance.

Enable ergo to watch for config changes and respond to it.

As a user, I want to point a custom location of my .ergo file with a flag:

ergo -config=/tmp/ergofile

This is a problem when we have a host defined as redis://redishost for example.
For this the returned value would be right now:
http://redis://redishost.dev

This issue is needed in order to enable run ergo using a custom TDL like .test or any other the user would like to use.

We need to decide the name of this file and its format.

Now that is possible to config ergo through Environment Variables, would be nice to mention it on the README.

By this I mean adding compliance checking with the go coding standards ( gofmt, go vet and golint).

Add a ergo show [name] command for showing the url for the given name. It will enable us to do things like this:

# OSX
open $(ergo show foo)

# Linux
xdg-open $(ergo show foo)

# or
curl $(ergo show foo)

I quickly set up the following wrapper--maybe something like it should be baked in, and triggered on a flag? This one is Ubuntu-only:

function finish {
  # these should probably revert to original settings
  gsettings set org.gnome.system.proxy mode "none"
  gsettings set org.gnome.system.proxy.http host ""
  gsettings set org.gnome.system.proxy.http port "8080"
}
trap finish EXIT

gsettings set org.gnome.system.proxy mode 'auto'
gsettings set org.gnome.system.proxy autoconfig-url http://127.0.0.1:2000/proxy.pac 

$GOPATH/bin/ergo run

In a discussion on slack channel I got some good feedback about the code and one of the ideas is instead of using a slice of Services use a map. This makes the function GetService practically not necessary plus it guarantees the services to be Unique.

The setup command aims to auto-configure your system to use ergo as the proxy. Currently, we only support two systems OSX and Linux-gnome. Would be nice to expand this support to more systems/flavors.

Here is the implementation:
https://github.com/cristianoliveira/ergo/blob/master/commands/setup.go

Your task is:

• Discover how to set up a proxy by command line. For instance for gnome we do the follow commands: 👨‍🔬 👩‍🔬

gsettings set org.gnome.system.proxy mode 'auto'
gsettings set org.gnome.system.proxy autoconfig-url http://127.0.0.1:2000/proxy.pac 

• Implement that on commands/setup.go 🛠
• Test if it actually works 👍
• Make the pull request. 📦
• Scores up toward your goal that is the T-shirt we all want 🎉

Create a command that would do the same as echo "service http://service.de" >> .ergo just for convenience.

The idea is a command like this:

ergo add <localname> <url:port>

Once we call this, I hope the app +url will be included into .ergo file

Thanks to @adiclepcea and his great work with the integration tests ergo is mostly covered by tests! 🥂

Although we are mostly covered, it doesn't reflect on codecov.io cause it gets only test coverage by "unit tests" and on the integration tests we use a compiled ergo's binary.

Today at codecov we have the follow report:

Would be nice to improve coverage in proxy folder and has at least some in commands 😅

It would be nice to have support to https. It will need a self-signed key.
For more examples on how to implement this, please take a look on that:

https://gist.github.com/denji/12b3a568f092ab951456

I got another feedback from the Slack Channel:

WatchConfigFile can run in parallel and change Services by itself behind a mutex, because that's basically what happens right not anyways.

Other:

Anyone access this config can be synchronized using the same mutex. The caller already has the goroutines. This is not an orchestration problem but a synchronization problem.

So I am thinking that we can get rid of the channels and just work with the mutex + gorotine for this case.

We want to add coverage metrics and docs in order to add Ergo to the AwesomeGo.

https://github.com/avelino/awesome-go/blob/master/CONTRIBUTING.md#quality-standards

Today we have the add command for adding new services into .ergo file. Would be nice to have a remove/delete service as well.

It must have:

• Option to remove by name or url
• A message in case of service not found
• Unit tests

I am having an error when running ergo without providing a command.

Expected behavior:
Show USAGE

Actual behavior:

ergo
panic: runtime error: index out of range

goroutine 1 [running]:
main.command(0x12da13e)
        /Users/cristianoliveira/go/src/github.com/cristianoliveira/ergo/main.go:45 +0xc8c
main.main()
        /Users/cristianoliveira/go/src/github.com/cristianoliveira/ergo/main.go:127 +0xb5

The problem is here. We should look if os.Args has more than 1 item before creating the command

func command() func() {
	config := proxy.NewConfig()
>>>	command := flag.NewFlagSet(os.Args[1], flag.ExitOnError)
	configFile := command.String("config", "./.ergo", "Set the services file")
	command.Parse(os.Args[2:])

It should be easy to solve.

Please also provide a test for it

Hello, fellow Hacktoberfest contributor!

I am opening this issue especially for Hacktoberfest. I am not a native English speaker and I am so used to read the project readme that I am not able to see all typos it might have.

So I invite you to take a look at it and if you find out either a typo or a weird sentence please feel free to point it out and open a PR with a fix for it. 😄

Also feel free to suggest description changes to make it easier to understand the purpose of this project/app 😃

Please make sure you point this issue on your PR (refers #37)

⚠️UPDATE: We are having a lot of PR for this. Please make sure you took a look at others Open PRs before opening a new one :)

Please feel free to ask anything.