Giter VIP home page Giter VIP logo

goatcounter's Introduction

GoatCounter is an open source web analytics platform available as a (free) hosted service or self-hosted app. It aims to offer easy to use and meaningful privacy-friendly web analytics as an alternative to Google Analytics or Matomo.

There are two ways to run this: as hosted service on goatcounter.com, or run it on your own server. The source code is completely Open Source/Free Software, and it can be self-hosted without restrictions.

See docs/rationale.md for some more details on the "why?" of this project.

There's a live demo at https://stats.arp242.net.

Please consider contributing financially if you're using goatcounter.com to pay for the server costs.

Features

  • Privacy-aware; doesn’t track users with unique identifiers and doesn't need a GDPR notice. Fine-grained control over which data is collected. Also see the privacy policy and GDPR consent notices.

  • Lightweight and fast; adds just ~3.5K of extra data to your site. Also has JavaScript-free "tracking pixel" option, or you can use it from your application's middleware or import from logfiles.

  • Identify unique visits without cookies using a non-identifiable hash (technical details).

  • Keeps useful statistics such as browser information, location, and screen size. Keep track of referring sites and campaigns.

  • Easy; if you've been confused by the myriad of options and flexibility of Google Analytics and Matomo that you don't need then GoatCounter will be a breath of fresh air.

  • Accessibility is a high-priority feature, and the interface works well with assistive technology such as screen readers.

  • 100% committed to open source; you can see exactly what the code does and make improvements, or self-host it for any purpose.

  • Own your data; you can always export all data and cancel at any time.

  • Integrate on your site with just a single script tag:

    <script data-goatcounter="https://yoursite.goatcounter.com/count"
            async src="//gc.zgo.at/count.js"></script>
    
  • The JavaScript integration is a good option for most, but you can also use a no-JavaScript image-based tracker, integrate it in your backend middleware, or parse log files.

Getting data in to GoatCounter

There are three ways:

  1. Add the JavaScript code on your site; this is the easiest and most common method. Detailed documentation for this is available at https://www.goatcounter.com/code

  2. Integrate in your middleware; send data to GoatCounter by calling the API from your backend server middleware. Detailed documentation for this is available at https://www.goatcounter.com/api#backend-integration

  3. Parse logfiles. GoatCounter can parse logfiles from nginx, Apache, CloudFront, or any other HTTP middleware or proxy. See goatcounter help import for detailed documentation on this.

Running your own

Note this README is for the latest master and may be inaccurate for the latest released version; use the release-2.5 branch for the 2.5 README.

The release page has binaries for Linux amd64, arm, and arm64. These are statically compiled, contain everything you need, and should work in pretty much any Linux environment. The only other thing you need is somewhere to store a SQLite database file or a PostgreSQL connection.

GoatCounter should run on any platform supported by Go, but there are no binaries for them (yet) as cross-compiling SQLite is somewhat complex. You'll have to build from source if you want to run it on e.g. FreeBSD or macOS.

Generally speaking only the latest release is supported, although critical fixes (security, data loss, etc.) may get backported to previous releases.

Deploy scripts and such

Building from source

You need Go 1.21 or newer and a C compiler (for SQLite). If you compile it with CGO_ENABLED=0 you don't need a C compiler but can only use PostgreSQL.

You can install from source to $GOBIN (go env GOBIN) with:

% git clone --branch=release-2.5 https://github.com/arp242/goatcounter.git
% cd goatcounter
% go build -ldflags="-X zgo.at/goatcounter/v2.Version=$(git log -n1 --format='%h_%cI')" ./cmd/goatcounter

Which will produce a goatcounter binary in the current directory.

The -ldflags=[..] sets the version; this isn't strictly required as such, but it's recommended as it's used to "bust" the cache for static files and may also be useful later when reporting bugs. This can be any string and doesn't follow any particular format, you can also set this to the current date or banana or anything you want really.

To use the latest development version switch to the master branch.

To build a fully statically linked binary:

% go build -tags osusergo,netgo,sqlite_omit_load_extension \
    -ldflags="-X zgo.at/goatcounter/v2.Version=$(git log -n1 --format='%h_%cI') -extldflags=-static" \
    ./cmd/goatcounter

It's recommended to use the latest release as in the above command. The master branch should be reasonably stable but no guarantees, and sometimes I don't write detailed release/upgrade notes until the actual release so you may run in to surprises.

You can compile goatcounter without cgo if you're planning to use PostgreSQL and don't use SQLite:

% CGO_ENABLED=0 go build \
    -ldflags="-X zgo.at/goatcounter.Version=$(git log -n1 --format='%h_%cI')" \
    ./cmd/goatcounter

Functionally it doesn't matter too much, but builds will be a bit easier and faster as it won't require a C compiler.

Running

You can start a server with:

% goatcounter serve

The default is to use an SQLite database at ./db/goatcounter.sqlite3, which will be created if it doesn't exist yet. See the -db flag and goatcounter help db to customize this.

Both SQLite and PostgreSQL are supported. SQLite should work well for most smaller sites, but PostgreSQL gives better performance. There are some benchmarks over here to give some indication of what performance to expect from SQLite and PostgreSQL.

GoatCounter will listen on port *:80 and *:443 by default. You don't need to run it as root and can grant the appropriate permissions on Linux with:

% setcap 'cap_net_bind_service=+ep' goatcounter

Listening on a different port can be a bit tricky due to the ACME/Let's Encrypt certificate generation; goatcounter help listen documents this in depth.

You can create new sites with the db create site command:

% goatcounter db create site -vhost stats.example.com -user.email [email protected]

This will ask for a password for your new account; you can also add a password on the commandline with -password. You must also pass the -db flag here if you use something other than the default.

Updating

You may need to run the database migrations when updating. Use goatcounter serve -automigrate to always run all pending migrations on startup. This is the easiest way, although arguably not the "best" way.

Use goatcounter db migrate <file> or goatcounter db migrate all to manually run migrations; generally you want to upload the new version, run migrations while the old one is still running, and then restart so the new version takes effect.

Use goatcounter db migrate pending to get a list of pending migrations, or goatcounter db migrate list to show all migrations.

PostgreSQL

To use PostgreSQL run GoatCounter with a custom -db flag; for example:

% goatcounter serve -db 'postgresql+dbname=goatcounter'
% goatcounter serve -db 'postgresql+host=/run/postgresql dbname=goatcounter sslmode=disable'

This follows the format in the psql CLI; you can also use the PG* environment variables:

% PGDATABASE=goatcounter PGHOST=/run/postgresql goatcounter serve -db 'postgresql'

The database will be created automatically if possible; if you want to create it for a specific user you can use:

% createuser --interactive --pwprompt goatcounter
% createdb --owner goatcounter goatcounter

You can manually import the schema with:

% goatcounter db schema-pgsql | psql --user=goatcounter --dbname=goatcounter

See goatcounter help db and the pq docs for more details.

Development/testing

You can start a test/development server with:

% goatcounter serve -dev

The -dev flag makes some small things a bit more convenient for development; TLS is disabled by default, it will listen on localhost:8081, the application will automatically restart on recompiles, templates and static files will be read directly from the filesystem, and a few other minor changes.

See .github/CONTRIBUTING.md for more details on how to run a development server, write patches, etc.

goatcounter's People

Contributors

0xedward avatar adam-lynch avatar addono avatar apie avatar arp242 avatar bomberstudios avatar brianclemens avatar canrau avatar crocmagnon avatar cweiske avatar davidventura avatar delenamalan avatar dersaidin avatar dmbaturin avatar fwdekker avatar hjr265 avatar kirubakaran avatar kronopath avatar larsvonqualen avatar lorenzoancora avatar lukethacoder avatar mic92 avatar milespernicious avatar oliverpool avatar oppodelldog avatar realpixelcode avatar samwhited avatar shreyas-reddy avatar wopian avatar worm-emoji avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

goatcounter's Issues

Improve UA parsing

github.com/mssola/user_agent isn't always reliable. I took a look in to fixing it, but it's not so easy.

I just noticed there's also https://github.com/avct/uasurfer, which may give better results.

This would also allow storing just the calculated result ("Firefox 70.0") instead of the full UA string, which sometimes contains quite a lot of information.

Filter paths

Right now it's somewhat hard to see stats for specific paths. Add a filter textbox.

Purge and ignore pages

Add ability to purge pages (remove all occurrences), which is useful if a page gets added by accident or whatnot.

Maybe also add ability to ignore page, although I'm less sure about that, since it could also be done in the JS client(?)

Don't care about the protocol in refs

Right now I have:

27 	https://www.dragonflydigest.com
12 	http://www.dragonflydigest.com

Which is just the same URL, really.

I think the protocol can just be stripped. Also saves some space in the refs view.

Be clearer about scale differences

The scaling of the chart can vary wildly from page per page:

screenshot_2019-08-19-01-11-06

There's a little number, but it's not very clear.

Maybe use the colours of bars or background colour? Or something else? Not sure...

Really Excited for this project!

I saw this repository linked from an issue from fathom analytics. I am really excited to see this project grow. Fathom is good but not good enough. I'm not a fan of how their Open Source version is half the quality of their paid version.

Record unique visits

Right now every page hit is just that: a page hit. We don't record any information if a person visited a site before or not.

This is not a bad thing, but it would be useful to have some information about unique visitors, and to do this we need to temporarily store some information in the browser.

The way Fathom does this is by setting a short-lived (30 mins, IIRC) cookie. This way we can get some information about someone's current visit, but we'll have forgotten about it tomorrow.

This would also enable reporting bounce rate and time on page.

Sign in from multiple browsers/locations

Right now the key is stored at users.login_key, allowing only one login. Make a new table for this so there can be multiple login keys per user.

Also:

  • Quick overview of current sessions, and button to log out all other sessions.
  • Keys should get reset when the user changes email address.

Wrong "Show more" link

screenshot_2019-09-03-17-16-51

Happens when there are entries in the hits table that are not yet in the hit_stats table.

Add ability to ignore own views

Can be done in backend by adding field to ignore IP addresses.

Or maybe JS option by setting cookie/localStorage? Can be set by visiting example.com?goatcounter_ignore (tracker script can pick up on this).

Maybe both?

Allow adding domains to a site

Allow adding domains to sites:

Domain:                 [ arp242.net    ]
Highlight color bg/fg:  [#000000] / [#ffffff]
[x] Ignore

Example uses:

  • Add your own domain to ignore or highlight different.
  • Ignore everything from refs you don't care about
  • Highlight refs you care about, or mute refs that are less important
  • Show [go] button to visit the page.
  • Perhaps also restrict which domains can send stats.

Align the numbers

The numbers in the Libre Baskerville font aren't aligned, which is rather ugly/distracting in some tables.

This can be fixed by modifying the font and adding tabular numbers (font-variant-numeric: tabular-nums;, or font-feature-settings: 'tnum' on;), but this isn't so easy for me to do as this isn't my expertise.

Shouldn't be too hard for someone experienced with this kind of stuff; just need to find someone.

Can cache a lot of stats better

Specifically, we can just cache the generated HTML. We don't need to template data from last month every time since it never changes.

Add browser stats

We already collect browser User-Agent information, but don't display it.

The data needs to be processed, since displaying the raw User-Agent strings isn't all that useful. We already import the github.com/mssola/user_agent package.

Also: add browser screen size.

https server

Right now the GoatCounter server doesn't provide https; goatcounter.com relies on a hitch + varnish stack.

We should make it easy to use the Go https server; it works well, and makes it much easier to self-host. Make it easy to set up let's encrypt, too.

Kind of depends on #58

Use SVG to render the chart

Right now the chart is generated with HTML divs in a flexbox. This works quite well, but creates quite large files (although they compress very well due to high repetition, 237K -> 9K, and 1.8M -> 112K) and large DOM trees. Especially the large DOM trees probably isn't great for performance.

Using SVG should give better render performance.

Turns out that doing this well is not so easy; in particular, I couldn't figure out how to get a good responsive view. Would also have to redo the popup JS.

That being said, the current implementation seems to work quite well, so not a huge priority unless people report problems.

Prettier errors

Display pretty errors on:

  • 404
  • "Could not login, perhaps token expires"
  • Unexpected panics, return err

Display in local timezone

Right now everything is in UTC. Should have the option to display in local TZ.

Note: the frontend is done in local TZ now, whereas the backend is in UTC (noticeable when filling in e.g. "show all for today", which may actually be the next day in UTC).

Referrer parsing improvements

  • Could perhaps group some more.
  • Not sure if we still need the ref_params column; I added it mainly because of all the junk ?umta=... stuff, but we just remove that now.
  • Perhaps also remove # from refs? Or put in ref_params (if we don't remove it?)

Problem with query and hashes is that they can either be useless junk or meaningful navigation.

Add option for sans-serif font

I rather like my Baskerville serif font, but it's also somewhat idiosyncratic. Might be a good idea to give people an option to switch it to a sans-serif font in the settings. Should be easy enough to do.

TypeError: e is undefined in count.min.js

Visiting my website with goatcounter gives this error:
2019-12-06-183356_1920x1080_scrot

I only looked through the code very cursorily but 85f3c30 looks suspicious — count now takes one argument count_vars instead of none, but the invocation of count() on load at the bottom passes in zero arguments.

Signup page frontend bug

Tested on qutebrowser (qtwebengine), chromium and firefox.

When sigining up, the inputbox titled "Code" has a message that says "You will access your account at https://code.localhost:8081.", even though that's not true unless the domain name is code. I'm gussing this is not intended behavior?

Review GDPR compliance

Re-review GDPR compliance. In particular, I'm not 100% sure if collecting location data is GDPR-compliant, or if we need to ask consent (this data wasn't collected when I initially reviewed it).

If consent is needed, we should create a toggle to disable collecting on a per-site basis.

Also write a more detailed write-up of the results.

Slight typo in dashboard

Hello!
I've just tried out the service, and I notice that the dashboard says "GoatCounter hasn't receive any data yet". I think it should be "received":
Screenshot from 2019-09-02 13-18-57

Record events

e.g.

<a href="#signup" data-gc-event="click-signup">Sign up</a>

Don't store count_ref for every hit

We only record this as a rough indication to check whether or not it's used a commercial site. Can just make a table which increments a counter; this would also allow adding an audit column.

Changed location of count.min.js?

It looks like the old site code used a script at //static.goatcounter.com/count.min.js, but when I check my site code now it uses //gc.zgo.at/count.min.js and the old script URL is 404ing. Was this intentional? Was there some kind of notification about the change? I think a bunch of recent stats are missing from my sites because I didn't realize the site code had changed and didn't update my static sites...

Multiple domains per account

This might be on your roadmap already, but I run a number of sites and it would be great to have them all under one account. Currently it appears an account supports a single domain only.

Login with subsites and custom domain fails

My main domain is arp242.goatcounter.com, which has stats.arp242.net as custom domain. Logging in on bestasciitable.goatcounter.com gives me a link to stats.arp242.net, which sets the cookie for that domain.

(this only affects me at the moment, since no one else can create custom domains)

Login URL isn't displayed on signup in dev

Not sure if this is how it's supposed to look, but it took me a while to realize I could only login by ommiting the equals sign on the first line of the cut off url.

1566441090

Database migrations

Right now we don't have any tool for database migrations. Need to:

  1. Provide a way to run migrations. This doesn't need to be very complex, and can just be a simple shell script or the like for the time being. There are also some existing Go solutions we can check out.

  2. Make sure the app errors (or maybe even refuses to start) if there are pending migrations. We already have a versions table for this.

Custom domain support

Users should be able to add a CNAME to stats.example.com to example.goatcounter.com.

Not hard but needs support for CSP etc. Need to add a setting for it.

Docker

I would love to make a pull request but I'm really not quiet strong with the Dockerness. I think having a Dockerfile, docker-compose.yml, and an official docker image might make it easier for certain people to use and work on goatcounter.

Multiple users per site

Right now a site can only have one user; change this so that administrators can add new users.

Much of the infrastructure is already there; just needs UI and an endpoint or two.

Make favicon

I just copied my arp242.net one ... Should make a different one.

Doesn't work with go-sqlite 1.11.0

SiteSettings.Scan panics with:

panic: ERROR: interface conversion: interface {} is string, not []uint8

But if I change it to string I get the reverse 🤔:

panic: ERROR: interface conversion: interface {} is []uint8, not string

Not sure if bug of if we're doing something wrong here. Don't see anything odd in the ChangeLog that would affect this, but resetting the version to 1.10.0 fixed it.

Don't show the full current day in the chart

The current day is always shown in full on the stats. If it's 04:00 it'll show 20 more hours for the rest of the day.

Don't do that. Need to change how this is generated in the cron job.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.