API using Go-lang struct tags to load SQL data and implement JSON endpoints.
Demonstrate how simple it is to prototype and modify an API in Go.
By simply adjusting or adding a field to a Go struct, you automatically update both how the server loads from the database, as well as what it outputs for the API endpoint.
- This project builds a MySQL database Docker image, initialized with sample data
- The sample data uses photos from https://clients3.google.com/cast/chromecast/home
- It also builds an api-server application in Go, which it installs on a Docker image
It uses govvv to provide the Github version string in the application. Install as follows:
go get "github.com/ahmetb/govvv"
Then clone the tag_api
project:
go get "github.com/DavidSantia/tag_api"
cd $GOPATH/src/github.com/DavidSantia/tag_api
First install govvv, clone this project, and start in the tag_api
directory as shown above.
Next, run the build.sh
script to compiles the apps and images.
cd docker
./build.sh
Then type docker images
, and you should see the following images:
REPOSITORY TAG IMAGE ID CREATED SIZE
tagdemo/api-server latest 0fc6d6e09ab2 About an hour ago 8.98MB
tagdemo/mysql latest 9d795daac22c About an hour ago 255MB
There is also a clean.sh
script to remove containers and images from your previous builds.
./clean.sh
Finally, start the database, NATS server, and api-server as follows:
docker-compose up
The NATS server handles communication between the services. In this Basic example, the API server server subscribes to NATS, but it doesn't do anything further.
Once you have the API server up and running, use your browser to authenticate and access data. The following endpoints are defined:
router.Handle("GET", "/authenticate", handleAuthTestpage)
router.Handle("POST", "/authenticate", makeHandleAuthenticate(cs))
router.Handle("GET", "/keepalive", makeHandleAuthKeepAlive(cs))
router.Handle("GET", "/image", makeHandleAllImages(cs))
router.Handle("GET", "/image/:Id", makeHandleImage(cs))
router.Handle("GET", "/user", makeHandleUser(cs))
By browsing to localhost:8080/authenticate, you will see a test page with two buttons.
Each button authenticates you as a particular user from the sample database, either in the Basic or Premium group. Once authenticated, your browser will have a Session cookie to allow you to continue using the API.
You can then browse to
- localhost:8080/image to see images you have access to
- localhost:8080/image/4 to see image id 4
- localhost:8080/user to see your user profile data
The API server and the MySQL container both have integrations with New Relic. If you have a license key for this, you can enable monitoring simply by setting a docker environment variable.
cd docker
cp newrelic.template.env newrelic.env
Then edit newrelic.env
and set the license key on the first line:
NEW_RELIC_LICENSE_KEY=YOUR_LICENSE_KEY_HERE
Running docker-compose will automatically use this key, and you should see additional lines in the logs like so:
api-server | INFO: 2019/02/24 01:18:32 New Relic monitor started
and
tagdemo-mysql | INFO: New Relic monitor started
The build.sh
script creates an empty stub of newrelic.env
if you are not using monitoring.
For developing, you can run this server locally without Docker.
Build the database manually as follows:
cd docker
docker build -t tagdemo/mysql ./mysql
You can then start the MySQL container as follows:
docker run --name tagdemo-mysql --rm -p 3306:3306 tagdemo/mysql
As shown above, we are mapping the MySQL default port 3306 from the container, to 3306 on localhost.
- If this conflicts with a local installations of MySQL server, specify a different port
- If you change the MySQL port, also specify
-dbport
in the api-server/Dockerfile entrypoint.
The database will be ready after you see the message:
[Entrypoint] MySQL init process done. Ready for start up.
If you need to stop the MySQL container, use
docker stop tagdemo-mysql
In a separate terminal, build the API server as follows:
cd apps/api-server
govvv build
Use the help option to get command-line usage
./api-server -h
Usage of ./api-server:
-bolt string
Specify BoltDB filename (default "./content.db")
-dbhost string
Specify DB host
-dbport string
Specify DB port (default "3306")
-debug
Debug logging
-host string
Specify Api host
-log string
Specify logging filename
-nhost string
Specify NATS host
-nport string
Specify NATS port (default "4222")
-port string
Specify Api port (default "8080")
Then run the server as follows:
./api-server -dbhost 127.0.0.1 -dbload -debug
The database loader uses the Go reflect package to auto-generate SELECT statements from the struct tags.
type Image struct {
Id int64 `json:"id" db:"id"`
Width int64 `json:"width" db:"width"`
Height int64 `json:"height" db:"height"`
Url string `json:"url" db:"url"`
Title *string `json:"title" db:"title"`
Artist *string `json:"artist" db:"artist"`
Gallery *string `json:"gallery" db:"gallery"`
Organization *string `json:"organization" db:"organization"`
}
Tags shown above are interpreted as follows:
- json: field name returned in API
- db: field name in SQL
- sql: optional SQL for SELECT
The sql tag is useful when
- using joined statements with otherwise ambiguous field names
- you want to insert an IFNULL or other logic
func (data *ApiData) MakeQuery(dt interface{}, query string, v ...interface{}) (finalq string)
This takes two inputs:
- dt: the struct you are loading data into
- query: the FROM and WHERE part of a query
It can also take optional v parameters. If using these, include a format 'verb' (see the Go fmt package) in your query for each parameter.
It returns one output, the generated query.
query = makeQuery(Image{},"FROM images i WHERE i.media IS NOT NULL"),
This combines the fields in the Image() struct, along with the FROM clause provided, as follows:
SELECT id, width, height, url, title, artist, gallery, organization
FROM images i WHERE i.media IS NOT NULL
The api-server -debug
flag reveals the generated SQL queries.
DEBUG: 2019/05/14 16:31:59 GroupQuery: SELECT id, name, sess_seconds
FROM groups g
INFO: 2019/05/14 16:31:59 Load Groups: 3 entries total
DEBUG: 2019/05/14 16:31:59 ImageQuery: SELECT id, width, height, url, title, artist, gallery, organization
FROM images i WHERE i.media IS NOT NULL
INFO: 2019/05/14 16:31:59 Load Images: 12 entries total
Using the sqlx package, the code loads each object with a single rows.StructScan() step as shown:
for rows.Next() {
err = rows.StructScan(&image)
if err != nil {
fmt.Printf("Load Images: %v\n", err)
continue
}
bs.ImageMap[image.Id] = image
}
Above we see all the parameters from makeQuery() are loaded into an image object.