Summary
If you have landed here, it is most likely because you attempted to implement the subject and ran into errors. In my use case, I ran into errors when splitting up my Golang app into packages and added unit testing to it. To compound it, I was automatically deploying via triggers with CloudBuild.
This builds upon the article IP Subnet Calculator on Google App Engine. Reading that article will help give you a good reference point for this article.
This article is not a “How To” on actual unit testing but getting it to work under this unique combination of tools. A great unit testing article I found is here – https://blog.alexellis.io/golang-writing-unit-tests/
How To Setup
Before we talk about what might have gone wrong, let’s talk about how to set this up properly. There has been a lot of confusion since Golang went to version 1.11 and started supporting modules. After supporting modules it appears using GOPATH is a less of a supported method. With that said some tools still look for it.
Directory Structure
To get a final state visual of my directory structure, it ended up as follows
tools.woohoosvcs.com
tools.woohoosvcs.com/app.yaml
tools.woohoosvcs.com/go.mod
tools.woohoosvcs.com/subnetcalculator
tools.woohoosvcs.com/subnetcalculator/subnetcalculator.go
tools.woohoosvcs.com/subnetcalculator/subnetcalculator_test.go
tools.woohoosvcs.com/cloudbuild.yaml
tools.woohoosvcs.com/html
tools.woohoosvcs.com/html/root.html
tools.woohoosvcs.com/html/view.html
tools.woohoosvcs.com/README.md
tools.woohoosvcs.com/static
tools.woohoosvcs.com/static/submitdata.js
tools.woohoosvcs.com/static/subnetcalculator.css
tools.woohoosvcs.com/root
tools.woohoosvcs.com/root/root.go
tools.woohoosvcs.com/main.go
Modules
Modules help with the dependency needs of Golang applications. Previously, private packages were difficult to manage as well as specific versioned public packages. Modules helps many of these issues. Here is a good article on modules – https://blog.golang.org/using-go-modules
The key to my issues seemed to be involving modules. Google App Engine’s migrating to 1.11 guide recommends the following.
The preferred method is to manually move all
http.HandleFunc()
calls from your packages to yourmain()
function in yourmain
package.Alternatively, import your application’s packages into your
https://cloud.google.com/appengine/docs/standard/go111/go-differencesmain
package, ensuring eachinit()
function that contains calls tohttp.HandleFunc()
gets run on startup.
My app’s directory is “tools.woohoosvcs.com” so I named the root, the same. Run “go mod init”
$ go mod init tools.woohoosvcs.com
go: creating new go.mod: module tools.woohoosvcs.com
dwcjr@Davids-MacBook-Pro tools.woohoosvcs.com % cat go.mod
module tools.woohoosvcs.com
go 1.13
Importing Private Packages
This then lets us to refer to packages and modules under it via something similar.
import ( "tools.woohoosvcs.com/subnetcalculator" )
CloudBuild
My cloudbuild.yaml ended up as follows
steps:
- id: subnetcalculator_test
name: "gcr.io/cloud-builders/go"
args: ["test","tools.woohoosvcs.com/subnetcalculator"]
#We use modules but this docker wants GOPATH set and they are not compatible.
env: ["GOPATH=/fakepath"]
- name: "gcr.io/cloud-builders/gcloud"
args: ["app", "deploy"]
Interesting tidbit is the “name” is a docker image from Google Cloud Repository or gcr.io
The first id runs the “go” image and runs “go test tools.woohoosvcs.com/subnetcalculator”. It seems the go image wants GOPATH but go test fails with it set so I had to set it to something fake.
It then uses the gcloud deployer which consumes app.yaml
How I Got Here
Unit tests drove me to wanting to split out logic, particularly the subnetcalculator (check it out via https://tools.woohoosvcs.com/subnetcalculator )
Before implementing modules I could get it to run locally by importing ./subnetcalculator but then “gcloud app deploy” would fail.
2019/11/22 01:39:13 Failed to build app: Your app is not on your GOPATH, please move it there and try again.
building app with command '[go build -o /tmp/staging/usr/local/bin/start ./...]', env '[PATH=/go/bin:/usr/local/go/bin:/builder/google-cloud-sdk/bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=ace17fcba136 HOME=/builder/home BUILDER_OUTPUT=/builder/outputs DEBIAN_FRONTEND=noninteractive GOROOT=/usr/local/go/ GOPATH=/go GOPATH=/tmp/staging/srv/gopath]': err=exit status 1, out=go build: cannot write multiple packages to non-directory /tmp/staging/usr/local/bin/start.
The error was vague but I noticed it was related to the import path. I tried moving the folder to $GOPATH/src and it seemed to deploy via “gcloud app deploy” but then failed via CloudBuild automated trigger.
------------------------------------ STDERR ------------------------------------
2019/11/21 21:31:39 staging for go1.13
2019/11/21 21:31:39 GO111MODULE=auto, but no go.mod found, so building with dependencies from GOPATH
2019/11/21 21:31:39 Staging second-gen Standard app (GOPATH mode): failed analyzing /workspace: cannot find package "tools.woohoosvcs.com/subnetcalculator" in any of:
($GOROOT not set)
/builder/home/go/src/tools.woohoosvcs.com/subnetcalculator (from $GOPATH)
GOPATH: /builder/home/go
--------------------------------------------------------------------------------
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/gcloud" failed: exit status 1
It was like a balancing act with a 3 legged chair! Once I initialized the module though and adjusted the imports it worked great
Final Worlds
Here we worked through a best practice of using modules and an internal package to do automated build deploy on Google App Engine. Unit testing with automated deploys are important so that broken builds do not get pushed to production.
One thought on “Unit Testing Golang on App Engine”
Comments are closed.