Packages in Go

Something you import inside your project and expect it to work, without asking too many questions on how it works under the hood. Right? 😸

Hello and welcome everyone, my name is Steve and I’m a YouTuber @ SteveHook. Consider checking it out before reading this article.
There is also a video tutorial version of this article, so be sure to check Packages in Go tutorial out.

So before we begin any coding or diagrams, let me show you where exactly you can find all the resources used inside this article and the video tutorial.

As you may expect everything is hosted on GitHub so make sure to check out Go Basics repository for more information. Also make sure to check out the presentations used inside the video tutorial and here in this article inside Packages directory.

Alright, so talk is cheap, that’s why without further ado, let’s go ahead and jump into the meat.

Package syntax

As soon as you create a file with the extension .go, the only required thing inside that file in order for it to compile successfully is the package declaration

package net

And that’s it, the job is done. Now you have a package called net and the compiler will not complain about it when building or running your program using go run or go build 😺

Package structure

Packages in Go are organized across files. So in other words a package can consists of one or more files. So basically it can look like this:

It’s as simple as that, nothing fancy. As opposed to other languages where a file is considered a package.
In Go a file is considered a single piece from one package. If you’re familiar to terms like namespace, that’s pretty much what Go calls a package.

Package types

Generally speaking in Go there are 2 types of packages

• Executable or main
• Non executable or non-main

So in case of the main package, you have to keep in mind that you need to have declared the main function, which is the function which starts your program when ran. That’s why it’s called executable in the first place, there’s something which needs to be executed 😋

Also keep in mind, only executable packages generate binaries when compiled or built. Trying to run or build a non-main package will result in a compile error

So check out the following code. We have the main package, which imports NewS1 function from services package.

In this specific case, the services package is a non-main package, which provides a certain functionality only to be imported by other packages, including the package main.

On the other hand we have the main package, whose only job is to run the main func which fires your program.

You can compile and run this program by simply typing:

# compiles your main program
# and all the packages which main imports
# and creates a binary in a temp location
# then runs that binary
go run main.go

or

# compiles your main program
# and all the packages which main imports
# and generates the executable binary
go build -o executable

Package naming & structure

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

So as soon as you start creating files, packages and whole applications in pretty much any programming language, the 2 problems you will encounter when your applications start to grow are:

• Naming
• Separation

These 2 are always reasons for holy wars and endless debates across any team of developers, and this is no exception to Go

So why is this so important, and why we still don’t have a standard way of naming things or separating things? And how to do it right? 🤷‍♂️

Really a tricky question, and I still learn day by day how to do it better with experience.🤔

So enough wi the talk, let’s dive into some naming and separation best and bad practices (which are recommendations on how to write clean Go code)

Name your package the same way you name your directory

There are unfortunately still good libraries out there which do quite the opposite, they name the packages one way and the directories those packages live in another way.

This brings a lot of confusion and it is especially true when dealing with import paths, then you must know the name of that package or you have to alias the import path.

So this becomes weird quickly. Have a look at this import path

newrelic "github.com/newrelic/go-agent"

How do we know which is which? Well, apparently the package was named newrelic but the directory it lives in was names go-agent when the import could have looked like this

"github.com/newrelic/newrelic"

or

"github.com/newrelic/nr"

Which clearly tells the consumer, we have a package named nr or newrelic
So no need for aliasing the import or guessing the package name.

Avoid naming your package using underscores or camelCase (multi words) in general

So the Go team in general says, naming your package that way, it’s just an example of poor naming.

A package name should be short, concise and very specific to what it does.
If a package has camelCase or under_score_here_and_there it only means that package is not clear and does more than it needs to do.

It also looks weird in the import path and inside the code

import "github.com/steevehook/repo/some_weird_pkg"
...
some_weird_pkg.SomeFunc()

If the package is only tied to one single responsibility and good naming comes just as a bonus on top of that.

So, name your packages short, concise, one word (preferably) and to the point (exactly what the package )

Avoid naming like utils or misc

Again, just like in case of camelCase or under_score naming, misc and utils are considered bad practices, avoid them as much as possible.

It again proves that packages named this way do more than one specific thing and are likely to always change. This is a tradition which comes from other languages, and it’s also considered a bad practice in other languages too.

Avoid nesting directories and packages too much

So I’m not saying nesting is bad or something. Just keep in mind that it’s recommended to have your directory and package structure as flat as possible.

This way it makes your code more readable and understandable, it avoids friction and unnecessary engineering.

As a rule of thumb, try to keep it 1,2 or 3 levels nested maximum. In general the more flat the better, so think twice before nesting. Is that necessary? If not then just keep things simple.

Have a file named as your package name when it makes sense

So, usually and in most of the cases having a file for example named network.go inside the network directory, where the network package is located is a good practice mainly for placing common functionality for the entire package in that file.

I usually create one, and common helpers and thing used across the entire package show up pretty quickly.

The last thing you want is an error which says:
'someHelper' is redeclared in this package

Because you forgot you declared inside another file, but in fact it’s actually you intend to use inside multiple files of the same package.

It’s a good practice, so make sure to have one file like this where makes sense, it keeps it more organized and more readable and understandable.

You cannot have multiple package declaration inside the same directory

I know this is an obvious one, but if you try this it’s just going to result in a compile error. Make sure all the packages inside a directory only belong to one and only one package.

You cannot do it otherwise so keep that in mind.

Package imports

When importing a package in Go, the import path looks very specific. In most of the projects you will see something like this:

So as you can see nothing fancy here. In Go we are not tied to a specific central package registry. We can fetch packages from anywhere simply by importing them like this and using go get to download them

# downloads httprouter package from github
# and stores it inside $GOPATH
go get github.com/julienschmidt/httprouter

or use the following command inside your project

# downloads every package which is referenced
# inside import paths from CWD (working dir)
go get ./...

It’s as simple as that, for more info make sure to check out go help packages

$GOPATH

Now the previous syntax for import path very strongly related to something which in Go is called $GOPATH

This is a very short explanation on what $GOPATH is but if you want to know more about it make sure to check out Demystifying $GOPATH tutorial on Youtube, where I explain more about it.

So what is $GOPATH in general?

$GOPATH in simple words is the root directory for pretty much any Go project. It holds source code which you develop (not necessarily if you use Go modules), it also contains binaries and third party packages.

So here is a small diagram of how $GOPATH really looks, it’s a simple as this:

So as you can see $GOPATH is nothing fancy, just 3 directories bin , pkg and src

So in the end you’re gonna pay most of the attention to thesrc which in most of the cases will be the place where your source code will live.

Cross reference

So when importing packages in Go, you can very easily import one package into the other and vice versa, causing an import loop. However Go got you covered, basically when you’ll compile your code it’s simply gonna fail.

So to give you a more concrete example of import loop aka cross reference
take a look at the following diagram:

So basically from the main package we import package p1 and it imports p2 , but also notice p2 imports p1 which will make the program fail at compile time.

As I said, as opposed to other languages, import loops or cross reference is not allowed in Go, so keep that in mind. 😉

File globals & locals

So when you create any file in Go, you have to declare certain things in absolutely every file, others you only have to declare only once in at least one file of the same package and it will be visible inside the entire package (every file of the package)

So the first category I call them locals. Things you have to declare in every file of the same package. Basically in this category fall

• Package declaration
• Import paths

The second category I call globals, things you have to declare only once to use everywhere inside the package, and cannot be redeclared in another file and they are:

• Constants
• Variables
• Functions
• Types

So basically declaring any of these in at least one file once, will make them available inside the entire package

Package visibility

So just like in any other language, we have to be able to have private symbols (which are only visible for the internals) and public symbols or public API, code which can safely be consumed by other packages.

So Go handles this too, but a little bit taking a different approach. So in Go we do not have reserved keywords like public , private or protected
In fact we do not have classes either.

So in order to make something public or visible outside of the package we simply must Uppercase the symbol, whether it’s a constant, variable, function, type or field of a struct. That in Go is named exported

Anything else that does not start with an uppercase tells Go it’s private or un-exported

Here’s a small example to make things a little more clear:

The same rules apply when Marshaling or Unmarshaling JSON. And sometimes it’s tricky to catch the mistake, so it may just be an Uppercase issue. But also be mindful of what needs and does not need to be exported

So keep in mind, exported (public) requires Uppercase, anything else is un-exported. Especially pay attention at struct fields.

Test files

So similar to regular packages, in Go there are also test packages, or packages that are inside files which end up with _test.go

So any file whose name ends in _test.go represents a test file in Go

A very important thing to keep in mind is even if you have exported symbols and they are inside a test package, you CANNOT IMPORT them inside another package, they are only available across the same package

So in case there is a need to create a common package for tests, which shares useful functionality across all tests, it’s advised to create a non-test package and store those helpers there.

go get

We barely covered go get and how it really works behind the scenes. Before I show you a small diagram I wanted to say that go get is as well very related to $GOPATH So here’s a small diagram explaining how it works:

So if we take the same example with the import paths and run go get ./... inside the project. The steps are simple:

1. Go ahead and fetch the package from github.com/steevehook/repo
2. Save the package inside $GOPATH/src or inside $GOPATH/pkg if you’re using a package manager or vgo (Go modules)

It’s as simple as that, here’s a small HTTP server example using a third party router, simply to illustrate how go get works.

Special directories

When developing Go projects, there are some directories in Go, which are interpreted by the go tool a little different. Some of these have a special meaning to the language others are more conventional and used widely by the community.

So the first 2, which I said have a special meaning to the language are:

• vendor
• internal

vendor

The vendor directory is an experiment introduced into the language since Go 1.5 version, and basically it serves the purpose of vendoring a project’s dependencies.

So when I talked about $GOPATH and go get I said that if you don’t use any sort of package manager it downloads the source code of third party packages and places it under $GOPATH/src

In the case when you use a package manager, you would typically use the vendor directory. So the vendor directory, looks exactly like $GOPATH/src

When you import a certain package, Go will pull it from the vendor directory as opposed to GOPATH/src if you have one inside your project.

internal

When it comes to internal directory, it is more suitable when developing libraries. Because of how it works.

So speaking of libraries, when you develop something thus provide a public API to the consumer of the package, because of the way Exported/un-exported works in Go you may accidentally expose something which is exported to your library packages but only meant to be used internally.

To avoid these kind of incidents, when you place your code which have exported symbols inside internal directory, those symbols will only be visible inside the library packages, and will not get exposed outside of the library, or to the consumer of the library.

So before making anything public think about it twice, maybe it’s better to just place it inside internal

pkg

When it comes to pkg it is more of a conventional directory, which is there to emphasize that these are the packages which are publicly and safely available for consumption. So we could also say that this is as well more suitable for libraries.

Again this directory is not interpreted anyhow by the Go tool, so it’s more of a convention.

cmd

The cmd directory is the same more of a conventional directory and you only need if it makes sense for your application. So you would want to place different commands there.

So if your application has multiple use cases, you could have multiple main packages and generate a binary for each specific case.

And in the end you would place those main commands inside cmd directory.

.dir, _dir and testdata

Also directories that start with . or _ are ignored, such as testdata directories. And this is what the official help tool says if you run

go help packages

However I’m not sure yet, in what ways these directories are ignored, other than the fact that dot directories and files in UNIX are invisible.

I tried running go install and go build from within these directories, and it definitely generated the binaries and the archives, so not sure how will they benefit my workflow. But I just wanted you to know they also exist

Conclusion

If you found this article useful, or the video tutorial useful, make sure to like, share, subscribe to know more about future upcoming content. Stay tuned. Peace! 🚀🚀🚀