go get private GitLab with group and subgroup (Golang modules)

go get private GitLab with group and subgroup (Golang modules).md

Problem

The go command line tool needs to be able to fetch dependencies from your private GitLab, but authenticaiton is required.

This assumes your private GitLab is hosted at privategitlab.company.com.

Environment variables

The following environment variables are recommended:

export GO111MODULE=on
export GOPRIVATE=privategitlab.company.com

The above lines might fit best in your shell startup, like a ~/.bashrc.

Explanation

GO111MODULE=on tells Golang command line tools you are using modules. I have not tested this with projects not using Golang modules on a private GitLab.

GOPRIVATE=privategitlab.company.com tells Golang command line tools to not use public internet resources for the hostnames listed (like the public module proxy).

Get a personal access token from your private GitLab

To future proof these instructions, please follow this guide from the GitLab docs. I know that the read_api scope is required for Golang command line tools to work, and I may suspect read_repository as well, but have not confirmed this.

Set up the ~/.netrc

In order for the Golang command line tools to authenticate to GitLab, a ~/.netrc file is best to use.

To create the file if it does not exist, run the following commands:

touch ~/.netrc
chmod 600 ~/.netrc

Now edit the contents of the file to match the following:

machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE

Where USERNAME_HERE is replaced with your GitLab username and TOKEN_HERE is replaced with the access token aquired in the previous section.

Common mistakes

Do not set up a global git configuration with something along the lines of this:

git config --global url."[email protected]:".insteadOf "https://privategitlab.company.com"

I beleive at the time of writing this, the SSH git is not fully supported by Golang command line tools and this may cause conflicts with the ~/.netrc.

Bonus: SSH config file

For regular use of the git tool, not the Golang command line tools, it's convient to have a ~/.ssh/config file set up. In order to do this, run the following commands:

mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/config
chmod 600 ~/.ssh/config

Please note the permissions on the files and directory above are essentail for SSH to work in it's default configuration on most Linux systems.

Then, edit the ~/.ssh/config file to match the following:

Host privategitlab.company.com
  Hostname privategitlab.company.com
  User USERNAME_HERE
  IdentityFile ~/.ssh/id_rsa

Please note the spacing in the above file matters and will invalidate the file if it is incorrect.

Where USERNAME_HERE is your GitLab username and ~/.ssh/id_rsa is the path to your SSH private key in your file system. You've already uploaded its public key to GitLab. Here are some instructions.

Hey Micah.

I've been trying to get the ssh solution working with Gitlab since I've seen it listed in different places as well as the official go docs. The examples I find, however, always use Github. This lead me to try and figure out if there was an issue with setting it up with Gitlab, which lead me to your gist. However, I haven't really been able to confirm anywhere else that it indeed does not work with GitLab. Have you been able to confirm that the problem is indeed with Gitlab? Also, do you have any idea why it wouldn't work with them?

Thanks

@Sol1du2 Hello! Have you found a solution?

I try to build golang app in docker using private gitlab repo. In local build that working fine.
Steps to build docker image on local host:

1. Add ssh keys and configure ~/.ssh/config
2. git config --global url."git@${gitlab_domain}:".insteadOf "https://${gitlab_insted_domain}/"
   At the beginning when I started git clone <some-private-repo>, the process stopped at a moment "compression", but if I restart git clone, it pulled successfully. Problem was in my vpn connection.

Now I trying to do the same thing with gitlab-ci, but it feels like golang just doesn't see ~/.gitconfig and show this error

 dial tcp <some-ip>:443: i/o timeout

@pepsi1k I have not. We ended up using ~/.netrc as the solution for now.

Verify Github on Galaxy. gid:iGj7FA8dCWPUxUvXuX3RjY

you save me !

You are a SAVIOUR @MicahParks !! 🙏

in netrc file, you can assign any name as username:

machine privategitlab.company.com login USERNAME_HERE password TOKEN_HERE

and there's a NETRC env:

NETRC=$(pwd)/.netc go mod download -x privategitlab.company.com/all/project/[email protected]

older gitlab version does not support subgroup go-get, use replace command helps go to identify correct respository path:

replace privategitlab.company.com/all/project/tools => privategitlab.company.com/all/project/tools.git v1.0.0

I try to fix this issue and then solved with this...

you need to set up 3 ENV on your local machine

GONOPROXY=privategitlab.company.com
GONOSUMDB=privategitlab.company.com
GOPRIVATE=privategitlab.company.com

this will solved your issue

Gitlab ci job token also worked:

echo -e "machine xxxxx\nlogin gitlab-ci-token\npassword ${CI_JOB_TOKEN}\n" > ~/.netrc

from https://copyprogramming.com/howto/unable-to-set-up-gitlab-ci-for-golang.

Does somebody know how to handle major versions > 1.0.0? I am currently struggling with some repo which I tagged with v5.0.0, I set up the module correctly too, but once I want to use my mod file:
replace gitlab.../path/v5 => gitlab.../path.git v5.0.0
it hits me with invalid: should be v0 or v1, not v5 probably since the path should end on v5 but since v5 is only a directory in the repo, I can't really satisfy what it wants.

Does somebody know how to handle major versions > 1.0.0? I am currently struggling with some repo which I tagged with v5.0.0, I set up the module correctly too, but once I want to use my mod file: replace gitlab.../path/v5 => gitlab.../path.git v5.0.0 it hits me with invalid: should be v0 or v1, not v5 probably since the path should end on v5 but since v5 is only a directory in the repo, I can't really satisfy what it wants.

maybe try replace gitlab.../path/v5 => gitlab.../path.git/v5 v5.0.0