Skip to content

Instantly share code, notes, and snippets.

@komuw
Last active April 9, 2022 20:31
Show Gist options
  • Select an option

  • Save komuw/639f3811ad10a3453190fb9b4d971b37 to your computer and use it in GitHub Desktop.

Select an option

Save komuw/639f3811ad10a3453190fb9b4d971b37 to your computer and use it in GitHub Desktop.
Go module proxy implemented in python/django
import json
from django.http import HttpResponse
class Modules(APIView):
"""
For documentation, see; https://github.com/golang/go/blob/master/src/cmd/go/internal/modfetch/proxy.go
call this like;
sudo rm -rf ~/go/src/mod/ && \
export GOPROXY=http://localhost:4000 && \
go get -v github.com/pkg/errors
to use go mod without hitting network
go mod -vendor && \
go build -getmode=vendor
"""
def get(self, request):
path = request.path
req_content_type = request.content_type
req_GET = request.GET
print(
"\n path={0} \nreq_content_type={1} \nreq_GET={2}".format(
path, req_content_type, req_GET
)
)
if path == "/github.com/pkg/errors/@v/list":
return HttpResponse(content_type="text/plain", content="v0.8.0")
elif path == "/github.com/pkg/errors/@v/v0.8.0.info":
# see; https://github.com/golang/go/blob/88aa208024f04845381a86f2d5679d2e520b1ff6/src/cmd/go/internal/modfetch/proxy.go#L64-L67
resp = {"Version": "v0.8.0", "Time": "2016-09-29T01:48:01Z"}
return HttpResponse(content_type="text/plain", content=json.dumps(resp))
elif path == "/github.com/pkg/errors/@v/v0.8.0.mod":
resp = "module github.com/pkg/errors\n"
return HttpResponse(content_type="text/plain", content=resp)
elif path == "/github.com/pkg/errors/@v/v0.8.0.zip":
"""
zipped package in specified format here;
https://github.com/golang/go/blob/master/src/cmd/go/internal/modfetch/proxy.go#L66-L73
The archive uses slash-separated paths, and every file path in the archive must
begin with <module>@<version>/, where the module and version are
substituted directly, not case-encoded. The root of the module
file tree corresponds to the <module>@<version>/ prefix in the archive.
so there is /github.com/pkg/errors/@v/v0.8.0.zip which if you opened up
it should look like:
github.com
└── pkg
└── errors@v0.8.0
├── LICENSE
├── README.md
├── appveyor.yml
├── bench_test.go
├── errors.go
├── errors_test.go
├── example_test.go
├── format_test.go
├── stack.go
└── stack_test.go
if you download from https://github.com/pkg/errors/archive/v0.8.0.zip it looks like
errors-0.8.0
├── LICENSE
├── README.md
├── appveyor.yml
├── bench_test.go
├── errors.go
├── errors_test.go
├── example_test.go
├── format_test.go
├── stack.go
└── stack_test.go
so we need to read that and reconstruct it into a new zip file that looks like;
github.com
└── pkg
└── errors@v0.8.0
├── LICENSE
├── README.md
└──etc
"""
return HttpResponse(
content_type="text/plain",
status=599,
content="endpoint /github.com/pkg/errors/@v/v0.8.0.zip is not implemented.",
)
#!/usr/bin/env bash
# https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md
if test "$BASH" = "" || "$BASH" -uc "a=();true \"\${a[@]}\"" 2>/dev/null; then
# Bash 4.4, Zsh
set -euo pipefail
else
# Bash 4.3 and older chokes on empty arrays with set -u.
set -eo pipefail
fi
# usage:
printf """\n\n example usage: \n
bash goproxy.sh \
/tmp/myGoProxy \
/tmp/zipper \
github.com/pkg/errors \
v0.0.8"""
GOPROXY_LOCATION=${1:-/tmp/myGoProxy}
ZIPPING_LOCATION=${2:-/tmp/zipper}
printf "\n\n GOPROXY_LOCATION=$GOPROXY_LOCATION \n"
printf "\n\n ZIPPING_LOCATION=$ZIPPING_LOCATION \n"
MODULE_NAME=${3:-notSet}
if [ "$MODULE_NAME" == "notSet" ]; then
printf "\n\n MODULE_NAME should not be empty\n"
exit
fi
MODULE_VERSION=${4:-notSet}
if [ "$MODULE_VERSION" == "notSet" ]; then
printf "\n\n MODULE_VERSION should not be empty\n"
exit
fi
# 1.
printf "\n\n downloading: \nmodule=$MODULE_NAME \nversion=$MODULE_VERSION \n\n"
mkdir -p $ZIPPING_LOCATION
wget -nc --directory-prefix=$ZIPPING_LOCATION https://$MODULE_NAME/archive/$MODULE_VERSION.zip
unzip $ZIPPING_LOCATION/$MODULE_VERSION.zip -d $ZIPPING_LOCATION
mkdir -p $ZIPPING_LOCATION/$MODULE_VERSION
mv $ZIPPING_LOCATION/*/* $ZIPPING_LOCATION/$MODULE_VERSION
mkdir -p $ZIPPING_LOCATION/$MODULE_NAME@$MODULE_VERSION
cp -r $ZIPPING_LOCATION/$MODULE_VERSION/* $ZIPPING_LOCATION/$MODULE_NAME@$MODULE_VERSION
zip /tmp/$MODULE_VERSION.zip $ZIPPING_LOCATION/$MODULE_NAME@$MODULE_VERSION/*
# 2.
printf "\n\n adding module to $GOPROXY_LOCATION: \nmodule=$MODULE_NAME \nversion=$MODULE_VERSION \n\n"
mkdir -p $GOPROXY_LOCATION
mkdir -p $GOPROXY_LOCATION/$MODULE_NAME/@v
echo "${MODULE_VERSION}" >> $GOPROXY_LOCATION/$MODULE_NAME/@v/list
echo "{"\"Version"\":"\"$MODULE_VERSION"\", "\"Time"\":"\"2016-09-29T01:48:01Z"\"}" >> $GOPROXY_LOCATION/$MODULE_NAME/@v/$MODULE_VERSION.info
echo "module $MODULE_NAME" >> $GOPROXY_LOCATION/$MODULE_NAME/@v/$MODULE_VERSION.mod
cp /tmp/$MODULE_VERSION.zip $GOPROXY_LOCATION/$MODULE_NAME/@v/
# 3. clean up
printf "\n\n cleaning up \n\n"
rm -rf $ZIPPING_LOCATION/*
# 4. set proxy env
printf "\n\n setting up GOPROXY env variable for \n\n"
export GOPROXY=file:///$GOPROXY_LOCATION
from django.urls import re_path
from django.views.decorators.csrf import csrf_exempt
from .goproxy import Modules
urlpatterns = [
# allow any uri
re_path(r"^.*", csrf_exempt(Modules.as_view()), name="modules"),
]
@komuw
Copy link
Author

komuw commented Sep 4, 2018

To create a file-based Go module proxy, you can run the following commands.
I turned it into a repo: https://github.com/komuw/goproxy

@charles-dyfis-net
Copy link

charles-dyfis-net commented Apr 9, 2022

Note that the advice to use errexit, given in https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md, conflicts with BashFAQ #105. The general best practice from the bash community (as solidified in the #bash IRC channel formerly of Freenode, now of Libera) is to use explicit || return / || exit extensively.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment