I'm currently using Pinto at $orkplace. Pinto is becoming increasingly painful to use.
More and more often we're seeing:
The repository is currently in use -- please try again later (Timed out waiting for blocking lock)
There's added fun when Pinto fails because backpan is down.
Speaking to MST on IRC it looks like opan might be a production suitable alternative.
FROM perl:5.28.2
RUN cpanm App::opan
RUN apt-get update && apt-get install -y apt-utils tree
WORKDIR /tmp
RUN useradd --create-home --user-group opan
USER opan
WORKDIR /home/opan
RUN opan init
RUN opan fetch
ENV OPAN_AUTH_TOKENS something987654
ENV OPAN_RECURRING_PULL 1
ENTRYPOINT ["opan"]
CMD ["prefork", "-l", "http://0.0.0.0:8030"]
version: "3.7"
services:
opan-server:
build: .
image: docker.zoopla.com/zpg-opan
environment:
- OPAN_AUTH_TOKENS=something987654
ports:
- 8030:8030
volumes:
- "pans:/home/opan/pans"
opan-tree:
image: docker.zoopla.com/zpg-opan
entrypoint: ["tree"]
command: ["/home/opan/pans"]
volumes:
- "pans:/home/opan/pans"
volumes:
pans:
I'm lucky; we backup our Pinto data to an S3 bucket.
I only wanted to seed with the most recent version of each (internal) module.
This is a quick script I wrote to point at my directory of dist tarballs to give me a list of the latest version of each thing there:
#!/usr/bin/env perl
use strict;
use warnings;
use Cwd 'abs_path';
use Dist::Metadata;
use File::Find::Rule;
use version;
my $dir = $ARGV[0] || die "need a directory to scan\n";
my $abs_dir = abs_path($dir) || die "$dir: can't resolve to absolute path\n";
if ( ! -d $abs_dir ) {
die "$abs_dir: directory not found\n";
}
warn "Processing $abs_dir...\n";
my @files = File::Find::Rule->file()
->name( '*.tar.gz' )
->in( $abs_dir );
my $results;
my $count = 0;
foreach my $file (@files) {
$count++;
my $dist = Dist::Metadata->new(file => $file);
my $description = sprintf "Dist %s (%s)", $dist->name, $dist->version;
# something new
if (!exists $results->{$dist->name}) {
warn "first time: adding: $description\n";
$results->{$dist->name} = $dist;
}
# compare with existing
else {
my $stored_dist = $results->{$dist->name};
# what we have stored is older than the current tarball
if ( version->parse($stored_dist->version) < version->parse($dist->version) ) {
warn 'found newer: stored version (' . $stored_dist->version . ') older than ' . $description . "\n";
$results->{$dist->name} = $dist;
}
# we're already storing the newer of the two
else {
warn 'ignoring: ' . $description . "\n";
}
}
#last if $count > 25;
}
foreach my $dist (sort keys %{$results}) {
say $results->{$dist}->{file};
}
#!/bin/bash
set -euo pipefail
function opan-upload() {
curl -H 'Authorization:Basic base64zzzauthzzzstringssazzxx==' -F "dist=@${1}" https://opan.wherever:8030/upload;
}
aws s3 sync s3://the-s3-bucket/authors/id/Z/ZP/ pinto-authors-id/
# find the latest version of each normal ZPG Thing
# it's a bit slow, but shouldn't need running very often
# we'll only run it if we don't have the file already
if [ ! -f tarball.list ]; then
bin/find-latest-of-each-dist pinto-authors-id/ZPGPACKAGE/ |tee tarball.list
fi
# we can then loop through them and add them
for F in $(cat tarball.list); do
opan-upload $F
done
I purposely omitted how/where the service and container are running.