Skip to content

Instantly share code, notes, and snippets.

@marzdgzmn
Last active March 9, 2019 11:34
Show Gist options
  • Save marzdgzmn/73f805abd043e1f524be21ceadcb7d58 to your computer and use it in GitHub Desktop.
Save marzdgzmn/73f805abd043e1f524be21ceadcb7d58 to your computer and use it in GitHub Desktop.
rsync: connection unexpectedly closed (194303538710 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(605) [receiver=3.0.9]
rsync: writefd_unbuffered failed to write 4 bytes to socket [generator]: Connection reset by peer (104)
rsync error: error in rsync protocol data stream (code 12) at io.c(1532) [generator=3.0.9]
+do_rsync:43> rvbash='' rvzsh=12 rr=12
+do_rsync:55> (( rr == 0 ))
+do_rsync:57> (( rr == 24 ))
+do_rsync:60> (( rr == 5 || rr == 10 || rr == 23 || rr == 30 || rr == 35 ))
+do_rsync:74> logit E 'rsync returned 12.'
+logit:2> local 'item=E'
+logit:3> shift
+logit:4> local 'err='
+logit:5> [[ E == E ]]
+logit:5> err=ERR:
+logit:6> [[ E == e ]]
+logit:8> [[ $LOGITEMS -regex-match $item ]]
+logit:9> logwrite ERR: 'rsync returned 12.'
+logwrite:2> [[ -n '' ]]
+logwrite:4> [[ -n /var/log/mirror_manager/fedora.log && -w /var/log/mirror_manager/fedora.log ]]
+logwrite:5> date '+%b %d %T'
+logwrite:5> echo 'Mar 09 04:21:02' ERR: 'rsync returned 12.'
+logit:11> (( VERBOSE >= 3 ))
+logit:12> db3 Log: ERR: 'rsync returned 12.'
+db3:0> (( VERBOSE >= 3 ))
+db3:0> echo '>>' Log: ERR: 'rsync returned 12.'
>> Log: ERR: rsync returned 12.
+do_rsync:75> echo 'rsync returned 12.'
rsync returned 12.
+do_rsync:76> [[ -f ./rsync-err-9VIerr.log ]]
+do_rsync:76> cat ./rsync-err-9VIerr.log
+do_rsync:78> return 12
+/usr/local/bin/quick-fedora-mirror:1259> rsyncreturn=12
+/usr/local/bin/quick-fedora-mirror:1260> [[ 12 -ne 0 ]]
+/usr/local/bin/quick-fedora-mirror:1261> echo 'rsync finished with nonzero exit status.\nWill not check in or delete anything.'
rsync finished with nonzero exit status.
Will not check in or delete anything.
+/usr/local/bin/quick-fedora-mirror:1262> logit E skipping further operations due to rsync failure
+logit:2> local 'item=E'
+logit:3> shift
+logit:4> local 'err='
+logit:5> [[ E == E ]]
+logit:5> err=ERR:
+logit:6> [[ E == e ]]
+logit:8> [[ $LOGITEMS -regex-match $item ]]
+logit:9> logwrite ERR: skipping further operations due to rsync failure
+logwrite:2> [[ -n '' ]]
+logwrite:4> [[ -n /var/log/mirror_manager/fedora.log && -w /var/log/mirror_manager/fedora.log ]]
+logwrite:5> date '+%b %d %T'
+logwrite:5> echo 'Mar 09 04:21:02' ERR: skipping further operations due to rsync failure
+logit:11> (( VERBOSE >= 3 ))
+logit:12> db3 Log: ERR: skipping further operations due to rsync failure
+db3:0> (( VERBOSE >= 3 ))
+db3:0> echo '>>' Log: ERR: skipping further operations due to rsync failure
>> Log: ERR: skipping further operations due to rsync failure
+/usr/local/bin/quick-fedora-mirror:1263> finish with_err
+finish:2> db1 '========================='
+db1:1> (( VERBOSE >= 2 ))
+db1:2> echo '========================='
=========================
+finish:3> date
+finish:3> db1 'Mirror finished: Sat Mar 9 04:21:02 +08 2019'
+db1:1> (( VERBOSE >= 2 ))
+db1:2> echo 'Mirror finished: Sat Mar 9 04:21:02 +08 2019'
Mirror finished: Sat Mar 9 04:21:02 +08 2019
+finish:4> logit R 'Run end.'
+logit:2> local 'item=R'
+logit:3> shift
+logit:4> local 'err='
+logit:5> [[ R == E ]]
+logit:6> [[ R == e ]]
+logit:8> [[ $LOGITEMS -regex-match $item ]]
+logit:9> logwrite 'Run end.'
+logwrite:2> [[ -n '' ]]
+logwrite:4> [[ -n /var/log/mirror_manager/fedora.log && -w /var/log/mirror_manager/fedora.log ]]
+logwrite:5> date '+%b %d %T'
+logwrite:5> echo 'Mar 09 04:21:02' 'Run end.'
+logit:11> (( VERBOSE >= 3 ))
+logit:12> db3 Log: 'Run end.'
+db3:0> (( VERBOSE >= 3 ))
+db3:0> echo '>>' Log: 'Run end.'
>> Log: Run end.
+finish:5> [[ -z with_err ]]
+finish:7> [[ $1 -regex-match 'with_err' ]]
+finish:8> cat /srv/tmp/quick-mirror.ddQe7499h2/output
+finish:9> exit 1
Mar 09 00:00:14 ERR: No completed run since Sun Feb 24 00:00:12 +08 2019.
Mar 09 04:21:02 ERR: rsync returned 12.
Mar 09 04:21:02 ERR: skipping further operations due to rsync failure
Mar 09 04:21:02 Run end.
Mar 09 05:00:05 Run start: cfg /etc/quick-fedora-mirror.conf, tmp /srv/tmp/quick-mirror.TU1X8kn09Z
Mar 09 05:01:36 Generating file list start: fedora-enchilada
Mar 09 05:06:31 cleaning up previous aborted run: 88699 file(s).
Mar 09 05:10:22 Generating file list end: fedora-enchilada
Mar 09 05:11:12 Generating file list start: fedora-epel
Mar 09 05:11:48 cleaning up previous aborted run: 1415 file(s).
Mar 09 05:11:52 Generating file list end: fedora-epel
Mar 09 10:00:06 ERR: No completed run since Sun Feb 24 00:00:12 +08 2019.
Mar 09 18:00:06 ERR: No completed run since Sun Feb 24 00:00:12 +08 2019.
# This file is sourced by the shell and must be in valid sh syntax.
#### Required settings
# Required: The the directory holding your copy of all of the modules you
# mirror. Does not include any module name
DESTD=/srv/mirror/fedora
# Required: The file in which to store the last mirror time.
# Note: this really should not be in the repository itself.
TIMEFILE=/var/log/mirror_manager/fedora_timefile
# Other settings
# The remote host to rsync from, not including a module name
REMOTE=rsync://mirrors.kernel.org
# The master module, which holds the other modules
# MASTERMODULE=fedora-buffet
# Tier 1 Fedora mirrors should uncomment the following to get the proper
# pre-bitflip content.
# MASTERMODULE=fedora-buffet0
# PREBITFLIP=1
# Define if the entire repository (all modules under fedora-buffet) should be
# mirrored. If set, MODULES (below) is ignored
# MIRRORBUFFET=
# An array containing the modules to be mirrored
# MODULES=(fedora-enchilada fedora-epel)
# The name of the file holding the file list on the mirror host Note: the
# string '$mdir' will be replaced with the name of the current module directory
# in context, and so the '$' must be escaped or the string quoted.
# FILELIST='fullfiletimelist-$mdir'
# An array of extra file lists to be transferred. They won't be processed, but
# will implicitly be included in every transfer because file lists can't be
# included in the file lists. '$mdir' is substituted as above.
# Note that if you change this, you will want to do a run with -a to pick up
# those extra files even in unchanged modules.
# EXTRAFILES=(fullfilelist imagelist-\$mdir)
# Mapping of MODULES to directory names, as an associative array
# MODULEMAPPING=(fedora-alt alt fedora-archive archive
# fedora-enchilada fedora fedora-epel epel)
# rsync binary
# RSYNC=/usr/bin/rsync
# curl binary (only if MirrorManager checkins are enabled; see below).
# CURL=/usr/bin/curl
# Array of default options to pass to rsync
# Will be modified automatically according to VERBOSITY level; no need to set
# -q, -v or --info here.
#
# You can add excludes here, but the script will always detect those files as
# missing and will add them back to the file list. This may generate
# complaints from rsync, but should not cause any problems, though it is almost
# certainly better to use the provided FILTEREXP to do exclusion instead.
#
# Note that some of these options may be required for proper operation of the
# script. You really should be careful if you change things here, as the
# default value is carefully crafted and rsync can react in unexpected ways to
# seeminly compatible sets of arguments.
# RSYNCOPTS=(-aSH -f 'R .~tmp~' --stats --preallocate --delay-updates --out-format='@ %i %n%L')
# By default quick-fedora-mirror will try to detect and recover from an aborted
# rsync run by moving any already downloaded files into place before
# processing, Define NORSYNCRECOVERY (to anything) to prevent this.
# NORSYNCRECOVERY=
# Define KEEPDIRTIMES (to anything) to make a third rsync call which restores
# the timestamps of any directories which were modified after file removal.
# This won't ensure that timestamps are always up to date, but its good enough
# if you don't modify your repository locally. Maintaining directory
# timestamps isn't important for mirroring in any case.
# KEEPDIRTIMES=
# DEFINE CHECKIN_SITE and CHECKIN_PASSWORD to perform a mirrormanager checkin
# at the completion of the run if rsync succeeded without error. These values
# should match the master site name and site password you have configured in
# mirrormanager.
# CHECKIN_HOST will default to the output of the hostname command; you must set
# it manually if that does not match the hostname you have configured in
# mirrormanager.
# The "curl", "bzip2" and "base64" commands must be installed in order to
# perform mirrormanager checkins.
CHECKIN_SITE=Rise
CHECKIN_PASSWORD=risesysad123
CHECKIN_HOST=mirror.rise.ph
# If you have configured multiple mirrormanager hostnames to virtually
# spread the modules/categories you mirror between them, then you can configure
# per-module checkin hosts with variables liks the following.
# If a specific CHECKIN_HOST_* setting is not configured, then CHECKIN_HOST
# above will be used.
# Note that the module name is used, not the mirrormanager category, and that
# '-' in the module name is replaced by '_' to make a legal variable name.
# Most sites will hnot need this; it is only necessary when you mirror too many
# modules for mirrormanager to handle in a single checkin or crawl.
# CHECKIN_HOST_fedora_archive=
# CHECKIN_HOST_fedora_alt=
# Verbosity levels - info sent to stdout; usually this gets mailed to root when
# being run by cron
# 0 - quiet
# 1 - quiet if no changes, otherwise basic transfer info
# 2 - no -q to rsync
# 3 - very slightly more verbosity
# 4 - One -v to rsync
# 5 - Another -v to rsync
# 6 - Output of all settings
# 7 - Add --progress to rsync
# 8 - Shell script trace
# VERBOSE=0
# Define (to anything) to enable logging to the systemd journal (via
# systemd-cat). the identifier "quick-fedora-mirror" is used, so logs can be
# retrieved with: journalctl -t quick-fedora-mirror
# LOGJOURNAL=
# Define to a full path to enable logging to that file.
# The provided file must already exist and be writable.
# Is only considered if LOGJOURNAL above is not defined.
LOGFILE=/var/log/mirror_manager/fedora.log
# Logging fields - Each character selects a piece of information to log.
# @ - Absolutely everything.
# a - aborted run recovery
# A - each recovered file from an aborted run
# c - rsync calls
# C - rsync call completions
# d - File/directory deletion start/end
# D - all file/directory deletes
# e - minor errors
# E - serious errors
# F - all transferred files (not impl)
# g - file list generation start/end
# k - lock contention
# l - per-module local file list generation (recursive find) start/end
# m - mirrormanager checkin
# M - mirrormanager checkin detail
# n - lack of updates in a run
# N - lack of updates in a module
# o - remote file list download start/end
# p - module processing start/end
# p - per-module module processing start/end
# r - run start
# R - run end
# s - Basic transfer statistics
# S - Detailed transfer statistics
# t - directory time updates (not impl)
# LOGITEMS=aeElrRs
# When q-f-m fails to run becuse it is already running, it checks the time
# since the last successful run. If that is larger than this value, it logs a
# serious error. Thid helps to detect a hung run or issues with slow
# transfers.
# WARNDELAY=$((60 * 60 * 24)) # One day
# When q-f-m encounters an error calling rsync, it may (depending on the error)
# sleep and retry. It will always sleep with exponential backup. Set
# MAXRETRIES to limit the number of times it retries.
# MAXRETRIES=10
# mktemp will be called after this file is sourced to make a temporary
# directory. This directory can contain a large amount of data, and that data
# is specified by the server. If your /tmp is small and you are concerned
# about the server potentially sending extra-huge files and filling things up,
# you can set TMPDIR here.
TMPDIR=/srv/tmp
# A regular expression used to filter the file lists. It must be quoted (or
# very carefully escaped). Entries matching this expression will not be synced
# and are expected not to be present locally. They will also be ignored by
# quick-fedora-hardlink. Cannot contain commas. Run against the file list that
# includes sizes (by quick-fedora-mirror) and the fullfiletimelist (by
# quick-fedora-hardlink), so don't use expressions that would match those
# metadata (which are digit strings and single characters). Example is a heavy
# filter which gives you an x86_64-only mirror with source packages, debuginfo
# packages, Alpha and Beta releases, and most image files excluded.
# FILTEREXP='(/i386|/armhfp|/source|/SRPMS|/debug/|\.iso|\.img|\.qcow2|\.raw\.xz|\.box|/releases/test)'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment