Forked from prithv1/gist:83655754abc13ed80e06bfcc662177c9
Last active
January 11, 2021 08:00
-
-
Save fulldeck/e24e4c6156999ac8f57c3ba5b2aecb91 to your computer and use it in GitHub Desktop.
/bin/tar: Argument list too long
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
@url https://major.io/2007/07/05/bintar-argument-list-too-long/ | |
If you find yourself stuck with over 30,000 files in a directory (text files in this example), packing them into a tar file can be tricky. | |
Largest number of files completed 507,000 files of 355bytes each | |
NOTE: This will tap out all your servers resources and find the weak link; so be careful. CPU or Disk I/O? | |
You can get around it with this: | |
find . -name '*.txt' -print >/tmp/test.manifest | |
tar -cvzf textfiles.tar.gz --files-from /tmp/test.manifest | |
find . -name '*.txt' | xargs rm -v | |
# remove files all in one go ( have not tested which is faster) | |
find . -name '*.txt' -print >/tmp/test.manifest | |
tar -cvzf textfiles.tar.gz --files-from /tmp/test.manifest --remove-files | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment