I have a directory with too much files in it.
I want to compress first 5 thousand files in that directory to become file.tar.gz
and then 5001 - 10000...and so on
how to do it?
I have a directory with too much files in it.
I want to compress first 5 thousand files in that directory to become file.tar.gz
and then 5001 - 10000...and so on
how to do it?
Use ls to generate the list of names and head and tail to filter them. Here's a one-liner that does it in a loop. You'll need to know the number of files in the directory (ls | wc -l will tell you).
for ii in $(seq -w 5000 5000 NUMBER_OF_FILES) ; do echo $ii ; ls | head -n $ii | tail -n 5000 | tar -f ../ARCHIVE_FILE_$ii.tar.gz -czv -T - ; done
Replace the bits in capitals with what you want.
This script gradually adds all files to the archive, and numbering the archive. Rename ARCHIVE_NAME and '5000'.
$ COUNT_MOD=0; for i in *; do tar -r -f ARCHIVE_NAME`expr $COUNT_MOD / 5000`.tar $i; ((COUNT_MOD++)) ; done
This script is not optimized, so there are a few rules:
You could build a set of files that list each 5000 filenames and use them with the -T arguments for tar. Something like this might work:
ls -1 | split -l 5000 - tarlist
count=0
for f in tarlist*
do
tar -czf save.$count.tar.gz -T $f
count=`expr $count + 1`
done