Copy only certain subfolders and contents from a folder structure automated, preserving the folder structure.

Hi there,

long time no see. So, we needed to to give some folders and files over to another company. We were supposed to keep the folder structure, but of course we should only copy certain folders, not all. And, of course, these were subfolders inside a list of project folders.

Do it by hand? Well, when you are dealing with 32K of folders and are supposed to copy only some subfolders out of 15K of them that is no fun…

This is the script I cooked up and it does the job just fine, except we ended up using a colleagues script instead of mine 😉

(sorry for the format, it used to look nicer!)

#!/bin/bash
#
# maybeageek
# Version 0.6, 09/July/2014
# UseCase: Copy certain subfolders (one layer depth) and their contents from a source drive,
# ignoring folders that are not written in the files.txt list.
# This will work for one hierarchy of folders/subfolders.
# To have it work with a deeper folder structure, you need to add more for-loops.
#
# Beware though that this will exponentially increase the time this script takes to finish, as it cycles
# through every subfolder checking for every entry in the files.txt.
#
# Attention: Under Windows you need to cd to the destination directory for this to work!
# Otherwise rsync will give you an error.
#
# Also: Even under Windows the files.txt has to be UNIX format, or it won’t work!

LOG=/path/to/log.txt
# a nice formatted output log of this script where you can see what it did for every folder.
FAILED=/path/to/failed.txt
# the full rsync log. This is why we put rsync in -v mode.
SOURCE=/path/to/Source
# your data source.
DEST=/path/to/Destination
# your destination. As this was running under Windows using cygwin, we needed this variable AND
# needed to cd into the destination folder!
FILES=/path/to/list/of/folders.txt
# This file is used to determine which subfolders the script should copy.

for F in $(ls $SOURCE); do
mkdir $DEST/$F;
for S in $(cat $FILES); do
echo \  >> $LOG;
echo “###### Begin ######” >> $LOG;
date >> $LOG;
if test -d $SOURCE/$F/$S;
then
echo “Copy folder:” >> $LOG;
echo $F/$S >> $LOG;
cd $DEST/$F;
rsync -avz –log-file=$FAILED $SOURCE/$F/$S . >> $LOG ;
else
echo “Folder not found:” >> $LOG;
echo $F/$S >> $LOG;
date >> $LOG;
fi
echo “###### End ######” >> $LOG;
done
done

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s