Skip to content

Commit

Permalink
Allow chunk copying to be disabled.
Browse files Browse the repository at this point in the history
  • Loading branch information
Guy Coates committed Nov 12, 2014
1 parent 3159647 commit 2ffb6a0
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 2 deletions.
2 changes: 2 additions & 0 deletions README
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,8 @@ alter the chunk size with the -b flag. The size is in megabytes. For optimal
performance the chunk size should be a whole multiple of the lustre stripe size
(typically 1MB) and lustre block size (typically 2MB).

You can disable the chunk copy feature by setting the chunk size to 0.


Checksum
--------
Expand Down
13 changes: 11 additions & 2 deletions pcp
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,10 @@ machines as possible to prevent local network bottlenecks.

args = parser.parse_args()

# setting blocksize = 0 will disable chunk copying.
if args.b == 0:
args.b = INFINITY

if not args.SOURCE and not args.R:
print "You must specify a source directory!"
parser.print_help()
Expand Down Expand Up @@ -1186,7 +1190,7 @@ comm = MPI.COMM_WORLD
rank = comm.Get_rank()
workers = comm.size
hostname = os.uname()[1]

INFINITY = float("inf")
STARTEDCOPY = False # flag to see whether we can start checkpointing.
signal.signal(signal.SIGUSR1, handler)
try:
Expand Down Expand Up @@ -1265,7 +1269,12 @@ try:
print "Will checkpoint every %i minutes to %s" %(DUMPINTERVAL, DUMPDB)
if LSTRIPE:
print "Will copy lustre stripe information."
print "Files larger than %i Mbytes will be copied in parallel chunks." %args.b

if args.b < INFINITY:
print "Files larger than %i Mbytes will be copied in parallel chunks." %args.b
else:
print "Chunk copying disabled: files will be copied in one go."

if FORCESTRIPE:
print "Will force stripe all files."
if (LSTRIPE or FORCESTRIPE) and NODIRSTRIPE:
Expand Down

0 comments on commit 2ffb6a0

Please sign in to comment.