Updating script for admin scripts moving to sbin

[patmcdonough]

No description provided.

[shivaram]

Hey @patmcdonough -- Do you know what Spark version was used in the sparksummit1 branch ? I just want to make sure that the version on the AMI has sbin instead of bin

[patmcdonough]

Hey @shivaram - good question. @pwendell and I were talking about that earlier as I hadn't realized the latest branch was ampcamp4 (although that's obvious in hindsight).

I did build a cluster earlier off of the sparksummit1 branch and that's when I found this change was required. But I could have sworn we ran Spark Summit off of v0.8. Is the script downloading the latest version somehow, or maybe it's using a newer AMI?

[shivaram]

It gets the AMI from http://s3.amazonaws.com/ampcamp-amis/latest-ampcamp3 -- I am pretty sure all the AMPCamp and Spark Summit clusters have used a pre-installed Spark instance on the AMI and don't download anything during launch.

I am not very sure, but did the S3 symlink changed sometime after the summit ?

[patmcdonough]

Ah, that explains it I think. I just went to the AWS console and searched for that AMI (ami-19474270 right now). The title of the AMI is "ampcamp4-4," so I'm guessing that symlink changed inadvertently.

It probably makes the most sense to close this PR if we can fix the symlink so this correctly represents the snapshot from Spark Summit, and so the online training docs will still work.

[shivaram]

Okay. Do you or @pwendell have the old AMI ID handy somewhere ? I can update it in S3, but don't know what the value should be.