After using this for a day or two I see a couple of issues and figured I'd start a ticket to capture any other defects/desires found after using this first attempt for a bit.

• Investigate the use of EBS backed instances. Are we limited to instance types that use instance store disks?
• In our internal use of this feature we wrap the call to the API in a Jenkins job. Expose more options like instance type to the user when running the job to provision the cluster
• For a general purpose cluster to explore new functionality what we have is fine but for the case of standing up a cluster to conduct benchmarks more investigation needs to be done on the intialization of disks, this first attempt of mine is broken.
  • It looks like EBS no longer needs the old "pre-warming" step but we don't know if we can use EBS backed instances yet.
  • I thought the faster "read-once" method of restoring a snapshot image using the fio tool could be used for instance store volumes but the docs say, "first write to any location on an instance store volume performs more slowly than subsequent writes" and they recommend using the "dd" tool which, "Initialization can take a long time (about 8 hours for an extra large instance)"
  • The way the bootstrap script is currently written there is no way for the $INIT_VOLUMES flag to get set to true so it's never going to to run the initialize_volumes step.

Instance store disk init: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/disk-performance.html
EBS Volume Snapshot init: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html