Define the environment in which this instance of CloudMan will be launched.
Besides providing the credentials, optionally provide the cloud object. This object must define the properties required to establish a boto connection to that cloud. See this method’s implementation for an example of the required fields. Note that as long the as provided object defines the required fields, it can really by implemented as anything (e.g., a Bunch, a database object, a custom class). If no value for the cloud argument is provided, the default is to use the Amazon cloud.
Check if the url is alive (i.e., remote server returns code 200(OK) or 401 (unauthorized)).
A convenience method used to compose and properly format the user data required when requiesting an instance.
user_provided_data is the data provided by a user required to identify a cluster and user other user requirements.
Find an EC2 region zone that supports the requested instance type.
By default, the method will return a single (first match) availability zone that matches the requirements. If want to get a list of all the zones that match the requirements, set get_all to True.
We do this by checking the spot prices in the potential availability zones for support before deciding on a region: http://blog.piefox.com/2011/07/ec2-availability-zones-and-instance.html
Note that, currently, this only applies to AWS. For other clouds, all of the zones are returned.
Create and return an EC2-compatible connection object for the given cloud.
See _get_cloud_info method for more details on the requirements for the cloud parameter. If no value is provided, the class field is used.
Create a security group with all authorizations required to run CloudMan. If the group already exists, check its rules and add the missing ones. Return the name of the created security group.
Create a key pair with the provided key_name. Return the name of the key or None if there was an error creating the key.
Given the instance_id, check on the instance state. Return a state dict with the current instance_state, public_dns, and placement keys, which capture the current state (the values for those keys defualt to empty string if no data is available from the cloud).
This method also allows the requires ec2_conn connection object to be provided at invocation time. If the object is not provided, the class field is used (this helps in case of stateless method invocations).
Check all the prerequisites (key pair and security groups) for launching a CloudMan instance, compose the user data based on the parameters specified in the arguments and the cloud properties as defined in the object’s cloud field.
For the current list of user data fields that can be provided via kwargs, see http://wiki.g2.bx.psu.edu/CloudMan/UserData
Return a dict containing the properties and info with wich an instance was launched, namely: sg_names containing the names of the security groups, kp_name containing the name of the key pair, kp_material containing the private portion of the key pair (note that this portion of the key is available and can be retrieved only at the time the key is created, which will happen only if no key with the name provided in the key_name argument exists), rs containing the boto ResultSet object, instance_id containing the ID of a started instance, and error containing an error message if there was one.
Create an instance of the CloudMan API class.
The url is a string defining the address of CloudMan, for example “http://127.0.0.1:42284”. The password is CloudMan’s password, as defined in the user data sent to CloudMan on instance creation.
Add worker nodes to the cluster.
The num_nodes parameter defines the number of worker nodes to add.
Adjust the autoscaling configuration parameters.
The number of worker nodes in the cluster is bounded by the optional minimum_nodes (default is None) and maximum_nodes (default is None) parameters. If a parameter is not provided then its configuration value does not change.
Disable autoscaling, meaning that worker nodes will need to be manually added and removed.
Enable cluster autoscaling, allowing the cluster to automatically add, or remove, worker nodes, as needed.
The number of worker nodes in the cluster is bounded by the minimum_nodes (default is 0) and maximum_nodes (default is 19) parameters.
Initialize CloudMan. This needs to be done before the cluster can be used.
The type, either ‘Galaxy’ (default), ‘Data’, or ‘SGE’, defines the type of cluster to initialize.
Reboot a specific worker node.
The instance_id parameter defines the ID, as a string, of a worker node to reboot.
Remove a specific worker node from the cluster.
The instance_id parameter defines the ID, as a string, of a worker node to remove from the cluster. The force parameter (defaulting to False), is a boolean indicating whether the node should be forcibly removed rather than gracefully removed.