Blend

About

Blend is a Python (2.6 or higher) library for interacting with BioCloudCentral.org, CloudMan, and Galaxy‘s API. Conceptually, it makes it possible to script and automate the process of cloud infrastrucutre provisioning and scaling, as well as running of analyses within Galaxy. In reality, it makes it possible to do things like this:

  • Create a CloudMan compute cluster, direclty from your local machine:

    from blend.bcc import BCC
    user_data = {}
    user_data['cluster_name'] = 'Started from blend'
    user_data['access_key'] = 'Your cloud access key'
    user_data['secret_key'] = 'Your cloud secret key'
    c = BCC.launch(ud=user_data, image_id='ami-e23943jd')
    
  • Manipulate your CloudMan instance and react to the current needs:

    from blend.cloudman import CloudMan
    cm = CloudMan("instance IP", "password")
    cm.initialize(type="Galaxy")
    cm.add_nodes(3)
    cluster_status = cm.get_status()
    cm.remove_nodes(2)
    
  • Interact with Galaxy via a straighforward API:

    from blend.galaxy import GalaxyInstance
    gi = GalaxyInstance('<Galaxy IP>', key='your API key')
    libs = gi.libraries.get_libraries()
    gi.workflows.show_workflow('workflow ID')
    gi.workflows.run_workflow('workflow ID', input_dataset_map)
    

Note

Although this library allows you to blend these three services into a cohesive unit, the library itself can be used with any single service irrespective of the rest. For example, you can use it to just manipulate CloudMan clusters or to script the interactions with an instance of Galaxy running on your laptop.

Installation

Stable releases of Blend are best installed via pip or easy_install from PyPI using something like:

$ pip install blend

Alternatively, you may install the most current source code from our Git repository, or fork the project on Github. To install from source, do the following:

# Clone the repository to a local directory
$ git clone https://github.com/afgane/blend.git
# Install the library
$ cd blend
$ python setup.py install

After installing the library, you will be able to simply import it into your Python environment with import blend. For details on the available functionality, see the API documentation.

Usage

To get started using Blend, it’s probably best to take a look at the example scripts in docs/examples source directory and browse the API documentation. Beyond that, it’s up to your creativity :).

Development

Anyone interested in contributing or tweaking the library is more then welcome to do so. To start, simply fork the Git repository on Github and start playing with it. Then, issue pull requests.

API Documentation

Blend’s API foucuses around and matches the services it wraps. Thus, there are three top-level sets of APIs, each corresponding to a separate serivce and a corresponding step in the automation process. Note that each of the service APIs can be used completely independently of one another.

Effort has been made to keep the structure and naming of those API’s consistent across the library but because they do bridge different services, some discrepancies may exist. Feel free to point those out and/or provide fixes.

BioCloudCentral.org API

API used to request instantiation of cloud resoruces (e.g., request a new CloudMan cluster).

CloudMan API

API used to manipulate the instantiated infrastucture. For example, scale the size of the compute cluster, get infrastructure status, get service status.

Galaxy API

API used to manipulate genomic analyses within Galaxy, including data management and workflow execution.

Testing

The unit tests, in the tests folder, can be run using nose. From the project root:

$ nosetests

Getting help

If you’ve run into issues, found a bug, or can’t seem to find an answer to your question regarding the use and functionality of Blend, please use Github Issues page to ask your question

Indices and tables