Search This Blog

Thursday, November 22, 2012

Fabric as a python module


Introduction

Fabric is a powerful tool to automate running the same command on many remote server. It is written in python, but authors concentrate on usage as command line tool rather as a python module.

There are some documentation how to do this. You can also find some examples in the internet, but  examples never too much, so below another one.

Scenario

We would like to run a command on each of our webservers, ideally in batches, but in a specific order. Our webserver are grouped in 'cells'. Each cell consist of 3 servers in different 'zones' (zone are called a, b, and c). For convince there is following convention of naming servers: "role-cell#-zone" i.e. web-04-a. To make things more interesting some cells are missing, so we cannot make simple loop. On the other hand, there is a service holding various servers information in json file accessible over http. Additionally, we have a python module called 'server' to locally access those information from our workstation. In the 'server' module we have class 'webserver' to deal with webservers. One of its method is 'get_all_server'. It return a list of all webserver.

Script

Servers

First we create the list of all servers using our external tool.

from servers import webserver
allservers = webserver.get_all_severs()


The order mentioned above is to run first all 'c' boxes next 'b' and finally 'a'. To achieve that, let creates 3 lists (in a function). Nothing magic, a bit of 're'.

def make_lists(servers):
    import re # if not imported in the main part


    wa = []
    wb = []
    wc = []
   
    for server in servers:
        if re.search('web-\d+-a', server):
            wa.append(server)
        elif re.search('web-\d+-b', server):
            wb.append(server)
        elif re.search('web-\d+-c', server):
            wc.append(server)

    separation = (wa, wb, wc)
    return separation


Fabric

So now we can start to use fabric. Let import necessary pieces:

# Fabric part
from fabric.api import *
from fabric.network import disconnect_all


set up environment

env.skip_bad_hosts = True
env.warn_only = True
env.parallel = True


and finally call the function:


@parallel(pool_size=10)
def run_remotely():
    with hide('running', 'status'):
       sudo(task)


Main part

Prepare the lists of servers (see paragraph servers):

lists = make_lists(get_servers())

Get the command to run. I don't remember why, but writing my script decided not to pass variable to 'fabric; function, but use global variable.

global task
task = raw_input("Type/paste command to run >")


Now run the provided command in selected order:

for i in [3, 2, 1]:
    execute (run_remotely, hosts=lists[i-1])


For any case disconnect all connection. (It might be not necessary).

disconnect_all

Wednesday, November 21, 2012

My way for binding ssh-agent with zshell

The following lines I put into my .zshrc might not be very pretty, but it is short, seems to work and I don't need to use any of graphical tools to ssh somewhere with my key (with passphrase). So it works fine with my zshell and E17.

SSHPID=`ps ax|grep -c "[s]sh-agent"`
if (( $SSHPID == 0 ))
then
    ssh-agent > ~/.ssh-env
    source ~/.ssh-env
    ssh-add
else
    source ~/.ssh-env
fi