Last month I needed to install runit in some servers to supervise a couple of services. Unfortunately my management interface cannot handle the services anymore, so I decided to write a small module in python to solve this handicap, and that is the result!.

With this module you can handle in python environment a number of runit scripts. I think that this might be work for daemontools too, but I do not test yet. Let’s see an example


>>> import supervise
>>> c = supervise.Service("/var/service/httpd")
>>> print s.status()
{'action': None, 'status': 0, 'uptime': 300L, 'pid': None}
>>> if s.status()['status'] == supervise.STATUS_DOWN: print "service down"
service down
>>> s.start()
>>> if s.status()['status'] == supervise.STATUS_UP: print "service up"
service up


Personally I use this module with rpyc library to manage remotely the services running in a host, but it too easy making a web interface, for example using bottle:


import supervise
import simplejson
from bottle import route, run

from bottle import PasteServer
run(server=PasteServer)


Now you can stop your service just only point your browser http://localhost/service/down/httpd (to down http service in this case).

Enjoy!

Last week I was working on libnss_map, aNSS library module to map user credentials to existent user in the system. This module is intended to be used in high virtualized environment like cloud computing or embedded systems which require a lot of users.

When a new user has been authenticated by PAM or other authentication mechanism, then the nss_map module create a virtual user when credentials mapped to an existent user. For example, suppose here are a user virtual, created a la standard way on /etc/passwd:

    virtual:x:15000:15000:virtual user for nss_map:/dev/null:/sbin/nologin

Then edit the /etc/nssmap.conf
file:

    virtual:x:15000:15000:virtual user for nss_map:/home/virtual:/bin/bash

Note that the user directory is really a base dir in nssmap, each new user can search their home in /home/virtual/logname, where logname is the name used by user to login, and the /home/virtual is the prefix setted in nssmap.conf.

As usual, you can get the project from http://connectical.com/projects/libnss-map.

Enjoy!

Really the sysfs and /proc filesystems are worlds of magic and fantasy. Each day I discover a new trick using this filesystems. So, I decided to post a short summary of my favorites ones. Enjoy and feel free to add your tricks in comments, maybe we can a /proc and /sys knownledge database in a post

1. Scanning for LUNs in attached FC
echo "- - -" > /sys/class/scsi_host/hostX/scan

2. CPU hotplug
echp 0 > /sys/devices/system/cpu/cpuX/online

Obviously when run echo 1, put the CPU online again

3. Enable dmesg timestamp
echo Y > /sys/modules/prinkt/parameters/time

4. Restore a removed file when is still in use
cat /proc/pid/fd/descriptor number > /tmp/myfile_restored

5. Get the IO operations for a process:
cat /proc/pid/io
The syscr and the syscw are the accumulated read and write IO operations that process do where running.

6. Increase size of IO scheduler queue:
echo 10000 > /sys/block/device/queue/nr_request

7. Get the current IO scheduler enabed to a specific device:
cat /sys/block/device/queue/scheduler

8. Get the threads of pdflush process which are running:
cat /proc/sys/vm/nr_pdflush_threads

9. Set the percentage threshold for memory to start to flushd data to disk:
echo XX > /proc/sys/vm/dirty_background_ratio

10. Set the sleep time for pdflush checking (in centisecs):
echo XXX > /proc/sys/vm/dirty_writeback_centisecs

11. Set the time to live for a data in buffer, when raises, data will commit to disk (in centisecs):
echo XXX > /proc/sys/vm/dirty_expire_centisecs

For last months I needed to maintain a number of heterogeneous servers for mi work, I need to do some usually actions, like update a config file, restart a service, create local users etc.

For this purposes there are a lot of applications, like dsh (or full csm), pysh, shmux and many others (only need to perform a search in google using phrase “distributed shell”). Unfortunately for me, I want a easy-to-parse solution, because I’ve a big (really big) number of servers, and I want a single cut-based/awk parsing, and also I need to do some actions as other users (like root, for example) via sudo. Althought many of the existants solutions offers me a subset of this features, I cannot found a complete solution. So I decided to create one

You can find the code, and some packages in the dtools development site. I was use this solution in production environment from months with excelent results, and you can feel free to use.

Of course, its free (of freedom) software, distributed under MIT license.

Enjoy and remember: feedback are welcome

Yerterday (monday 23 of Feb) the GNU team released a new version of bash.

This new version contains a lot of interesting features, for example asociative arrays (yep!), autocd … and more!

The notices was published in the bash mailing list.

This week I publish a tiny library to manage netstrings, as described in Bernstein’s paper. This library is wirten in C and provide a very basic implementation for small applications. I use this library in some projects for months, and now a publish the code over MIT License in launchpad.

You can visit the page of the project in launchpad, or get the code via bzr:

bzr get lp:netstr

Last days I reinstalled my Gentoo in my desktop computer, a Dell Dimension C521 using the new package handler pkgcore.

The last months I’ve use official portage, and also new replacement called paludis. The main advantage of this one is being written entirely in C++, so paludis is very fast, but also you can detect some problems when you need to compile some “specials” packages, such qemu, which requires to be compiled with gcc3. If you compile paludis with another version of gcc, you can find a beauty error related to dynamic linking. Obviously, you can solve this problem by hand or with sonme tricks , but I don’t like tricks in production machines.

This week I finished to install my desktop computer using pkgcore. Pkgcore is another replacement of portage, written in python, but with some critical pieces in C (yes, the old reliable C), and I was surprised with the status of the development. The last time I had seen pkgcore was really unstable and very confusing for me, so I was decided to try paludis, which was more stable and mature. But, in this case, pkgcore exceeded my expectations.

In my opinion the strengths of pkgcore are:

• fast (similar to paludis, in my experience)
• easy to use (portage compatible syntax)
• portage compatible configuration
• agnostics about compiler (avoid the qemu problem)
• well colored

But nobody is perfect, so pkgcore has some disadvantages also:

• Arg! configuration file is the hell!
• Errors has special syntax, you need some practice to understand
• Very strict with no-clean packages in portage (for example when install into /lib instead to use get_lib_dir). I’m not sure if it’s a real disadvantage.

I put my pkgcore.conf file in pastebin. I hope to be usefull

I see in the “People of the Web” the last interview to the Professor Walter Lewin, at MIT. I have no comments about him and his classes, only two comments: 1) the great phrase: “Physics work!”, and 2) can other people regarded teacher? I don’t need to think more about it…

Last days I was read the book titled “Entanglement: The greatest mystery in physics”, which is written by Ph.D. Amir D. Aczel, famous for his books about physics. This one is about quantum physics, particularly the entanglement phenomenon. The entanglement is an effect of quantum objects (if we can call them “objects”) consists in a relation between the quantum state of implicated objects, so when we alter one of them, the others “know” automagically the change, even though the objects are spatially separated.

I’m surprised at how easy is to read, it’s funny and clearly (as much as quantum theory allows) and I recommended to people who are interesting in quantum theory and it’s history.

In some situations i like to use INI files as configuration files, as python do. But bash do not provide a parser for these files, obviously you can use a awk code or a couple of sed calls, but if you are bash-priest and do not want to use nothing more, then you can try the following obscure code:

cfg.parser () {
    IFS=$'\n' && ini=( $(<$1) )              # convert to line-array
    ini=( ${ini[*]//;*/} )                   # remove comments
    ini=( ${ini[*]/#[/\}$'\n'cfg.section.} ) # set section prefix
    ini=( ${ini[*]/%]/ \(} )                 # convert text2function (1)
    ini=( ${ini[*]/=/=\( } )                 # convert item to array
    ini=( ${ini[*]/%/ \)} )                  # close array parenthesis
    ini=( ${ini[*]/%\( \)/\(\) \{} )         # convert text2function (2)
    ini=( ${ini[*]/%\} \)/\}} )              # remove extra parenthesis
    ini[0]=''                                # remove first element
    ini[${#ini[*]} + 1]='}'                  # add the last brace
    eval "$(echo "${ini[*]}")"               # eval the result
}

And then you can parse your ini files as following:

# parse the config file called 'myfile.ini', with the following
# contents::
#   [sec2]
#   var2='something'
cfg.parser 'myfile.ini'

# enable section called 'sec2' (in the file [sec2]) for reading
cfg.section.sec2

# read the content of the variable called 'var2' (in the file
# var2=XXX). If your var2 is an array, then you can use
# ${var[index]}
echo "$var2"

Unfortunately, the cfg.parser() function do no support embedded spaces
in section names… yet