Integer conversions in bash

Since version 2, bash support a single aritmethic operations. Altough bash is not a mathematical shell (use bc instead), you can perform certain conversions using the bash arithmetic logic.

For example you can remove the left zeroes in a decimal number without require any external utility or print formats, let’s suppose that you want to strip zeroes from the number 007, which is stored in bond variable.

$ echo $bond
007
$ let nozeros=10#$bond
$ echo $nozeros
7

In many forums and mailing list, people need to use ugly sed expressions, or awk invokation, but (with bash) it’s just simply 🙂

Using the same trick, you can perform a base conversions, for example:

$ let i=0x10
$ echo $i
16
$ let i=2#10000
$ echo $i
16

Or create an easy number checking:

$ is_decimal () { let i=10#$1 2>/dev/null; }
$ is_decimal 'a' || echo Nop
Nop
$ is_decimal 56 && echo 'Yep'
Yep

Enjoy!

Update dot files with git

For a years I was using a custom created scripts to keep my dot files updated. I had a local repository in bazaar and a script which check differences between home dot files and files stored in the repository. This solutions works fine for years, but now I want to do some changes…

The first one is moving my dot files to git (and probably pushed them to github), and the second one is to create a hook for git to update my dot files. I known that there are a lot of similar solutions, one more complex, other more easy, but this is mine 🙂

So, I created a post-commit hook script for git, which perform the modifications that I need. Now I just only do this steps:

1. Create a new git repo:

mkdir mydots_repo
cd my_dots_repo && git init

2. Put the hook:

wget -O .git/hooks/pre-commit  http://2tu.us/2scm
chmod 755 .git/hooks/pre-commit

Or just put this content to pre-commit hook:

#! /bin/bash
# (c) 2010 Andres J. Diaz <ajdiaz@connectical.com>
# A hook to git-commit(1) to update the home dot files link using this
# repository as based.
#
# To enable this hook, rename this file to "post-commit".

for dot in $PWD/*; do
 home_dot="$HOME/.${dot##*/}"

 if [ -L "${home_dot}" ]; then
 if [ "${home_dot}" -ef "$dot" ]; then
 echo "[skip] ${home_dot}: is already updated"
 else
 rm -f "${home_dot}" && \
 ln -s "$dot" "${home_dot}" && \
 echo "[done] updated link: ${home_dot}"
 fi
 else
 if [ -r "${home_dot}" ]; then
 echo "[keep] ${home_dot}: is regular file"
 else
 ln -s "$dot" "${home_dot}" && \
 echo "[done] updated link: ${home_dot}"
 fi 
 fi
done
true

3. Copy old files:

cp ~/old/bzr_repo/* .
git add *
 

4. Commit and recreate links:

git commit -a -m'initial import'

And it’s works 🙂

New version of dtools

Today I was released a new version of dtools. Distributed tools, aka dtools is a project written in bash coding to create a suite of programs to allow running different UNIX comamnds parallelly in a list of tagged hosts.

Features

• Fully written in bash, no third party software required (except ssh, obviously).
• Based in module architecture, easy to extend.
• Full integration with ssh.
• Easy to group hosts by tags or search by regular expression.
• Manage of ssh hosts
• Parseable output, but human-readable
• Thinking in system admin, no special development skills required to extend the software.

Short Example

$ dt tag:linux ssh date
okay::dt:ssh:myhostlinux1.domain:Mon Nov 16 23:54:04 CET 2009
okay::dt:ssh:myhostlinux3.domain:Mon Nov 16 23:54:04 CET 2009
okay::dt:ssh:myhostlinux2.domain:Mon Nov 16 23:54:04 CET 2009

As usual, you can download the code from the project page, or if you wish you can download the code via git:

git clone git://git.connectical.com/ajdiaz/dtools

Enjoy!

Distributed tools

For last months I needed to maintain a number of heterogeneous servers for mi work, I need to do some usually actions, like update a config file, restart a service, create local users etc.

For this purposes there are a lot of applications, like dsh (or full csm), pysh, shmux and many others (only need to perform a search in google using phrase “distributed shell”). Unfortunately for me, I want a easy-to-parse solution, because I’ve a big (really big) number of servers, and I want a single cut-based/awk parsing, and also I need to do some actions as other users (like root, for example) via sudo. Althought many of the existants solutions offers me a subset of this features, I cannot found a complete solution. So I decided to create one 😀

You can find the code, and some packages in the dtools development site. I was use this solution in production environment from months with excelent results, and you can feel free to use.

Of course, its free (of freedom) software, distributed under MIT license.

Enjoy and remember: feedback are welcome 😉

bash4 is out!

Yerterday (monday 23 of Feb) the GNU team released a new version of bash.

This new version contains a lot of interesting features, for example asociative arrays (yep!), autocd … and more!

The notices was published in the bash mailing list.

bash ini parser

In some situations i like to use INI files as configuration files, as python do. But bash do not provide a parser for these files, obviously you can use a awk code or a couple of sed calls, but if you are bash-priest and do not want to use nothing more, then you can try the following obscure code:

Update:Added support spaces between keys and values, described in comment #364

Update:Added a writer function, described in comment #370 with examples.

Update:Fix bug when values or keys contains a ‘#’ sign, as described in comment #397. Now only comments at the beginning of line are allowed.

Update:Fix bug in section evaluation when a file which match section expression exists in cwd, as described in comment #436. Fixed also a bug related with
parser when brackets are escaped (read #437.

cfg_parser ()
{
	ini="$(<$1)"                # read the file
	ini="${ini//[/\[}"          # escape [
	ini="${ini//]/\]}"          # escape ]
	IFS=$'\n' && ini=( ${ini} ) # convert to line-array
	ini=( ${ini[*]//;*/} )      # remove comments with ;
	ini=( ${ini[*]/\	=/=} )  # remove tabs before =
	ini=( ${ini[*]/=\	/=} )   # remove tabs be =
	ini=( ${ini[*]/\ =\ /=} )   # remove anything with a space around =
	ini=( ${ini[*]/#\\[/\}$'\n'cfg.section.} ) # set section prefix
	ini=( ${ini[*]/%\\]/ \(} )    # convert text2function (1)
	ini=( ${ini[*]/=/=\( } )    # convert item to array
	ini=( ${ini[*]/%/ \)} )     # close array parenthesis
	ini=( ${ini[*]/%\\ \)/ \\} ) # the multiline trick
	ini=( ${ini[*]/%\( \)/\(\) \{} ) # convert text2function (2)
	ini=( ${ini[*]/%\} \)/\}} ) # remove extra parenthesis
	ini[0]="" # remove first element
	ini[${#ini[*]} + 1]='}'    # add the last brace
	eval "$(echo "${ini[*]}")" # eval the result
}

cfg_writer ()
{
    IFS=' '$'\n'
    fun="$(declare -F)"
    fun="${fun//declare -f/}"
    for f in $fun; do
        [ "${f#cfg.section}" == "${f}" ] && continue
        item="$(declare -f ${f})"
        item="${item##*\{}"
        item="${item%\}}"
        item="${item//=*;/}"
        vars="${item//=*/}"
        eval $f
        echo "[${f#cfg.section.}]"
        for var in $vars; do
            echo $var=\"${!var}\"
        done
    done
}

And then you can parse your ini files as following:

# parse the config file called 'myfile.ini', with the following
# contents::
#   [sec2]
#   var2='something'
cfg.parser 'myfile.ini'

# enable section called 'sec2' (in the file [sec2]) for reading
cfg.section.sec2

# read the content of the variable called 'var2' (in the file
# var2=XXX). If your var2 is an array, then you can use
# ${var[index]}
echo "$var2"

Unfortunately, the cfg.parser() function do no support embedded spaces
in section names… yet

comida

comida is a bash script designed to create a FTP (or HTTP, or rsync or…) mirror and maintain the archive synchronized with the source. I design this tool when I was working at the Free Software Office (OSL) in the University of A Corunna, and decided to create a new mirror in Spain which host a lot of free software projects. But, we wanted a single tool to manage all mirrors in the archive, with full integration in our web page (yes, we wanted on-the-fly status page).

You can download the source code from launchpad project page or using bzr version control system:

$ bzr get lp:comida