Latest Updates: tips RSS

  • bash ini parser 

    ajdiaz 7:42 pm on 9 February 2008 Permalink | Reply
    Tags: , tips

    In some situations i like to use INI files as configuration files, as python do. But bash do not provide a parser for these files, obviously you can use a awk code or a couple of sed calls, but if you are bash-priest and do not want to use nothing more, then you can try the following obscure code:

    cfg.parser () {
        IFS=$'\n' && ini=( $(<$1) )              # convert to line-array
        ini=( ${ini[*]//;*/} )                   # remove comments
        ini=( ${ini[*]/#[/\}$'\n'cfg.section.} ) # set section prefix
        ini=( ${ini[*]/%]/ \(} )                 # convert text2function (1)
        ini=( ${ini[*]/=/=\( } )                 # convert item to array
        ini=( ${ini[*]/%/ \)} )                  # close array parenthesis
        ini=( ${ini[*]/%\( \)/\(\) \{} )         # convert text2function (2)
        ini=( ${ini[*]/%\} \)/\}} )              # remove extra parenthesis
        ini[0]=''                                # remove first element
        ini[${#ini[*]} + 1]='}'                  # add the last brace
        eval "$(echo "${ini[*]}")"               # eval the result
    }
    

    And then you can parse your ini files as following:

    # parse the config file called 'myfile.ini', with the following
    # contents::
    #   [sec2]
    #   var2='something'
    cfg.parser 'myfile.ini'
    
    # enable section called 'sec2' (in the file [sec2]) for reading
    cfg.section.sec2
    
    # read the content of the variable called 'var2' (in the file
    # var2=XXX). If your var2 is an array, then you can use
    # ${var[index]}
    echo "$var2"
    

    Unfortunately, the cfg.parser() function do no support embedded spaces
    in section names… yet

     
    • Shawn 5:27 pm on 18 February 2008 Permalink | Reply

      Uh-oh:
      eval: line 2: syntax error near unexpected token `}’
      eval: line 2: `}’

      Hopefully this is just a typo in the code above as I’d be super happy to get this to work and I’m just not quite good enough to figure out what is wrong.

      Thanks for posting this!

    • ajdiaz 6:22 pm on 18 February 2008 Permalink | Reply

      Mmm, the code works for me :D

      There are my ‘myfile.ini’:

      [~]$ cat myfile.ini
      [sec2]
      var2=XXX

      And the result:
      [~]$ ./ini.sh
      XXX

      Maybe you need bash >= 3.0?, I’m using:

      [~]$ echo ${BASH_VERSION}
      3.2.17(1)-release

      If you want, send me the result of the command:

      $ bash -x ini.sh

      And I try to help you :)

      Thanks for your comments!! :D

    • Greg 1:58 pm on 3 March 2008 Permalink | Reply

      This script also doesn’t work if the ini file is not fully standard, like samba’s smb.conf. In that, there are some white spaces on both sides of the equation sign.

    • ajdiaz 10:50 pm on 8 March 2008 Permalink | Reply

      Sure, the script do not work fine with non stardard files, becaus the script parse file variables as bash variables and spaces are not allowed in bash variables. But, you are free to replace spaces with another character. :)

    • Tomas 2:24 pm on 29 April 2008 Permalink | Reply

      This works great!

      Do you have any functionality for updating the functions array dynamically without re-parsing the ini-file?

      Example: Add new section or add/update field within a section.

      I’m considering trying to code this, but it would of course save me lots of time if it already exists :-)

    • ajdiaz 3:34 pm on 1 May 2008 Permalink | Reply

      Sorry, I don’t code any functions to update the array. Really this hack do not use an array. We read the ini and create a couple of variables with the same name as variable in the file, and change the section creating a function called the section name. So if you need to add a new section, you might do:

      cfg.section.MYNEWSECTION ()
      {
      MYNEWVAR1=”MYNEWVALUE1″
      # any other variable
      }

      Now I’m working in a dump function to write the ini from specified functions.

    • Emanuele 3:16 pm on 8 May 2008 Permalink | Reply

      Hi,
      I’ve only copy the function in my bash script and I receive the following error:
      ‘cfg.parser’:not a valid identifier

      someone know why?

      BASH_VERSION -> 3.1.17(1)

    • ajdiaz 5:04 pm on 10 May 2008 Permalink | Reply

      Yes, probably your bash is running with POSIX option (check the value of $HELLOPTS), in this case the dot (.) is not a valid character for function name.

      If you really require a POSIX shell compatibility, you can replace cfg.parser and cfg.sections by cfg_parser and cfg_sections, the underline is always a valid character :D

    • Emanuele 10:00 am on 12 May 2008 Permalink | Reply

      Thanks! now I try!

    • Henning 2:37 pm on 20 June 2008 Permalink | Reply

      The source listing is somehow garbled. There’s something wrong with quotes (backticks?). The show up as a single character (&#8221 resp. &#8220=

    • ajdiaz 2:48 pm on 20 June 2008 Permalink | Reply

      Sure, the wordpress format is not very readable with selected theme, use the pastebin code in http://pastebin.com/f61ef4979 instead :)

    • Stephen 10:13 pm on 27 June 2008 Permalink | Reply

      The following will parse out a file that has spaces between the ‘=’ as well as both ; and # comments. Just thought I would share it.

      function cfg.parser ()
      {
      IFS=$’\n’ && ini=( $(<$1) ) # convert to line-array
      ini=( ${ini[*]//;*/} ) # remove comments ‘;’
      ini=( ${ini[*]//\#*/} ) # remove comments ‘#’
      ini=( ${ini[*]/\ =\ /=} ) # remove anything with a space around ‘ = ‘
      ini=( ${ini[*]/#[/\}$'\n'cfg.section.} ) # set section prefix
      ini=( ${ini[*]/%]/ \(} ) # convert text2function (1)
      ini=( ${ini[*]/=/=\( } ) # convert item to array
      ini=( ${ini[*]/%/ \)} ) # close array parenthesis
      ini=( ${ini[*]/%\( \)/\(\) \{} ) # convert text2function (2)
      ini=( ${ini[*]/%\} \)/\}} ) # remove extra parenthesis
      ini=( ${ini[*]/#\ */} ) # remove blank lines
      ini=( ${ini[*]/#\ */} ) # remove blank lines with tabs
      ini[0]=” # remove first element
      ini[${#ini[*]} + 1]=’}’ # add the last brace
      #printf “%s\n” ${ini[*]}
      eval “$(echo “${ini[*]}”)” # eval the result
      }

    • Lars 7:31 pm on 1 September 2008 Permalink | Reply

      Thanks for your script!

      Is there any chance to get the available variables of a section?

      For example: if I know that there’s a section [sec2], how can I print out the content of this section?

      I think I need something like this:

      for var in cfg.section.sec2; do
      print “variable: %s\n” $var;
      print “value: %s\n” $var[*];
      done

      but that doesn’t work for me.

    • ajdiaz 1:01 pm on 2 September 2008 Permalink | Reply

      declare -f cfg.section.sec2 | grep ‘=’ must be work Lars :)

    • frank 9:31 pm on 16 October 2008 Permalink | Reply

      Just a quick question… At one point I had this working, however, now all but the first section has been replaced by the number ‘1′. It’s replace right after this line:

      IFS=$’\n’ && ini=( $(<$1) )

      I’m using bash 3.0. Any thoughts?

      Thanks for the help.

    • Maciej Pasternacki 11:02 pm on 19 November 2008 Permalink | Reply

      One more fix: script changes $IFS and doesn’t restore previous value. Took me quote a moment of debugging, why the hell after reading config word splitting doesn’t work as they used to (e.g. LS=”ls -a”;$LS doesn’t work, because bash tries to find and execute “ls -a” command, including space and parameter). Fixed version:

      function cfg.parser ()
      {
      _old_IFS=”$IFS” # save $IFS
      IFS=$’\n’ && ini=( $(<$1) ) # convert to line-array
      ini=( ${ini[*]//;*/} ) # remove comments `;’
      ini=( ${ini[*]//\#*/} ) # remove comments `#’
      ini=( ${ini[*]/\ =\ /=} ) # remove anything with a space around `=`
      ini=( ${ini[*]/#[/\}$'\n'cfg.section.} ) # set section prefix
      ini=( ${ini[*]/%]/ \(} ) # convert text2function (1)
      ini=( ${ini[*]/=/=\( } ) # convert item to array
      ini=( ${ini[*]/%/ \)} ) # close array parenthesis
      ini=( ${ini[*]/%\( \)/\(\) \{} ) # convert text2function (2)
      ini=( ${ini[*]/%\} \)/\}} ) # remove extra parenthesis
      ini=( ${ini[*]/#\ */} ) # remove blank lines
      ini=( ${ini[*]/#\ */} ) # remove blank lines with tabs
      ini[0]=” # remove first element
      ini[${#ini[*]} + 1]=’}’ # add the last brace
      # printf “%s\n” ${ini[*]}
      eval “$(echo “${ini[*]}”)” # eval the result
      if [ -z "$_old_IFS" ] # restore old $IFS
      then unset IFS
      else IFS=”$_old_IFS”
      fi
      }

    • Brooks Moses 12:59 am on 16 January 2009 Permalink | Reply

      Maceij: Why not just use “local IFS” at the top of the function?

    • Piotr Gabryjeluk 12:57 pm on 23 February 2009 Permalink | Reply

      Yeah, should be

      local IFS instead of just IFS

      the rest of the script seemed a bit magic to me at first, but anyway nice peace of work. (I would suggest not using a dot in name of function).

    • Markus E. 1:34 pm on 7 May 2009 Permalink | Reply

      nice work!

      the error described in first post occurs if a section does not contain values.
      this can be solved by adding any command between the brackets which are evaluated by eval().

      i think, there could be a better solution, but following works;
      replace:
      ni[${#ini[*]} + 1]=’}’ # add the last brace

      with something like:
      ini[${#ini[*]} + 1]=’echo -n; }’

    • neofutur 5:17 am on 20 May 2009 Permalink | Reply

      useful post, but the wordpress format makes it a hell to use it.

      what about packaging this in a .tar.gz and give the download link ?

    • bld 2:56 pm on 21 May 2009 Permalink | Reply

      This Bash ini parser may be easier to cut & paste:

      http://codesnippets.joyent.com/posts/show/2060

      cheers,

      bld

    • nic 11:04 pm on 17 August 2009 Permalink | Reply

      Hello!

      I tried the wonderful script and I’m even starting to understand what it does. I would like to modify the two areas where the text is converted to function and key names to do this: uppercase the names and in the case of the section name replace blank characters with underscore. I don’t know how to do this, especially since the script runs the regex for the whole file in one swing (IE not line by line).

      Best regards

      • ajdiaz 8:11 pm on 19 August 2009 Permalink | Reply

        Hi!

        To put the ini contents in uppercase is easy, just change the line:
        IFS=$'\n' && ini=( $(<$1) )
        by the line:
        IFS=$'\n' && ini=( $(tr a-z A-Z <$1) )

        This will be work ok to you. About the spaces in sections, there are no a easy way to do this AFAIK.

        I hope to help you a bit.

        Regards,
        Andres

  • ajdiaz 10:28 pm on 1 February 2008 Permalink | Reply
    Tags: , , tips

    tcptraceroute was another friend of the network administrator. Probably you known classical traceroute, which use the TTL field in IP header to determinate the hops in the route to a specific destination. In each hop the TTL value is decreasing (according to internet protocol), and when TTL is equal to cero, a ICMP is returned to sender IP. So, the classical traceroute technique, send a UDP packet with TTL field setted to 1, and get the IP address of the first hop from returned ICMP, and likewise for other hops.

    Unfortunately, today many host are firewalled and ICMPs are blocking. The classical traceroute design fails, and we only obtain a list of useless “*”. The tcptraceroute use TCP packets instead of UDP packets, and try to connect to usual port enabling the SYN flag. If port is closed, a RST flag is returned, and if port is open then return an ACK flag. So we don’t need ICMPs anymore.

     
  • ssh socket 

    ajdiaz 5:11 pm on 30 January 2008 Permalink | Reply
    Tags: , tips

    When work using a “cheap” wireless network (yes, I still have kind neighbours), each new ssh connection takes a lot of time, because a new authentication is required from the peer, but what happens if I already connected? In theory no new re-authentication is necessary, you can use existing socket to send data over the same channel of the previous connection. To enable socket manager, put the following lines in your ~/.ssh/config file:

    Host *
      ControlMaster auto
      ControlPath ~/.ssh/socket-%r@%h:%p
    

    Some situations may freeze your ssh connection, for example when the network goes down before close connection and timeout is reached, in this case the socket will also be frozen, and new connections to the same destination are no possible. Only need to remove the socket file in ~/.ssh/ directory and kill the previous session.

     
    • hario 11:08 pm on 31 January 2008 Permalink | Reply

      Great tip! I wonder why this does not come preconfigured with OpenSSH… it is savaing me a lot of time at home :D

    • Quatro por Quatro 10:21 am on 4 April 2008 Permalink | Reply

      Use a command to terminate an active ssh connection:

      ssh -S “${control_socket}” -O exit
      “${REMOTE_HOST}”

c
compose new post
j
next post/next comment
k
previous post/previous comment
r
reply
e
edit
o
show/hide comments
t
go to top
l
go to login
h
show/hide help
esc
cancel