Utilities

A collection of scripts and the like which I have come to rely on.

#!/usr/bin/env python
"""
URL-click forwarder (Windows -> Linux).
 
This script catches URL clicks (actually iexplore.exe invokations) and 
forwards them over TCP/IP to a Linux machine which launches them in the 
default browser (e.g. Firefox).
 
This was written so that clicking a URL in, say, MS Outlook in a WinXP VM 
would open the page in the host Linux's browser. (Yes, VMware Fusion can do 
that via VMwareOpenHost.exe, but VMware Player doesn't seem to want to do 
that, so this was written.)
 
WINDOWS-SIDE INSTALLATION
=========================
 
    1. Change ``giHOST`` to match your server-side hostname
    2. Call this script with --install
 
LINUX-SIDE INSTALLATION
=======================
 
    1. Run the script without arguments
 
TODO: Use Python-Win32 modules to implement a Win32 DDE server to catch 
special invocations, such as "start http://example.org"
 
TCP/IP PROTOCOL
===============
 
The protocol uses endpoint readiness to signal when processing has been 
completed. This relies on the behaviour of ``recv()``, which blocks until data 
is available (returns a non-empty string) or the local READ endpoint has been 
closed by the remote end (returns an empty string).
 
    1. Client:
 
        a. Connects to server
        b. Transmits URL data as ASCII-encoded text
        c. Closes WRITE end-point
        d. Starts spinning on recv()
 
    2. Server:
 
        a. Waits for a connection
        b. Spins on recv() until it returns an empty string
        c. Closes WRITE end-point
        d. Spins on recv()
        e. Closes connection
 
    3. Client:
 
        a. Finishes spinning on recv()
        b. Closes connection.
"""
__docformat__ = "restructuredtext en"
 
giPORT = 43233  # That's IE2FF in T9 (predictive text, as on a phone)
giHOST = "e102928-lin"
 
# -----------------------------------------------------------------------------
def regedit(rRegistryFragment):
    import os
    import tempfile
 
    import subprocess as sp
 
    assert os.name == "nt", "You're not running Windows!"
 
    rRegistryFragment = os.linesep.join( rRegistryFragment.splitlines() )
 
    rTF = tempfile.mktemp()
 
    try:
        with file(rTF, "wb") as sTF:
            sTF.write( rRegistryFragment.encode("utf-16") )
            sTF.flush()
 
        # Seems we must close the file or regedit cannot read it. Go figure.
 
        sPH = sp.Popen([
            "regedit",
            "/S",   # Silent
            sTF.name])
 
        iRetVal = sPH.wait()
 
    finally:
        if os.path.exists(rTF):
            os.unlink(rTF)
 
    return iRetVal == 0
 
# -----------------------------------------------------------------------------
def install():
    import os
    import sys
    from textwrap import dedent
 
    # Note that backslashes have to be escaped, and double-escaped between a 
    # drive letter and a path.
 
    rRegistryFragment = dedent(R"""
    Windows Registry Editor Version 5.00
 
    [HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\iexplore.exe]
    "Debugger"="%s %s"
 
    """).lstrip() % (
            (os.path.abspath( sys.executable )
                .replace(":\\", ":\\\\")
                .replace("\\", "\\\\")),
 
            (os.path.abspath( install.func_code.co_filename )
                .replace(":\\", ":\\\\")
                .replace("\\", "\\\\")),
            )
 
    if regedit(rRegistryFragment):
        print "Installation was successful"
    else:
        print "Installation failed, `regedit.exe` returned %d" % iRetVal
 
# -----------------------------------------------------------------------------
def uninstall():
    from textwrap import dedent
 
    # Note that backslashes have to be escaped, and double-escaped between a 
    # drive letter and a path.
 
    rRegistryFragment = dedent(R"""
    Windows Registry Editor Version 5.00
 
    [-HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options\iexplore.exe]
 
    """).lstrip()
 
    if regedit(rRegistryFragment):
        print "Uninstallation was successful"
    else:
        print "Uninstallation failed, `regedit.exe` returned %d" % iRetVal
 
# -----------------------------------------------------------------------------
def server():
    """
    Launch URLs sent via TCP/IP using ``gnome-open``.
    """
    import socket
    import subprocess as sp
 
    sSock = socket.socket()
    sSock.bind( ("", giPORT) )
    sSock.listen(1)
 
    print "Listening on", giPORT
 
    while True:
        sConn, tAddr = sSock.accept()
 
        rMessage = ""
        while True:
            rData = sConn.recv(1024)
            if rData == "":
                break
            rMessage += rData
 
        sConn.shutdown( socket.SHUT_WR )     # Indicate finished
 
        # Spin until remote side is also finished
        while sConn.recv(1024) != "":
            pass
 
        sConn.close()
 
        print rMessage
 
        sPH = sp.Popen([
            "gnome-open",
            rMessage])
 
        sPH.wait()
 
# -----------------------------------------------------------------------------
def client():
    R"""
    ``sys.argv`` should look like so:
 
        0. ``H:\IE2FF.py``
        1. ``C:\Program Files\Internet Explorer\IEXPLORE.EXE``
        2. May be absent. If present, should be one of:
 
            a. ``-nofile``, indicates DDE usage (TODO)
            b. ``http://...`` (or other URL)
    """
    import sys
 
    if len(sys.argv) < 3 or sys.argv[2] == "-nofile":
        # TODO: Implement fallback behaviour (start local browser?)
        return
 
    rMessage = sys.argv[2]
 
    import socket
    import subprocess as sp
 
    sConn = socket.socket()
    sConn.connect( (giHOST, giPORT) )
    sConn.sendall( rMessage )
    sConn.shutdown( socket.SHUT_WR )    # Indicate finished
 
    # Spin until remote side is also finished
    while sConn.recv(1024) != "":
        pass
 
    sConn.close()
 
# -----------------------------------------------------------------------------
def main():
    import platform
    from optparse import OptionParser
 
    sParser = OptionParser(
            description=" ".join(
                x.strip() for x in
                """\
                This script catches URL clicks (actually iexplore.exe 
                invocations) on a Windows machine and forwards them over 
                TCP/IP to a Linux machine (also running this script) which 
                launches them in the default browser (e.g. Firefox).
                """.splitlines()
            )
        )
 
    sParser.add_option(
        "--install",
        action="store_true",
        dest="yInstall",
        help="Intercept all invocations of iexplore.exe. Internet Explorer "\
             "will not be available until --uninstall is used.",
        default=False)
 
    sParser.add_option(
        "--uninstall",
        action="store_true",
        dest="yUninstall",
        help="Disable interception of iexplore.exe. Internet Explorer will "\
             "be available again.",
        default=False)
 
    # Parse command line
    sOptions, lArgs = sParser.parse_args()
 
    if sOptions.yInstall:
        install()
 
    if sOptions.yUninstall:
        uninstall()
 
    if platform.system() == "Linux":
        server()
    else:
        client()
 
# -----------------------------------------------------------------------------
if __name__ == "__main__":
    main()

Runs the command given, but colours STDERR red, making it easier to spot problems when there is a lot of output. Particularly useful with “make” commands.

# Red STDERR
# rse <command string>
function rse()
{
        # We need to wrap each phrase of the command in quotes to preserve arguments that contain whitespace
        # Execute the command, swap STDOUT and STDERR, colour STDOUT, swap back
        ((eval $(for phrase in "$@"; do echo -n "'$phrase' "; done)) 3>&1 1>&2 2>&3 | sed -e "s/^\(.*\)$/\1/") 3>&1 1>&2 2>&3
}
export -f rse # Export function, so it is available to subshells

Examples:

rse find /etc -iname \*.conf

...
/etc/debconf.conf
/etc/ca-certificates.conf
/etc/gssapi_mech.conf
find: /etc/ssl/private: Permission denied
/etc/pam.conf
/etc/nsswitch.conf
/etc/sestatus.conf
...

Allows renaming of multiple files using pattern matching.

# Multi-Move
# mm [file list/glob] [regexp match] [regexp replace]
function mm
{
        [ "$#" -eq 0 ] && echo "mm [file list/glob] [regexp match] [regexp replace]" >&2 && return
        for file in $1; do mv "$file" "$(echo $file | sed -e "s/$2/$3/")"; done
}
export -f mm

Example:

# Files before:
$ ls
1.dat  1.txt  10.txt  2.dat  2.txt  3.dat  3.txt  4.dat  4.txt  5.txt  6.txt  7.txt  8.txt  9.txt
 
# Rename *.txt -> *.txt.bak
$ mm '*.txt' '.txt' '.txt.bak'
 
# Files after:
$ ls
1.dat  1.txt.bak  10.txt.bak  2.dat  2.txt.bak  3.dat  3.txt.bak  4.dat  4.txt.bak  5.txt.bak  6.txt.bak  7.txt.bak  8.txt.bak  9.txt.bak

# Files before:
$ ls
a_1.html  a_2.html  a_3.html  b_1.html  b_2.html  b_3.html  c_1.html  c_2.html  c_3.html
a_1.txt   a_2.txt   a_3.txt   b_1.txt   b_2.txt   b_3.txt   c_1.txt   c_2.txt   c_3.txt
 
# Pad numbers with zeros
# Use escaped parenthesis to capture one or more characters in the range 0-9, and replace them with "0" + "previous contents"
$ mm '*' '\([0-9]\+\)' '0\1'
 
# Files after:
$ ls
a_01.html  a_02.html  a_03.html  b_01.html  b_02.html  b_03.html  c_01.html  c_02.html  c_03.html
a_01.txt   a_02.txt   a_03.txt   b_01.txt   b_02.txt   b_03.txt   c_01.txt   c_02.txt   c_03.txt

Creates a cscope database for the current directory, suitable for use with a vim which has been compiled with +cscope.

#!/bin/bash
echo Creating file list...
find `pwd` "$@" -name \*.c -o -name \*.h -o -name \*.cpp -o -iname Makefile -o -name \*.mak | sed -e 's/^/"/g' -e 's/$/"/g'> cscope.files
echo Building database...
cscope -b -q
echo Done

Simple script to push my config files from the current host to all others in my list

#!/bin/bash
 
# Copies local .vimrc, .screenrc and other settings to all my other accounts
# TODO: Possibly account for my tablet PC, which isn't always on
 
cd ~
FILELIST="
    .vimrc
    .screenrc
    .bashrc
    .bash_profile
    editor.sh
    less.sh
    propagate_rc_files.sh
    .vim
"
if [ $# -gt 0 ]; then
    HOSTLIST="$@"
else
    HOSTLIST="
        meermanr@ikari.robmeerman.co.uk:
        meermanr@ryoga.robmeerman.co.uk:
        meermanr@ryo-ohki.robmeerman.co.uk:
    "
fi
 
function IPof()
{
    # POSIX
    if [ -x "$(which traceroute 2>/dev/null)" ]; then
        traceroute -m 1 -w 2 $1 2>&1 \
        | head -n1 \
        | cut -d " " -f 4 \
        | tr -d "(),"
 
    # Windows (Cygwin)
    elif [ -x "$(which tracert 2>/dev/null)" ]; then
        tracert -h 1 -w 1 $1 \
        | head -n2 \
        | tail -n1 \
        | cut -d " " -f 5 \
        | tr -d "[]\015"        # Snip ^M off the end, stupid Windows<->Cygwin interworking!
 
    fi
}
 
# Find local IP
if [ -x "/sbin/ifconfig" ]; then
    LOCAL_IP=$(/sbin/ifconfig | head -n2 | tail -n1 | sed -e 's/^[^0-9]\+//' | cut -d ' ' -f1)
elif [ -x "/cygdrive/c/WINDOWS/system32/ipconfig" ]; then
    LOCAL_IP=$(ipconfig | grep -C3 "Local Area Connection" | grep "IP Address" | tail -n1 | cut -d : -f 2 | sed -e 's/\s\+\(\S\+\)\s\+/\1/')
else
    LOCAL_IP=""
fi
 
echo LOCAL_IP = \"$LOCAL_IP\"
 
for HOST in $HOSTLIST; do
    TARGET_HOST=$(echo $HOST | cut -d '@' -f2 | cut -d ':' -f1)
    TARGET_IP=$( IPof $TARGET_HOST )        # Function call
    echo $TARGET_HOST = \"$TARGET_IP\"
    [ "$LOCAL_IP" = "$TARGET_IP" ] && echo "Skipping..." && continue
    [ "127.0.1.1" = "$TARGET_IP" ] && echo "Skipping..." && continue
 
    echo Uploading to $HOST
    scp -r ${FILELIST} $HOST
    echo ___
    echo
done

Runs a command in a Microsoft Platform Builder environment.

#!/usr/bin/bash
# vim: set ft=sh:
 
# Runs a command in a Microsoft Windows CE Platform Builder environment
 
 
 
# If you don't have a buildenv.bat you can create one by running
#
#   pbxmlutils /getbuildenv > ../../buildenv.bat
#
# from inside a build window (Visual Studio -> Build -> Open Release Directory
# in Build Window).
 
 
 
# Note the parenthesis below, all commands inside them are run in a
# sub-shell, which prevents the export polluting the current shell
(
 
    PB_BUILDENV_BAT=/cygdrive/c/WINCE600/OSDesigns/mwin/WINCE600/OSDesigns/mali-win-ebmpcore/buildenv.bat
 
    # Convert all "set" commands in $PB_BUILDENV_BAT to "export" commands,
    # and evaluate them (updating this sub-shell's environment)
    eval \
        $( grep '^set' $PB_BUILDENV_BAT \
            | sed \
                -e 's/^set /export /' \
                -e 's/=/="/' \
                -e 's/$/"/' \
                -e 's/\\/\\\\/g' \
        )
 
    # Convert paths in $PATH from Win32 style ("c:\foo;c:\bar") to  Cygwin
    # style ("/cygdrive/c/foo:/cygrive/c/bar) using the `cygpath` utility
    # NB: Must use absolute paths here, because the effective $PATH is trashed!
    eval \
        $( echo export PATH=\"$PATH\" \
            | /usr/bin/sed -e 's/;/\n/g' \
            | /usr/bin/xargs --delimiter="\n" /usr/bin/cygpath -u \
            | /usr/bin/tr "\n" ":"
        )
 
    # Run arguments in this altered sub-shell
    "$@"
)

Example:

# What happens without the "pb" command
$ build
-bash: build: command not found

# And with the "pb" command
$ pb build
/home/robmee01/usr/bin/pb: line 32: export: `WECVERSIONFORROSEBUD.1488=2': not a valid identifier
BUILD: [Thrd:Sequence:Type  ] Message
BUILD: [00:0000000000:PROGC ] Build started with parameters:
BUILD: [00:0000000001:PROGC ] Build started in directory: c:\work\WinCE\estimate_current_directory
...

Convenience script which picks the appropriate build method. Suited for use as a one-fire make command from vim. :set makeprg=~/usr/bin/mymake

Written primarily to allow me compile GNU-Make and Microsoft-Platform-Builder¹⁾ driven sources using a single command which works out which to call, and mangles the output from Platform Builder so it looks more like “traditional” compiler output which I vim can parse.

#!/usr/bin/bash
# Wrapper script to find the most appropriate build method depending on where
# it is called from
 
# Configure pre/post scripts
# Note that ":" is a BASH built-in which returns true (0)
if [ -r ./mymake-pre ]; then
    PREEXEC=./mymake-pre
else
    PREEXEC=:
fi
if [ -r ./mymake-post ]; then
    POSTEXEC=./mymake-post
else
    POSTEXEC=:
fi
 
if [ -e sources -o -e dirs ]; then
    # WindowsCE
    # Mangle the output so it resembles every other compiler on the planet,
    # and thus can be intepreted by decent text editors
    $PREEXEC && \
    pb build "$@" 2>&1 \
        | tr -d "\\015" \
        | grep -e "WARNN\\|ERRORE" \
        | sed -e "s@^.*] @@" -e "s@ : @:@" \
    && $POSTEXEC
    exit
fi
 
for filename in ./mymake ../mymake; do
    if [ -x $filename ]; then
        $PREEXEC && exec $filename "$@" && $POSTEXEC
    elif [ -r $filename ]; then
        $PREEXEC && source $filename "$@" && $POSTEXEC
        exit
    fi
done
 
if [ -e Makefile -o -e GNUMakefile ]; then
    $PREEXEC && make "$@" && $POSTEXEC
    exit
fi
 
echo "Don't know how to make this directory, sorry."

Runs a python script using the Win32 version of Python from within Cygwin. Only useful if you have both Cygwin-python and Win32-python installed on the same machine.

#!/usr/bin/bash
# Wrapper to allow shell scripts to be run using the Windows (not Cygwin) version of Python
# Specifically, translates the first argument into a Windows-friendly path, and hands-off to Python
SCRIPT=$1; shift
exec /cygdrive/c/Python25/python.exe $(cygpath -w "$SCRIPT") "$@"

Jump to a topic in the Microsoft Visual Studio 2005 Documentation explorer (dexplore.exe) via the command line. I wrote this so I could use vim's manpage-lookup feature when editing WindowsCE source code.

Hard-coded to only search within the documentation for “Windows CE 6.0”.

Note: This script requires Python's win32com module, which is part of Python for Windows extensions.

Note: dexplore.exe must already be running for this script to work. Code to start dexplore.exe if needed may be added later.

Note: This uses the python-win script (above) as the interpreter to ensure it is run using the Win32 installation of Python

#!/home/robmee01/usr/bin/python-win
# Jump to keyword in DExplore
# Huge thanks to Rob Chandler and his page http://www.helpware.net/mshelp2/dexplore/dexplorer.htm
import win32com.client, sys
h = win32com.client.Dispatch("DExplore.AppObj")
h = h.Help  # Probably not necessary
h.SetCollection("ms-help://MS.VSCC.v80/MS.VSIPCC.v80/MS.WindowsCE.v60.en", "Windows Embedded CE 6.0")
if sys.argv[1][0:10] == "ms-help://":
    h.DisplayTopicFromURL(sys.argv[1])
else:
    h.DisplayTopicFromKeyword(sys.argv[1])

Example:

# Jump to the topic "GetModuleFileName" in an existing instance of dexplore.exe
$ dexplore-jump.py GetModuleFileName