====== Unixy Stuff ======
I run Linux at home, and use it constantly at work. On the those few occasions then I have to work on a Windows box I use [[http://www.cygwin.com/|Cygwin]] (a [[wp>unix-like]] shell for Windows(TM)).

//**Note:** This are also includes web-related things, such as Apache, which arn't really specific to unix & linux systems//

  * [[unix:lvm_recovery]] --- Recovering my LVM volume stored on my RAID array.
  * [[unix:flash]] --- Getting fullscreen performance out of Adobe Flash under Ubuntu
  * [[unix:x|My Perfect X11 setup]] --- Portrait PC monitor with compiz + HDTV without compiz
  * [[unix:xinput|Logitech Marble Mouse scroll-wheel emulation]] --- How to configure a trackball mouse with extra button to pretend it has a scroll-wheel
  * [[unix:256colours|Enabling 256 colour text in terminals]] --- Guide on how to get gnome-terminal/PuTTY + GNU screen + Vim to work in 256-colour mode
  * [[unix:xbmc|XBMC Media Center (on Ubuntu)]] --- Workarounds and scripts for using XBOX Media Center on Linux
  * [[unix:unison|Using Unison with DokuWiki]]
  * [[unix:Webmastering]] --- My personal tips and experiences with being a webmaster
  * [[unix:gateway|Linux Gateway]] --- Easy-to-use firewall/NAT script I created and similar efforts
  * [[unix:Debian]] --- Notes for my own benefit about how I've installed Debian at home
  * [[unix:USB Booting]] --- Wanted to reflash my BIOS, without resorting to floppies or CD-ROMs.

====== Backups ======

Use [[https://pypi.org/project/duplicity/]]

===== Cygwin =====
I use Cygwin so much, that I need to keep some tips on it here.

  * [[unix:cygwin:xserver]] --- Fixing up the clipboard in the Cygwin/X Xserver.

===== Finding directories NOT containing a specific file =====
<code bash>
(find -mindepth 2 -maxdepth 2 | grep UNWANTED_FILE_NAME | cut -d/ -f2; command ls) | sort | uniq -u
</code>

This creates to output streams, one from the ''find'' command and one from ''ls'', that are combined, sorted and finally filtered to remove any entries that occurs more than one.

The streams are combined by the parenthesis which create a sub-shell.

===== Finding directories, without examining .snapshots =====
<code bash>
find . -type d \( -name .snapshot -prune -o -name .git \)
</code>

Looking in the current directory, ''.'', find entries of type ''d'' (directories) for which: if the name is ''snapshot'' remove it from consideration, if the name is ''.git'' carry on (to implicitly ''-print'' the entries to STDOUT).
===== Renaming =====
There is actually a ''rename'' command, which can be used to change extensions thus:
  rename .rpmnew '' *.rpmnew

This would perform the following:
  main.cf.rpmnew -> main.cf
  master.cf.rpmnew -> master.cf

===== History Substitution =====
(Explained to me by [[http://www.catonmat.net/blog/the-definitive-guide-to-bash-command-line-history/|The Definitive Guide to Bash Command Line History]])

If you want to reuse the arguments from your last command, you can do so like so:

<code bash>
svn status file1 file2 file3
svn diff !!:2  # Equivalent to "svn diff file1 file2 file3"
</code>

So that's an "event designator" of ''!!'' to select the previous command, coupled with a "word designator" of ''2'' to select arguments 2 onwards (zero indexed, of course). These are separated by a colon.

===== Make STDERR red =====
Add this function to your BASH environment (e.g. copy'n'paste it to your ''~/.bash_profile''), and you will be able to make STDERR output from your commands appear red, making them easier to see.

<code bash>
# Red STDERR
# rse <command string>
function rse()
{
	# We need to wrap each phrase of the command in quotes to preserve arguments that contain whitespace
	# Execute the command, swap STDOUT and STDERR, colour STDOUT, swap back
	((eval $(for phrase in "$@"; do echo -n "'$phrase' "; done)) 3>&1 1>&2 2>&3 | sed -e "s/^\(.*\)$/$(echo -en \\033)[31;1m\1$(echo -en \\033)[0m/") 3>&1 1>&2 2>&3
}
</code>

==== Usage ====
Typical usage is as simple as placing the characters "''rse ''" infront of the command you want to affect the output of.

<note warning>
Avoid colouring the output of highly interactive programs, such as ncurses-based program (aptitude, cscope etc) or bash. You will get unexpected side-affects.

For instance, try ''rse bash'' and you will notice that it "works" in the strict sense, but isn't useful: You will not see a prompt, nor will tab-completion work or other shell-shortcuts (such as UP/DOWN to scroll history), however you will be able to run commands in the usual way and see their output coloured.

If someone knows a work-around for this, [[robert.meerman@gmail.com|please let me know]]!
</note>

Suppose you have a script called **''rse_test.sh''**:
<code bash>
echo STDOUT
echo STDERR >&2
</code>

you would colour its output like so:

<code>rse ./rse_test.sh</code>

and see

<html> <pre>
STDOUT
<font color="red">STDERR</font>
</pre> </html>

However, the following does **not** colour the output:

<code bash>rse echo ERROR >&2</code>

"ERROR" will be printed in the usual colour, because the redirection directive is intepreted by the shell, and not by the "rse" function. Adding parenthesis would group the commands like so:

<code>( rse ( echo ERROR ) ) >&2</code>

==== How it works ====
''rse'' wraps all lines send to STDERR with control characters which mean "set foreground colour to red" and "restore default foreground colour".

=== Colouring ===

The control characters are [[wp>ANSI_escape_code|ANSI escape code sequences]] which most((Microsoft's ''cmd.exe'' being a notable exception, but then I suppose it's a stretch to call it "modern" anyway.)) modern terminals understand.

The escape sequences in the code are:
<code>
^[[31;1m -- Set foreground to red (32), set bold (1)
^[[0m    -- Restore default style
</code>

<note>I have written "^[" to mean the ESC character, but I have done this by typing two seperate characters - therefore copy'n'pasting this text will have no effect.

To type an ESC character at your shell, press CTRL+V and then hit ESC. You will notice that if you hit backspace both the "^" and "[" characters are removed as once.</note>

In my ''~/.bash_profile'' I have raw ESC characters, but these cannot be (reliably)((Firefox 2.x could /almost/ do it, it printed a little "left-arrow" icon which looked about right, but then it would truncate the page it was displaying...)) displayed in browsers. So, as a work-around, I have used command-substitution in the code above.

The following text is replaced with an ESC character when it is evalutated by BASH:

<code>$(echo -en \\033)</code>

The "-e" passed to ''echo'' tells it to evaluate the text it will be echo'ing, in this case meaning that "\\033" becomes a single character with octal value 33; the ESC char. The "-n" stops ''echo'' printing a newline.

Wrapping lines is achieved by using the [[wp>Sed|sed command]]:

<code bash>sed -e "s/^\(.*\)$/$(echo -en \\033)[31;1m\1$(echo -en \\033)[0m/"</code>

We provide a sed-script on the command line and ask ''sed'' to evaluate it with the "-e" option. The script is a single substitute command, of the form: "s/before/after/".

Our "before" patterns is a match of the entire line: "^.*$" where we capture the line using escaped-parenthesis: "^\(.*\)$".

Our "after" pattern is essentially: "{RED}\1{NORMAL}" where "\1" is the text that was captured. RED and NORMAL are as described above when explaining ANSI escape code sequences.

=== Applying the colouring to STDERR ===
Every process has three file-handles by default:

^ Number ^ Name   ^ Purpose ^
| 0      | STDIN  | This is how keystrokes are communicated to the program |
| 1      | STDOUT | This is where "normal" output is sent |
| 2      | STDERR | This is where exceptional output, such as errors, are sent |

If you are running only a single program, then the distinction between STDOUT and STDERR is arbitrary - they will both appear on your screen. They become significant when you have a pipe-line of programs.

For example, suppose you want to know how many files are in the directory tree under your current directory. You can list all the files using the "find" command, and you will output something like this:

<code>
find: ./ssl/private: Permission denied
find: ./cups/ssl: Permission denied
find: ./vpnc: Permission denied
.
./acpi
./acpi/ac.d
./acpi/ac.d/85-anacron.sh
./acpi/ac.d/cpufreq.sh
./acpi/ac.d/hal-disable-polling.sh
./acpi/ac.d/mount-noatime.sh
./acpi/ac.d/vm-dirty-writeback.sh
./acpi/ac.d/wired-enable-auto-negoatiation.sh
./acpi/ac.d/wireless_set_power.sh
  ...
</code>

And you can count the number of lines printed with the word-count utility by specifying the "-l" option for lines: "wc -l".

Putting these together we get "find | wc -l", where the output from "find" will be sent as input to "wc -l". When I run this I get:

<code>
find: ./ssl/private: Permission denied
find: ./cups/ssl: Permission denied
find: ./vpnc: Permission denied
2636
</code>

Note that the errors from "find" are still displayed, but so is a line-count from "wc". This is because the errors were printed on STDERR, which is //not// sent to "wc" as input.

When two programs are piped together, STDOUT is connected to STDIN. STDERR, on the other hand, remains connected to the parent process, which is our shell in this case. Therefore error messages are not processed by the pipe-line. In our example this was useful, we ended up counting the number of files which we have access to. If error messages were printed to STDOUT, then our count-number would be inaccurate.

How is this relevant? Well, it is not possible to manipulate STDERR in a pipeline, so an alternative approach was required.

Although you cannot manipulate STDERR, you can ask your shell to redirect it. The most common use of this is to suppress errors by redirecting them to /dev/null. However, you can do more than redirect them to files.

''rse'' works by swapping STDOUT and STDERR, modifying the new STDOUT, and then swapping them back. This is achieved by nesting subshells, and having each subshell instruct its parent to swap the streams:

<code bash>
( ( {USER-CMD} ) 3>&1 1>&2 2>&3 | {MODIFY-NEW-STDOUT} ) 3>&1 1>&2 2>&3
</code>

Where "3>&1 1>&2 2>&3" instructs the parent to swap STDOUT/STDERR. This is parsed from right-to-left, and uses the numbers in the table above. This works by shifting both STDOUT/STDERR up by one number and then moving the top-most one to the bottom; effectively a swap. Therefore it reads: "Renumber STDERR as #3, Renumber STDOUT as STDERR, Renumber #3 as STDOUT".

Note that each redirection directive appears after a closing parenthesis. Commands listed within parenthesis are executed in a new sub-shell, and therefore a parent-child relationship is created. This allows us to instruct the parent to manipulate the child's output streams.

=== Running the user's command ===
In order for all this magic to work, the user's command must be run in the deepest sub-shell. This is achieved by re-assembling the arguments passed to the rse() function into a command-line and then executing them.

<code bash>
eval $(for phrase in "$@"; do echo -n "'$phrase' "; done)
</code>

"eval" is a BASH built-in which executes a (single) string passed to in the same way which command you enter at the prompt are.

''"$@"'' is the original list of arguments passed to the function. The code iterates over it and stitched them togther so that ''eval'' has a single command strint to execute.

<note>You may see scripts that use ''$*'' where I have used ''"$@"''. These differ in how they treat white-space. Consider the command

<code bash>
echo "Hello" "World" "How are you?"
</code>

''"$@"'' would tell you the arguments are:
  * Hello
  * World
  * How are you?

while ''$*'' would tell you they are:
  * Hello
  * World
  * How
  * are
  * you

Noting that there are quotation marks in the resulting lists.
</note>

===== Ctrl + S in PuTTY =====
If you ever accidently press ^S when using PuTTY, you'll know that this causes it to stop responding, which I need not mention is highly annoying, but I will because it's //that// annoying :-).

To get your session to respond again hit Ctrl + Q.

**//Quoted from [[http://www.plug.org/pipermail/plug/2004-April/005872.html]]//**
>> Every time I accidentally press Ctrl+s in Putty when connected to any 
>> Linux box via ssh, my session stops responding. How do I restore it?
>
>This is software flow control - left over from the days of serial
>connections. Control+S sends XOFF and Control+Q sends XON and together they
>make XON/XOFF, or "software," flow control. Hardware flow control is known
>as RTS/CTS and involves switching the signal on a couple of the pins
>connected to the serial port.

To disable flow control add this to your shell's init script (e.g. ''.bashrc''):

<code bash>
# Disable software flow control (XON / XOFF or ^S / ^Q)
stty -ixon
</code>

===== Disabling CapsLock =====
Caps Lock is utterly useless. It is extremely seldom that I want to type a lot of stuff in upper-case, and when I do I'll just let my text editor upper-case it after I've written it.

More to the point, I use glorious [[http://www.vim.org|vim]], and the difference between pressing "ZZ" and "zz" is closing the editor without saving or scrolling my window so the cursor is in the middle.

Adapted from [[http://alecthegeek.wordpress.com/2007/01/05/handy-hacks-disable-caps-lock/|Handy Hack: Disable Caps Lock under GNOME]].

Create/modify ''~/.Xmodmap'' so it contains the following:
<code>
remove lock = Caps_Lock
</code>

That's it. Next time X is started, CapsLock will be disabled. If you want the change to take affect immediately (i.e. without logging out), use the following command:

<code>xmodmap ~/.Xmodmap</code>

===== SSH port-forwarding =====
http://www.stearns.org/doc/ssh-techniques-two.current.html

===== X11: "Client is not authorized to connect to Server" =====

From: https://wiki.tcl-lang.org/page/Client+is+not+authorized+to+connect+to+Server

Copy xauth token from somewhere with access:

  xauth extract - $DISPLAY

and import it to the place that's having trouble:

 xauth merge -

For example, to allow a Docker container access to the X11 display:

  xauth extract - $DISPLAY | base64

and copy the result to the clipboard, and then in the container shell:

  base64 -d <<<'BASE64_GOES_HERE' xauth merge -