Headlines | Linux | Apps | Coding | BSD | Admin | News
Information for Linux System Administration 

Scripting: Bash Array Tutorial


An excellent introduction to bash arrays including 15 examples...
$ cat
#! /bin/bash
Unix[1]='Red hat'

echo ${Unix[1]}

Red hat
permapage | score:9219 | -Ray, June 7, 2010

Pattern matching in shell scripting


This article is excerpted from the book Beginning Portable Shell Scripting.
Shell programming is heavily dependent on string processing. The term string is used generically to refer to any sequence of characters; typical examples of strings might be a line of input or a single argument to a command. Users enter responses to prompts, file names are generated, and commands produce output. Recurring throughout this is the need to determine whether a given string conforms to a given pattern; this process is called pattern matching. The shell has a fair amount of built-in pattern matching functionality.
mail this link | permapage | score:9048 | -Ray, January 1, 2009

Scripting: Put a clock in your bash terminal


In the original version, the cursor positioning didn't work on my Mac OS X system. If that happens to you, try this simplified variant:
while true
tput sc
tput cup 0 60
echo -en `date +"%H:%M:%S %F"`
tput rc
sleep 1
Also, note that you'll need to run either script in the background to use your terminal.
The script saves the current cursor position with an ANSI escape sequence instruction. Then, using the tput command, the cursor is sent to row 0 (the top of the screen) and the last column minus 19 characters (19 is the length of HH:MM:SS YYYY-MM-DD). The formatted date command is displayed in green inverted color. The cursor is then sent back to its original position with another ANSI sequence that restores the original saved position.
mail this link | permapage | score:9007 | -Ray, January 22, 2008

Scripting: A parallel Linux backup script


This example bash shell script demonstrates a simple method of creating backups of multiple filesystems to multiple tape devices simultaneously. While the script presented writes to four tape drives in parallel, it can easily be modified to write to other device types and to create a different number of backup streams. The script is set up for the bash shell under Linux, but modifying it for another variety of Unix should simply be a matter of changing the locations of utility files such as tar, echo, cp, and sleep.

The script can be downloaded from Download the file now and load it into an editor as this article will refer to it frequently. Also, you may want to modify bits of it to match your filesystem names and your devices.

The first line of the script looks like this:
If the bash shell isn’t in the /bin directory on your system, you’ll need to modify this line. Enter the command which bash now to verify the location of bash. My Fedora Linux system and my Mac OS X system both have bash in /bin, but my FreeBSD system does not. If you have a non-Linux flavor of Unix, you’ll probably need to use the ‘which’ command to verify the locations of each command used in the script. The commands used are:
Note that ‘wait’ and ‘cd’ are usually implemented as internal shell commands and may not have external commands associated with them. If that is true for your system, leave ‘cd’ and ‘wait’ with no directory prefix just as they are in the original script.

Now, the first command in the script resets the current working directory to ‘/’:
 cd /
Since the script precedes each directory to be backed up with a ‘.’ to represent the current working directory, starting out at ‘/’ is necessary. The reason for this precaution is that some implementations of the tar command will only load files from a tar archive into the exact directory that was specified when the file was backed up. By prefixing the names with a ‘.’ we preserve the ability to recover the files into any subdirectory we want, without overwriting the original files.

Immediately after the ‘cd /’ command is where you would put any commands to shut down all services that must be quieted prior to a backup. The example script has a (commented out) command to initiate an Oracle database shutdown followed by a ‘sleep’ command to allow time for the shutdown to complete. The example database shutdown and the following delay probably don’t apply to your system. Obviously, you’ll have to add commands yourself to stop any applications that might interfere with the backup.

Next, we use the ‘date’ command to create two sets of four tiny files to stick at the start and end of each tape. Note that the presence of a ‘date.#’ file at the beginning of each tape lets you quickly find out when a tape was created and on which drive. The ‘zzzz.#’ files, appended to the end of each tape, only serve to let you easily verify that a backup completed without overrunning the end of the tape.

Next, we start the four actual ‘tar’ backup commands, each with sample directories named ‘./dir1’, ‘./dir2’, etc. Of course, you’ll need to modify the list of directories to match the actual directories you wish to back up. Note that you’ll probably want to balance the directory sizes so that all of the largest directores aren’t on the same tape. Also, note that each ‘tar’ command is run in the background and logs to a tar.#.log file in the /tmp directory. Obviously, you might want to put the logfiles somewhere else.

After each ‘tar’ command there is an entry like this: ‘TASK=$0’, or ‘TASK=$1’. These arbitrarily-named ‘TASK’ variables are used to store the process ID of each ‘tar’ command so that the script can wait for them with the four ‘wait’ commands that follow in the next block of code. There, we have the four ‘wait’ commands waiting on the $TASK0, etc, variables. (The addition of the ‘$’ to each TASK# shell variable is not a typo -- it’s necessary to read back the contents of the variable.)

Next, after the script has waited for the completion of each of the four ‘tar’ commands, it appends some information to a history file for later reference. It stores the date of the backup, the filesize of the logfile, and the number of files backed up on each tape to each of four history files. While the script will overwrite the logfiles (tar.#.log) each time it is run, it will append these three lines to each of the four history files (tar.#.history).

The final steps in the script are commented out. Those are the commands necessary to restart any applications that were brought down for the backup. Again, in the example we assume an Oracle database needs to be restarted. You’ll need to add the commands necessary to start any applications that were stopped at the beginning of the script.
mail this link | permapage | score:8940 | -Ray, April 10, 2005

The Vim Scripting editor


Vimscript is a mechanism for reshaping and extending the Vim editor. Scripting allows you to create new tools, simplify common tasks, and even redesign and replace existing editor features. Start with Part 1 and get the basic elements of Vimscript. In Part 2 learn how to create and deploy new functions in the Vimscript. In Part 3 Explore Vimscript's support for lists and arrays. read more...
mail this link | permapage | score:8624 | -solrac, February 4, 2010

The Squirrel shell and portable scripting language


If you don't want to commit to the idiosyncrasies of a specific shell running on a particular platform, try the Squirrel Shell. The Squirrel Shell provides an advanced, object-oriented scripting language that works equally well on UNIX, Linux, Mac OS X, and Windows systems. Write a script once, and run it anywhere. read more...
permapage | score:8458 | -solrac, March 19, 2009

bash scripting: Looping through a list


This is an example of a Bash shell script used to loop through a list to compare to text strings found in logs. The script is used to detect and block attacks on a web site. read more...
permapage | score:8371 | -aweber, December 26, 2011

Comparison Matrix: Scripting Language Survey


This comparison includes sh (bash), Perl, Ruby, Ch, Python, Pike, Tcl, Awk, merd, Lua, OCaml, Scheme, JavaScript, Haskell, Smalltalk, tcc, C, C#, and Java...
The Scriptometer tries to measure whether a programming language can be easily used for SOP (Script-Oriented Programming).

For this:
  • the programming environment is checked: ability to compile and run in one command, REPL (Read-Eval-Print Loop)...
  • some typical SOP tasks are written in each programming language, and the length of the resulting program is measured.
mail this link | permapage | score:8312 | -Ray, June 14, 2004

Mac Shell Scripting Tutorial


A tutorial on scripting for the Mac, from Apple.
This document assumes that you already have some basic understanding of at least one procedural programming language such as C. It does not assumes that you have very much knowledge of commands executed from the terminal, though, and thus should be readable even if you have never run the Terminal application before.

The techniques in this document are not specific to Mac OS X, although this document does note various quirks of certain command-line utilities in various operating systems. In particular, it includes information about some cases where the Mac OS X versions of command-line utilities behave differently than other commonly available versions such as the GNU equivalents commonly used in Linux and some BSD systems.
mail this link | permapage | score:8244 | -Ray, October 10, 2006

Sysadmin Shell Scripting


This three-part series covers basic, intermediate and advanced shell scripting techniques for system administrators.
A working knowledge of shell scripting is vital if someone wants to become good at system administration tasks. Since this tutorial tackles topics that assume a basic understanding of shell scripting, we strongly urge you to take a look at our Shell Scripting: The Basics article first…

One important aspect of shell scripting is file-oriented utility. A file-oriented utility is basically used as a filter in a pipe.

We can add a ‘-’ to get a more useful result. That is, when we have ‘file -’, the shell waits for the user input and analyses it.
mail this link | permapage | score:8204 | -Ray, December 17, 2010

Scripting: Convert your gzip files to bzip2


A 'rezip' script that converts your .gz files to .bz2 for better compression and maybe better error recovery -- at a substantial cost in compression time, of course. Rezip...
Uses a simple text file of paths and filenames for input -- so you can save the results of "find" to a file, run rezip, and the files will be re-compressed one at a time, with a running log and no user intervention (as long as there's free space on the destination drive.) Example:

$ find /mnt/bkps -name *.gz > ~/rezipp-files.txt && rezip
mail this link | permapage | score:8172 | -Ray, February 13, 2006

Scripting: Bash floating point


Is the lack of floating point math putting a crimp in your scripting style?
When you think about it, it's surprising how many programming tasks don't require the use of floating point numbers. If you're an embedded systems programmer, you'd probably get fired for using "double" in a C program. If you write PHP or JavaScript, quick, do they even support floating point? One language that doesn't support it is Bash, but let's not let that stop us...

The obvious candidate for adding floating point capabilities to bash is bc.
mail this link | permapage | score:8168 | -Ray, August 5, 2008

Scripting: Command line PHP in Linux


Learn how to better integrate scripts with command-line tools. Examine using shell_exec(), exec(), passthru(), and system(); safely passing information to the command line; and safely retrieving information from it. See how to integrate closely with underlying shell commands and folding any return values into your interfaces and processes. read more...
permapage | score:8161 | -solrac, May 20, 2009

Scripting: Bash sub shells


Sometimes you need to run scripts in the background...
Creating sub-shells in bash is simple: just put the commands to be run in the sub-shell inside parentheses. This causes bash to start the commands as a separate process. This group of commands essentially acts like a separate script file, their input/output can be collectively redirected and/or they can be executed in the background by following the closing parenthesis with an ampersand.
permapage | score:8093 | -Ray, November 17, 2008

Shell Scripting: A Low-precision Visual Timer


A low-precision timer script for your Linux / Unix box...
For many years, I've wanted a simple command-line timer program to use in conjunction with short human tasks — the sort of thing my wristwatch timer works for, but I'm not always wearing that watch. I want something that provides some indication of elapsed time or time remaining. Amazingly, the commercial Unixes I'm familiar with do not provide such a utility; nor does one seem to be readily available with the standard X Window tools.
mail this link | permapage | score:8085 | -Ray, May 10, 2004

Bash Shell Scripting String Functions


This intermediate bash shell scripting article provides some useful functions, in source code of course.
In C, defines strcat(3), strcpy(3), strlen(3), and strcmp(3) for string concatenation, copy, size, and test operations respectively. Such basic operations are needed constantly when programming in any language, and shell scripting is no exception.
permapage | score:8037 | -Ray, November 12, 2004

Scripting: Date arithmetic with shift_date


Wouldn't it help your scripts if you could easily add or subtract a few days from a date?
shift_date is a Korn shell script to add or subtract a number of days from a date. It allows you to specify any date beween 0001 and 9999 AD, and add or subtract any whole number of days, e.g:

shift_date 20031225 -30

I wrote it to solve a problem I was having with an archiving script, where I wanted a way to take a date and subtract, for example, 90 days, the way you can in SQL (for example, "SYSDATE -90" in Oracle). Although there are ways of testing this kind of condition using find's "-mtime" option, I wanted to get an actual YYYYMMDD string that I could use in file names, log entries, interactive prompts, and so on.
mail this link | permapage | score:8035 | -Ray, January 27, 2004

Scripting: wait vs. sleep


When coordinating parallelized processes, wait is the essential command.
The wait command man page simply states that it pauses script execution until a specified process ID completes. The man page for the sleep command is a single line stating that it causes a script to pause for specified period of time.

How do you decide which is appropriate for the script at hand? In some cases it won't make any difference to the successful completion of the script. Other times it can be the difference between success and total failure to produce the expected results.
mail this link | permapage | score:8014 | -Ray, March 29, 2005

Shell Scripting Tutorial: Reading Cursor, Function Keys


This tutorial includes examples of how to read arrow keys and function keys with the bash shell...
My preferred shell for interactive scripting is bash because of the readline library and the extensions to the read command. In this article, I will forsake portability for the clarity of bash (version 2 or later). If you want to use stty and dd, the principles are the same. You can find out more about using dd in Mouse Reporting in a Shell Script and Trapping Special Characters in the Korn Shell; the techniques in this article are similar to the ones discussed there.
mail this link | permapage | score:8002 | -Ray, November 6, 2005

Monitor a Linux service with watchdog scripting


Monitor your web server, or just possibly, that program that mysteriously disappears occasionally...
Old Unix hands already know this, but new Unix (Linux) users may be asking, ‘What is a “watchdog script”?’ Basically it is a bash or other script that is run via cron periodically to check on a persistent service. The watchdog script takes actions based on the state of the service it monitors.
permapage | score:8001 | -Ray, May 19, 2010
More articles...
Abstract Art by Ray Yeargin

Recent headlines

Tutorial: Create an NFS-like Storage Server with GlusterFS on Ubuntu 12.10

10 basic examples of Linux ps command

LG 8-inch G Pad 8.3 Android tablet

Tutorial: Webcam streaming your desktop plus audio with ffmpeg, crtmpserver, Flowplayer

How to install HotShots on Fedora 19 and Ubuntu 13.04

OpenVZ Mount Tutorial: Mount Host Devices, Partitions, Directories in Containers (Debian, Ubuntu)

OpenBSD Tutorial: Configure Ralink USB Wireless Adapter

Tutorial: Debian 7 Samba Server with tdbsam

CoolShip Android all-in-keyboard computer

Install openQRM 5.1 on Debian 7

Dual-boot Windows 7 and Ubuntu 12.04 on a PC with UEFI board, SSD and HDD

Linux find command examples: finding files

Virtual Users/Domains with Postfix/Courier/MySQL/SquirrelMail (Debian 7)

Ubuntu Edge: Is there life after an unsuccessful crowd-funding campaign?

Tutorial: Install Debian 7 (testing) with debootstrap from a Grml live Linux

Upgrade Fedora 18 to 19 With FedUp

Epoptes – Open source computer lab management and monitoring tool

Elementary OS 0.2 Luna review

Tutorial: Fedora 19 Samba server with tdbsam

Setup Nginx + php-FPM + apc + MariaDB on Debian: The perfect LEMP server

Linux mail command examples: send mails from command line

Pipelight – Using Silverlight in Linux browsers

Unix: Shell Script Wrapper Examples

Using Multiple PHP Versions (PHP-FPM FastCGI) With ISPConfig 3 (Ubuntu 12.04)

Sagemath in the Cloud and Sagemath 5.11

Tutorial: Replace Windows with Ubuntu 13.04

Tutorial: Running CS-Cart on Nginx on Debian 7/Ubuntu 13.04

Hand of Thief trojan and your favorite Linux distribution

OpenShift Online: a non-developer guide

Plasma Media Center 1.1 and digiKam 3.3

Linux Iptables Examples

Lynis: Security and system auditing tool

Nuvola Player: Enjoy all your Cloud music services from one interface

Tutorial: Automatically add a disclaimer to emails with alterMIME (Postfix on Debian 6)

Set up Oneiric PVHVM at Xen 4.1.2 Ubuntu 11.10 Dom0

Build a Mail Server on CentOS 6 with Postfix, Dovecot, PostgreSQL

Ubuntu is not a community distribution

Tahoe Least-Authority File System for secure, distributed data storage

TuxOnIce: Hibernate Linux

Tutorial: Ubuntu Jeos 12.04 LTS Spam Filter Gateway Server


Firefox sidebar

Site map

Site info

News feed


(to post)


Articles are owned by their authors.   © 2000-2012 Ray Yeargin