Shell Scripting Recipes: A Problem-Solution Approach

by Chris F.A. Johnson

Apress, 2005

448 Pages

About the Author	xv
About the Technical Reviewer	xvii
Acknowledgments	xix
Introduction	xxi

CHAPTER 1:
The POSIX Shell and Command-Line Utilities

Shell Commands		1
Parameters and Variables		7
	Positional Parameters	7
	Special Parameters	7
standard-vars — A Collection of Useful Variables		9
Patterns		9
	Filename expansion	10
	Regular Expressions	10
Parameter Expansion		12
	The Bourne Shell Expansions	12
	POSIX Parameter Expansions	14
	Shell-Specific Expansions, bash2, and ksh93	15
Shell Arithmetic		16
Aliases		16
Sourcing a File		17
Functions		17
	Functions Are Fast	17
	Command Substitution Is Slow	17
	Using the Functions in This Book	18
standard-funcs: A Collection of Useful Commands		19
	1.1 get_key — Get a Single Keystroke from the User	19
	1.2 getline — Prompt User to Enter a Line	20
	1.3 press_any_key — Prompt for a Single Keypress	21
	1.4 menu1 — Print a Menu and Execute a Selected Command	22
	1.5 arg — Prompt for Required Argument If None Supplied	23
	1.6 die — Print Error Message and Exit with Error Status	24
	1.7 show_date — Display Date in D[D] MMM YYYY Format	25
	1.8 date_vars — Set Date and Time Variables	26
	1.9 is_num — Is This a Positive Integer?	28
	1.10 abbrev_num — Abbreviate Large Numbers	29
	1.11 commas — Add Thousands Separators to a Number	30
	1.12 pr1 — Print Arguments, One to a Line	32
	1.13 checkdirs — Check for Directories; Create If Necessary	33
	1.14 checkfiles — Check That a Directory Contains Certain Files	33
	1.15 zpad — Pad a Number with Leading Zeroes	34
	1.16 cleanup — Remove Temporary Files and Reset Terminal on Exit	35
The Unix Utilities		36
Summary		40

CHAPTER 2
Playing with Files: Viewing, Manipulating, and Editing Text Files

End of the Line for OS Differences: Converting Text Files		42
	2.1 dos2unix — Convert Windows Text Files to Unix	42
	2.2 unix2dos — Convert a Unix File to Windows	43
	2.3 mac2unix — Convert Macintosh Files to Unix	44
	2.4 unix2mac — Convert Unix Files to Mac Format	44
	2.5 dos2mac — Convert Windows Files to Macintosh	45
	2.6 mac2dos — Convert Macintosh Files to Windows	45
Displaying Files		46
	2.7 prn — Print File with Line Numbers	46
	2.8 prw — Print One Word per Line	49
	2.9 wbl — Sort Words by Length	51
Formatting File Facts		52
	2.10 finfo — File Information Display	52
	2.11 wfreq — Word Frequency	56
	2.12 lfreq — Letter Frequency	56
	2.13 fed — A Simple Batch File Editor	58
Summary		60

CHAPTER 3
String Briefs

Character Actions: The char-funcs Library		61
	3.1 chr — Convert a Decimal Number to an ASCII Character	62
	3.2 asc — Convert a Character to Its Decimal Equivalent	63
	3.3 nxt — Get the Next Character in ASCII Sequence	64
	3.4 upr — Convert Character(s) to Uppercase	65
	3.5 lwr — Convert Character(s) to Lowercase	67
String Cleaning: The string-funcs Library		69
	3.6 sub — Replace First Occurrence of a Pattern	69
	3.7 gsub — Globally Replace a Pattern in a String	70
	3.8 repeat — Build a String of a Specified Length	72
	3.9 index, rindex — Find Position of One String Within Another	74
	3.10 substr — Extract a Portion of a String	76
	3.11 insert_str — Place One String Inside Another	78
Summary		79

CHAPTER 4
What's in a Word?

Finding and Massaging Word Lists		82
wf-funcs: WordFinder Function Library		82
	4.1 write_config — Write User's Information to the Configuration File	82
	4.2 do_config — Check For and Source Default Configuration File	83
	4.3 set_sysdict — Select the Dictionary Directory	84
	4.4 mkwsig — Sort Letters in a Word	85
	4.5 wf-clean — Remove Carriage Returns and Accents	86
	4.6 wf-compounds — Squish Compound Words and Save with Lengths	87
	4.7 wf-setup — Prepare Word and Anagram Lists	89
Playing with Matches		91
	4.8 wf — Find Words That Match a Pattern	91
	4.9 wfb — Find Words That Begin with a Given Pattern	93
	4.10 wfe — Find Words That End with a Given Pattern	94
	4.11 wfc — Find Words That Contain a Given Pattern	95
	4.12 wfit — Find Words That Fit Together in a Grid	97
	4.13 anagram — Find Words That Are Anagrams of a Given Word	100
	4.14 aplus — Find Anagrams of a Word with a Letter Added	101
	4.15 aminus — Remove Each Letter in Turn and Anagram What's Left	103
Summary		104

CHAPTER 5
Scripting by Numbers

The math-funcs Library		105
	5.1 fpmul — Multiply Decimal Fractions	106
	5.2 int — Return the Integer Portion of a Decimal Fraction	110
	5.3 round — Round the Argument to the Nearest Integer	110
	5.4 pow — Raise a Number to Any Given Power	111
	5.5 square — Raise a Number to the Second Power	113
	5.6 cube — Raise a Number to the Third Power	113
	5.7 calc — A Simple Command-Line Calculator	114
Adding and Averaging		115
	5.8 total — Add a List of Numbers	115
	5.9 mean — Find the Arithmetic Mean of a List of Numbers	116
	5.10 median — Find the Median of a List of Numbers	118
	5.11 mode — Find the Number That Appears Most in a List	119
	5.12 range — Find the Range of a Set of Numbers	120
	5.13 stdev — Finding the Standard Deviation	122
Converting Between Unit Systems		123
	5.14 conversion-funcs — Converting Metric Units	124
	5.15 conversion — A Menu System for Metric Conversion	131
Summary		135

CHAPTER 6
Loose Names Sink Scripts: Bringing Sanity to Filenames

What's in a Name?		137
	POSIX Portable Filenames	138
	OK Filenames	138
Functioning Filenames: The filename-funcs Library		139
	6.1 basename — Extract the Last Element of a Pathname	139
	6.2 dirname — Return All but the Last Element of a Pathname	141
	6.3 is_pfname — Check for POSIX Portable Filename	142
	6.4 is_OKfname — Check Whether a Filename Is Acceptable	143
	6.5 pfname — Convert Nonportable Characters in Filename	144
	6.6 OKfname — Make a Filename Acceptable	145
	6.7 is_whitespc — Does the Filename Contain Whitespace Characters?	146
	6.8 whitespc — Fix Filenames Containing Whitespace Characters	146
	6.9 is_dir — Is This a Directory I See Before Me?	147
	6.10 nodoublechar — Remove Duplicate Characters from a String	148
	6.11 new_filename — Change Filename to Desired Character Set	149
	6.12 fix_pwd — Fix All the Filenames in the Current Directory	150
	6.13 fixfname — Convert Filenames to Sensible Names	152
Summary		154

CHAPTER 7
Treading a Righteous PATH

The path-funcs Library		157
	7.1 path — Display a User's PATH	157
	7.2 unslash — Remove Extraneous Slashes	158
	7.3 checkpath — Clean Up the PATH Variable	159
	7.4 addpath — Add Directories to the PATH Variable	161
	7.5 rmpath — Remove One or More Directories from $PATH	163
Summary		164

CHAPTER 8
The Dating Game (Sample chapter: PDF or HTML)

The date-funcs Library		166
	8.1 split_date — Divide a Date into Day, Month, and Year	166
	8.2 is_leap_year — Is There an Extra Day This Year?	167
	8.3 days_in_month — How Many Days Hath September?	169
	8.4 date2julian — Calculate the Julian Day Number	170
	8.5 julian2date — Convert Julian Back to Year, Month, and Day	170
	8.6 dateshift — Add or Subtract a Number of Days	172
	8.7 diffdate — Find the Number of Days Between Two Dates	174
	8.8 day_of_week — Find the Day of the Week for Any Date	175
	8.9 display_date — Show a Date in Text Format	176
	8.10 parse_date — Decipher Various Forms of Date String	179
	8.11 valid_date — Where Was I on November 31st?	183
Summary		184

CHAPTER 9
Good Housekeeping: Monitoring and Tidying Up File Systems

	9.1 dfcmp — Notify User of Major Changes in Disk Usage	185
	9.2 symfix — Remove Broken Symbolic Links	189
	9.3 sym2file — Converts Symbolic Links to Regular Files	191
	9.4 zrm — Remove Empty Files	193
	9.5 undup — Remove Duplicate Files	194
	9.6 lsr — List the Most Recent (or Oldest) Files in a Directory	196
Summary		198

CHAPTER 10
POP Goes the E-Mail

	10.1 popcheck — Display Number of Messages in POP3 Mailbox	199
	10.2 pop3-funcs — Functions for Managing a POP3 Mailbox	200
	10.3 pop3list — Examine a POP3 Mailbox	215
	10.4 pop3filter — Delete and Retrieve Messages from a POP3 Server	222
	10.5 pflog — Print a Summary of a Day's Activity	224
	10.6 viewlog — Continuously View the pop3filter Log File	225
Summary		227

CHAPTER 11
PostScript: More Than an Afterthought

	11.1 ps-labels — Produce Mailing Labels from a List	231
	11.2 ps-envelopes — Print Envelopes from List	237
	11.3 ps-grid — Convert a Crossword Grid to PostScript	242
Summary		249

CHAPTER 12
Screenplay: The screen-funcs Library

	12.1 screen-vars — Variables for Screen Manipulation	252
	12.2 set_attr — Set Screen-Printing Attributes	254
	12.3 set_fg, set_bg, set_fgbg — Set Colors for Printing to the Screen	255
	12.4 cls — Clear the Screen	256
	12.5 printat — Position Cursor by Row and Column	257
	12.6 put_block_at — Print Lines in a Column Anywhere on the Screen	260
	12.7 get_size — Set LINES and COLUMNS Variables	262
	12.8 max_length — Find the Length of the Longest Argument	263
	12.9 print_block_at — Print a Block of Lines Anywhere on the Screen	264
	12.10 vbar, hbar — Print a Vertical or Horizontal Bar	265
	12.11 center — Center a String on N Columns	267
	12.12 flush_right — Align String with the Right Margin	268
	12.13 ruler — Draw a Ruler Across the Width and Height of the Window	269
	12.14 box_block, box_block_at — Print Text Surrounded by a Box	271
	12.15 clear_area, clear_area_at — Clear an Area of the Screen	272
	12.16 box_area, box_area_at — Draw a Box Around an Area	273
	12.17 screen-demo — Saving and Redisplaying Areas of the Screen	275
Summary		277

CHAPTER 13
Backing Up the Drive

	13.1 bu — Backup Directories	280
	13.2 bin-pack — Fill Fixed-Capacity Directories with Files	285
	13.3 unpack — Extract All Files from One or More Archives	290
Summary		292

CHAPTER 14
Aging, Archiving, and Deleting Files

	14.1 date-file — Add a Datestamp to a Filename	293
	14.2 rmold — Remove Old Files	297
	14.3 keepnewest — Remove All but the Newest or Oldest Files	300
Summary		302

CHAPTER 15
Covering All Your Databases

	15.1 lookup — Find the Corresponding Value for a Key	304
	shdb-funcs: Shell Database Function Library	306
	15.2 load_db — Import Database into Shell Array	306
	15.3 split_record — Split a Record into Fields	307
	15.4 csv_split — Extract Fields from CSV Records	309
	15.5 put_record — Assemble a Database Record from an Array	310
	15.6 put_csv — Assemble Fields into a CSV Record	311
	15.7 db-demo — View and Edit a Password File	312
PhoneBase: A Simple Phone Number Database		317
	15.8 ph — Look Up a Phone Number	318
	15.9 phadd — Add an Entry to PhoneBase	319
	15.10 phdel — Delete an Entry from PhoneBase	320
	15.11 phx — Show ph Search Results in an X Window	320
Summary		321

CHAPTER 16
Home on the Web

Playing with Hypertext: The html-funcs Library		323
	16.1 get_element — Extract the First Occurrence of an Element	323
	16.2 split_tags — Put Each Tag on Its Own Line	326
	16.3 html-title — Get the Title from an HTML File	327
HTML on the Fly: The cgi-funcs Library		329
	16.4 x2d2 — Convert a Two-Digit Hexadecimal Number to Decimal	329
	16.5 dehex — Convert Hex Strings (%XX) to Characters	330
	16.6 filedate — Find and Format the Modification Date of a File	332
Creating HTML Files		333
	16.7 mk-htmlindex — Create an HTML Index	333
	16.8 pretext — Create a Wrapper Around a Text File	341
	16.9 text2html — Convert a Text File to HTML	342
	16.10 demo.cgi — A CGI Script	343
Summary		344

CHAPTER 17
Taking Care of Business

	17.1 prcalc — A Printing Calculator	345
	17.2 gle — Keeping Records Without a Shoebox	350
Summary		360

CHAPTER 18
Random Acts of Scripting

The rand-funcs Library		361
	18.1 random — Return One or More Random Integers in a Given Range	362
	18.2 toss — Simulate Tossing a Coin	365
	18.3 randstr — Select a String at Random	366
A Random Sampling of Scripts		367
	18.4 rand-date — Generate Random Dates in ISO Format	367
	18.5 randsort — Print Lines in Random Order	369
	18.6 randomword — Generate Random Words According to Format Specifications	370
	18.7 dice — Roll a Set of Dice	375
	18.8 throw — Throw a Pair of Dice	378
Summary		379

CHAPTER 19
A Smorgasbord of Scripts

	19.1 topntail — Remove Top and Bottom Lines from a File	381
	19.2 flocate — Locate Files by Filename Alone	383
	19.3 sus — Display a POSIX Man Page	384
	19.4 cwbw — Count Words Beginning With	385
	19.5 cci — Configure, Compile, and Install from Tarball	386
	19.6 ipaddr — Find a Computer's Network Address	387
	19.7 ipaddr.cgi — Print the Remote Address of an HTTP Connection	388
	19.8 iprev — Reverse the Order of Digits in an IP Address	389
	19.9 intersperse — Insert a String Between Doubled Characters	389
	19.10 ll — Use a Pager for a Directory Listing Only If Necessary	390
	19.11 name-split — Divide a Person's Full Name into First, Last, and Middle Names	391
	19.12 rot13 — Encode or Decode Text	392
	19.13 showfstab — Show Information from /etc/fstab	393
	19.14 unique — Remove All Duplicate Lines from a File	395
Summary		395

CHAPTER 20
Script Development Management

	20.1 script-setup — Prepare the Scripting Environment	397
	20.2 cpsh — Install Script and Make Backup Copy	402
	20.3 shgrep — Search Scripts for String or Regular Expression	403
	20.4 shcat — Display a Shell Script	404
Summary		405

APPENDIX
Internet Scripting Resources

Introductions to Shell Scripting	407
Intermediate and Advanced Scripting	408
Collections of Scripts	408
Home Pages for Shells	408
Regular Expressions, sed, and awk	408
Miscellaneous Pages	409
History of the Shell	409

Shell Scripting Recipes: A Problem-Solution Approach

by Chris F.A. Johnson

Apress, 2005

Contents

CHAPTER 1:
The POSIX Shell and Command-Line Utilities

CHAPTER 2
Playing with Files: Viewing, Manipulating, and Editing Text Files

CHAPTER 3
String Briefs

CHAPTER 4
What's in a Word?

CHAPTER 5
Scripting by Numbers

CHAPTER 6
Loose Names Sink Scripts: Bringing Sanity to Filenames

CHAPTER 7
Treading a Righteous PATH

CHAPTER 8
The Dating Game (Sample chapter: PDF or HTML)

CHAPTER 9
Good Housekeeping: Monitoring and Tidying Up File Systems

CHAPTER 10
POP Goes the E-Mail

CHAPTER 11
PostScript: More Than an Afterthought

CHAPTER 12
Screenplay: The screen-funcs Library

CHAPTER 13
Backing Up the Drive

CHAPTER 14
Aging, Archiving, and Deleting Files

CHAPTER 15
Covering All Your Databases

CHAPTER 16
Home on the Web

CHAPTER 17
Taking Care of Business

CHAPTER 18
Random Acts of Scripting

CHAPTER 19
A Smorgasbord of Scripts

CHAPTER 20
Script Development Management

APPENDIX
Internet Scripting Resources

INDEX

Shell Scripting Recipes: A Problem-Solution Approach

by Chris F.A. Johnson

Apress, 2005

Contents

CHAPTER 1: The POSIX Shell and Command-Line Utilities

CHAPTER 2 Playing with Files: Viewing, Manipulating, and Editing Text Files

CHAPTER 3 String Briefs

CHAPTER 4 What's in a Word?

CHAPTER 5 Scripting by Numbers

CHAPTER 6 Loose Names Sink Scripts: Bringing Sanity to Filenames

CHAPTER 7 Treading a Righteous PATH

CHAPTER 8 The Dating Game (Sample chapter: PDF or HTML)

CHAPTER 9 Good Housekeeping: Monitoring and Tidying Up File Systems

CHAPTER 10 POP Goes the E-Mail

CHAPTER 11 PostScript: More Than an Afterthought

CHAPTER 12 Screenplay: The screen-funcs Library

CHAPTER 13 Backing Up the Drive

CHAPTER 14 Aging, Archiving, and Deleting Files

CHAPTER 15 Covering All Your Databases

CHAPTER 16 Home on the Web

CHAPTER 17 Taking Care of Business

CHAPTER 18 Random Acts of Scripting

CHAPTER 19 A Smorgasbord of Scripts

CHAPTER 20 Script Development Management

APPENDIX Internet Scripting Resources

INDEX

CHAPTER 1:
The POSIX Shell and Command-Line Utilities

CHAPTER 2
Playing with Files: Viewing, Manipulating, and Editing Text Files

CHAPTER 3
String Briefs

CHAPTER 4
What's in a Word?

CHAPTER 5
Scripting by Numbers

CHAPTER 6
Loose Names Sink Scripts: Bringing Sanity to Filenames

CHAPTER 7
Treading a Righteous PATH

CHAPTER 8
The Dating Game (Sample chapter: PDF or HTML)

CHAPTER 9
Good Housekeeping: Monitoring and Tidying Up File Systems

CHAPTER 10
POP Goes the E-Mail

CHAPTER 11
PostScript: More Than an Afterthought

CHAPTER 12
Screenplay: The screen-funcs Library

CHAPTER 13
Backing Up the Drive

CHAPTER 14
Aging, Archiving, and Deleting Files

CHAPTER 15
Covering All Your Databases

CHAPTER 16
Home on the Web

CHAPTER 17
Taking Care of Business

CHAPTER 18
Random Acts of Scripting

CHAPTER 19
A Smorgasbord of Scripts

CHAPTER 20
Script Development Management

APPENDIX
Internet Scripting Resources