Every Solution Starts With “FOR /F”

Our team has recently grown and as part of that process I’ve been showing how I’ve tended to do some of the sysadmin type stuff, such as deployment, managing log files[*], checking system health etc. For a brief period it felt as if the answer to every question started with “FOR /F” - the Swiss Army Knife of Windows batch file programming. One week it also ended up being the answer to two questions via Twitter!

Given that I work on Distributed Systems is that such a shock? After all, whatever I want to do, I’m likely to want to do the same thing to every server in the farm and that implies use of some looping construct either to generate a batch file with the same command multiple times:-

@echo off
psexec \\SERVER-1 cmd.exe /c . . .
psexec \\SERVER-2 cmd.exe /c . . .
psexec \\SERVER-3 cmd.exe /c . . .
. . .

Or just one command executed multiple times, controlled by a simple list:-

for /f %h in(machines.txt) do @psexec \\%h cmd.exe /c . . .

This latter variant requires a text file (called machines.txt) with one server hostname per line like so:-

SERVER-1
SERVER-2
SERVER-3

Of course rather than crafting one on-the-fly every time I have a number of files, one called DEV-machines.txt, another called UAT-machines.txt and finally a PROD-machines.txt which are pre-configured.

Deleting Folders

Whilst PSEXEC pretty much makes an appearance in every one-liner I write, usually to invoke a command on a remote host, the simpler task of cleaning up folders in a non-sequential way doesn’t feature it for once. Sometimes you’d like a little more progress than a straight “RMDIR /S /Q” will give you (i.e. none) and so a sprinkling of FOR /D allows you to iterate a bunch of folders and output their name before recursively deleting them:-

for /d %d in (folder\2010_??_??) do @echo %d & rmdir /s /q %d

A little extra parallelism can then also easily be achieved by firing up a few more command prompts with START (which will default to the CWD of the parent) and splitting up the work:-

Prompt-1> for /d %d in (folder\2010_01_*) do @echo %d & rmdir /s /q %d

Prompt-2> for /d %d in (folder\2010_02_*) do @echo %d & rmdir /s /q %d

For those that have successfully avoided batch file programming the “&” separates commands. A “&&” ensures the second is only executed if the first succeeded, whilst a “|” does the opposite and executes the second only if the first fails.

Reading Variables

Probably one of the most unintuitive aspects of Windows batch file programming is creating variables from data in a file. For example our current Heath Robinson overnight batch scheduler uses a combination of the Windows Task Scheduler and batch files to sequence it[#]. The “batch date” is stored in a text file (BatchDate.txt) like so:-

2011-01-01

To read this into a variable so that it can then be passed onto the child scripts and processes requires the following:-

for /f %v in (\\dfs\share\BatchDate.txt) do @set BatchDate=%v

Who would of thought you need a loop to read a single value! If you want to store it as a “key=value” pair instead you need a little more magic to parse the line:-

for /f "delims== tokens=1,2" %i in (c:\temp\test.txt) do @set %i=%j

There are various options that you can pass to FOR to control the parsing, such as “eol=;” if you want to allow comments in your input file like so:-

D-SERVER-1
D-SERVER-2
; NB: Server being repaired
; D-SERVER-3

PowerShell Integration

PowerShell is clearly a better tool for doing any serious scripting under Windows but it suffers one major drawback when you’re using it as we are to sequence a set of operations - PowerShell can’t modify the variables of the calling process. If a PowerShell script forms the root process this isn’t a problem, but when old school batch files are leading the way it’s harder work. If it wasn’t for the annoying problems PowerShell has with doing simple stdout redirection[$] it would be a no brainer to switch.

Anyway, going back to our previous example of reading in variables, you can invoke a PowerShell one-liner and read the output into a variable, as I showed in this StackOverflow answer to get yesterday’s date:-

for /f "usebackq" %i in (`PowerShell ^(get-date^).adddays^(-1^).tostring^('yyyy-MM-dd'^)`) do set Yesterday=%i

However there is a fair bit of ugliness in this technique because you need to escape the parenthesis with the ^ (hat) character and to allow the date format string to be passed through correctly (it either needs to be in single or double quotes) you need to enclose the command with the ` (backtick ) character instead of the usual ‘ (apostrophe).

You Can Remember This Stuff?

Even after all this time I still can’t remember half the switches and options that you can use with “FOR” so it’s lucky that I do remember this command to bring up the manual:-

help for

Bizarrely enough it also lists the set of modifiers you can use on the loop variables to munge paths, such as getting the parent folder, file extension etc.

 

[*] This should of course be automated, but with limited resources it’s been well down the priority list.

[#] See [*] for the reasons why this came about :-)

[$] I already have a post queued up on this as I think it’s one of the most annoying things you discover when trying to switch from using batch files to PowerShell scripts.