Guest Author - Julie L Baumler
Batch processing consists of scheduled tasks, usually called jobs, where you submit you program and data and receive any results or program success / failure notifications at a later time. Once upon a time in the early days of computing, everything was done via batch processing. Jobs would scheduled based on the availability of computer time or other needed resources as well as when the results were required. For instance, the payroll job has to be finished in time to distribute the checks on payday. Things had to be done that way because early computers had very limited memory and no local storage. Programs and data, including operating systems were stored separately on things like stacks of punch cards, reel to reel tapes and later disk packs. If you needed to run, say a FORTRAN program, first the operators would load the FORTRAN compiler and then all the FORTRAN programs would be run, one at a time. An output or errors would be returned to the job owner. If the program failed, it would have to be fixed and resubmitted
Then came timesharing and interactive computers which allowed the computer to do more than one thing at a time and give feedback right away, respectively. This allowed first interactive programming (and particularly the added efficiency of interactive debugging) and then interactive programs. Batch processing became an option, not a requirement.
Fredrick Brooks said in 1975 that "Interactive systems will never displace batch systems for some applications." He reiterated that this was "still true" in 1995. (Brooks, p 245) Today (and this was already true in 1995), interactive applications are so ubiquitous that this seems unrealistic at first glance. However, when you start thinking about things like payroll and billing, those will always be cyclical for most companies. For instance, if you are an affiliate of, say, Amazon.com, they don't pay you every time you refer a sale, rather at a set time each month they determine who has reached a set minimum amount and issue payments to those people. Legal and regulatory accounting requirements call for the books to be closed and certain reports created at set intervals. Holiday card lists are created once a year. Backups by their nature are batch processes. These things are unlikely to be done interactively and if they are, they are likely a boring requirement for the poor employee who has to handle them.
What has changed since the mid-70's and even mid-90's is the tools and expertise available for managing batch jobs. In the mainframe world there are specialists in scheduling and specialized tools to optimize batch processing, manage the interaction of dependent and interacting batches and generally make sure that batches run correctly and efficiently. Today, even in companies that use mainframes, many of those experts have retired and not been replaced. Most non-mainframe shops don't have anyone with those skills. Those skills have become rare and poorly understood. Batch processing is mostly ignored. This is a shame because it causes more resource contention between scheduled jobs and scheduled jobs and interactive processing. Additionally, jobs that are good candidates for batch processing are often not recognized as such and work that could be done in advance is not. Another problem is that because batch processing is not recognized as a key business need, applications that would be good candidates for batch processing are written to require human interaction. While batch scheduling is a career path for very few computer professionals today, all of us can be more effective by thinking about how it can be used in our work and how our work might be used in batch processing.
Brooks, Frederick P. Jr. The Mythical Man-Month: Essays on Software Engineering. Anniversary Edition with Four New Chapters. (Reading, MA: Addison-Wesley, 1995.)