What Is Batch Processing?
Batch processing simply refers to the processing of transactions in a group or batch. This is different from real-time transaction processing, which happens one transaction at a time. Consider the example of an ecommerce purchase—behind the customer’s real-time transaction is a series of ancillary tasks associated with orders being placed, which might not happen right away.
These tasks may include inventory replenishment that may not occur immediately, especially for products that are purchased early in the day and/or sell by the tens or hundreds of thousands daily. Other examples include order boxing and shipping, which benefit from assembling orders. There are numerous reasons—often related to cost and warehouse efficiencies—that these processes occur in bulk as part of batch processing jobs once orders reach a certain threshold.
Batch processing is the key to many of these supporting back-end processes. But at precisely the time the proliferation of modern apps is placing greater demands on batch processing, the industry is also facing a retiring workforce of batch operators. It is imperative for batch processing to modernize alongside the real-time software products that are emerging from DevOps. The characteristics of batch modernization include the following considerations:
Automation: Manual system operation of batch processing must evolve to automation via policy-based controls. (Organizations can assess their current operation using the Mainframe Batch Maturity Model at https://resources.compuware.com/batch-maturity_level_lp). SLAs should dictate job initiation and prioritize CPU access—in other words, job queues should be automatically organized such that the most urgent job is the next job to initiate. Prioritized access to datasets and readiness of job resource requirements available for processing should also be automatic, all helping to prevent unnecessary time delays.
Cost Awareness: The costs associated with larger and more frequent batch workloads can send monthly license charges (MLC) skyrocketing if organizations aren’t careful. MLC costs are based on Peak 4-Hour Rolling Average (4HRA)—the highest utilization determined from the sum of the Peak 4HRAs for all logical partitions (LPARs) in which a particular product runs. The key to reining in these costs is to execute jobs in a cost-aware manner that deliberately reduces the Peak 4HRA. So, in addition to factoring which jobs are the most time-sensitive, job execution must be automatically manipulated in a manner that exploits savings opportunities while sustaining maximum throughput.
Continuous Improvement: Once job execution is automated and operational costs are minimized without detriment to SLAs, operators have an opportunity to focus on continuous performance management and improvement. Urgent and critical jobs should be monitored and compared with historical performance levels. Any deviations from normal wall clock time or excessive resource consumption should stimulate automation to capture symptom data and ticket the incident. This creates a proactive opportunity for operators to make tweaks that help optimize performance of the system while also avoiding errors and failed jobs that could put the business at risk.
Customer Service-Focused: When batch processing is automated and optimized, from both a cost and job performance perspective, an improved level of customer service emerges as the natural byproduct. Per the ecommerce example, jobs are orchestrated in a manner that leads to satisfied customers through products that are readily available and delivered on time and cost-effectively.
The Evolution of Batch Processing
Customer demand to engage with services and products through mobile apps and the web, whenever and wherever they wish, is dramatically accelerating the speed of software development and delivery. This increased activity on the front end ultimately triggers more frequent, more compute-intensive, and more critical and complex mainframe batch processing work on the back end, including inventory management, warehouse and shipping management,
customer billing, and more.
A batch approach is often optimal for these scenarios, which means batch can no longer be simply dismissed as old-school overnight processing that can be ignored or deprioritized. As mainframe-based batch processing continues to grow in volume and complexity, it must evolve. Organizations that fail to recognize this as part and parcel of their DevOps approaches are ultimately putting at jeopardy their ability to compete.