End Users Benefit from Batch Processing Too

Almost every ERP implementation brings with it the need to schedule and run operations in batch mode. A typical example is the regular reservation of stock or invoicing of completed sales. But batch processing is also useful for end users who need to run computationally demanding operations just once – for instance, generating a report or running the period-end close.

Microsoft Dynamics AX has supported batch processing since its earliest versions. Each new release has brought a number of important changes – while Axapta 3.0 processed batch jobs through a dedicated client, AX 2009 introduced the concept of server-side processing. AX 2012 then delivered another significant performance boost by moving batch processing into the .NET framework world.

At Blue Dynamic we have extensive experience running batch jobs across different environments for different customers. In this article we want to share, in the form of five tips, some of the valuable insights and experience we picked up along the way. Throughout the text we’ll refer to the Blue Dynamic Batch package – a set of application customizations we developed while solving the problems that come with running batches in production.

Tip 0: A Bit of Theory Never Hurt Anyone

Every batch job consists of a header (the Batch job) and one or more lines.

Okay, that’s not really a tip yet. But before we go further, it helps to get the terminology straight. Every batch job consists of a header (a Batch job) and one or more lines (Batch tasks). We’ll stick to the English terms throughout this article, since they describe the concept far more precisely than any literal translation could.
The header is called a Batch job, and each line is a Batch task – keep that distinction in mind as you read on.

What matters is understanding what Batch jobs and Batch tasks are for, and how Microsoft Dynamics AX treats them. Batch jobs define the behavior of the batch as a whole – its current status, the processing schedule including recurrence, and the date and time of the next run. Batch tasks, by contrast, represent the individual steps within the batch – they define which actions run and with what parameters (this is where, for example, input filters are set). Dependencies can then be defined between individual steps to set the correct processing order, or to allow parallel processing.

Tip 1: Good Naming as a Foundation

If you’re running just five batches in your Dynamics AX solution, their names hardly matter – it’s easy to keep an overview of what’s running in the system. Over time, though, things tend to get more complicated. Some batches should only run during the day, others only at night. Some batches you want to run every few minutes in a “process changes only” mode, and then once a day, just to be safe, in a “process everything” mode (a good example is transferring retail master data from headquarters to individual stores). Some batches take so long that you decide to ease the load on the system and process only a slice of the data at a time (for instance, when hunting for duplicates in a customer list a million records long, you might start with those whose surname begins with A). Sooner or later you’ll find that the list of active batches stretches across several screens.

So it makes sense to bring order to the chaos right from the start. We’ve found it works well to name batches using different prefixes, which can even have several levels. As an example, here’s a two-level naming scheme based on:

• Whether it runs during the day (D) or at night (N)
• Which area (module) it relates to – e.g. logistics (LOG), finance (FIN), administration (ADM), etc.

A batch might then be named, for example, N-FIN-Update balances for dimension set Ucet.

The benefits of this approach are obvious – sorting by name groups thematically related batches together. What’s more, it’s easy to tell batches that are properly scheduled and configured apart from ones a regular user launched ad hoc. Last but not least, the batch name itself tells you something about its content and its run schedule.

Tip 2: The Administrator Is on Vacation

Let’s say we have a Batch job with thirty Batch tasks, each with some complex filter set up, and with intricate dependencies defined between them. After a while, a request comes in to add one more Batch task. Petr, the administrator who defined the original Batch job, is on vacation, so Pavel gets asked to handle it. He gradually discovers that to finish the task he needs to be logged in as an administrator, and that the header status has to be set to Withhold. In the end, though, he finds he can’t complete the task at all – when creating a new Batch tasks record, he gets the thoroughly unhelpful error message “Cannot create a record in Batch transactions (Batch). The corresponding AOS validation failed.” He’d hit the exact same wall – or rather, get nowhere at all – if he only wanted to change the dependencies between existing Batch tasks. So now what?

• the batch owner, i.e. whoever created the original batch definition
• a system administrator
• neither of the above

The first and third cases are the two extremes. The batch owner can do whatever they like with their own batch. An ordinary user without administrator rights, on the other hand, can’t do anything with someone else’s batch. The interesting case is the second one – an administrator who isn’t the batch’s author. They can change settings related to how the batch actually runs – pausing it, say, or scheduling it differently. What they can’t do is change the batch’s substance – they can’t create new Batch tasks, change the parameters of existing Batch tasks (such as input filters), or change the action itself (i.e. the class name). The reason for these restrictions is to enable an audit trail of changes – Pavel simply can’t be allowed to change the batch’s configuration in a way that leads to unwanted behavior, all under Petr’s name. So Pavel now has two options: track down Petr, or build his own Batch job definition from scratch.

So that we wouldn’t undermine the audit principles while still letting administrator Petr go on vacation in peace, we built new functionality into the Blue Dynamic Batch package. Instead of relying on the “Created by” field, we use a new field for audit purposes – “Batch Job Owner.” Its value matches the “Created by” field when the batch is first created, but it can be changed using the “Take Ownership” function.

Once he’s taken ownership, Pavel can make the required changes – under his own name, of course. The “Batch Job Owner” value also carries through into the batch processing history, so it’s easy to see exactly when ownership changed.

Tip 3: What Batch Groups Are – and Aren’t – Good For

Let’s start with what batch groups aren’t good for – organizing the work of individual Batch jobs. The reason is simple: a batch group is a flag on the Batch task record, not on the Batch job. If you need to organize Batch jobs better, what you need is a proper naming convention instead – see Tip 1.

So what are batch groups good for? Their main purpose is letting you process different Batch tasks on different AOS servers. In installations where a single AOS server handles all batch processing, one batch group is enough (AX automatically creates an unnamed group for this purpose right after installation). But there are situations where splitting tasks across AOS servers comes in handy – for example, computationally demanding batches, or batches that need some special resource available only on certain AOS servers.

At Blue Dynamic we thought about how batch groups could serve us even better. One problem we often run into when operating batches is two or more concurrently running batches interfering with each other. For example, the retail batches “Post inventory” and “Post statement” sometimes collide. “Post inventory” is scheduled to run every 10 minutes, while “Post statement” needs to run once a day, after the registers close. Yet “Post inventory” keeps running “empty” after the registers close – logically, no new transactions are coming in at that point. If we could set “Post inventory” to run only until a certain hour and then not again until the next morning, we could easily prevent this kind of collision – and save ourselves the unpleasant task of figuring out how the two batches interfere with each other and rewriting them to account for running in parallel.

In standard AX, the problem above can only be solved by using multiple application servers. We create two batch groups – DAY and NIGHT. We assign “Post inventory” to the DAY group and “Post statement” to the NIGHT group. We configure the first AOS to run from 8:00 AM to 10:00 PM and process only the DAY batch group. The second AOS is then configured to run from 10:15 PM to 7:45 AM (leaving a 15-minute buffer for longer-running batches to finish) and process only the NIGHT batch group.

But what if using an additional AOS server isn’t an option? We solve the problem by adding the ability to set a schedule at the batch group level.

With this customization, part of the Blue Dynamic Batch package, a single AOS server is enough, and the conflict never happens.

Tip 4: When Batches Start “Multiplying on Their Own” – and the “Nameless” Batch Group

In this tip we return to our “Post statement” batch from the previous one. The customer runs a chain of stores and uses the AX Retail module. During the day they sell and record revenue in the POS. In the evening the registers are closed out and the data is transferred to AX. AX then generates a statement that needs to be posted. Since we want everything to run automatically, the posting task is scheduled as a nightly job. Right after scheduling, we have a Batch job called “Post statement,” with a single Batch task of the same name, assigned to the NIGHT batch group. But once this batch starts processing, new Batch tasks suddenly appear that weren’t there before. What are they?

Some batch jobs are written so that they split the overall work into smaller chunks, which are then processed in parallel across multiple Batch tasks. Dependencies can even be defined between these dynamic Batch tasks – for instance, a final Batch task can be added at the end to handle cleanup work that should only run once all the other Batch tasks have finished.
There’s one catch to this otherwise nice piece of functionality. Dynamically created Batch tasks don’t necessarily inherit the same batch group as the original Batch task – the exact behavior is determined by the developer when building the batch. What this can lead to is obvious – if the dynamically created Batch tasks end up assigned to a group whose processing is paused, they’ll never run, and the batch as a whole will stay “stuck” in the In Progress state.

The following screenshot illustrates the situation – the last three Batch tasks are assigned to the nameless batch group. If processing for the nameless batch group is paused, the statement will never finish posting.

This is actually a typical case – dynamically created Batch tasks will most likely end up with the same batch group as the main Batch task, or with none at all. Since you can’t always control the internal logic behind how dynamic Batch tasks get created, it’s a good idea to leave processing for the nameless batch group unrestricted. That matters for another reason too – one-off batches launched ad hoc by end users very often end up in the nameless batch group, since less experienced users frequently forget to specify one.

Tip 5: How to Sleep Soundly

Batches that run overnight often need to finish successfully for the company to function properly the next day – for example, the master plan generating purchase order proposals and store replenishment, or a batch preparing outbound picking work at the central warehouse.

These are generally fairly complex tasks, with plenty of pitfalls that can keep them from completing successfully every time. What’s more, these batches often run for a long time, so there may not be enough time to fix things the next morning. That’s why it’s so important to respond quickly to any problems related to running these batches.

These are, on the whole, fairly complex tasks that may not complete successfully. It’s important to respond quickly to any problems related to these batches.

Standard AX offers the use of alerts for this purpose. For every batch, you can have AX generate an alert whenever one of these events occurs:

• the batch completes successfully
• the batch ends with an error
• the batch is canceled

An administrator can view alerts within the AX application, or have them emailed to a predefined address.

Alerts are undoubtedly useful, but on their own they’re not enough to guarantee a good night’s sleep. The problem arises when a batch never starts processing at all, or when its processing takes an unusually long time. A common cause of extended run times is a negative interaction with another computationally demanding job.

AX is also often not the only system in a company that contributes to a successful next day. Companies frequently run a separate CRM, an e-shop, POS systems, and so on. In such cases keeping an eye on the whole ecosystem becomes more demanding, and it makes sense to start monitoring the infrastructure from one central place. Alerts then start to look like a piecemeal solution, and it’s better to hook AX batches into central monitoring instead.

Blue Dynamic successfully uses the Check MK system for batch monitoring. We’ve developed scripts that let us track critical batches – whether processing started at the scheduled time, what progress looks like, and whether they’ll finish within the expected timeframe – and respond to critical situations by sending an SMS to the support team’s phone. We’re currently working on generalizing this and making it possible to configure monitoring parameters directly within AX.

Our Batch Jobs Manager Solution

Our Batch Jobs Manager solution for Dynamics AX is available to purchase as an Add-on for Dynamics AX on the marketplace for Dynamics AX add-ons and applications – Addons.Blue.

Ondřej Liberda works as a Dynamics AX Developer/Architect at Blue Dynamic and has experience managing the rollout of development batches.