ML Fails in Supply Chain Planning: The Case for AD3’s Optimal Machine Learning

The performance of supply chains has become a major challenge in today’s turbulent environment. Disruptions, delays and shortages are now common in many industries. Managers looking to alleviate the crisis are searching for answers that will enable their companies to develop and implement strategies to achieve both agility and resilience. We believe that the key to achieving these goals is through data driven decisions.

Some of you are reading this article because you’re interested in harnessing the power of Machine Learning (ML) in your supply chain planning process for the first time. Others of you may have been previously burned by a poor application of ML in your supply chain and are now wondering if ML is pure hype. In this article, we explain why standard ML approaches fail in the context of supply chain planning and how AD3’s novel Optimal Machine Learning – a paradigm shift relative to current approaches – leads to robust, principled, and agile data driven decisions.

The end goal in a supply chain context is to make decisions to achieve an objective (e.g., capacity and inventory decisions such as how many units are ordered, where & when those units should be positioned, how those units should be transported, etc. to maximize profits). Historically, the way this problem has been solved – and as professors the way we’ve taught our students for decades – is to use a multi-step process.

This multi-step process, however, has significant drawbacks:

• The standard approach for generating forecasts is through statistical methods which make strong mathematical assumptions about the distribution of demand/sales.
• Statistical forecasts use a limited set of data features, making the forecasts inaccurate. Some companies create rule-based “overrides” to adjust the forecasts; however, this is incredibly labor intensive and can make the forecasts subjective.
• Business models, which capture the supply chain dynamics, are highly stylized models that don’t capture the intricacies of the supply chain dynamics. This leads to idealistic results that do not perform well in the real world.
• The algorithms used to solve these optimization problems generally use heuristics / greedy algorithms, leading to sub-optimal results.
• Errors in any one of these steps will propagate (and get compounded) through the multi-step process. This can lead to poor results despite using an "optimizer."

Machine learning allows computers to automatically learn a specified behavior or task purely based on underlying data. Virtually all of the most successful ML models in 2021 use supervised learning: a framework where a model is trained by 1) making a prediction 2) measuring the error between the prediction and the intended result 3) updating the model in order to penalize any errors. These models have excelled in prediction environments which have large amounts of data, such as text translation, speech recognition, and image processing.

Because of the powerful predictive capabilities of supervised ML, the supply chain planning community has incorporated it into the first step of the supply chain planning process. All supply chain optimization providers which claim to use ML do so by making ML driven forecasts. However, they keep all the other steps in the process the exactly the same.

Our experience has taught us that this updated process has not addressed many of the inefficiencies in the supply chain planning process.

• Forecasting via ML (usually with neural networks) significantly reduces interpretability, which is essential in high-stakes environments such as supply chain planning.
• Even if someone was able to create a crystal ball which perfectly predicted the future, the final decision generated by this multi-step process would likely still be wrong due to assumptions and inaccuracies in the business model and algorithm. In other words, even if your forecasts are perfect, your decisions are likely still sub-optimal.
• Forecasts will always be wrong, as shown by the 2020 crisis. The relationship between the forecast and the decision, however, is generally non-linear. This means that a 10% forecast error can lead to a much larger error in decisions.
• Errors in any one of these steps will propagate (and get compounded) through the multi-step process. This can lead to poor results despite using an "optimizer."

The real solution to this problem is not to replace the forecasting step with ML but rather make the whole process a single-step machine learning model. Unfortunately, this is easier said than done. We’re proud to say that we’ve solved this problem after three years of effort combining our decades long experience in supply chain modeling, deep understanding of modern machine learning, and expertise in high performance computing. We’ve coined the term “Optimal Machine Learning” to describe our solution, which is unique relative to current ML approaches.

Neural Networks have become incredibly popular over the last couple of years due to their incredible success across a diverse set of applications (from text translation all the way to computer vision). A commonality among all these popular models is that they are trained using supervision: a framework where the user feeds the “correct” output ahead of time so that the model can “learn” the relationship between input data and output predictions.

Unfortunately, the supervised ML framework does not work in the context of making supply chain decisions because it relies on knowing what the correct decision is ahead of time. For example, pretend that we wanted to find an inventory policy which maximizes revenue. A supervised model would require us to know the optimal inventory policy ahead of time so that our model can learn the best decisions. Notice that supervised ML creates a paradox: in order to figure out the optimal inventory policy, we are trying to create a model that links data inputs to the inventory policy, and this model needs to know the optimal inventory policy.

Clearly, we need to break out of the mentality of making predictions. Instead, we must use an unsupervised approach – one that does not rely on ground truth.

Our end goal is to be able to create a model that can support “conflicting” goals. For example, it should be able to minimize stocking level investment but guarantee certain service requirements; or, the model should be able to maximize revenue while adhering to a specified budget.

We achieve this goal by creating a model which determines the optimal relationship between decisions, D, and input data, x. The optimal solution determines values for weights, w, to be applied to the data inputs, to compute decisions D, i.e., D = f(x,w). We can also quantify KPIs that these decisions lead to based on the decisions themselves and the intricate supply chain dynamics (e.g., the amount of time it takes inventory to move around the network, the number of units at a particular location currently, etc). By integrating the power of our Digital Twin, OML is able relate the decision D to the specified business objectives and constraints. Through this formulation, we are able to directly find the optimal weights, w, that when applied to the input data x, leads to the best decision, D, which satisfies the specified goals.

The granularity of our system allows us to compute and optimize any number of KPIs (service based metrics, resiliency metrics, etc). Using AD3’s OML, managers now have access to optimal decisions that work for them; e.g., Do you have special locations or products that need to hit certain performance targets? In under five clicks you can update the model and a new decision – one that meets your business requirements – will be waiting for you.

OML gives managers this power to optimize and “shape the solution” without making sacrifices on the optimality of the decisions or on training speed. Achieving this feat was incredibly hard as the dynamics, and as a result the KPIs, are messy functions that can’t be optimized effectively by the standard strategies used in the ML field. Instead, we use methodologies developed in the field of Operations Research that are tried and trusted. Not only does this approach produce optimal results, but it also gives us incredible computational efficiency and allows us to scale to problems across global supply chains with a massive number of products/parts.

For example, we worked with a leading manufacturer of an advanced consumer electronic product where OML recommended decisions for $300M of inventory. Unlike the existing multi-step planning process, these decisions were made in a principled data driven fashion which captures the interactions between multiple stakeholders in the end-to-end supply chain (including contract manufacturers, internal actors, and customers at both the Distribution Center and Retail Store levels).

The resulting decisions from OML are also interpretable. The stacked graph below shows the recommended inventory stocking decision at a particular warehouse over time. Each layer represents the contribution of a particular type of feature (ex: promotions) on the recommended decision.

We have also worked with a major producer of semiconductor equipment where we applied OML to make recommendations for $600M of spare parts inventory across their global network. The graph below illustrates the efficient frontier for inventory investments required to achieve a target fill rate. The solution recommended by the existing state-of-the-art software, achieves a fill rate of 77% for an investment in parts inventory of over $135 million. The points on the curve to right represent the efficient frontier generated by the AD3 software. As we can see, our efficient frontier delivers higher service at a lower cost, e.g. a reduction of over $20 million, at the fill rate achieved by the customer planning solution or a fill rate improvement of 4% at the inventory cost of the customer planning solution.

• Using our Optimal Machine Learning (OML) approach, we have observed improvements of 25% in inventory and logistics cost reductions while maintaining the level of customer service when compared to the state-of-the-art systems currently available.
• OML can recommend decisions which enable a 10% increase in customer service (availability) with the same inventory cost.
• We also have seen the potential for $100’s million increases in revenue in POCs we’re engaged in.
• OML leads to a supply chain planning solution that does not depend on forecast accuracy. This is incredibly valuable as most organizations’ forecasts contain human biases. Our approach, instead, optimizes the use of all data that that has the potential to drive the end decision, e.g. installed base and promotions, internal and customer forecasts, product features and customer attributes. If your forecasts are biased, the model will simply assign a smaller weight to it!
• AD3 allows managers to " shape their solution": a process through which managers can refine decisions in order to find decisions that not only satisfy their business requirements, but also can be realistically implemented.
• The decisions recommended by the OML system are interpretable, making it easy for managers to monitor, understand, and incorporate.
• Since OML is a purely data-driven one-step approach, it is able to significantly expand the scope of data inputs while ensuring that the data is used optimally.
• Our investment in designing OML in computationally efficient manner allowed us to create a system that is able to ingest and respond to new information rapidly, enabling agility in supply chain planning.
• AD3 uses the cloud to deliver solutions that are scalable when applied to complex global supply chain networks.