Stress Testing Provide Chain Networks at Scale on Databricks

Introduction

Within the current commerce warfare, governments have weaponized commerce via cycles of retaliatory tariffs, quotas, and export bans. The shockwaves have rippled throughout provide chain networks and compelled corporations to reroute sourcing, reshore manufacturing, and stockpile essential inputs—measures that reach lead occasions and erode once-lean, just-in-time operations. Every detour carries a price: rising enter costs, elevated logistics bills, and extra stock tying up working capital. Because of this, revenue margins shrink, cash-flow volatility will increase, and balance-sheet dangers intensify.

Was the commerce warfare a singular occasion that caught international provide chains off guard? Maybe in its specifics, however the magnitude of disruption was hardly unprecedented. Over the span of only a few years, the COVID-19 pandemic, the 2021 Suez Canal blockage, and the continuing Russo-Ukrainian warfare every delivered main shocks, occurring roughly a 12 months aside. These occasions, tough to foresee, have brought on substantial disruption to international provide chains. 

What might be carried out to organize for such disruptive occasions? As a substitute of reacting in panic to last-minute modifications, can corporations make knowledgeable choices and take proactive steps earlier than a disaster unfolds? A well-cited paper by MIT professor David Simchi-Levi provides a compelling, data-driven strategy to this problem. On the core of his methodology is the creation of a digital twin—a graph-based mannequin the place nodes signify websites and services within the provide chain, and edges signify the circulate of supplies between them. A variety of disruption eventualities is then utilized to the community, and its responses are measured. Via this course of, corporations can assess potential impacts, uncover hidden vulnerabilities, and establish redundant investments.

This course of, generally known as stress testing, has been broadly adopted throughout industries. Ford Motor Firm, for instance, utilized this strategy throughout its operations and provide community, which incorporates over 4,400 direct provider websites, a whole bunch of 1000’s of lower-tier suppliers, greater than 50 Ford-owned services, 130,000 distinctive elements, and over $80 billion in annual exterior procurement. Their evaluation revealed that roughly 61% of provider websites, if disrupted, would don’t have any impression on earnings—whereas about 2% would have a big impression. These insights essentially reshaped their strategy to provide chain threat administration.

The rest of this weblog submit gives a high-level overview of easy methods to implement such an answer and carry out a complete evaluation on Databricks. The supporting notebooks are open-sourced and obtainable right here.

Stress Testing Provide Chain Networks on Databricks

Think about a state of affairs the place we’re working for a worldwide retailer or a shopper items firm and tasked with enhancing provide chain resiliency. This particularly means guaranteeing that our provide chain community can meet buyer demand throughout future disruptive occasions to the fullest extent attainable. To attain this, we should establish susceptible websites and services inside the community that might trigger disproportionate harm in the event that they fail and reassess our investments to mitigate the related dangers. Figuring out high-risk places additionally helps us acknowledge low-risk ones. If we uncover areas the place we’re overinvesting, we are able to both reallocate these sources to stability threat publicity or scale back pointless prices.

Step one towards attaining our objective is to assemble a digital twin of our provide chain community. On this mannequin, provider websites, manufacturing services, warehouses, and distribution facilities might be represented as nodes in a graph, whereas the sides between them seize the circulate of supplies all through the community. Creating this mannequin requires operational knowledge comparable to stock ranges, manufacturing capacities, payments of supplies, and product demand. By utilizing these knowledge as inputs to a linear optimization program—designed to optimize a key metric comparable to revenue or price—we are able to decide the optimum configuration of the community for that given goal. This allows us to establish how a lot materials must be sourced from every sub-supplier, the place it must be transported, and the way it ought to transfer via to manufacturing websites to optimize the chosen metric—a provide chain optimization strategy broadly adopted by many organizations. Stress testing goes a step additional—introducing the ideas of time-to-recover (TTR) and time-to-survive (TTS).

Time-to-recover (TTR)

TTR is among the key inputs to the community. It signifies how lengthy a node—or a bunch of nodes—takes to get better to its regular state after a disruption. For instance, if certainly one of your provider’s manufacturing websites experiences a fireplace and turns into non-operational, TTR represents the time required for that website to renew supplying at its earlier capability. TTR is usually obtained instantly from suppliers or via inside assessments.

With TTR in hand, we start simulating disruptive eventualities. Underneath the hood, this entails eradicating or limiting the capability of a node—or a set of nodes—affected by the disruption and permitting the community to re-optimize its configuration to maximise revenue or reduce price throughout all merchandise underneath the given constraints. We then assess the monetary lack of working underneath this new configuration and calculate the cumulative impression over the period of the TTR. This provides us the estimated impression of the particular disruption. We repeat this course of for 1000’s of eventualities in parallel utilizing Databricks’ distributed computing capabilities.

Under is an instance of an evaluation carried out on a multi-tier community producing 200 completed items, with supplies sourced from 500 tier-one suppliers and 1000 tier-two suppliers. Operational knowledge had been randomly generated inside affordable constraints. For the disruptive eventualities, every provider node was eliminated individually from the graph and assigned a random TTR. The scatter plot beneath shows whole spend on provider websites for threat mitigation on the vertical axis and misplaced revenue on the horizontal axis. This visualization permits us to shortly establish areas the place threat mitigation funding is undersized relative to the potential harm of a node failure (pink field), in addition to areas the place funding is outsized in comparison with the danger (inexperienced field). Each areas current alternatives to revisit and optimize our funding technique—both to boost community resiliency or to scale back pointless prices.

Time-to-survive (TTS)

TTS provides one other perspective on the danger related to node failure. Not like TTR, TTS will not be an enter however an output—a choice variable. When a disruption happens and impacts a node or a bunch of nodes, TTS signifies how lengthy the reconfigured community can proceed fulfilling buyer demand with none loss. The chance turns into extra pronounced when TTR is considerably longer than TTS. 

Under is one other evaluation carried out on the identical community. The histogram exhibits the distribution of variations between TTR and TTS for every node. Nodes with a destructive TTR − TTS are usually not a priority—assuming the offered TTR values are correct. Nevertheless, nodes with a constructive TTR − TTS might incur monetary loss, particularly these with a big hole. To boost community resiliency, we will reassess and probably scale back TTR by renegotiating phrases with suppliers, enhance TTS by constructing stock buffers, or diversify the sourcing technique.

By combining TTR and TTS evaluation, we are able to achieve a deeper understanding of provide chain community resiliency. This train might be carried out strategically on a yearly or quarterly foundation to tell sourcing choices, or extra tactically on a weekly or each day foundation to watch fluctuating threat ranges throughout the community—serving to to make sure easy and responsive provide chain operations.

On a light-weight four-node cluster, the TTR and TTS analyses accomplished in 5 and 40 minutes respectively on the community described above (1,700 nodes)—all for underneath $10 in cloud spend. This highlights the answer’s spectacular pace and cost-effectiveness. Nevertheless, as provide chain complexity and enterprise necessities develop—with elevated variability, interdependencies, and edge instances—the answer might require better computational energy and extra simulations to take care of confidence within the outcomes.

Why Databricks

Each data-driven resolution depends on the standard and completeness of the enter dataset—and stress testing isn’t any exception. Firms want high-quality operational knowledge from their suppliers and sub-suppliers, together with info on payments of supplies, stock, manufacturing capacities, demand, TTR, and extra. Gathering and curating this knowledge will not be trivial. Furthermore, constructing a clear and versatile stress-testing framework that displays the distinctive features of your enterprise requires entry to a variety of open-source and third-party instruments—and the flexibility to pick the appropriate mixture. Particularly, this contains LP solvers and modeling frameworks. Lastly, the effectiveness of stress testing hinges on the breadth of the disruption eventualities thought of. Working such a complete set of simulations calls for entry to extremely scalable computing sources.

Databricks is the perfect platform for constructing the sort of resolution. Whereas there are a lot of causes, crucial embody:

1. Delta Sharing: Entry to up-to-date operational knowledge is important for growing a resilient provide chain resolution. Delta Sharing is a robust functionality that permits seamless knowledge trade between corporations and their suppliers—even when one social gathering will not be utilizing the Databricks platform. As soon as the information is out there in Databricks, enterprise analysts, knowledge engineers, knowledge scientists, statisticians, and managers can all collaborate on the answer inside a unified, knowledge clever platform.
2. Open Requirements: Databricks integrates seamlessly with a broad vary of open-source and third-party applied sciences, enabling groups to leverage acquainted instruments and libraries with minimal friction. Customers have the flexibleness to outline and mannequin their very own enterprise issues, tailoring options to particular operational wants. Open-source instruments present full transparency into their internals—essential for auditability, validation, and steady enchancment—whereas proprietary instruments might supply efficiency benefits. On Databricks, you will have the liberty to decide on the instruments that finest fit your wants.
3. Scalability: Fixing optimization issues on networks with 1000’s of nodes is computationally intensive. Stress testing requires working simulations throughout tens of 1000’s of disruption eventualities—whether or not for strategic (yearly/quarterly) or tactical (weekly/each day) planning—which calls for a extremely scalable platform. Databricks excels on this space, providing horizontal scaling to effectively deal with advanced workloads, powered by sturdy integration with distributed computing frameworks comparable to Ray and Spark.

Abstract

International provide chains usually lack visibility into community vulnerabilities and battle to foretell which provider websites or services would trigger essentially the most harm throughout disruptions—resulting in reactive disaster administration. On this article, we introduced an strategy to construct a digital twin of the availability chain community by leveraging operational knowledge and working stress testing simulations that consider Time-to-Get better (TTR) and Time-to-Survive (TTS) metrics throughout 1000’s of disruption eventualities on Databricks’ scalable platform. This methodology allows corporations to optimize threat mitigation investments by figuring out high-impact, susceptible nodes—much like Ford’s discovery that solely a small fraction of provider websites considerably have an effect on earnings—whereas avoiding overinvestment in low-risk areas. The result’s preserved revenue margins and diminished provide chain prices.

Databricks is ideally suited to this strategy, because of its scalable structure, Delta Sharing for real-time knowledge trade, and seamless integration with open-source and third-party instruments for clear, versatile, environment friendly and cost-effective provide chain modeling. Obtain the notebooks to discover how stress testing of provide chain networks at scale might be carried out on Databricks.