A Sliding-Window-Based Reinforcement Learning for Dynamic Assembly Flow Shop Scheduling with Multi-Product Delivery
06:00 · July 7, 2026 · arXiv cs.AI RSS
Multi-product kitting delivery imposes significant challenges for real-time scheduling in hybrid manufacturing systems that integrate processing and assembly, as dynamic order arrivals simultaneously alter supply dependencies and the set of feasible job-machine assignments. This paper proposes a sliding-window-based reinforcement learning (SWRL) framework for end-to-end online scheduling in the flexible assembly flow shop scheduling problem with complex kitting constraints. The problem is formulated as a heterogeneous graph-based Markov decision process that captures the dual-layer kitting structure and the tail-product bottleneck dynamics that produce a sparse reward landscape. To address the resulting challenges, SWRL integrates a sliding-window filtering mechanism that filters inactive nodes and prioritizes kitting-critical operations, a spatiotemporal graph encoding network that tracks bottleneck shifts across consecutive decision states, and a dynamic action mapping module with a constrained waiting strategy that adapts to the changing action space under variable topologies. Experiments on real-world instances from a home appliance manufacturer demonstrate that SWRL achieves consistent tardiness reductions over classical dispatching rules and existing deep reinforcement learning methods, and exhibits robust performance across varying resource configurations, order loads, and arrival concentrations.
Summary
This paper presents a sliding-window-based reinforcement learning (SWRL) framework for real-time scheduling in the dynamic flexible assembly flow shop problem with multi-product delivery. The approach targets hybrid manufacturing environments that combine processing and assembly stages, where multi-product kitting requires all components of each product to be ready before final assembly and all products within an order to finish simultaneously. Dynamic order arrivals continuously alter both supply dependencies and the set of feasible job-machine assignments, creating shifting bottlenecks that classical static optimization methods cannot address at operational timescales.
The problem is cast as a heterogeneous graph-based Markov decision process. This formulation encodes the dual-layer kitting structure—part-level coupling within products and order-level synchronization across products—along with the tail-product dynamics that determine order tardiness. The resulting reward landscape is sparse because most job-machine decisions produce no immediate change in tardiness. To manage this, SWRL incorporates three elements: a sliding-window filter that removes inactive nodes and emphasizes kitting-critical operations, a spatiotemporal graph encoding network that tracks how bottlenecks move between successive decision states, and a dynamic action mapping module paired with a constrained waiting strategy that adjusts to evolving graph topologies and action spaces.
Experiments conducted on instances drawn from a home appliance manufacturer show that SWRL reduces tardiness relative to both classical dispatching rules and prior deep reinforcement learning schedulers. The method maintains performance across different resource levels, order volumes, and arrival patterns while requiring fewer training episodes than comparable end-to-end approaches. Ablation results confirm that each of the three components contributes measurably to these gains.
Why it matters
The research provides advanced reinforcement learning methodologies for dynamic scheduling, which is highly applicable to the Netherlands' robust high-tech manufacturing and logistics sectors (e.g., Brainport region). AI researchers and practitioners can leverage these graph-based MDP techniques to optimize complex assembly lines and supply chains.
