Multi-Agent Deep Reinforcement Learning for Multi-Echelon Inventory Management PDF Download
Are you looking for read ebook online? Search for your book and save it on your Kindle device, PC, phones or tablets. Download Multi-Agent Deep Reinforcement Learning for Multi-Echelon Inventory Management PDF full book. Access full book title Multi-Agent Deep Reinforcement Learning for Multi-Echelon Inventory Management by Xiaotian Liu. Download full books in PDF and EPUB format.
Author: Xiaotian Liu
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
We apply Multi-Agent Deep Reinforcement Learning (MADRL) to inventory management problems with multiple echelons and evaluate MADRL's performance to minimize the overall costs of a supply chain. We also examine whether the upfront-only information-sharing mechanism used in MADRL helps alleviate the bullwhip effect in a supply chain. We apply Heterogeneous-Agent Proximal Policy Optimization (HAPPO) on the multi-echelon inventory management problems in both a serial supply chain and a supply chain network. Our results show that policies constructed by HAPPO achieve lower overall costs than policies constructed by single-agent deep reinforcement learning and other heuristic policies. Also, the application of HAPPO results in a less significant bullwhip effect than policies constructed by single-agent deep reinforcement learning where information is not shared among actors. Somewhat surprisingly, when applying HAPPO, the system achieves the lowest overall costs when the minimization target for each actor is a combination of its own costs and the overall costs of the system, and the fully self-interested reward target performs near-optimally, while one would expect using the overall costs of the system as a reward target for each actor would be optimal in training the models. Our results provide a new perspective on the benefit of information sharing inside the supply chain that helps alleviate the bullwhip effect and improve the overall performance of the system. Upfront information sharing and action coordination in model training among actors are essential, with the former more essential, for improving a supply chain's overall performance when applying MADRL. Neither actors being fully self-interested nor actors being fully system-focused leads to the optimal performance of policies learned and constructed by MADRL. Our results also verify MADRL's potential in solving various multi-echelon inventory management problems with complex supply chain structures and in non-stationary market environments.
Author: Xiaotian Liu
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
We apply Multi-Agent Deep Reinforcement Learning (MADRL) to inventory management problems with multiple echelons and evaluate MADRL's performance to minimize the overall costs of a supply chain. We also examine whether the upfront-only information-sharing mechanism used in MADRL helps alleviate the bullwhip effect in a supply chain. We apply Heterogeneous-Agent Proximal Policy Optimization (HAPPO) on the multi-echelon inventory management problems in both a serial supply chain and a supply chain network. Our results show that policies constructed by HAPPO achieve lower overall costs than policies constructed by single-agent deep reinforcement learning and other heuristic policies. Also, the application of HAPPO results in a less significant bullwhip effect than policies constructed by single-agent deep reinforcement learning where information is not shared among actors. Somewhat surprisingly, when applying HAPPO, the system achieves the lowest overall costs when the minimization target for each actor is a combination of its own costs and the overall costs of the system, and the fully self-interested reward target performs near-optimally, while one would expect using the overall costs of the system as a reward target for each actor would be optimal in training the models. Our results provide a new perspective on the benefit of information sharing inside the supply chain that helps alleviate the bullwhip effect and improve the overall performance of the system. Upfront information sharing and action coordination in model training among actors are essential, with the former more essential, for improving a supply chain's overall performance when applying MADRL. Neither actors being fully self-interested nor actors being fully system-focused leads to the optimal performance of policies learned and constructed by MADRL. Our results also verify MADRL's potential in solving various multi-echelon inventory management problems with complex supply chain structures and in non-stationary market environments.
Author: Xiaotian Liu
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
The boom of the e-commerce industry in recent years prompts the focus of inventory management into large-scale problems with multiple products and multi-echelon supply chains. This work introduces a simulation-driven solution for large-scale inventory problems, where deep reinforcement learning (DRL) is used as the central technique and deep learning (DL) is exploited to assist the training of the associated neural network. We first investigate a single-echelon multi-product problem as a representative of relatively simple inventory models with ex-post optimal or sub-optimal policies. Using training samples generated by simulation, a DL model is first trained by imitating a target policy, after which a DRL procedure is applied to fine-tune the DL model for further improvement. The numerical results on real-life data from a leading e-commerce company show that our method outperforms conventional base-stock policies and an existing DL method with regard to average operational cost. Then, we formulate a multi-echelon multi-product problem with a practical two-level warehouse network and shared storage resources as a representative of hard inventory models without available heuristic solutions. In this case, a DRL model is trained based on feedback from simulation. The numerical results on real-life data show that our method is capable of constructing intelligent ordering policies that involve coordination among stages and outperforms three combined heuristics adapted to this problem in operational cost.
Author: Pavithra Harsha
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
Reinforcement Learning has lead to considerable break-throughs in diverse areas such as robotics, games and many others. But the application to RL in complex real-world decision making problems remains limited. Many problems in Operations Management (inventory and revenue management, for example) are characterized by large action spaces and stochastic system dynamics. These characteristics make the problem considerably harder to solve for existing RL methods that rely on enumeration techniques to solve per step action problems. To resolve these issues, we develop Programmable Actor Reinforcement Learning (PARL), a policy iteration method that uses techniques from integer programming and sample average approximation. Analytically, we show that the for a given critic, the learned policy in each iteration converges to the optimal policy as the underlying samples of the uncertainty go to infinity. Practically, we show that a properly selected discretization of the underlying uncertain distribution can yield near optimal actor policy even with very few samples from the underlying uncertainty. We then apply our algorithm to real-world inventory management problems with complex supply chain structures and show that PARL outperforms state-of-the-art RL and inventory optimization methods in these settings. We find that PARL outperforms commonly used base stock heuristic by 51.3% and RL based methods by up to 9.58% on average across different supply chain environments.
Author: Stefano V. Albrecht
Publisher: MIT Press
ISBN: 0262049376
Category : Computers
Languages : en
Pages : 0
Get Book Here
Book Description
The first comprehensive introduction to Multi-Agent Reinforcement Learning (MARL), covering MARL’s models, solution concepts, algorithmic ideas, technical challenges, and modern approaches. Multi-Agent Reinforcement Learning (MARL), an area of machine learning in which a collective of agents learn to optimally interact in a shared environment, boasts a growing array of applications in modern life, from autonomous driving and multi-robot factories to automated trading and energy network management. This text provides a lucid and rigorous introduction to the models, solution concepts, algorithmic ideas, technical challenges, and modern approaches in MARL. The book first introduces the field’s foundations, including basics of reinforcement learning theory and algorithms, interactive game models, different solution concepts for games, and the algorithmic ideas underpinning MARL research. It then details contemporary MARL algorithms which leverage deep learning techniques, covering ideas such as centralized training with decentralized execution, value decomposition, parameter sharing, and self-play. The book comes with its own MARL codebase written in Python, containing implementations of MARL algorithms that are self-contained and easy to read. Technical content is explained in easy-to-understand language and illustrated with extensive examples, illuminating MARL for newcomers while offering high-level insights for more advanced readers. First textbook to introduce the foundations and applications of MARL, written by experts in the field Integrates reinforcement learning, deep learning, and game theory Practical focus covers considerations for running experiments and describes environments for testing MARL algorithms Explains complex concepts in clear and simple language Classroom-tested, accessible approach suitable for graduate students and professionals across computer science, artificial intelligence, and robotics Resources include code and slides
Author: Arindam Dey
Publisher: John Wiley & Sons
ISBN: 1394214111
Category : Computers
Languages : en
Pages : 485
Get Book Here
Book Description
Provides a comprehensive understanding of the latest advancements and practical applications of machine learning techniques. Machine learning (ML), a branch of artificial intelligence, has gained tremendous momentum in recent years, revolutionizing the way we analyze data, make predictions, and solve complex problems. As researchers and practitioners in the field, the editors of this book recognize the importance of disseminating knowledge and fostering collaboration to further advance this dynamic discipline. How Machine Learning is Innovating Today's World is a timely book and presents a diverse collection of 25 chapters that delve into the remarkable ways that ML is transforming various fields and industries. It provides a comprehensive understanding of the practical applications of ML techniques. The wide range of topics include: An analysis of various tokenization techniques and the sequence-to-sequence model in natural language processing explores the evaluation of English language readability using ML models a detailed study of text analysis for information retrieval through natural language processing the application of reinforcement learning approaches to supply chain management the performance analysis of converting algorithms to source code using natural language processing in Java presents an alternate approach to solving differential equations utilizing artificial neural networks with optimization techniques a comparative study of different techniques of text-to-SQL query conversion the classification of livestock diseases using ML algorithms ML in image enhancement techniques the efficient leader selection for inter-cluster flying ad-hoc networks a comprehensive survey of applications powered by GPT-3 and DALL-E recommender systems' domain of application reviews mood detection, emoji generation, and classification using tokenization and CNN variations of the exam scheduling problem using graph coloring the intersection of software engineering and machine learning applications explores ML strategies for indeterminate information systems in complex bipolar neutrosophic environments ML applications in healthcare, in battery management systems, and the rise of AI-generated news videos how to enhance resource management in precision farming through AI-based irrigation optimization. Audience The book will be extremely useful to professionals, post-graduate research scholars, policymakers, corporate managers, and anyone with technical interests looking to understand how machine learning and artificial intelligence can benefit their work.
Author: Arup Kumar Sadhu
Publisher: John Wiley & Sons
ISBN: 1119699029
Category : Computers
Languages : en
Pages : 320
Get Book Here
Book Description
Discover the latest developments in multi-robot coordination techniques with this insightful and original resource Multi-Agent Coordination: A Reinforcement Learning Approach delivers a comprehensive, insightful, and unique treatment of the development of multi-robot coordination algorithms with minimal computational burden and reduced storage requirements when compared to traditional algorithms. The accomplished academics, engineers, and authors provide readers with both a high-level introduction to, and overview of, multi-robot coordination, and in-depth analyses of learning-based planning algorithms. You'll learn about how to accelerate the exploration of the team-goal and alternative approaches to speeding up the convergence of TMAQL by identifying the preferred joint action for the team. The authors also propose novel approaches to consensus Q-learning that address the equilibrium selection problem and a new way of evaluating the threshold value for uniting empires without imposing any significant computation overhead. Finally, the book concludes with an examination of the likely direction of future research in this rapidly developing field. Readers will discover cutting-edge techniques for multi-agent coordination, including: An introduction to multi-agent coordination by reinforcement learning and evolutionary algorithms, including topics like the Nash equilibrium and correlated equilibrium Improving convergence speed of multi-agent Q-learning for cooperative task planning Consensus Q-learning for multi-agent cooperative planning The efficient computing of correlated equilibrium for cooperative q-learning based multi-agent planning A modified imperialist competitive algorithm for multi-agent stick-carrying applications Perfect for academics, engineers, and professionals who regularly work with multi-agent learning algorithms, Multi-Agent Coordination: A Reinforcement Learning Approach also belongs on the bookshelves of anyone with an advanced interest in machine learning and artificial intelligence as it applies to the field of cooperative or competitive robotics.
Author: Eduardo Lalla-Ruiz
Publisher: Springer Nature
ISBN: 3030597474
Category : Computers
Languages : en
Pages : 780
Get Book Here
Book Description
This book constitutes the proceedings of the 11th International Conference on Computational Logistics, ICCL 2020, held in Enschede, The Netherlands, in September 2020. The 49 papers included in this book were carefully reviewed and selected from 73 submissions. They were organized in topical sections named: maritime and port logistics; vehicle routing and scheduling; freight distribution and city logistics; network design and scheduling; and selected topics in logistics. Due to the Corona pandemic ICCL 2020 was held as a virtual event.
Author: Ginni Devi
Publisher:
ISBN:
Category :
Languages : en
Pages : 0
Get Book Here
Book Description
This study is an attempt to present a brief survey of several excellent works done by various authors in the field of multi-agent learning as well as multi-agent deep learning towards improving coordination and learning efficiency in the same. Based on the review of the existing work and research findings, we have proposed a framework to address coordination and learning issues in multi-agent learning. In this paper, we present a Networked-Deep Multi-agent Learning framework (N-DMAL), based upon implementation of deep reinforcement learning with social networks, that will result in improved learning efficiency of agents while interacting in a networked multi-agent system. The presented approach extends the traditional deep reinforcementlearning algorithm for agents' interaction with other neighbouring agents when they coordinate in a cooperative manner.
Author: Jung Kyu Canci
Publisher: Springer Nature
ISBN: 3031118146
Category : Business & Economics
Languages : en
Pages : 131
Get Book Here
Book Description
This open access book explores the field of life science business from a multidisciplinary perspective. Applying statistical, mathematical, game-theoretic, and data science tools to pharmaceutical and biotechnology business endeavors, the book describes value creation, value maintenance, and value realization in the life sciences as a sequence of processes using the quantitative language of applied mathematics. Written by experts from a variety of fields, the contributions illustrate the shift from a deterministic to a stochastic view of the processes involved, offering a new perspective on life sciences economics. The book covers topics such as valuing and managing intellectual property in life science, licensing in the pharmaceutical business, outsourcing pharmaceutical R&D, and stochastic modelling of a pharmaceutical supply chain. The book will appeal to scholars of economics and the life sciences, as well as to professionals in chemical and pharmaceutical industries.
Author: Felipe Felipe Leno da Silva
Publisher: Springer Nature
ISBN: 3031015916
Category : Computers
Languages : en
Pages : 111
Get Book Here
Book Description
Learning to solve sequential decision-making tasks is difficult. Humans take years exploring the environment essentially in a random way until they are able to reason, solve difficult tasks, and collaborate with other humans towards a common goal. Artificial Intelligent agents are like humans in this aspect. Reinforcement Learning (RL) is a well-known technique to train autonomous agents through interactions with the environment. Unfortunately, the learning process has a high sample complexity to infer an effective actuation policy, especially when multiple agents are simultaneously actuating in the environment. However, previous knowledge can be leveraged to accelerate learning and enable solving harder tasks. In the same way humans build skills and reuse them by relating different tasks, RL agents might reuse knowledge from previously solved tasks and from the exchange of knowledge with other agents in the environment. In fact, virtually all of the most challenging tasks currently solved by RL rely on embedded knowledge reuse techniques, such as Imitation Learning, Learning from Demonstration, and Curriculum Learning. This book surveys the literature on knowledge reuse in multiagent RL. The authors define a unifying taxonomy of state-of-the-art solutions for reusing knowledge, providing a comprehensive discussion of recent progress in the area. In this book, readers will find a comprehensive discussion of the many ways in which knowledge can be reused in multiagent sequential decision-making tasks, as well as in which scenarios each of the approaches is more efficient. The authors also provide their view of the current low-hanging fruit developments of the area, as well as the still-open big questions that could result in breakthrough developments. Finally, the book provides resources to researchers who intend to join this area or leverage those techniques, including a list of conferences, journals, and implementation tools. This book will be useful for a wide audience; and will hopefully promote new dialogues across communities and novel developments in the area.
