Dynamic surgical scheduling within a workday is a complicated decision-making process. The critical challenge is that the actual duration of surgery and the arrival process of emergency patients are uncertain and unknown in advance. In this work, we propose a two-level dynamic scheduling framework based on hierarchical reinforcement learning to solve dynamic surgical scheduling problems considering both elective and emergency patients. Specifically, with the realisation of uncertainty, the upper-level agent (UA) dynamically decides whether to trigger rescheduling to optimise the workday total cost. The lower-level agent (LA) aims at obtaining subscheduling solutions when rescheduling is triggered. The subproblem at the LA can be formulated as a mixed integer programming model, which can be generalised to unrelated parallel machine scheduling with machine eligibility restrictions and sequence-and machine-dependent setup times. Such problems can be solved in small-scale cases and suffers the combinatorial explosion in large scale cases. To address this issue, we propose a heuristic method that is built upon deep reinforcement learning to obtain high-quality solutions. We conduct extensive simulation experiments with real data to test the effective of our framework. The results for different scenarios show that our proposed framework outperforms existing methods in terms of overall performance and has strong generalisation ability.

Lixiang Zhao

Han Zhu

Min Zhang

Jiafu Tang

Yu Wang

International Journal of Production Research

国际生产研究杂志 (IJPR) 于 1961 年出版，是一本成熟、高度成功和领先的期刊，报道制造、生产和运营管理研究。 IJPR 每年出版 24 期，包括关于创新管理、产品设计、制造流程、生产和物流系统的论文。生产经济学、生产资源和系统的基本行为以及生产和物流系统的设计、管理和控制中出现的复杂决策问题都被考虑在内。制造战略、政策制定和评估以及新技术的贡献也是该期刊的主要关注点。还考虑了在计算机、决策和数学科学中开发的可用于生产和物流系统的设计、测量或操作的基本技术。 IJPR 的声誉建立在与工业应用的紧密联系之上。具有明确现实应用的令人信服的科学结果是选择论文的主要标准。也欢迎介绍新的和有趣的生产研究问题或/和新应用的教学文章。该杂志也发表重大的理论进展，只要清楚地解释它们的主要概念和有用性，以便整个生产研究界都能理解它们。为调查和讨论文件以及由领先专家发表的特邀文章保留了一个特殊的位置。 IJPR 是一本面向机械工程、工业和系统工程、运筹学和管理科学以及商业领域的研究人员和教授的期刊。对于希望提高其生产系统的效率和有效性的工业管理人员来说，它也是一个信息参考。所有提交的手稿均需经过编辑的初步评估，如果认为适合进一步考虑，则由至少三位独立的匿名专家审稿人进行同行评审。所有同行评审都是单盲的，并且通过 SCHOLARONE MANUSCRIPTS 在线提交。

The International Journal of Production Research (IJPR) has been a mature and leading journal since its inception in 1961, focusing on research in manufacturing, production, and operations management. IJPR publishes 24 issues annually, covering topics such as innovation management, product design, manufacturing processes, and production and logistics systems. The journal addresses fundamental behaviors in production economics, production resources, and systems, as well as complex decision-making problems that arise in the design, management, and control of production and logistics systems. Key areas of focus also include manufacturing strategies, policy formulation and evaluation, and contributions of new technologies. Furthermore, the journal considers fundamental techniques developed in computer science, decision science, and mathematics that can be applied to the design, measurement, or operation of production and logistics systems. IJPR has built its reputation on a close connection with industrial applications, and compelling scientific results with clear practical applications are the main criteria for selecting papers. We welcome new and interesting research questions in production or teaching articles on new applications. The journal also publishes significant theoretical advancements, provided they clearly explain their main concepts and usefulness for the broader production research community. A special section is reserved for investigation and discussion papers, as well as invited articles from leading experts. The journal targets researchers and professors in mechanical engineering, industrial and systems engineering, operations research, management science, and business fields. Additionally, it serves as an important information resource for industrial managers seeking to enhance the efficiency and effectiveness of their production systems. All submitted manuscripts undergo an initial assessment by the editor, and if deemed suitable for further consideration, they are reviewed by at least three independent anonymous experts. All peer reviews are single-blind and submitted online via SCHOLARONE MANUSCRIPTS.

国际生产研究杂志 (International Journal of Production Research, IJPR) 自1961年创刊以来，已成为一个成熟且领先的期刊，专注于制造、生产和运营管理领域的研究。IJPR每年出版24期，涵盖创新管理、产品设计、制造流程、生产和物流系统等主题。期刊关注生产经济学、生产资源和系统的基本行为，以及在生产和物流系统的设计、管理和控制中出现的复杂决策问题。制造战略、政策制定和评估，以及新技术的贡献也是该期刊的重要关注点。此外，期刊还考虑计算机、决策和数学科学中开发的基本技术，这些技术可用于生产和物流系统的设计、测量或操作。IJPR的声誉建立在与工业应用的紧密联系之上，具有明确现实应用的科学结果是选择论文的主要标准。我们欢迎新的和有趣的生产研究问题或新应用的教学文章。该杂志也发表重要的理论进展，只要这些进展能够清晰地解释其主要概念和实用性，以便整个生产研究界都能理解。IJPR特别保留了一个位置用于调查和讨论文件，以及由领先专家撰写的特邀文章。该期刊面向机械工程、工业和系统工程、运筹学和管理科学以及商业领域的研究人员和教授。同时，对于希望提高其生产系统效率和有效性的工业管理人员来说，IJPR也是一个重要的信息参考。所有提交的手稿均需经过编辑的初步评估，若适合进一步考虑，将由至少三位独立的匿名专家进行同行评审。所有同行评审均为单盲评审，并通过SCHOLARONE MANUSCRIPTS在线提交。

Category	Quartile
工程技术	2
工程技术, 工程工业	2
工程技术, 工程制造	2
工程技术, 运筹学与管理科学	2

Large-scale dynamic surgical scheduling under uncertainty by hierarchical reinforcement learning

ivySCI AI Smartly Parses PDF, Answers Researchers' Questions, and Helps You Understand Papers in Seconds

Journal Info

Category	Quartile
OPERATIONS RESEARCH & MANAGEMENT SCIENCE	1