Tackling the Conference Scheduling Conundrum: Exploring the Limits of AI Assistants
The Challenge: Creating an Optimal Schedule for a Five-Day Conference
Stephan Jansen, a renowned Java Champion and founder of Devoxx, embarked on a quest to utilize ChatGPT’s capabilities to craft the perfect conference schedule. While he achieved promising results for a single day, replicating this success across the entire five-day conference proved elusive. Geoffrey de Smet, the CTO and founder of Timefold (formerly OptaPlanner), attributed this challenge to the AI assistant’s inability to effectively handle soft constraints, a crucial aspect in scheduling optimization.
The scheduling problem’s complexity stems from the sheer number of variables involved. With 210 talks to be assigned across 40 time slots and 10 rooms, the possible combinations skyrocket, resulting in an astronomical 10^277 schedule combinations. de Smet aptly compared finding the ideal schedule to searching for a needle in a haystack.
The Computational Bottleneck: NP-Hardness and the Search for a Silver Bullet
The conference scheduling problem falls under the category of NP-hard problems, notorious for their computational intractability. Even algorithms capable of reducing the search space by 99% would still be faced with the daunting task of sifting through 10^279 combinations. de Smet emphasized the impossibility of finding a solution within a billion years, even with the combined computing power of the entire planet.
Seeking Clarity: Unraveling the Scheduling Conundrum
To gain deeper insights into this intricate problem, InfoQ reached out to Jansen and de Smet, seeking their perspectives on the role of Large Language Models (LLMs) in generating optimal schedules for five-day events.
InfoQ: Why is the scheduling problem so challenging to solve?
Jansen’s Perspective: Balancing Constraints and Creating a Coherent Experience
Jansen explained the multifaceted nature of the scheduling problem, highlighting the need to consider speaker availability, avoid topic overlap, and ensure a logical progression of sessions. Additionally, he emphasized the importance of accommodating popular talks in larger rooms to optimize audience engagement.
InfoQ: Can ChatGPT deliver a satisfactory solution?
Jansen’s Experience: Promising Results for a Single Day, Challenges for the Full Conference
Jansen’s experimentation with ChatGPT revealed its potential in generating code for scheduling, leading to a feasible schedule for a single day of the conference. However, extending this success to the entire five-day event proved challenging.
de Smet’s Analysis: The Limitations of ChatGPT in Handling Soft Constraints
de Smet attributed ChatGPT’s shortcomings to its inability to effectively manage soft constraints, a crucial aspect in scheduling optimization. He highlighted the contrast between ChatGPT’s approach and Timefold’s specialized planning library, which excels in handling soft constraints.
A Friendly Wager: Comparing AI-Generated and Hand-Written Models
Jansen and de Smet engaged in a friendly wager to determine whether an AI-generated approach could outperform a hand-written model in scheduling optimization. While the hand-written model currently holds a significant advantage, de Smet acknowledged that Jansen’s experiment opened his eyes to the potential of LLM-generated Timefold models.
Timefold’s Approach: Harnessing Functional Programming for Efficient Constraint Handling
de Smet elaborated on Timefold’s unique approach to constraint handling, which leverages functional programming on Talk and Speaker classes. This approach simplifies the creation and maintenance of constraints, making it more accessible and efficient compared to mathematical equation-based approaches used by other production solvers.
Timefold’s Hardware Requirements: Striking a Balance Between Performance and Resources
de Smet shed light on Timefold’s hardware requirements, emphasizing its CPU-bound nature. The solving process is I/O-free and typically utilizes 4 CPUs for 30 minutes per dataset. Larger enterprises may employ 40 CPUs across 5 hours to process 100 datasets per night.
Exploring Open-Source Alternatives: A Diverse Landscape of Scheduling Libraries
Jansen’s Recommendations: A Range of Open-Source Libraries for Scheduling
Jansen highlighted the availability of several open-source libraries for scheduling, including those developed by Google (OR-Tools) and Microsoft (Z3-Prover). He also mentioned presentations at Devoxx showcasing alternative solutions.
de Smet’s Insights: Java, Kotlin, and Python Libraries for Scheduling
de Smet pointed to a variety of scheduling libraries written in programming languages supported by Timefold, such as Java, Kotlin, and Python. He listed specific libraries like Choco-Solver, JaCoP, JOpt, Konopt, KOpt, KIE, CPLEX, and COIN-OR as notable options.
The Current State of AI Assistants in Scheduling: A Promising Start, but Room for Growth
While LLMs like ChatGPT hold promise in assisting developers with partial model generation, they are not yet capable of generating complete and optimal schedules for complex problems like conference scheduling. A combination of experience and the utilization of AI assistants can yield efficient solutions that balance outcomes and resource utilization.