PROSA · Program-Specific Agents

In recent years, novel AI coding assistants based on large language models (LLMs) have become extremely popular in creating code. However, at least 50% of the effort of software development goes into understanding and maintaining existing software, tasks that still pose grand challenges for LLMs. This is because most maintenance tasks are related to the dynamic behavior of programs, which is insufficiently captured by learning from static code alone. Understanding how the individual pieces of a large program work together can take human experts months to years.As part of our ERC Advanced Grant “S3—Semantics of Software Systems”, we have developed innovative techniques to systematically _test_software systems, comprehensively exploring their behavior. This allows us to capture the dynamic features of inputs, outputs, and executions and train programspecific machine learning models that relate them to each other. The resulting model can then answer maintenance questions such as “What configuration input do I need to make this button green?” or “What input triggers this error message?”Our underlying research has already shown the principal feasibility of this approach, but has been limited to small examples. The question is whether these approaches scale to the multitude of features that occur during executing a complex program. This is the aim of this proof of concept proposal: To create a demonstrator of a Program-Specific Agent that is automatically trained from a nontrivial piece of software and that can answer maintenance-related questions with high accuracy.Training such an agent on even a single piece of software will require substantial resources—hence this grant proposal. However, any such trained agent will act as an expert on the software’s behavior as long as it exists, promising substantial cost savings for the future of software development.