Finding Simulation Models With AI: A Cartoon Summary of Our Latest Research

We decided to change our blog style a bit. Today we debut a blog post in the form of a cartoon.

Yes, a cartoon like those you may find in a newspaper.

Inspired by the work Jhon is doing for his dissertation on the analysis and contextualization of editorial cartoons, we decided to cartoonize our latest accepted conference paper.

Often, scientific findings are summarized in the form of posters or infographics, which are academic in nature. But really, most academic papers can be explained (at a high level) with four sections: the problem; the solution; the results; and the implications of the solution/results for the bigger picture. We'll keep it to the first three for brevity and to encourage you to take a look at the paper for the larger implications of our work.

The Problem: Finding a Model is Harder Than it Sounds

Imagine working in an organization that has developed hundreds, or even thousands, of simulation models over many years.

Now imagine that someone asks:

"Do we already have a model that can help answer this new question?"

The answer is rarely obvious. Models are often expensive to build, so we would like to reuse them as they are or modify them slightly for a different application. In some cases, we may even want to combine multiple models, which is itself a challenging endeavor.

As such, searching for and finding models is important but complicated because they are often not documented, and even if they are, we are looking for them based on what we want to do.

The first panel of our cartoon illustrates this situation. Someone knows a model might exist somewhere, but it has become lost among a large collection of models, and manually finding it is complicated.

The Solution: Let AI Search For Us

To answer the question of how we can find simulation models, there are many possible approaches. In our paper, we explored how AI can help with this.

AI offers a semantic layer that goes beyond simple keyword matching by understanding the meaning of the user's query in relation to the available models. Most applications of AI for this task have focused on unstructured data, such as narratives; but, a system like this applied to simulation models has not yet been explored.

However, several design decisions affect how well such a system performs. We therefore experimented with three important components.

1) Data formats: Before a model can be searched, it may be represented in different ways. Should the model be stored in its native XML-based format? Should it be represented in JSON? Should additional non-structural information be included? Different representations expose different information to the AI system, and this can influence retrieval performance.

2) Embeddings: Computers cannot directly compare the meaning of sentences, so embeddings transform text into numerical vectors where semantically similar concepts are positioned close together. Different embedding models create these representations differently, which can also affect retrieval performance.

3) Retrieval strategies: When retrieving simulation models, the system needs a strategy for deciding which models are most relevant. Different strategies, such as the use of reranking or not, may lead to different performance.

We experimented with combinations of data formats, embedding models, and retrieval strategies to provide insights into retrieval performance and establish a baseline for future research. This is the first step toward doing something with the models we find. What we have in mind is that this facilitates the identification of models that can be combined because they complement each other in answering questions for which the original models were not designed.

The second panel of the cartoon captures this exploration. The AI is represented as a friendly dog enthusiastically fetching models from a large collection. The dog is a playful metaphor for an assistant that retrieves what we ask for without needing to understand simulation modeling the way a human expert does.

The results

Our experiments showed that some combinations of data formats, embeddings, and retrieval strategies consistently retrieved more relevant models than others. The system produces a ranked list where the most relevant models appear first. This greatly reduces the effort required by a modeler to identify useful candidates.

The third panel of the cartoon illustrates this outcome. Rather than searching through hundreds of boxes, the user receives a short ranked list with the most promising models at the top.

We hope this cartoon provides a simple and engaging introduction to our work. If it sparked your curiosity, we invite you to explore the full paper for the technical details in the pre-print version of our paper here: https://arxiv.org/abs/2606.30846