Program

The workshop will happen in person in Rio de Janeiro, Brazil on July 25th, 2025.

Invited Talks

• Language Model Control as Tractable Probabilistic Reasoning

  Kareem Ahmed (University of California, Irvine, USA)

  Abstract: Despite their unprecedented abilities, language models (LMs) struggle to generate outputs that adhere to syntactic or semantic constraints. We show how the task of controlling LM outputs can be cast as a simple, yet intractable, probabilistic reasoning problem. To that end, we combine sampling with tractable inference to induce a locally faithful, tractable distribution over all sentences. We can efficiently condition this distribution on arbitrary logical constraints, allowing us to guide the LM toward constraint-satisfying outputs during finetuning, and guaranteeing constraint satisfaction during generation. We extend this idea to semantic constraints defined in terms of neural verifiers, where we leverage a verifier's gradient information to efficiently reason over all generations that satisfy the target attribute, enabling semantic control of LMs. Our experiments show that our approaches outperform the baselines across a range of tasks leading to fluent generations that are guaranteed to syntactically satisfy the constraint, while satisfying semantic constraints with very high probability.

• Imbalances in Neurosymbolic Learning: Characterization and Mitigation

  Efthymia Tsamoura (Huawei Labs)

  Abstract: This talk considers one of the most popular neurosymbolic learning settings in which a neural classifier is learned assuming access to (1) a vector of inputs and (2) the result of applying a symbolic component, e.g., a logical theory, over the gold labels of the inputs. The gold labels themselves are hidden from the learner. This setting has been one of the most well-studied ones in the context of neurosymbolic learning, finding numerous applications, including fine-tuning language models, aligning video to text, and learning knowledge graph embeddings. We study, for the first time, an unexplored topic in this setting: the characterization and mitigation of learning imbalances, i.e., significant differences in errors that occur when classifying instances of different classes. Our theoretical analysis reveals a unique phenomenon: that the symbolic component can greatly impact learning imbalances. This result sharply contrasts with previous research on supervised and weakly supervised learning, which suggests that learning imbalances are only due to data imbalances. On the practical side, we introduce algorithms that mitigate imbalances at training and testing time by treating the marginal of the hidden labels as a constraint. Our empirical analysis shows that our techniques lead to performance improvements of up to 14% over strong baselines from the neurosymbolic and long-tailed learning literature.

• Structured and Unstructured Sparsity in Probabilistic Neurosymbolic AI

  Pedro Zuidberg Dos Martines (Örebro University, Sweden)

  Abstract: In order to perform learning, data has to exhibit a certain degree of sparsity. In neurosymbolic AI this sparsity is expressed explicitly via formal languages. In this talk I will sketch how such sparsity can be exploited efficiently using modern AI hardware in the context of probabilistic neurosymbolic AI.

• Credal Probabilistic Circuits

  Cassio De Campos (TU Eindhoven, NL)

  Abstract: Probabilistic Circuits (PCs) are versatile models that encode a joint probability distribution with the use of a discrete latent space. They allow for tractable computations of multiple types of queries. Credal PCs extend PCs by allowing the specification of convex sets of parameters, thus representing a set of PCs sharing the same structure. They have been shown useful in the analysis of robustness of inferences, treatment of complicated missing data, and improvement of privacy. We look at their capabilities for large-scale uncertainty treatment, discussing model properties and algorithms. Finally, we dive into the possibility of extending these models to leverage continuous latent spaces.