SymSeqBench: a unified framework for the generation and analysis of rule-based symbolic sequences and datasets

JuSER (Forschungszentrum Jülich)

Abstract

Sequential structure is a key feature of multiple domains of natural cognition and behavior, such as language, movement and decision-making. Likewise, it is also a central property of tasks to which we would like to apply artificial intelligence. It is therefore of great importance to develop frameworks that allow us to evaluate sequence learning and processing in a domain agnostic fashion, whilst simultaneously providing a link to formal theories of computation and computability. To address this need, we introduce two complementary software tools: SymSeq, designed to rigorously generate and analyze structured symbolic sequences, and SeqBench, a comprehensive benchmark suite of rule-based sequence processing tasks to evaluate the performance of artificial learning systems in cognitively relevant domains. In combination, SymSeqBench offers versatility in investigating sequential structure across diverse knowledge domains, including experimental psycholinguistics, cognitive psychology, behavioral analysis, neuromorphic computing and artificial intelligence. Due to its basis in Formal Language Theory (FLT), SymSeqBench provides researchers in multiple domains with a convenient and practical way to apply the concepts of FLT to conceptualize and standardize their experiments, thus advancing our understanding of cognition and behavior through shared computational frameworks and formalisms. The tool is modular, openly available and accessible to the research community.

Authors 8

1. Barna Zajzon
2. Younes Bouhadjar

   Forschungszentrum Jülich

   Affiliation as printed

   Peter-Grunberg Institute , Neuromorphic Software Ecosystems (PGI-15) , Jülich Research Centre , Germany

3. Maxime Fabre
4. Felix Schmidt Aachen

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Peter-Grunberg Institute , Neuromorphic Software Ecosystems (PGI-15) , Jülich Research Centre , Germany

   RWTH Aachen University , Aachen , Germany

5. Noah Ostendorf Aachen

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Institute for Advanced Simulation (IAS-6) , Jülich Research Centre , Germany

   RWTH Aachen University , Aachen , Germany

6. Emre Neftci Aachen Department of Electrical Engineering and Information Technology

   Forschungszentrum Jülich · RWTH Aachen University

   Affiliation as printed

   Department of Electrical Engineering and Information Technology , RWTH Aachen University , Aachen , Germany

   Peter-Grunberg Institute , Neuromorphic Software Ecosystems (PGI-15) , Jülich Research Centre , Germany

7. Abigail Morrison
8. Duarte Renato