Login Register
Login Register

We’re introducing AutumnSynth, an algorithm that synthesizes the source code of simple 2D video games from a small amount of observed video data. This represents a step forward toward systems that can perform causal theory discovery in real-world environments.

In light of this, we’re developing techniques of program synthesis—algorithms that generate programs—for theory discovery. Can we create an algorithm that takes in some observations and automatically generates the program (i.e. code) that produced them? Such a program would be a model or theory of the data in the sense that it describes the mechanisms or laws which govern how the data came about. The programming languages community, and more recently the ML community, has explored automated program synthesis and produced instances of human-level programming capabilities, e.g. in Sketch4, AlphaCode5, and AlphaTensor6. However, these systems are designed to be tools to improve programmer productivity, whereas our goal is to advance toward novel scientific discovery by producing algorithms that can perform theory discovery broadly.

In collaboration with Columbia, MIT, and Stanford, our latest paper7 presented at POPL 2023 develops a new algorithm, AutumnSynth, that can synthesize programs in a new programming language called Autumn. We show that AutumnSynth can generate the source code for a novel suite of interactive grid-world games. Autumn is a reactive language—it allows us to succinctly express the temporal dynamics of objects and specify changes that occur as consequences of events. Solving the problem of causal theory discovery in the context of Autumn games, which are visually simple yet contain complex causal dependencies, is an important step toward systems that can learn to discover causal theories in complex real-world environments.

Leave a Reply

Related

t2m-gpt

Text-to-motion is a thing now, demo on Hugging Face

This space uses T2M-GPT models based on Vector Quantised-Variational AutoEncoder (VQ-VAE) and Generative Pre-trained Transformer (GPT) for human motion generation from textural descriptions.

  • 27
  • 111
  • 5
programmingld

Measuring the impact of programming language distribution

Current benchmarks for evaluating neural code models focus on only a small subset of programming languages, excluding many popular languages such as Go or Rust. To ameliorate this issue, we present the BabelCode framework for execution-based evaluation of any benchmark in any language. BabelCode enables new investigations into the qualitative performance of models’ memory, runtime, […]

  • 60
  • 101
mPLUG-2

mPLUG-2 – A new unified paradigm with modularized design for multi-modal pretraining

Recent years have witnessed a big convergence of language, vision, and multi-modal pretraining. In this work, we present mPLUG-2, a new unified paradigm with modularized design for multi-modal pretraining, which can benefit from modality collaboration while addressing the problem of modality entanglement. In contrast to predominant paradigms of solely relying on sequence-to-sequence generation or encoder-based […]

  • 18
  • 98

Help us find great AI content

Newsletter

Never miss a thing! Sign up for our AI Hackr newsletter to stay updated.

About

AI curated tools and resources. Find the best AI tools, reports, research entries, writing assistants, chrome extensions and GPT tools.

Navigation
Follow

Submit