Talk outline: Imagine a machine, equipped with various sensors, receiving a stream of sensory information. Somehow, it must make sense of that sensory stream. But what, exactly, does “making sense” involve, and how can it be implemented in a machine? I argue that making sense of a sensory stream involves constructing objects that persist over time, with properties that change over time according to intelligible universal laws (represented as logic programs). Thus, making sense is a form of program synthesis - but it is unsupervised program synthesis from raw unstructured input. I will describe our system, the Apperception Engine, that was designed to make sense of raw sensory input by constructing programs that explain that input. I will show various examples of our system in action. This talk will draw on two of our recent papers published in the Artificial Intelligence Journal: “Making Sense of Sensory Input”, and the sequel “Making Sense of Raw Input”.
Talk outline: I’ll describe our efforts to establish a complete neuro-symbolic paradigm and its progress toward some open problems of AI, vs deep learning (DL) alone.
1) I’ll introduce the Logical Neural Network (LNN) model framework, and new first-of-a-kind theoretical results which address the need for provably correct and tractable full first-order logical inference with realistic (uncertain, partial and even inconsistent) knowledge while also maintaining the full learning capabilities of a modern neural network. Upon this foundation, we build effective learning of logical formulae, in ways that achieve more compactness for interpretability, and can utilize temporal relationships. More broadly, I’ll discuss our larger emerging theory around learning the compositional structure underlying a dataset and its implications for fundamentally reducing sample complexity.
2) I’ll describe our progress toward an alternative strategy for NLP which this framework enables, which is to translate sequences of words into logic statements representing their underlying semantics, using our state-of-the-art technologies for semantic parsing, entity linking and relation linking. While a difficult road, it holds the promise of “true” understanding of language, in particular when quizzed on examples far from what was observed in training data. The approach already reaches or exceeds the state of the art in question answering benchmarks, and more importantly can answer questions that are not otherwise answerable without reasoning. I’ll also show how we can use our machinery to constrain text generation by large language models to stay consistent with a chosen set of reference facts or policy constraints.
3) Finally, I’ll show how we can use the framework to realize a sequential decision making scheme which can leverage the aforementioned reasoning (for planning-like capability) and the aforementioned learning (for RL-like capability) in concert to perform more efficiently in very difficult open-ended domains with infinite actions such as text-based interactive fiction games (like Zork). In order to address the need for a rich world model for bootstrapping new applications in general, I’ll introduce our Universal Logic and Knowledge Base (ULKB), a first federation/alignment of all of the major public linguistic and ontological knowledge graphs comprising about 1 biliion facts, with a logical foundation in simple type theory. Putting these together with our NLP machinery, we show the ability to solve games that typical DL approaches are unable to complete.
Talk outline: AI can enhance programming experiences for a diverse set of programmers: from professional developers and data scientists (proficient programmers) who need help in software engineering and data wrangling, all the way to spreadsheet users (low-code programmers) who need help in authoring formulas, and students (novice programmers) who seek hints when stuck with their programming homework. To communicate their need to AI, users can express their intent explicitly—as input-output examples or natural-language specification—or implicitly—where they encounter a bug (and expect AI to suggest a fix), or simply allow AI to observe their last few lines of code or edits (to have it suggest the next steps).
The task of synthesizing an intended program snippet from the user’s intent is both a search and a ranking problem. Search is required to discover candidate programs that correspond to the (often ambiguous) intent, and ranking is required to pick the best program from multiple plausible alternatives. This creates a fertile playground for combining symbolic-reasoning techniques, which model the semantics of programming operators, and machine-learning techniques, which can model human preferences in programming. Recent advances in large language models like Codex offer further promise to advance such neuro-symbolic techniques.
Finally, a few critical requirements in AI-assisted programming are usability, precision, and trust; and they create opportunities for innovative user experiences and interactivity paradigms. In this talk, I will explain these concepts using some existing successes, including the Flash Fill feature in Excel, Data Connectors in PowerQuery, and IntelliCode/CoPilot in Visual Studio. I will also describe several new opportunities in AI-assisted programming, which can drive the next set of foundational neuro-symbolic advances.
Talk outline: Turing considered instructing machines by programming, but also envisaged ‘child’ machines that could be educated by learning. Today, we have very sophisticated programming languages and very powerful machine learning algorithms, but can we really instruct machines in an effective way? In this talk I claim that we need better prior alignment between machines and humans for machines to do what humans really want them to do –with as little human effort as possible. First, I’ll illustrate the reason why very few examples can be converted into programs in inductive programming and machine teaching. In particular, I’ll present a new teaching framework based on minimising the teaching size (the bits of the teaching message) rather than the classical teaching dimension (the number of examples). I’ll show the somewhat surprising result that, in Turing-complete languages, when using strongly aligned priors between teacher and learner, the size of the examples is usually smaller than the size of the concept to teach. This gives us insights into the way humans should teach machines, but also the way machines should teach humans, what is commonly referred to as explainable AI. Second, I’ll argue that the shift from teaching dimension to teaching size reconnects the notions of compression and communication, and the primitive view of language models, as originally introduced by Shannon. Nowadays, large language models have distilled so much about human priors that they can be easily queried with natural language ‘prompts’ combining a mixture of textual hints and examples, leading to ‘continuations’ that do the trick without any program or concept representation. The expected teaching size for a distribution of concepts presents itself as a powerful instrument to understand the general instructability of language models for a diversity of tasks. With this understanding, ‘prompting’ can properly become a distinctively new paradigm for instructing machines effectively, yet deeply intertwined with programming, learning and teaching.
Talk outline: Although AI and Machine Learning have significantly impacted science, technology, and economic activities several research questions remain challenging to further advance the agenda of trustworhty AI. Researchers have show that there is a need for AI and machine learning models that soundly integrate logical reasoning and machine learning. Neurosymbolic AI aims to bring together effective machine learning models and the logical essence of reasoning in AI. In recent years, technology companies have organized research groups toward the development of neurosymbolic AI technologies, as contemporary AI systems require sound reasoning and improved explainability. In this presentation, I address how neurosymbolic AI evolved over the last decades. I also address the recent contributions within the field to building richer AI models and technologies and how neurosymbolic AI can contribute to improved AI explainability and trust.
Talk outline: Starting from a reflection on the evolution of computing and on the crucial role of language in human cognition, I propose the adoption of a computable language to serve as a semantic metadata protocol.
A candidate already exists to play this role: IEML (Information Economy Metalanguage) is composed of a compact dictionary of 3000 words and a fully regular grammar. This artificial language is philological, i.e. it can evoke any concept, translate natural languages and is self-defining. It is univocal, i.e. it has no homonyms or synonyms. It promotes narrativity, i.e. it allows for the evocation of scenes and stories - including causal explanations. It is recursive, i.e. its sentences can fit into each other. The act of reference (to data) is explicit and included in the grammar. It is self-referential, which means that it can refer to and comment on its own expressions. On a formal level, IEML is an abstract algebra based on a stack of symmetry structures. On the user interface level, it is manipulated from its translations into natural languages, it is visualized in the form of tables and graphs. In terms of efficiency, it allows the programming of nodes and semantic links, i.e. the semi-automatic generation of ontologies and other data structures.
The adoption of this protocol could multiply the power of artificial intelligence, bring deep learning, blockchain and metavers into synergy, ensure the interoperability of databases and applications of all kinds, and finally lay the technical foundations for the emergence of a reflexive collective intelligence.
Talk outline: We, as robot engineers, have to think hard about our role in the design of robots and how it interacts with learning, both in “the factory” (that is, at engineering time) and in “the wild” (that is, when the robot is delivered to a customer). I will share some general thoughts about the strategies for robot design and then talk in detail about some work I have been involved in, both in the design of an overall architecture for an intelligent robot and in strategies for learning to integrate new skills into the repertoire of an already competent robot.
Bio: Leslie is a Professor at MIT. She has an undergraduate degree in Philosophy and a PhD in Computer Science from Stanford, and was previously on the faculty at Brown University. She was the founding editor-in-chief of the Journal of Machine Learning Research. Her research agenda is to make intelligent robots using methods including estimation, learning, planning, and reasoning.
She is not a robot.