[TOC]
↗ Language & Literature ↗ Mathematics
↗ Logic (and Critical Thinking) ↗ Mathematical Logic (Foundations of Mathematics)
↗ Mechanized (Formal) Reasoning & Automated Reasoning (Inference)
↗ RLM (Reasoning Language Model) & LRM (Large Reasoning Model) ↗ LLM Agentic Reasoning
🔗 https://en.wikipedia.org/wiki/Knowledge_representation_and_reasoning
Knowledge representation (KR) aims to model information in a structured manner to formally represent it as knowledge in knowledge-based systems whereas knowledge representation and reasoning (KRR, KR&R, or KR²) also aims to understand, reason, and interpret knowledge. KRR is widely used in the field of artificial intelligence (AI) with the goal to represent information about the world in a form that a computer system can use to solve complex tasks, such as diagnosing a medical condition or having a natural-language dialog. KR incorporates findings from psychology about how humans solve problems and represent knowledge, in order to design formalisms that make complex systems easier to design and build. KRR also incorporates findings from logic to automate various kinds of reasoning.
Traditional KRR focuses more on the declarative representation of knowledge. Related knowledge representation formalisms mainly include vocabularies, thesaurus, semantic networks, axiom systems, frames, rules, logic programs, and ontologies. Examples of automated reasoning engines include inference engines, theorem provers, model generators, and classifiers.
In a broader sense, parameterized models in machine learning — including neural network architectures such as convolutional neural networks and transformers — can also be regarded as a family of knowledge representation formalisms. The question of which formalism is most appropriate for knowledge-based systems has long been a subject of extensive debate. For instance, Frank van Harmelen et al. discussed the suitability of logic as a knowledge representation formalism and reviewed arguments presented by anti-logicists. Paul Smolensky criticized the limitations of symbolic formalisms and explored the possibilities of integrating it with connectionist approaches.
More recently, Heng Zhang et al. have demonstrated that all universal (or equally expressive and natural) knowledge representation formalisms are recursively isomorphic. This finding indicates a theoretical equivalence among mainstream knowledge representation formalisms with respect to their capacity for supporting artificial general intelligence (AGI). They further argue that while diverse technical approaches may draw insights from one another via recursive isomorphisms, the fundamental challenges remain inherently shared.
📖 Artificial Intelligence: A Modern Approach, 4th ed. RUSSELL & NORVIG Chapter 7
The central component of a knowledge-based agent is its knowledge base, or KB. A knowledge base is a set of sentences. (Here “sentence” is used as a technical term. It is related but not identical to the sentences of English and other natural languages.) Each sentence is expressed in a language called a knowledge representation language and represents some assertion about the world. When the sentence is taken as being given without being derived from other sentences, we call it an axiom.
There must be a way to add new sentences to the knowledge base and a way to query what is known. The standard names for these operations are TELL and ASK, respectively. Both operations may involve inference—that is, deriving new sentences from old. Inference must obey the requirement that when one ASKs a question of the knowledge base, the answer should follow from what has been told (or TELLed) to the knowledge base previously. Later in this chapter, we will be more precise about the crucial word “follow”. For now, take it to mean that the inference process should not make things up as it goes along.
Figure 7.1 shows the outline of a knowledge-based agent program. Like all our agents, it takes a percept as input and returns an action. The agent maintains a knowledge base, KB, which may initially contain some background knowledge.
Each time the agent program is called, it does three things. First, it TELLs the knowledge base what it perceives. Second, it ASKs the knowledge base what action it should perform. In the process of answering this query, extensive reasoning may be done about the current state of the world, about the outcomes of possible action sequences, and so on. Third, the agent program TELLs the knowledge base which action was chosen, and returns the action so that it can be executed.
The details of the representation language are hidden inside three functions that implement the interface between the sensors and actuators on one side and the core representation and reasoning system on the other. MAKE-PERCEPT-SENTENCE constructs a sentence asserting that the agent perceived the given percept at the given time. MAKE-ACTION-QUERY constructs a sentence that asks what action should be done at the current time. Finally, MAKE-ACTION-SENTENCE constructs a sentence asserting that the chosen action was executed. The details of the inference mechanisms are hidden inside TELL and ASK. Later sections will reveal these details.
The agent in Figure 7.1 appears quite similar to the agents with internal state described in Chapter 2. Because of the definitions of TELL and ASK, however, the knowledge-based agent is not an arbitrary program for calculating actions. It is amenable to a description at the knowledge level, where we need specify only what the agent knows and what its goals are, in order to determine its behavior.
For example, an automated taxi might have the goal of taking a passenger from San Francisco to Marin County and might know that the Golden Gate Bridge is the only link between the two locations. Then we can expect it to cross the Golden Gate Bridge because it knows that that will achieve its goal. Notice that this analysis is independent of how the taxi works at the implementation level. It doesn’t matter whether its geographical knowledge is implemented as linked lists or pixel maps, or whether it reasons by manipulating strings of symbols stored in registers or by propagating noisy signals in a network of neurons.
A knowledge-based agent can be built simply by TELLing it what it needs to know. Starting with an empty knowledge base, the agent designer can TELL sentences one by one until the agent knows how to operate in its environment. This is called the ==declarative approach== to system building. In contrast, the ==procedural approach== encodes desired behaviors directly as program code. In the 1970s and 1980s, advocates of the two approaches engaged in heated debates. We now understand that a successful agent often combines both declarative and procedural elements in its design, and that declarative knowledge can often be compiled into more efficient procedural code.
==We can also provide a knowledge-based agent with mechanisms that allow it to learn for itself.== These mechanisms, which are discussed in Chapter 19, create general knowledge about the environment from a series of percepts. A learning agent can be fully autonomous.
↗ Logic (and Critical Thinking) ↗ Mathematical Logic (Foundations of Mathematics)
↗ Mechanized (Formal) Reasoning & Automated Reasoning (Inference)
↗ Logic Programs & Symbolic AI
↗ Uncertain Knowledge & Probabilistic Reasoning (Decision Making)
↗ Uncertain Knowledge & Probabilistic Reasoning (Decision Making) ↗ Game Theory & Decision Making in Multi-Agents Environments
↗ Probabilistic Models (Distributions) & Stochastic Process
↗ Automated Planning and Scheduling (APS) & AI Planning
↗ Reinforcement Learning (RL) & Sequential Decision Making
[!links] ↗ Philosophy & Its History
↗ Inferential Statistics (Analysis) & Statistical Inference ↗ Causal Inference in Statistics
🔗 https://probml.github.io/pml-book/book2.html Probabilistic Machine Learning: Advanced Topics | 36. Causality by Kevin Patrick Murphy.
MIT Press, 2023.
📖 Artificial Intelligence: A Modern Approach, 4th ed. RUSSELL & NORVIG Chapter 10
In which we show how to represent diverse facts about the real world in a form that can be used to reason and solve problems.
The previous chapters showed how an agent with a knowledge base can make inferences that enable it to act appropriately. In this chapter we address the question of what content to put into such an agent’s knowledge base—how to represent facts about the world. We will use first-order logic as the representation language, but later chapters will introduce different representation formalisms such as hierarchical task networks for reasoning about plans (Chapter 11), Bayesian networks for reasoning with uncertainty (Chapter 13), Markov models for reasoning over time (Chapter 16), and deep neural networks for reasoning about images, sounds, and other data (Chapter 22). But no matter what representation you use, the facts about the world still need to be handled, and this chapter gives you a feeling for the issues.
Section 10.1 introduces the idea of a general ontology, which organizes everything in the world into a hierarchy of categories. Section 10.2 covers the basic categories of objects, substances, and measures; Section 10.3 covers events; and Section 10.4 discusses knowledge about beliefs. We then return to consider the technology for reasoning with this content: Section 10.5 discusses reasoning systems designed for efficient inference with categories, and Section 10.6 discusses reasoning with default information.
↗ Logic Programs & Symbolic AI
↗ Decision Trees (CART) ↗ Rule-Based Systems
↗ Probabilistic Graphical Models ↗ Artificial Neural Networks (ANN) & Deep Learning Methods
↗ Uncertain Knowledge & Probabilistic Reasoning (Decision Making) ↗ Statistical (Data-Driven) Learning & Machine Learning (ML)
↗ Uncertain Knowledge & Probabilistic Reasoning (Decision Making)
📖 Artificial Intelligence: A Modern Approach, 4th ed. RUSSELL & NORVIG IV Uncertain Knowledge and Reasoning
↗ Statistical (Data-Driven) Learning & Machine Learning (ML) "knowledge in learning" ↗ Artificial Neural Networks (ANN) & Deep Learning Methods
↗ AI Embodiment & World Model (WM)
📖 Artificial Intelligence: A Modern Approach, 4th ed. RUSSELL & NORVIG Chapter 20
In which we examine the problem of learning when you know something already.
In all of the approaches to learning described in the previous chapter, the idea is to construct a function that has the input–output behavior observed in the data. In each case, the learning methods can be understood as searching a hypothesis space to find a suitable function, starting from only a very basic assumption about the form of the function, such as “second-degree polynomial” or “decision tree” and perhaps a preference for simpler hypotheses. Doing this amounts to saying that before you can learn something new, you must first forget (almost) everything you know. ==In this chapter, we study learning methods that can take advantage of prior knowledge about the world==. In most cases, the prior knowledge is represented as general first-order logical theories; thus for the first time we bring together the work on knowledge representation and learning.