Constitutional AI
Constitutional AI (CAI) is a training methodology developed by Anthropic that aligns AI models with human values by having the AI evaluate and revise its own outputs against a written set of principles — a 'constitution' — rather than relying exclusively on human-labeled preference data.
Comprendre Constitutional AI
Introduced by Anthropic in 2022, Constitutional AI was designed to address scalability limitations of RLHF: as models become more capable, human evaluators may struggle to reliably judge which outputs are better. CAI replaces some human feedback with AI feedback: the model is prompted to critique its own responses against a constitution of principles (e.g., 'Is this response harmful?', 'Is this response honest?') and then revise them.
The process has two main phases. In supervised learning, the model generates responses, critiques them against constitutional principles, and revises them — creating a synthetic dataset of improved responses. In RL from AI Feedback (RLAIF), a separate AI model is trained as a preference model using AI-generated comparisons rather than human comparisons, which is then used to fine-tune the base model with reinforcement learning.
The 'constitution' itself is a human-authored document: a list of principles that describe what the AI should and should not do. Anthropic's constitution draws from sources including the UN Declaration of Human Rights and existing AI ethics frameworks. By encoding values explicitly in language rather than implicitly through human preference ratings, CAI makes the alignment process more interpretable and adjustable.
Constitutional AI is most associated with Claude, Anthropic's family of AI models. It complements rather than replaces RLHF — most deployed models use both techniques.
Comment GAIA utilise Constitutional AI
GAIA runs on leading models available through its supported providers (Gemini, OpenRouter, OpenAI, or a custom endpoint). Techniques like Constitutional AI shape how those underlying models behave — including refusing harmful actions or protecting user privacy — which carries through to how GAIA acts when it manages sensitive personal data across email, calendar, and task systems.
Concepts liés
Reinforcement Learning from Human Feedback (RLHF)
Reinforcement Learning from Human Feedback (RLHF) is a machine learning technique that trains AI models to produce outputs preferred by humans by learning from human-provided rankings or ratings rather than purely from raw data.
Humain dans la boucle
L'humain dans la boucle (HITL) est un modèle de conception dans lequel un système IA inclut une supervision et une validation humaines à des points de décision clés, garantissant que les actions sensibles ou à fort impact nécessitent une confirmation humaine avant exécution.
Modèle de langage volumineux (LLM)
Un modèle de langage volumineux (LLM) est une intelligence artificielle entraînée sur d'énormes quantités de textes et capable de comprendre, de générer et de raisonner sur le langage humain avec une remarquable aisance.
Ajustement fin
L'ajustement fin est le processus qui consiste à reprendre l'entraînement d'un modèle d'IA pré-entraîné sur un jeu de données plus petit et spécifique à une tâche afin d'adapter son comportement à un domaine ou une application particuliers.
Agent IA
Un agent IA est un système logiciel autonome qui perçoit son environnement, raisonne sur les actions à entreprendre et prend des mesures pour atteindre des objectifs spécifiques sans intervention humaine continue.
