Languages

Languages: The Protocols of AD4M

What a Language is

In AD4M, a Language is a pluggable protocol: it defines how a piece of data is stored, retrieved, addressed, and — when it is a shared-state Language — how changes to that data propagate between agents.

"Language" is a deliberate metaphor. Just as human languages are the medium through which people share meaning, AD4M Languages are the medium through which digital agents share data. If a group of people wants to interact, they pick a shared language; if a group of AD4M agents wants to coordinate around some data, they pick a shared Language.

A Language is a small piece of code that the AD4M runtime loads into a sandboxed isolate and calls via a well-defined interface. Nothing else about the runtime is Language-specific. Swap the Language, and the same agent can suddenly speak a different protocol — IPFS instead of Holochain, a Git repo instead of a SQL database, an inbox instead of a shared wiki — without changing anything else in the stack.

Why Languages: universal addressing across protocols

An Expression in AD4M has a globally unique URL of the form:

<language-hash>://<language-specific-address>

The language-hash is the content hash of the Language's compiled bundle — so every URL carries, in its authority component, a pointer to the exact protocol code that can resolve it. Any AD4M runtime that does not already have the Language installed can fetch it by hash, verify it against the URL, install it, and then resolve the address. This lets AD4M span protocols the way HTTP spans servers:

  • A Git commit: QmGitLang…://abcdef1234…
  • An IPFS CID: QmIPFSLang…://QmZXY…
  • A wiki page: QmWikiLang…://article-42
  • A DID document: did:key:z6Mk… (handled by a special built-in case)
  • Your own protocol: <your-hash>://<your-address>

AD4M doesn't hardcode which protocols exist. Protocols are Languages, Languages can be published, and publishing a Language is just minting a new piece of code and letting its hash be its identity.

Capabilities, not a monolith

A Language is not a single object that must implement one big interface. It is a module that exports whichever capabilities it supports, and the runtime introspects the export table at load time to decide what it can do. The capabilities are independent:

CapabilityWhat it means
expressionMints and serves Expressions by address
perspective-commitWrites link diffs into a shared Perspective
perspective-queryAnswers queries (SPARQL, link patterns, by-author, Prolog)
perspective-syncFull-replica CRDT-style bidirectional convergence
peersMembership fabric (local + remote agents)
telepresenceEphemeral signals between online agents
language-sourceServes the bundle of other Languages (language-language)
ui, interactionsOptional UI hints for clients

A few concrete shapes:

  • A static expression store exports expression (get only).
  • A read-only knowledge graph exports perspective-query (no perspective-sync, no perspective-commit — the server has the data, the client queries it remotely).
  • A full-replica link Language (today's perspective-diff-sync) exports perspective-commit + perspective-sync + peers.
  • A DM inbox exports perspective-commit only — senders write, the inbox owner reads through a separate replication mechanism.
  • A write-only drop box exports perspective-commit alone.

The runtime enforces "presence = capability" at load time: it scans the module's exports and caches which capabilities the Language actually implements. Downstream, anything that wants to use capability X first checks whether the Language advertises it, and routes around it if not.

The interface

v1.0 of the interface is specified in two places that agree with each other by design:

Conceptually, a Language is an ES module. It exports what the runtime calls; it imports what the runtime provides.

Exports: what the runtime calls on the Language

// Lifecycle (always required)
export async function init(): Promise<void> { /* ... */ }
export async function teardown(): Promise<void> { /* ... */ }
 
// Discovery (statically resolvable, before init())
export const name = "my-language";
export const version = "1.0.0";
 
// Capability exports — include only the ones you implement
export async function expressionCreate(content: string): Promise<string> { /* ... */ }
export async function expressionGet(addr: string): Promise<Expression | null> { /* ... */ }
export async function perspectiveCommit(diff: PerspectiveDiff): Promise<void> { /* ... */ }
export async function perspectiveSync(): Promise<PerspectiveDiff> { /* ... */ }
// ... etc.
 
// Optional event handlers the runtime calls when external events arrive
export function handleHolochainSignal(signal: HolochainSignal): void { /* ... */ }
export function handleTelepresenceSignal(payload: any, recipientDid?: string): void { /* ... */ }

There is no create(context) factory; there is no object returned from anywhere. The module itself is the Language instance. Each instance lives in its own Deno isolate (Languages are mutually sandboxed), and module-level state (e.g. let currentRevision = …) is per-instance.

Imports: what the Language calls on the runtime

All runtime services come in via named imports from a single module specifier, ad4m:host:

import {
    // Agent identity
    agentDid,
    agentSign,
    agentCreateSignedExpression,
 
    // Language context
    languageAddress,
    languageSettings,
 
    // KV storage (core — always available)
    storageGet,
    storagePut,
    storageDelete,
    storageListKeys,
 
    // Event emission — Language pushes to runtime
    emitPerspectiveDiff,
    emitSyncStateChange,
    emitTelepresenceSignal,
    emitSignal,
 
    // Holochain (optional extension)
    holochainRegisterDnas,
    holochainCall,
 
    // HTTP fetch (for HTTP-backed Languages)
    httpFetch,
} from "ad4m:host";

The ad4m:host module is a synthetic module the runtime registers before loading the Language. The full contract — which globals a host must install so these imports work, and what each method does — is in host-contract.md (opens in a new tab).

Core vs. optional extensions

The import surface is explicitly layered:

  • Core — every AD4M-compliant runtime implements this. Agent identity, language context, KV storage, event emission, HTTP fetch, runtime utilities. You can write a Language using only core imports and it will run on Deno, a browser, Node, or any other runtime that conforms.
  • Extensions — optional. A runtime that does not ship an extension installs neither the globals nor the extension's methods; Languages that import extension functions throw a clear error at call time on such a runtime.

Current extensions:

ExtensionExports in ad4m:hostRuntime needs
holochain-extholochainCall, holochainRegisterDnas, …A Holochain conductor
storage-fs-extreadStorageFile, writeStorageFileA filesystem-like backend

A Language declares which extensions it needs by targeting the matching WIT world — ad4m-language (core only), ad4m-language-holochain, ad4m-language-fs, or ad4m-language-holochain-fs. A runtime that does not support an extension refuses to load a Language targeting its world.

Storage: prefer the KV, reach for files only if you must

The core KV (storageGet / storagePut / storageDelete / storageListKeys) is the portable way to persist per-Language data:

  • It never throws — on runtimes without a persistent backend it degrades to in-memory-only within an isolate.
  • It is namespace-scoped by languageAddress() automatically: two Language instances cannot read each other's keys.
  • It works unchanged on Deno, in a browser with IndexedDB, behind an HTTP API to a remote store, etc.

The File I/O extension is for the cases the KV genuinely cannot express: custom storage layouts, very large blobs, or shared paths outside per-Language scope (the two test-fixture persistence Languages use it, because they need to share a path across agents). Every other Language should use the KV.

Authoring a Language

The typical authoring flow uses the ALDK (AD4M Language Development Kit), which provides typed wrappers, trait-based APIs, and bundlers that wire ad4m:host correctly. Two packages, one per language:

Both resolve to the same runtime interface. A Language compiled from Rust and a Language written in TypeScript are indistinguishable from the runtime's point of view.

TypeScript with defineLanguage

defineLanguage({…}) is a pure transform that takes a grouped spec and emits the flat named exports the runtime expects.

// index.ts — minimal expression Language in TypeScript
import {
    defineLanguage,
    agentCreateSignedExpression,
    hash,
    storageGet,
    storagePut,
} from "@coasys/ad4m-ldk";
 
export default defineLanguage({
    name: "note-store",
    version: "1.0.0",
 
    async init() {
        // Language-specific setup. Runtime imports (e.g. languageSettings)
        // are callable here.
    },
 
    expression: {
        async create(content) {
            const expr = agentCreateSignedExpression(content);
            const s = JSON.stringify(expr);
            const address = hash(s);          // canonical AD4M content hash
            storagePut(address, s);
            return address;
        },
 
        async get(address) {
            const raw = storageGet(address);
            return raw ? JSON.parse(raw) : null;
        },
    },
});

The ALDK also re-exports the full ad4m:host import surface with types, so you get auto-complete on agentDid, holochainCall, emitPerspectiveDiff, etc., and the TypeScript compiler can check your usage.

Rust with ad4m_language!

The Rust ALDK gives you a trait-based API plus a macro that wires the wasm-bindgen exports automatically:

// lib.rs — minimal HTTP-backed expression Language in Rust
use ad4m_ldk::imports as rt;
use ad4m_ldk::prelude::*;
 
pub struct NoteStore;
 
impl Language for NoteStore {
    fn name()    -> &'static str { "note-store" }
    fn version() -> &'static str { "1.0.0" }
 
    async fn init() -> LanguageResult<Self> {
        Ok(NoteStore)
    }
}
 
impl ExpressionCapability for NoteStore {
    async fn expression_create(
        &mut self,
        content: serde_json::Value,
    ) -> LanguageResult<Address> {
        let signed = rt::agent_create_signed_expression_typed(&content);
        let body = serde_json::to_string(&content).unwrap_or_default();
        // In a real Language you'd POST to a storage backend, hash
        // locally, or call Holochain. Here we POST and use the server's
        // returned URL as the address.
        rt::http_post_json("https://example.com/notes", &signed)
            .await
            .map_err(|e| LanguageError::internal(format!("POST failed: {:?}", e)))
    }
 
    async fn expression_get(
        &mut self,
        address: Address,
    ) -> LanguageResult<Option<Expression>> {
        let body = rt::http_get(&format!("https://example.com/notes/{}", address))
            .await
            .map_err(|e| LanguageError::internal(format!("GET failed: {:?}", e)))?;
        Ok(serde_json::from_str(&body).ok())
    }
}
 
ad4m_language! {
    language: NoteStore,
    capabilities: [expression],
    holochain_signal: false,
}

The ad4m_language! macro emits the required #[no_mangle] shims for exactly the capabilities you list — the WASM export table is minimal and export-presence-as-capability detection works correctly. The runtime sees the same flat interface it would from a TypeScript Language; nothing about the Rust path is special-cased.

For a production reference, see bootstrap-languages/centralized-agent-language/rust-impl/src/lib.rs (opens in a new tab) — the HTTP-backed agent Language — or bootstrap-languages/agent-language/rust-impl/ (opens in a new tab) for a Holochain-backed one.

Publishing, installing, using

Nothing here changed with v1.0: the Language-language still stores and serves bundles, and the AD4M client API is unchanged.

// Publish: returns the language-hash that becomes your Language's address
const meta = await ad4m.languages.publish(
    "/path/to/bundle.js",
    { name: "note-store", description: "A simple note store" },
);
 
// Install on another node (or the same one later)
await ad4m.languages.install(meta.address);
 
// Use: all Expression operations go through the universal address format
const exprAddr = await ad4m.expression.create("Hello World", meta.address);
const expr     = await ad4m.expression.get(`${meta.address}://${exprAddr}`);

A published Language is just a file whose content hash is its identity. Anyone with the hash can fetch it, verify it hashes to what the URL claims, install it, and speak the protocol.

The core Languages

Every AD4M node boots with a small set of core Languages. These are the ones the rest of the system actually relies on — without them, an agent cannot participate in a Neighbourhood at all:

Core LanguageRole
language-languagePublishes and serves other Language bundles. The root of addressing.
agent-languageResolves DIDs to agent profile Expressions
neighbourhood-languageServes shared Neighbourhood definitions
perspective-diff-syncThe default full-replica link Language used by most Neighbourhoods

The two agent Language variants (DHT-backed and HTTP-backed) ship as Rust/WASM end-to-end, built with the Rust ALDK and wasm-bindgen; the rest are TypeScript-authored with the JS ALDK. Both paths produce bundles the runtime treats identically.

Where to look next

Templates

The following template repos are starting points for your own Languages. They currently target the pre-v1.0 interface; expect a v1.0 template update when the migration lands on main.

Security model

Languages run in per-instance Deno isolates, mutually sandboxed.

Filesystem. Each isolate has a permission allow-list scoped to its own storage directory (typically {app_data}/ad4m/languages/{hash}/). System Languages (the bootstrap set) additionally get read/write access to the executor's working directory so they can read bundle paths passed in via languagePublish. Everything outside those paths is off-limits and fails with a permission error.

Network. Languages can call httpFetch and reach Holochain; this is necessary because most Languages are network protocols. A future release will scope this more tightly — today a Language can reach any host the runtime can reach, so trust decisions around which Languages to install matter (see below). Each httpFetch call, including reading the response body, times out after 10 seconds and rejects with an error naming the URL.

Other capabilities. FFI, subprocess spawning, reading arbitrary environment variables, and loading arbitrary remote ES modules are all disabled. agentSign never exposes the private key — the Language receives only the signature bytes. agentCreateSignedExpression is the only signing path available to Language code, so the attack surface is narrow.

Identity. A Language's identity is the content hash of its bundle, so a tampered Language is trivially detectable: its hash stops matching its address. Trust decisions (will I install this Language? will I accept Expressions authored with it?) are made per-agent, at the edges, not by any central registry.

AgentsExpressions