Two Categories of Memory Standard Memory Types

• Summary Memory — How It Actually Works

Vector Memory Types

• Hybrid Memory — The Best of Both

Per-Call Multi-Tenant Identity Routing Document Loaders Building a Complete RAG Pipeline

• Step 1: Ingest
• Step 2: Query
• Step 3: Hybrid for Production

Token Management Multiple Memories Per Agent The aiPopulate() BIF — Structured Memory Without Live CallsWhat's Next

BoxLang AI 3.0 Series · Part 6 of 7

A chatbot with no memory isn't a conversation — it's a series of isolated queries. Every message starts from scratch. The user has to re-explain who they are, what they're working on, and what was just said. It's exhausting, and it signals that the AI isn't really listening.

Memory is what separates a useful AI application from a toy. BoxLang AI ships with one of the most comprehensive memory systems in any AI framework — 20+ memory types across two major categories, vector embedding support for semantic retrieval, 30+ document loaders for RAG pipelines, and a per-call identity routing system that makes multi-tenant applications safe by default.

This post is a complete tour.

🧠 Two Categories of Memory

           +-----------------------------------+
           |         BoxLang AI Memory         |
           +-----------------------------------+
                        /           \
                       /             \
                      v               v

+--------------------------------+   +--------------------------------+
|        Standard Memory         |   |         Vector Memory          |
+--------------------------------+   +--------------------------------+
| Stores conversation history    |   | Stores semantic knowledge      |
| Sequential message thread      |   | Embeddings + retrieval         |
| Retrieves by recency/order     |   | Retrieves by meaning           |
| Example: remember prior fact   |   | Example: RAG knowledge lookup  |
+--------------------------------+   +--------------------------------+

                      \               /
                       \             /
                        v           v

         +-------------------------------------------+
         | Shared abstraction and usage model        |
         +-------------------------------------------+
         | IAiMemory interface                       |
         | aiMemory() BIF                            |
         | Per-call identity routing                 |
         | Minimal app-code changes between both     |
         +-------------------------------------------+

BoxLang AI memory breaks into two fundamentally different categories, solving two different problems.

Standard Memory stores conversation history — the sequential messages between user and assistant. It's what lets the agent remember "my name is Luis" from three messages ago.

Vector Memory stores semantic knowledge — embeddings of documents, past conversations, or domain content that can be retrieved by meaning, not by recency. It's what enables RAG: "find the three most relevant passages from our knowledge base for this query."

Both categories share the same IAiMemory interface, the same aiMemory() BIF, and the same per-call identity routing — your application code barely changes between them.

📋 Standard Memory Types

Create any memory with our lovely global function: aiMemory( type, config: {} ). Our default memory type is a window memory of 20 messages:

// Window memory — keeps the last N messages
mem = aiMemory( "window", config: { maxMessages: 20 } )

// Summary memory — auto-summarizes old messages to preserve context
mem = aiMemory( "summary", config: {
    maxMessages      : 30,
    summaryThreshold : 15,
    summaryModel     : "gpt-4o-mini"
} )

// Cache memory — CacheBox-backed, distributed-friendly
mem = aiMemory( "cache", config: { cacheName: "aiMemory" } )

// Session memory — scoped to the current web session
mem = aiMemory( "session" )

// File memory — persisted to disk for audit trails
mem = aiMemory( "file", config: { filePath: "/logs/conversations/" } )

// JDBC memory — stored in a database for enterprise multi-user scenarios
mem = aiMemory( "jdbc", config: {
    datasource : "myDB",
    table      : "ai_conversations"
} )

Summary Memory — How It Actually Works

The summary type deserves special attention. When the message count exceeds summaryThreshold, it calls the configured LLM to produce a one-paragraph summary of the oldest messages, replaces them with that summary as a single system message, then continues accumulating. Conversation context survives without the token cost of carrying the full history.

agent = aiAgent(
    name   : "support-bot",
    memory : aiMemory( "summary", config: {
        maxMessages      : 40,    // keep up to 40 messages
        summaryThreshold : 20,    // summarize when we hit 20
        summaryModel     : "gpt-4o-mini"  // use a cheap model for summarization
    } )
)

🔍 Vector Memory Types

Vector memory stores embeddings and retrieves by semantic similarity — the right tool when "find relevant context" matters more than "recall what was said recently."

// In-memory vectors — development and small datasets
mem = aiMemory( "boxvector" )

// ChromaDB — Python-based vector store
mem = aiMemory( "chroma", config: {
    collection       : "support_docs",
    embeddingProvider: "openai",
    embeddingModel   : "text-embedding-3-small"
} )

// PostgreSQL pgvector — works with your existing Postgres
mem = aiMemory( "postgres", config: {
    datasource       : "myDB",
    table            : "ai_embeddings",
    embeddingProvider: "openai"
} )

// Pinecone — managed cloud vector DB
mem = aiMemory( "pinecone", config: {
    apiKey     : "${Setting: PINECONE_API_KEY not found}",
    index      : "knowledge-base",
    namespace  : "support"
} )

// OpenSearch — AWS OpenSearch or self-hosted
mem = aiMemory( "opensearch", config: {
    host             : "https://my-opensearch:9200",
    index            : "ai_embeddings",
    embeddingProvider: "openai"
} )

Full vector memory roster:

Hybrid Memory — The Best of Both

hybrid combines a recent message window with semantic vector retrieval — you get recency and relevance:

mem = aiMemory( "hybrid", config: {
    recentLimit   : 5,        // keep last 5 messages always
    semanticLimit : 5,        // add 5 semantically relevant past messages
    vectorProvider: "chroma"  // backed by ChromaDB
} )

For most production support-bot or assistant scenarios, hybrid is the sweet spot — recent context for coherence, semantic retrieval for depth.

🏢 Per-Call Multi-Tenant Identity Routing

This is the architectural feature that makes BoxLang AI memory extensible. Memory instances are stateless and safe to use as singletons — userId and conversationId route each operation to the correct isolated conversation. Or you can create memories with seeded identities if you want a specific agent with specific memory; your choice.

Every memory operation accepts optional identity arguments:

sharedMemory = aiMemory( "cache" )

// Operations are fully tenant-isolated
sharedMemory.add( message, userId: "alice", conversationId: "sess-1" )
sharedMemory.add( message, userId: "bob",   conversationId: "sess-2" )

// Retrieval is scoped — alice never sees bob's messages
aliceHistory = sharedMemory.getAll( userId: "alice", conversationId: "sess-1" )
bobHistory   = sharedMemory.getAll( userId: "bob",   conversationId: "sess-2" )

// Clear only alice's conversation
sharedMemory.clear( userId: "alice", conversationId: "sess-1" )

In practice, you pass identity through AiAgent.run() options and it flows automatically to all memory operations:

sharedAgent = aiAgent( name: "support", memory: sharedMemory )

// One agent instance, many concurrent users — fully safe
sharedAgent.run( "Hello, I need help with my order",    {}, { userId: "alice", conversationId: "sess-1" } )
sharedAgent.run( "What did I just ask about?",          {}, { userId: "alice", conversationId: "sess-1" } ) // remembers
sharedAgent.run( "Can you help me reset my password?",  {}, { userId: "bob",   conversationId: "sess-2" } ) // isolated

No per-user agent factories. No thread-local hacks. No shared-state concurrency bugs. One instance, many tenants.

📚 Document Loaders

Document loaders are the ingestion layer for RAG pipelines. They normalize content from 30+ source types into the Document format that vector memory understands.

// Load a single PDF
docs = aiDocuments(
    source : "/path/to/product-manual.pdf",
    config : { type: "pdf" }
).load()

// Load all Markdown files in a directory (recursively)
docs = aiDocuments(
    source : "/knowledge-base",
    config : {
        type       : "directory",
        recursive  : true,
        extensions : [ "md", "txt", "pdf" ]
    }
).load()

// Load a live web page
docs = aiDocuments(
    source : "https://boxlang.ortusbooks.com/getting-started/overview",
    config : { type: "http" }
).load()

// Load from a database query
docs = aiDocuments(
    source : "SELECT title, content FROM articles WHERE published = 1",
    config : { type: "sql", datasource: "myDB" }
).load()

// Crawl an entire website
docs = aiDocuments(
    source : "https://docs.mycompany.com",
    config : {
        type     : "webcrawler",
        maxPages : 200,
        delay    : 500
    }
).load()

Built-in loaders:

🔗 Building a Complete RAG Pipeline

Here's the full picture — ingest documents into vector memory, then use an agent with that memory to answer questions grounded in your content.

Step 1: Ingest

// Create vector memory backed by ChromaDB
vectorMemory = aiMemory( "chroma", config: {
    collection       : "company_knowledge",
    embeddingProvider: "openai",
    embeddingModel   : "text-embedding-3-small"
} )

// Ingest everything in one call
result = aiDocuments(
    source : "/knowledge-base",
    config : {
        type       : "directory",
        recursive  : true,
        extensions : [ "md", "txt", "pdf" ]
    }
).toMemory(
    memory  : vectorMemory,
    options : { chunkSize: 1000, overlap: 200 }
)

// Rich ingestion report
println( "Documents loaded : #result.documentsIn#" )
println( "Chunks created   : #result.chunksOut#" )
println( "Vectors stored   : #result.stored#" )
println( "Duplicates skipped: #result.deduped#" )
println( "Estimated cost   : $#result.estimatedCost#" )

The toMemory() method handles chunking via aiChunk(), embedding via the configured provider, deduplication, and storage — everything in one fluent call with a detailed report back.

Step 2: Query

// Agent with the same vector memory — retrieves relevant chunks automatically
agent = aiAgent(
    name        : "knowledge-assistant",
    description : "Expert on all company documentation and policies",
    memory      : vectorMemory
)

// The agent retrieves semantically relevant chunks and grounds its answer
response = agent.run(
    "What is our refund policy for enterprise customers?",
    {},
    { userId: "support-team", conversationId: "ticket-12345" }
)

When the agent runs, vector memory retrieves the most semantically similar document chunks for the query and injects them as context before the LLM call. The LLM answers based on your actual content — not hallucinations.

Step 3: Hybrid for Production

For most production RAG scenarios, hybrid memory beats pure vector:

// Combines short-term conversation memory with long-term semantic retrieval
productionMemory = aiMemory( "hybrid", config: {
    recentLimit   : 8,
    semanticLimit : 6,
    vectorProvider: "chroma",
    collection    : "company_knowledge"
} )

agent = aiAgent(
    name   : "enterprise-assistant",
    memory : productionMemory
)

The first 8 messages keep conversations coherent. The semantic layer ensures relevant documentation is always surfaced. Together they handle both "what did I just ask?" and "what does our policy say about X?"

🔧 Token Management

Two BIFs help you reason about context window usage:

// Count tokens before sending (approximate)
tokenCount = aiTokens( "This is the text I want to count", { method: "words" } )

// Chunk a large document for ingestion
chunks = aiChunk( largeText, {
    chunkSize : 1000,  // tokens per chunk
    overlap   : 200    // overlap between chunks for context continuity
} )

aiChunk() is used internally by toMemory(), but you can call it directly when building custom ingestion pipelines.

🏗️ Multiple Memories Per Agent

Agents can have multiple memory instances simultaneously — useful when you want different retention policies for different types of information:

agent = aiAgent(
    name   : "research-assistant",
    memory : [
        // Short-term: current conversation
        aiMemory( "window", config: { maxMessages: 20 } ),
        // Long-term: semantic knowledge base
        aiMemory( "chroma", config: {
            collection       : "research_papers",
            embeddingProvider: "openai"
        } )
    ]
)

// Add another memory dynamically
agent.addMemory( aiMemory( "file", config: { filePath: "/audit/" } ) )

All memories are read from and written to in parallel. Messages retrieved from all memories are merged before each LLM call.

📦 The aiPopulate() BIF — Structured Memory Without Live Calls

One often-overlooked feature: aiPopulate() fills a typed BoxLang class from JSON without making any LLM call. This is essential for caching and testing:

class CustomerProfile {
    property name="name"         type="string";
    property name="tier"         type="string";
    property name="openTickets"  type="numeric";
}

// From a live AI call
profile = aiChat(
    "Extract the customer profile from: John Doe, Gold tier, 3 open tickets",
    { returnFormat: new CustomerProfile() }
)

// Cache it as JSON
cachedJson = jsonSerialize( profile )

// Later — restore the typed object without another LLM call
restoredProfile = aiPopulate( new CustomerProfile(), cachedJson )
println( restoredProfile.getName() ) // "John Doe"

Perfect for: pre-populated test fixtures, cached AI extractions, converting existing JSON data to typed objects.

What's Next

In Part 7 — the final post in the series — we go deep on MCP: how to consume tools from any MCP server, how MCPTool proxies work, and how to expose your own BoxLang functions as an enterprise MCP server with full security, CORS, API key validation, and rate limiting.

📖 Full Documentation 🌐 BoxLang AI Site 📦Install Today: install-bx-module bx-ai 🫶Professional Support

← Previous

Next ->

The post BoxLang AI Deep Dive — Part 6 of 7: Memory Systems & RAG — Building AI That Remembers appeared first on foojay.

article: Exclusive: Why are Chinese AI models dominating open-source as Western labs step back?
authors: Dashveenjit Kaur, AI News
date: 2026-02-09
desc.: A shift in what AI models are being used and where the models are being produced.
category: opinion

article: Machines of Loving Grace
authors: Dario Amodei
date: 2024-10-01
desc.: Although the article is older, it remains relevant for any author aiming to sketch a future in which everything with AI goes right. In light of recent developments, which appear to follow a sigmoid curve rather than exponential growth (marked by stagnation, with current models reaching a point where another breakthrough is required), the trajectory looks more measured than initially anticipated. Although the author discusses multiple risks (grandiosity, market forces, propaganda, sci-fi-like expectations, etc.), he also highlights the bright sides and explores areas where current AI may prove genuinely helpful. The question remains whether the current state of affairs can truly guarantee progress, rather than causing damage through non-deterministic outcomes (education, industry, human creativity etc.).
category: opinion

article: The Urgency of Interpretability
authors: Dario Amodai
date: 2025-04-01
desc.: The author describes lessons learned from current AI development and adds multiple valuable thoughts and facts to consider when interacting with AI models. The main point is that progress in the underlying technology is inexorable, driven by forces too powerful to stop, but what matters is the way in which it unfolds. Accepting that the current evolution of LLM-based AI cannot be halted, the author expresses hope that it may still be guided (this fact affect not only entire industry but also human kind thoughs and perception of reality), much like a bus controlled by a steering wheel, and warns of the dangers of ignorance, illustrating this through several concrete examples.
category: opinion

article: From Delegates to Trustees: How Optimizing for Long-Term Interests Shapes Bias and Alignment in LLM
authors: Suyash Fulay, Jocelyn Zhu, Michiel Bakker (MIT)
date: 2025-10-14
desc.: The article addresses the question of 'behavioral cloning', specifically, how accurately LLMs reproduce individuals' expressed preferences. Large language models have demonstrated promising accuracy in predicting survey responses and policy preferences, which has fueled growing interest in their potential to represent human interests across various domains. Drawing on theories of political representation, the article highlights an underexplored design trade-off: whether AI systems should act as delegates, mirroring expressed preferences, or as trustees, acting in users' broader interests. Models may align well with users' short-term preferences while failing to account for their long-term interests. Studies further indicate greater bias in topics where consensus is lacking.
category: research

article: DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science
authors: Fan Shu, Yite Wang, Ruofan Wu, Boyi Liu, Zhewei Yao, Yuxiong He, Feng Yan
date: 2026-02-27
desc.: The article addresses the challenge posed by fast-growing demand for Large Language Models (LLMs) to tackle complex, multi-step data science tasks, which has created an urgent need for accurate benchmarking. Two major gaps are identified in existing benchmarks: (i) the lack of standardized, process-aware evaluation that captures instruction adherence and process fidelity, and (ii) the scarcity of accurately labeled training data. While highlighting that even capable models (Anthropic, OpenAI, etc.) may struggle in performance, the article introduces the DARE-bench benchmark alongside supervised fine-tuning as approaches that may improve outcomes in specific applications. Although the results appear promising, they retain considerable potential for further improvement, as accuracy is not yet guaranteed.
category: research

article: Do LLMs Benefit From Their Own Words?
authors: Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, Jacob Andreas (MIT, IBM Research)
date: 2026-02-27
desc.: The article aims to answer the question of whether preserving past assistant responses is more beneficial than harmful. The study uses in-the-wild, multi-turn conversations and compares standard (full-context) prompting with a user-turn-only prompting approach that omits all previous assistant responses, evaluated across three open reasoning models and one state-of-the-art model. Surprisingly, omitting past assistant responses does not negatively affect response quality in a large fraction of turns and may also reduce token length. The article concludes with a discussion of findings and directions for future research.
category: research

article: SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems
authors: Jialiang Fan, Weizhe Xu, Mengyu Liu, Oleg Sokolsky, Insup Lee, Fangxin Kong
date: 2026-02-27
desc.: Safety-critical task planning in robotic systems remains a significant challenge: classical planners suffer from poor scalability, reinforcement learning (RL)-based methods generalize poorly, and base large language models (LLMs) cannot guarantee safety. To address this gap, the article proposes SafeGen-LLM, a safety-generalizable large language model framework. As part of this contribution, a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints is introduced, along with Supervised Fine-Tuning (SFT) on those constraints. Although the results appear optimistic, with minimal safety violations observed across tested domains, the approach still requires further research in more complex robotic settings.
category: research

article: LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics
authors: Antoine Peyronnet, Fabian Gloeckle, Amaury Hayat
date: 2026-02-27
desc.: Existing benchmarks largely rely on static, hand-curated sets of contest or textbook-style problems as proxies for mathematical research. The article introduces a novel approach leveraging state-of-the-art models (GPT-5, Gemini 2.5, Gemini 3, Claude Opus 4.5, and DeepSeek-R) by extracting lemmas from arXiv and updating them dynamically. This results in a benchmark that can be refreshed regularly with new problems drawn directly from current mathematical research, while previous instances can be used for training without compromising future evaluations. This approach achieves 10–15% accuracy in theorem proving and opens a new frontier for future research. Although the process may appear fully automated, a human in the loop, such as the article's author or reviewer, remains critically necessary to produce high-quality inputs and to effectively use LLM models.The results also indicate that it is considerably easier for a model to validate an existing proof than to produce one.
category: research

article: Task Complexity Matters: An Empirical Study of Reasoning in LLMs for Sentiment Analysis
authors: Donghao Huang, Zhaoxia Wang
date: 2026-02-27
desc.: It is a well-established narrative that reasoning in large language models (LLMs) universally improves performance across language tasks. This article aims to test that claim through a comprehensive evaluation of 504 configurations across seven models, considering different reasoning architectures such as adaptive, conditional, and reinforcement-based approaches. The findings reveal that the effectiveness of reasoning is strongly task-dependent and degrades for simpler tasks. The article provides quantitative findings alongside error analysis and outlines directions for future research.
category: research

article: Benchmarking LLM Summaries of Multimodal Clinical Time Series for Remote Monitoring
authors: Aditya Shukla, Yining Yuan, Ben Tamo, Yifei Wang, Micky Nnamdi and others
date: 2026-03-02
desc.: Large language models (LLMs) can generate fluent clinical summaries of remote therapeutic monitoring time series, however, the impact of information bias on clinically significant events, such as sustained abnormalities, remains poorly understood. The article presents the Technology-Integrated Health Management (TIHM) framework to address these questions, introducing a protocol that measures abnormality recall, duration recall, and measurement coverage, while utilizing GPT-4o-mini as a proxy evaluator. Traditional models frequently exhibit near-zero abnormality recall, whereas the vision-based approach achieves the strongest event alignment, with 45.7% abnormality recall and 100% duration recall. These results underscore the need for event-aware evaluation methods in future research to ensure reliable clinical time-series summarization.
category: research

article: Full interview: Anthropic CEO responds to Trump order, Pentagon clash
authors: CBS News
date: 2026-02-28
desc.: Anthropic CEO Dario Amodei sat down with CBS News for an exclusive interview, hours after Defense Secretary Pete Hegseth declared the company a supply chain risk to national security, which restricts military contractors from doing business with the AI giant. Amodei called the move "retaliatory and punitive," and he said Anthropic sought to draw "red lines" in the government's use of its technology because "we believe that crossing those lines is contrary to American values, and we wanted to stand up for American values.". Response of the OpenAI striking a deal with Pentagon causes many questions.
category: youtube

article: Scary Agent Skills: Hidden Unicode Instructions in Skills ...And How To Catch Them
authors: Embrace The Red
date: 2026-02-11
desc.: Skills introduce common threats such as prompt injection, supply chain attacks, remote code execution (RCE), and data exfiltration, among others. This post discusses the fundamentals, highlights the most straightforward prompt injection vector, and demonstrates how a real Skill from OpenAI can be back-doored using invisible Unicode Tag code-points, a technique that certain models, including Gemini, Claude, and Grok, are known to interpret as instructions. From a security perspective, Skills present serious concerns, as they represent a typical supply chain risk with limited governance or security controls. The author identified that some Skills instruct the AI to embed API tokens directly in curl requests and similar constructs , a poor design practice. This means that credentials are passed through the LLM, making them susceptible to leakage and leaving them vulnerable to being overwritten by an attacker via indirect prompt injection.
category: tutorial

The post JC-AI Newsletter #14 appeared first on foojay.

While significant activity characterizes the AI field, many unresolved research, design, and implementation challenges continue to impact progress. Future advancement depends heavily on understanding the nature of these challenges to approach probabilistic problems from the appropriate directions. This JC-AI newsletter features insightful interviews with key figures in the field, enabling readers to ask the right questions and compare visions of an 'uncertain future' against current capabilities to maintain a grounded perspective.

article: Deep Researcher with Sequential Plan Reflection and Candidates Crossover (Deep Researcher Reflect Evolve)
authors: Saurav Prateek
date: 2026-01-28
desc.: This paper introduces Deep Researcher, a novel architecture that shifts the paradigm from latency-optimized parallel scaling to an accuracy-driven sequential refinement model. Within the development of Deep Research Agents (DRAs), two primary paradigms are considered, Parallel Scaling and Sequential Refinement. The Deep Researcher agent achieved an overall score of 46.21 on the Research Bench, demonstrating superior performance compared to existing agents, including Claude Researcher, Nvidia AIQ Research Assistant, Perplexity Research, Kimi Researcher, and Grok Deep Search. While these improvements are good, the field requires further research to address remaining challenges.
category: research

article: Manipulation in Prediction Markets: An Agent-based Modeling Experiment
authors: Bridget Smart, Ebba Mark, Anne Bastian, Josefina Waugh (University of Oxford)
date: 2026-01-28
desc.: The paper investigates the utilization of agentic systems in the economic field and their impact on prediction. First, the paper evaluates an agent-based model of a prediction market in which bettors with heterogeneous expertise, noisy private information, variable learning rates, and budgets observe the evolution of public opinion on a binary election outcome to inform their betting strategies in the market. The agentic system exhibits stability across experiments. The second area relates to experiments on how "whale" agents, a highly resourced minority with biased information, may distort market prices and for how long. The paper discusses interesting simulation results on how biased information may change the market from a long-term perspective.
category: research

article: Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents
authors: Qihao Wang, Yue Hu, Mingzhe Lu, Jiayue Wu, Yanbing Liu, Yuanmin Tang
date: 2026-01-28
desc.: While LLMs' ability to use external tools enables powerful real-world applications, current benchmarks focus on final accuracy rather than revealing the cognitive bottlenecks that limit their true capabilities. This paper presents a framework based on Cognitive Load Theory that aims to decompose tasks into two components: Intrinsic Load and Extraneous Load. The paper discusses performance inconsistencies as cognitive load increases, and demonstrates how the proposed framework enables the identification of capability boundaries in the examined examples.
category: research

article: Build a Prompt Learning Loop - SallyAnn DeLucia & Fuad Ali, Arize
authors: AI Engineer, Sally Ann Delucia, Fuad Alli (Arize)
date: 2026-01-06
desc.: This talk aims to provide ideas on how it is possible to improve LLM responses by using feedback loops. It's important to view this talk through the lens of current research results regarding the LLM hallucination phenomenon and other factors. The main reason to keep current research results in mind is to avoid ending up in an infinite loop of failure/error.
category: youtube

article: Stanford CS230 | Autumn 2025 | Lecture 8: Agents, Prompts, and RAG
authors: Stanford Online
date: 2025-11-11
desc.: For more information about Stanford’s Artificial Intelligence professional and graduate programs
category: youtube, tutorial

article: Developer Experience in the Age of AI Coding Agents – Max Kanat-Alexander, Capital One
authors: AiEngineer, Max Kanat-Alexander
date: 2025-12-23
desc.: It feels like every two weeks, the world of software engineering is being turned on its head. Are there any principles we can rely on that will continue to hold true, and that can help us prepare for the future, no matter what happens? Max uses research, data, and his 20+ years working in enterprise Developer Experience teams to talk through what we can do now that will prepare us for an agentic future, no matter what that future holds.
category: youtube, opinion

article: Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding
authors: Yifan Zhu, Huiqiang Rong, Haoran Luo
date: 2026-01-29
desc.: Hallucination is a recognized phenomenon in the LLM field that impacts applications such as Retrieval-Augmented Generation (RAG) and Reward Modeling (RM). This paper introduces Token-Guard, a self-checking mechanism designed to identify and control hallucinations at the token level. The experiments demonstrate improvements.
category: research

article: Reward Models Inherit Value Biases from Pretraining
authors: Brian Christian, Jessica A. F. Thompson, Elle Michelle Yang, Vincent Adam, Hannah Rose Kirk and others (University of Oxford, University Pompeu Farba)
date: 2026-01-28
desc.: Despite their importance in LLM alignment, reward models (RMs) remain under-researched. This paper provides evidence that RMs inherit biases from their base models, suggesting that the choice of an open-source model is a reflection of values as much as performance. The paper discusses limitations of experiments and offers avenues for future research.
category: research

article: Professor Geoffrey Hinton - AI and Our Future
authors: City of Hobart, Geoffrey Hinton
date: 2026-01-08
desc.: Professor Geoffrey Hinton, known as the "Godfather of AI", will discuss artificial intelligence - how it works, the risks it poses to our society, and how we might coexist with super-intelligent AI. Ideal for business leaders, creatives, researchers, educators, students and anyone curious about the future of intelligence and society.
category: opinion

article: Your MCP Server is Bad (and you should feel bad) - Jeremiah Lowin, Prefect
authors: AI Engineer, Jeremiah Lowin
date: 2026-01-12
desc.: Too many MCP servers are simply glorified REST wrappers, regurgitating APIs that were designed for SDKs rather than agents. This leads to confused LLMs, wasted tokens, and demonstrably poor performance. If you have ever pointed an MCP generator at an OpenAPI spec and called it a day, this talk is your wake-up call.
category: youtube

article: Frontier Models & AI | Sam Altman, CEO & Co-Founder, OpenAI
authors: Cisco
date: 2026-02-04
desc.: Although Sam Altman, CEO and Co-Founder of @OpenAI, explores ideas about future possibilities and potential developments, he is asked during the interview to align his vision with the current state of research and existing technological capabilities. The interview, however, does not present clear data demonstrating how Codex outperforms alternatives or what 'better' specifically means in this context. The responses to questions may appear to be non-deterministic in nature. The interview relies heavily on thoughts about an "undefined future" that would require a deterministically defined foundation. It is interesting how the interview examined frontier AI models and their implications for economies, institutions, and global systems.
category: opinion

article: How to build secure and scalable remote MCP servers
authors: Den Delimarsky (Microsoft)
date: 2025-07-25
desc.: The tutorial provides insights into how to build a reliable Model Context Protocol (MCP) server, enabling AI agents to connect to external tools. It covers several crucial areas and provides valuable resources and ideas for tackling the challenge.
category: tutorial

The post JC-AI Newsletter #13 appeared first on foojay.

Although there have been many small steps toward the field's evolution, it often feels that development is stagnating, applying known or slightly tweaked strategies to non-deterministic problems while expecting deterministic results. This includes the often misleading benchmarking strategies (deterministic) performed on synthetic datasets.

The first New Year edition of the JC-AI Newsletter aims to shed light on new approaches and movements in the field, including the directions of its evolution.

Let's jump in and happy reading!

article: Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response
authors: Aron Distelzweig, Yiwei Wang, Faris Janjoš and others
date: 2025-12-03
desc.: Autonomous driving, specifically decision-making, remains a significant challenge. While routine scenarios yield nearly perfect plans using multi-agent collaboration, dense urban traffic presents considerable difficulties, particularly for vehicle lane changes. This paper presents the Bayesian Iterative Best Response (BIReR) framework, which aims to unify motion prediction and planning based on game theory. The framework demonstrates an 11% improvement in lane change performance compared to classical approaches.
category: research

article: PBFuzz: Agentic Directed Fuzzing for PoV Generation
authors: Haochen Zeng, Andrew Bao, Jiajun Cheng, Chengyu Song
date: 2025-12-04
desc.: Proof-of-Vulnerability (PoV) input generation is a critical task in software security. Generating a PoV input requires solving two sets of constraints: (1) reachability constraints for reaching the vulnerable code location(s), and (2) triggering constraints for activating the target vulnerability. Despite dramatic advancements in the LLM field, fuzzing models struggle to solve these constraints effectively. This paper proposes the PBFuzz framework, composed of four layers and enabling property-based directed fuzzing. Although PBFuzz underperformed in several scenarios, it outperforms conventional fuzzers overall.
category: research

article: DSPy: The End of Prompt Engineering - Kevin Madura, AlixPartners Enhancement
authors: AI Engineer, Kevin Madura
date: 2026-01-08
desc.: Applications developed for enterprise environments need to be rigorous, testable, and robust. The same is true for AI-powered applications, but LLMs can make this challenging. In other words, users need to be able to program with LLMs, not just tweak prompts. This talk covers why DSPy may be all users need when building applications with LLMs. Although the talk dives into some real-world examples, the audience is encouraged to explore the DSPy tool themselves to determine whether it fits their particular needs.
category: youtube

article: From Vibe Coding To Vibe Engineering – Kitze, Sizzy
authors: AI Engineer, Ryan Florence
date: 2025-12-14
desc.: Web development has always moved in cycles of hype, from frameworks to tooling. With the rise of large language models, we're entering a new era of "vibe coding," where developers shape software through collaboration with Al rather than syntax. This talk explores what that means for the future of coding, especially in frontend development, and how it echoes the past while redefining what comes next.
category: youtube

article: The AI Bubble Should Have Never Existed In The First Place
authors: Will Lockett
date: 2025-12-07
desc.: The article elaborates on the existence of an AI bubble, arguing that so much money has been poured into AI that we have effectively bet the entire economy on its success. Regardless of whether an AI bubble exists or in what form, the article formulates valid points that should be taken into account when considering future developments.
category: opinion

article: We Let AI Run Our Office Vending Machine. It Lost Hundreds of Dollars
authors: The Wall Street Journal (Antropic)
date: 2025-12-18
desc.: In a research case study supported by Anthropic, the Claudius Agent was developed to manage vending machine operations. Testing revealed multiple exploitable vulnerabilities that allowed users to obtain goods without payment. Real-world trials consistently resulted in operational failures, with the system dispensing free products while automatically reordering inventory, a combination that would lead to bankruptcy in commercial-like deployment.
category: youtube

article: When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
authors: Yaqi Duan, Yichun Hu, Jiashuo Jiang
date: 2025-12-31
desc.: Inventory control (encompassing cash management, storage, order quantities, etc.) presents a stochastic control challenge where minor structural errors result in recurring costs. Direct interaction with LLM models may produce plausible yet systematically suboptimal or even inconsistent results. This paper proposes using LLMs not as problem solvers but as language interfaces to enhance optimization through a hybrid agentic approach.
category: research

article: Memory in LLMs: Weights and Activations - Jack Morris, Cornell
authors: AI Engineer, Jack Morris
date: 2025-12-29
desc.: This work examines memory mechanisms in large language models through the lens of weights and activations. Jack Morris addresses the limitations of current Large Language Models (LLMs) in handling niche, long-tail knowledge that falls outside their training data or beyond knowledge cutoffs. He critiques the reliance on massive context windows and Retrieval Augmented Generation (RAG), citing their high computational cost and latency due to the quadratic complexity of self-attention. The core thesis advocates for a third paradigm: training knowledge into weights, efficiently injecting specific knowledge directly into model parameters. This approach treats weights as a memory storage mechanism, conceptually distinct from the working memory represented by activations.
category: youtube

article: There are no new ideas in AI — only new datasets
authors: Jack Morris
date: 2025-07-06
desc.: This article provides a comprehensive overview of progress in the AI field over recent years. All four major breakthroughs in LLMs occurred because researchers unlocked new sources of data. The question remains: what will be the next breakthrough?
category: opinion

article: VL-JEPA: Joint Embedding Predictive Architecture for Vision-language
authors: Delong Chen, Mustafa Shukor, Theo Moutakanni, Willy Chung, Jade Yu, Tejaswi Kasarla, Allen Bolourchi, Yann LeCun, Pascale Fung
date: 2025-12-11
desc.: This paper introduces the Joint Embedding Predictive Architecture for Vision-Language models (VL-JEPA). Current Vision-Language Models (VLMs) are straightforward but inadequate for two main reasons. First, VLMs are expensive to develop. Second, real-time tasks involving live streaming video (e.g., live action tracking) require sparse and selective decoding. The paper empirically validates the advantages of this newly introduced approach against token-generative VLMs. VL-JEPA delivers consistently higher performance on zero-shot captioning and classification while improving inference-time efficiency during the training phase. Although improvements remain in the experimental stage, the work demonstrates clear benefits from scaling both parameters and dataset size.
category: research

article: Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
authors: Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly (Carnegie Mellon Univeristy, Apple)
date: 2024-01-29
desc.: Although this paper is older, it may shed light on the approaches chosen for training LLM models and provide better understanding of their evolution. The paper proposes Web Rephrase Augmented Pre-training (WRAP), which uses an off-the-shelf instruction-tuned model to rephrase noisy input data. It offers insights into how the structure of training data impacts LLM performance.
category: research

article: When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
authors: Laksh Advani
date: 2026-01-01
desc.: This paper investigates the reasoning performance of agentic systems based on small language models (Mistral-7B, Llama-3-8B, Qwen-2.5-7B). The findings reveal statistically significant evidence that RAG systems may improve reasoning performance while simultaneously increasing the likelihood of hallucination due to the Right-for-Wrong-Reason (RWR) phenomenon. The paper introduces the Reasoning Integrity Score (RIS) approach to identify hidden flaws in reasoning processes.
category: research

The post JC-AI Newsletter #12 appeared first on foojay.

At Quantexa, we love Scala. This may be the first dedicated article on Foojay.io about Scala. I hope my colleagues and I can add more in due course.

Scala is built on the Java Virtual Machine (JVM). For foojay.io readers, this is "stating the bleeding obvious," but it means it compiles to Java bytecode and runs on the same runtime environment as Java. This brings significant enterprise scalability, resilience and big data ecosystem benefits.

Scala is extremely popular in data engineering (e.g., Apache Spark), concurrent systems (e.g., Akka), and functional programming. It is a first class citizen for operating with Spark, while tools like Spark MLlib, Akka Streams, and Scala-based ETL frameworks make Scala a solid choice for end-to-end data engineering workflows. Through the JVM, it can interoperate with Hadoop, Kafka, Flink, Elasticsearch, Solr and more, those classics of the OpenJDK and JVM-based big data ecosystem.

At Quantexa, we extensively use Spark for batch processing. We also deploy Elasticsearch and OpenSearch for dynamic in-memory processing. Elsewhere, we use Python, e.g. for innovation and data science capabilities, but Scala is our production language of choice. It is the glue which binds our stack, and provides the foundation for our key capabilities. If you are enthusiastic about Scala, data engineering and/or Spark, Quantexa is a great place to work.

Quantexa's application layer is predicated on entity resolution, graphs and scoring capabilities. They underpin highly contextualized and accurate data and analytics products that service human-in-the-loop "Decision Intelligence" use cases. Quantexa's customers apply them for tangible outcomes, for example to detect financial and non-financial crimes, to perform know your customer (KYC) and to sharpen customer intelligence. They also deploy it in their enterprise layer to improve data management (through entity quality) upstream and populate AI pipelines with AI-ready data downstream.

Functional Programming Overview

In our experience, developers should seek to understand Scala in conjunction with Functional Programming.

While not unique to Scala, functional programming is a paradigm in which we try to bind everything in pure mathematical functions style. It is a declarative type of programming, where the main focus is on “what to solve” in contrast to an imperative style where the main focus is “steps to solve”. It uses expressions instead of statements - an expression is evaluated to produce a value whereas a statement is executed to assign variables. The Scala courses below will introduce you to the functional programming paradigm using Scala.

Some best practice guidelines:

• use sensible and explicit variable names
• its all about readability - your code should tell a story
• write functional reusable code - if you are repeating a lot of code, there is probably a better way to write it
• error handling is there to handle edge cases and help with debugging

Online Scala Courses

The following courses will give you a good foundation in Scala. It should take around 1-2 days to understand the basics.

The first is taught by Martin Odersky, the creator of the Scala language. From this, you will gain a knowledge base of essential syntax and concepts.

The second is half theory, half practice with hands on coding exercises. It provides a balanced and thorough introduction to the whole language and its concepts, as well as the practical skills to code in Scala. There is a small fee attached to this course. Course 1 should suffice, but if you wish to build on your knowledge and apply it to some more practical examples, you can undertake the second course

The two Scala courses are listed below.

Scala Course 1: Functional Programming Principles in Scala taught by Martin Odersky
Functional Programming Principles in Scala

This course is offered by the École Polytechnique Fédérale de Lausanne (EPFL). You will need to create an id to log on, but from that point you can 'audit' the course without doing the exercises. Once in, navigate past the setup pages to the first video of Martin Odersky. Then, we recommend reviewing these sections.

1. Getting Started
Week 1 - Introduction 1m
Lecture 1.1 - Programming Paradigms 2m
Lecture 1.2 - Elements of Programming 12m
Lecture 1.3 - Evaluation Strategies and Termination 4m
Lecture 1.4 - Conditionals and Value Definitions 8m

2. Higher Order Function
Week 2 - Introduction 30sec
Lecture 2.1 - Higher-Order Functions 8m
Lecture 2.2 - Currying 15m
Lecture 2.4 - Scala Syntax Summary 4m
Lecture 2.5 - Functions and Data 10m
Lecture 2.7 - Evaluation and Operators 13m

3. Data and Abstraction
Week 3 - Introduction 1m
Lecture 3.1 - Class Hierarchies 17m
Lecture 3.2 - How Classes Are Organized 9m
Lecture 3.4 - Objects Everywhere 14m
Lecture 3.5 - Functions as Objects 5m

4. Types and Pattern Matching
Lecture 4.2 - Pattern Matching 11m
Lecture 4.5 - Subtyping and Generics 9m

This can be used in conjunction with Martin Odersky’s book: Programming in Scala
https://people.cs.ksu.edu/~schmidt/705a/Scala/Programming-in-Scala.pdf

Scala Course 2: Scala Applied
https://www.udemy.com/stairway-to-scala-applied-part-1/learn/v4/overview

Key Concepts
These videos will give an introduction to some of the key features used in Scala: Case Classes, Traits and Pattern Matching.

Case Classes Explained (Youtube): Here
- Explains some of the extra features Case Classes give.
Scala Traits use case (Youtube): Here
- An example of a single use case to understand what the trait is
Case Class and Pattern Matching (Youtube): Here
- Walk-through of Pattern Matching with Case Class

Further Scala Materials
The following articles, links and videos will help give you an understanding of some key Scala features we enjoy at Quantexa.

Case Classes
Case Classes have the benefit that they are immutable compared to regular classes and also offer additional methods.

Useful Links:
Case Class Tutorial (Youtube): Here
- Example of using a Case Class
Case Classes: Here
- Official Scala overview of Case Classes
Learning Scala Chapter 8 & 9 (eBook - O’Reilly): Here

Traits
A trait encapsulates method and field definitions, which can then be reused by mixing them into classes. Unlike class inheritance, in which each class must inherit from just one superclass, a class can mix in any number of traits.

Useful Links:
Scala Traits Tutorial (Youtube): Here
- Gives a high-level example of how traits work across classes.
Traits: Here
- Official Scala overview of traits
Learning Scala Chapter 9 (eBook - O’Reilly): Here

Pattern Matching
Pattern Matching is used to check a value against a specific pattern. It can be used in place of several if-else statements.

Useful Links

• Walk-through of Pattern Matching with Case Class Pattern Matching: Here and Here
• Official Scala overview of Pattern Matching, and a Baeldung.com overview

Key Learning Points

• Do you understand the difference between a Class, Case Class and an Object? What are the advantages to using a Case Class?
• In what cases is Pattern Matching better than IF statements and why is it better in these cases?

And Finally!

Like the Java User Groups (JUGs) frequented by Foojay.io readers, there are many active Scala communities to join, learn and network.

Quantexa's London-based developers collaborate with and support, for example, the London Scala User Group.

In Spain, my friends at Habla Computing contribute to Scala Programming Madrid. There are many more.

To learn more about Open Source at Quantexa, including in areas such as NLP and graph projects not discussed in this article, read this Quantexa Community article.

The post Getting Started With Scala appeared first on foojay.