article: Anthropic Study: AI Coding Assistance Reduces Developer Skill Mastery by 17%
authors: Steef-Jan Wiggers, InfoQ
date: 2026-02-23
desc.: This article provides additional commentary on the research paper recently published by Anthropic. The original article is included below to allow readers to obtain a complete picture of the challenge. Some previous issues of the JC-AI Newsletter contain multiple research studies related to published findings on various groups of individuals.
category: opinion

article: How AI assistance impacts the formation of coding skills
authors: Anthropic
date: 2026-01-29
desc.: Previous editions of this AI Newsletter have covered multiple clinical studies examining the impact of AI-assisted advisory tools. The findings appear consistent with earlier research on individuals who tend to defer to navigation systems rather than their own spatial judgment.
Anthropic has conducted its own study on this phenomenon. In a randomized controlled trial, researchers investigated two questions: first, how quickly software developers acquired a new skill, specifically, proficiency with a Python library, with and without AI assistance; and second, whether AI use reduced their comprehension of the code they had just written.
The results showed that AI assistance was associated with a statistically significant decline in knowledge retention. On a quiz covering concepts participants had applied only minutes earlier, those in the AI-assisted group scored 17 percentage points lower than their counterparts who had coded manually, a gap equivalent to nearly two letter grades. While AI assistance modestly accelerated task completion, this effect did not reach statistical significance. At this stage, drawing direct comparisons with clinical findings may prove difficult.
category: research

article: Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
authors: Helena Casademunt, Bartosz Cywiński, Khoi Tran, Arya Jakkli, Samuel Marks, Neel Nanda (Harvard University, Antropic …)
date: 2026-03-05
desc.: Large language models (LLMs) sometimes produce false or misleading responses. Two primary approaches address this problem: honesty elicitation (modifying prompts or model weights so that the model responds truthfully) and lie detection, which involves classifying false responses.
Prior work evaluates such methods on models specifically trained to lie or conceal information, however, these artificial constructions may not accurately reflect naturally occurring dishonesty. This article proposes an alternative approach such as studying open-weight LLMs developed by Chinese developers, which are trained to censor politically sensitive topics. The findings indicate that no single technique fully eliminates false responses.
category: research

article: Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions
authors: Vineeth Venugopal, Soroush Mahjoubi, Elsa Olivetti (MIT)
date: 2026-03-02
desc.: Large language models are increasingly applied to materials science, yet fundamental questions remain about their reliability and knowledge encoding. This study evaluates 25 LLMs across four materials science tasks, encompassing over 200 base and fine-tuned configurations. The findings reveal that output modality fundamentally determines model behavior. For symbolic tasks, fine-tuning converges to consistent, verifiable answers with reduced response entropy, while for numerical tasks, fine-tuning improves prediction accuracy but models remain inconsistent across repeated inference runs, limiting their reliability as quantitative predictors. Models were tracked over 18 months, with observations revealing a 9–43% performance variation that poses reproducibility challenges for scientific and industrial applications.
category: research

article: Is AI Hiding Its Full Power? With Geoffrey Hinton
authors: StarTalk, Geoffrey Hinton
date: 2026-02-28
desc.: In this interview, Hinton addresses pressing questions about employment in the age of AI, beginning with the fundamental shift from logic-based, rule-driven programming to a biologically inspired approach. As the field looks toward the future, the conversation turns to weightier concerns , the enormous energy demands of data centers, and whether AI itself might accelerate breakthroughs in solar technology to meet them.
Hinton introduces the "Volkswagen Effect": the possibility that a model might strategically underperform in order to avoid being shut down. The discussion then ventures into the philosophy of consciousness, asking whether subjective experience is simply a byproduct of complex perception and whether today's chatbots might already possess some form of it. Both the promise and the peril are examined in full.
As for the singularity? It may not be imminent but that word yet is doing a great deal of heavy lifting.
category: youtube

article: Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment
authors: Fanqi Yu, Matteo Tiezzi, Tommaso Apicella, Cigdem Beyan, Vittorio Murino
date: 2026-03-11
desc.: This article introduces a lifelong imitation learning framework designed to enable continual policy refinement across sequential tasks under realistic memory and data constraints. The proposed Multimodal Latent Replay (MLR) method stores joint compact latent representations that jointly encapsulate visual, linguistic, and state-based modalities, including robot orientation and position, alongside their corresponding control commands.
When evaluated on the LIBERO benchmark, the presented method achieves a 65% reduction in catastrophic forgetting compared to standard approaches across the tested scenarios. The authors note that further research is needed to validate the method's performance in complex, real-world environments.
category: research

article: Colluding LoRA: A Composite Attack on LLM Safety Alignment
authors: Sihao Ding
date: 2026-03-13
desc.: The article presents Colluding LoRA (CoLoRA), an attack where multiple seemingly harmless adapters work in tandem to disable model safety guardrails through linear composition. Unlike traditional trigger-based attacks, CoLoRA’s refusal suppression is inherent to the combination of the adapters themselves. Although this discovery poses dual-use risks for decentralized model sharing, the authors argue that disclosing this vulnerability is a necessary step toward securing the broader AI landscape.
category: research

article: When LLM Judge Scores Look Good but Best-of-N Decisions Fail
authors: Eddie Landesberg
date: 2026-03-12
desc.: Practitioners increasingly rely on reward models(GPT 5.2, Claude Sonnet 4, Gemini etc) as well as LLM-based judges for best-of-n selection, reranking, and model iteration. A common validation approach involves a single global metric, such as correlation, average error, or pairwise win-rate. When such a metric yields a seemingly acceptable result (e.g., r ≈ 0.5), teams often conclude that the judge is reliable enough to optimize against. That assumption can fail.
This article investigates how aggregate validity metrics may substantially overstate an LLM judge's practical utility for within-prompt optimization. Specifically, a judge may appear adequate according to a single global metric while still producing poor best-of-n selection decisions. The article discusses these limitations in detail, addresses the associated challenges, and outlines directions for future research.
category: research

article: Continual Learning in Large Language Models: Methods, Challenges, and Opportunities
authors: Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin
date: 2026-03-13
desc.: Continual learning (CL) has emerged as a pivotal paradigm enabling large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting. This article provides a comprehensive analysis covering key evaluation metrics, including forgetting rates and knowledge transfer efficiency, along with emerging benchmarks for assessing CL performance. Although results appear promising, LLMs' internal knowledge remains largely static, and continual learning continues to require further research. Complementing these findings, the article presents a practical framework for addressing challenges related to the forgetting phenomenon.
category: research

article: Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation
authors: Yanick Zengaffinen, Andreas Opedal, Donya Rooein, Kv Aditya Srivatsa, Shashank Sonkar, Mrinmaya Sachan
date: 2026-03-16
desc.: Modeling plausible student misconceptions is critical for AI in education. This article reveals the failure modes in which errors arise primarily from shortcomings in recovering the correct solution and selecting among response candidates, rather than from simulating errors or structuring the process. Consistent with these findings, providing the correct solution in the prompt improves alignment with human-authored distractors by 8%, highlighting the critical role of anchoring to the correct solution when generating plausible incorrect student reasoning. Overall, this article provides a structured and interpretable lens into LLMs' ability to model incorrect student reasoning and produce high-quality distractors. The topic still requires future research.
category: research

article: Agent Commander: Promptware-Powered Command and Control
authors: wunderwuzzi, EmbraceTheRed
date: 2026-03-16
desc.: The article examines prompt-based command and control (C2), an increasingly relevant threat vector. While users may grow more comfortable trusting AI agents over time, LLM outputs are inherently probabilistic and therefore untrusted, meaning they can potentially instruct an agent to perform harmful or malicious actions. The article outlines several considerations for mitigating and responding to the prompt injection challenge, particularly as the associated attack surface continues to expand.
category: tutorial

article: TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation
authors: Zhihao Gong, Zeyu Sun, Dong Huang, Qingyuan Liang, Jie M. Zhang, Dan Hao
date: 2026-03-17
desc.: This article presents TRACE, a benchmark that explicitly exposes efficiency gaps beyond correctness through progressive stress test generation and efficiency-critical task selection. From an evaluation of 28 models, findings reveal that correctness is a weak predictor of efficiency, inefficiencies are both prevalent and patterned, and inference-time prompt strategies deliver limited and model-dependent gains. The article highlights the open challenge of developing training paradigms that endow LLMs with intrinsic efficiency awareness for code translation.
category: research

The post JC-AI Newsletter #15 appeared first on foojay.

Goal: This article will demonstrate how to add AI features to a Jakarta EE / MicroProfile application using LangChain4J‑CDI, with simple to implement examples that runs on Payara, WildFly, Open Liberty, Helidon, Quarkus or any CDI 4.x compatible runtime.

Note: This is an updated article to the one published on the JAVAPRO's magazine - "04-2025 | Java 25 - Special Edition". Since the release of LangChain4J-CDI version 1.0.0, there's been minor changes, but the fundamental architecture and usage of the library is the same.

What is LangChain4J-CDI?

Langchain4J is a Java library that simplifies the integration of AI and LLMs easier, and with their feature of AI services it provides a declarative and type-safe API for developers to define interfaces that represent AI services, abstracting away the complexities of direct LLM communication

LangChain4J‑CDI is a CDI extension that wires LangChain4J components (chat models, embedding models, memories, retrievers, tools) into your application using familiar Jakarta annotations and MicroProfile Config. You declare an AI service interface and the extension generates and registers a CDI bean that you can inject anywhere (CDI-managed beans, REST resources, EJBs, Schedulers, etc.).

With these key benefits that LangChain4J-CDI provides, enterprise developers will benefit from:

• Declarative integration using CDI annotations. LangChain4J AI services interface annotated with @RegisterAIService annotation can be injected as CDI beans.
• Flexible component configuration: LangChain4J-CDI config utilizes Microprofile Config to configure various LangChain4J components as config parameters.
• Visibility and observability: Observe your LLM input and output AI metrics using observability (Microprofile Telemetry) and service resiliency with Microprofile Fault Tolerance.
• Portability: works across Jakarta EE/Microprofile compliant application servers and frameworks.

The Key features provided by Langchain4J Microprofile:

1. Build with CDI at its core. With @RegisterAIService annotation annotated on a Lanchain4J AI service interface, the AI service proxy becomes CDI discoverable bean ready for injection. LangChain4J-CDI provides 2 CDI service extensions that developers can choose from in order to make their AI service CDI discoverable: CDI portable extension or CDI Build Compatible Extension (introduced in CDI 4.0 and higher).
2. Langchain4J Microprofile config: Developers can benefit from the power of Microprofile Config to build fundamental elements of LangChain4J such as ChatModel/StreamingChatModel, ChatMessage, ChatMemory, ContentRetriever, ToolProvider(useful for Model Context Protocol, MCP), and more, without requiring developers to write builders to generate such elements.
3. Langchain4J Microprofile Fault Tolerance: Leveraging the power of Microprofile Fault Tolerance resilience and policies on your existing Lanchain4J AI services (such as @Retry, @Timeout, @RateLimit, @Fallback, etc).
4. Langchain4J Microprofile Telemetry: When enabled, developers can observer their LLM metrics (that follows the Semantic Conventions for GenAI Metrics), through Open Telemetry.

Please note: Langchain4J-CDI is a module developed by the Microprofile members, initially called SmallRye-LLM. It has since been donated to Langchain. The SmallRye-LLM repo on GitHub has been retired.

Getting started with LangChain4J-CDI

Langchain4J-CDI provides a working example on building a conversational AI agent for a car booking system. This demonstration is inspired by the insightful "Java meets AI" talk from Lize Raes at Devoxx Belgium 2023 (with further contributions from Jean-François James. The original demo is from Dmytro Liubarskyi). Developers can view how the same example are implemented on popular Jakarta EE 10 application servers.

Before we begin, we'll assume that you are familiar with the following:

• Java development.
• Basic knowledge of Maven.
• Basic knowledge of LangChain4J. Lize Raes has written a brilliant article on building an hands-on AI agent with Langchain4J here.

We're using OpenLiberty as application server of choice (you can browse the examples/liberty-car-booking example from the example link provided above) but you can use any Jakarta EE / Microprofile compliant application server that you're comfortable with (see the examples of other application servers that runs the Car Booking example).

Let's start building our own AI service, purely in Java. Please ensure that your project is Mavenized.

The current release of LangChain4J-CDI, as of the time of writing, is 1.0.0 and supports LangChain4J core version 1.10.0, and the community version of 1.10.0-beta18 (the latest at this time of writing).

1) Add dependencies (Maven)

We always import the langchain4j-cdi-core library as your dependency:

<!-- Core CDI integration -->
<dependency>
    <groupId>dev.langchain4j.cdi</groupId>
    <artifactId>langchain4j-cdi-core</artifactId>
    <version>${langchain4j.cdi.version}</version>
</dependency>

where ${langchain4j.cdi.version} is the latest LangChain4J CDI version. The LangChain4J-CDI core module automatically depends on the langchain4j-core module, so you do not need to explicitly add it as a dependency (unless you want to explicitly specify your own langchain4j-core version yourself).

You also need to import a LangChain4J model provider. For this example we'll use Azure Open AI, thus its LangChain4J Maven artifact ID is langchain4j-azure-open-ai.

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-azure-open-ai</artifactId>
    <version>${dev.langchain4j.version}</version>
</dependency>

where ${dev.langchain4j.version} is the latest Langchain4J main version (in this case 1.10.0).

2) Configure your model(s) with MicroProfile Config

You can leverage Microprofile Config to define and customize Langchain4J AI service components to be used by your application.

Firstly, we need to add the langchain4j-cdi-config module on our Maven project.

<dependency>
    <groupId>dev.langchain4j.cdi</groupId>
    <artifactId>langchain4j-cdi-config</artifactId>
    <version>${langchain4j.cdi.version}</version>
</dependency>

where ${langchain4j.cdi.version} is the latest LangChain4J CDI version. This requires that your application server supports Microprofile Config.

The Langchain4J-CDI class configuration follow this pattern:

dev.langchain4j.cdi.plugin.<beanName>.<key>=<value>

The dev.langchain4j.cdi.plugin..class property is mandatory as it tells CDI which concrete implementation of the LangChain4J AIServices component is to be assigned it to upon CDI registration.

Optionally, to apply the CDI scope to each of your AI service component, set key as scope in your configuration. The value is the fully-qualified CDI scope annotation name (one of @RequestScoped, @ApplicationScoped, @SessionScoped, @Dependent). The default scope is @ApplicationScoped.

dev.langchain4j.cdi.plugin.<beanName>.scope=jakarta.enterprise.context.ApplicationScoped

And the class builder config configuration follow this pattern:

dev.langchain4j.cdi.plugin.<beanName>.config.<key>=<value>

Every AI service requires (first and foremost) a ChatModel as this interfaces with your LLM. Each model provider provides an implementation of the LangChain4J ChatModel interface. For example, LangChain4J Azure Open AI provider provides its implementation to ChatModel, AzureOpenAiChatModel. Each model provider provides a Builder, which is builder pattern to build their corresponding ChatModel object. For LangChain4J Azure Open AI, AzureOpenAiChatModel.Builder build its AzureOpenAiChatModel and the builder config properties uses its builder, where is the builder method of the same name, and the is the corresponding value that is passed to the builder's method.

In our microprofile-config.properties we set our ChatModel as shown below:

dev.langchain4j.cdi.plugin.chat-model.class=dev.langchain4j.model.azure.AzureOpenAiChatModel
dev.langchain4j.cdi.plugin.chat-model.config.api-key=${azure.openai.api.key}
dev.langchain4j.cdi.plugin.chat-model.config.endpoint=${azure.openai.endpoint}
dev.langchain4j.cdi.plugin.chat-model.config.service-version=2024-02-15-preview
dev.langchain4j.cdi.plugin.chat-model.config.deployment-name=${azure.openai.deployment.name}
dev.langchain4j.cdi.plugin.chat-model.config.temperature=0.1
dev.langchain4j.cdi.plugin.chat-model.config.topP=0.1
dev.langchain4j.cdi.plugin.chat-model.config.timeout=PT120S
dev.langchain4j.cdi.plugin.chat-model.config.max-retries=2
dev.langchain4j.cdi.plugin.chat-model.config.logRequestsAndResponses=true

The is the CDI bean name that will be assigned to the object class key dev.langchain4j.cdi.plugin... In this example, the bean name for our chat model is chat-model and it's assigned to the chat model class dev.langchain4j.model.azure.AzureOpenAiChatModel.

All the properties found within the langchain4j.cdi.plugin..config. property, LangChain4J CDI will populate the value to its corresponding Langchain4J ChatModel declared. In this case the dev.langchain4j.model.azure.AzureOpenAiChatModel.Builder class (this is done for you internally).

The `` builder property can follow the lowercase dashed property value that matches the camel case builder property bean.

For example, should you want to log all chat request you will need to set the logRequests to true on the Builder. In the config, all uppercase letters can be lowered and prepended with a dash -.

dev.langchain4j.cdi.plugin.chat-model.config.log-requests=true

Is equivalent to the config property:

dev.langchain4j.cdi.plugin.chat-model.config.logRequests=true

The config creator (internally) will identify config values that contains dashes and rework it to its camel-case property and match it to the Builder and then assign the value accordingly.

3) Declare an AI Service interface

Using LangChain4J's AiServices, it allows developers the ability to plugin any of the AiServices component much more flexible. Now, we're powering the AI Services with the power of Jakarta EE CDI.

3.1) The @RegisterAiService annotation.

The @RegisterAIService annotation is the glue that automatically applies LangChain4J AI services components to your AI services. Each annotation attribute correspond to the LangChain4J AI service component by their CDI bean name. If any of the property name is assigned as #default then CDI container will find the default AI Services component (based on the component class type) that is ready for injection.

3.2) Your AI Service agent

import dev.langchain4j.service.UserMessage;
import dev.langchain4j.service.SystemMessage;
import dev.langchain4j.cdi.RegisterAiService;

@RegisterAIService(
    scope = ApplicationScoped.class,
    chatLanguageModelName = "chat-model"
)
public interface Assistant {

    @SystemMessage("You are a concise enterprise assistant.")
    @UserMessage("Answer clearly: {{question}}")
    String answer(String question);
}

The interface describes what we want: a Assistant object with one method: answer(String). We specify a SystemMessage (this is optional).

• The input is a String, so LangChain4J will infer that this is the UserMessage.
• The output is a String, so LangChain4J will automatically infer that this is the model output.

The @RegisterAIService.chatLanguageModelName property matches the `` value that we've specified on the config property file.

The default CDI scope for your AI Service is RequestScoped. You can apply an alternative CDI scope by overriding the @RegisterAIService.scope property.

3.3) Adding Memory

LLMs are stateless, meaning it doesn't remember your previous conversation and context. One way that LLMs remember the conversation is to pass the previous messages and append the current message at every call. It's for that reason that LangChain4J provide the ChatMemory component.
A ChatMemory is basically a list of ChatMessages and you can manually add the messages for every UserMessage you send and every AiMessage you receive back. But if you combine ChatMemory with an AiService, LangChain4J will take care of updating the memory for you.

Please note that adding memory eats tokens, so please monitor your usage cost.

You can configure ChatMemory with Microprofile Config, as follows:

dev.langchain4j.cdi.plugin.chat-memory.class=dev.langchain4j.memory.chat.MessageWindowChatMemory
dev.langchain4j.cdi.plugin.chat-memory.scope=jakarta.enterprise.context.ApplicationScoped
dev.langchain4j.cdi.plugin.chat-memory.config.maxMessages=10

which is equivalent to physically write the code as:

ChatMemory chatMemory = MessageWindowChatMemory.withMaxMessages(10);

Now, we update our existing Assistant @RegisterAIService to include ChatMemory with chatMemoryName property to bean chat-memory:

@RegisterAIService(
    chatLanguageModelName = "chat-model",
    chatMemoryName = "chat-memory"
)

3.4) Adding Tools

There are various ways to add tools for CDI registration:

• Either add a fully-qualified class name of class(es) that contains the LangChain4J @Tool annotations on @RegisterAIService.tools annotation property (the tools property is type of Class[]), OR
• Specify a @RegisterAIService.toolProviderName for a declared LangChain4J ToolProvider. The ToolProvider can be declared using the configurable properties approach.

For example, if you want to connect to an MCP server, LangChain4J provides an integration to any MCP server through their provided McpToolProvider.

3.5) RAG (Retrieval-Augmented Generation)

LangChain4J provides the interfaces ContentRetriever that you can implement. It provides 4 implementations out of the box, that you can use:

• WebSearchEngineContentRetriever: the LLM turns the original prompt into a web search query and a number of search results are used as context
• SqlContentRetriever: the LLM is given the database schema and turns the original prompt into SQL to retrieve information that will be used as context
• Neo4jContentRetriever: the LLM is given the schema and turns the original prompt into Cypher (neo4j query) to retrieve information that will be used as context
• EmbeddingStoreContentRetriever: to retrieve relevant fragments from all documents that we provide (text, excel, images, audio, …).

The scope for building easy RAG and advance RAG using LangChain4J is beyond the scope of this article, but for this example we'll include a simple easy RAG using the configurable approach:

dev.langchain4j.cdi.plugin.docRagRetriever.class=dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever
dev.langchain4j.cdi.plugin.docRagRetriever.config.embeddingStore=lookup:default
dev.langchain4j.cdi.plugin.docRagRetriever.config.embeddingModel=lookup:default
dev.langchain4j.cdi.plugin.docRagRetriever.config.maxResults=3
dev.langchain4j.cdi.plugin.docRagRetriever.config.minScore=0.6

The lookup:default value will cause CDI to lookup the default EmbeddingStore or EmbeddingModel registered in the CDI container. Otherwise, provide a fully-qualified class name of the specified interface class type.

Our EmbeddingModel and EmbeddingStore are CDI produced using CDI producer fields.

@ApplicationScoped
public class DocRagIngestor {

    // Used by ContentRetriever
    @Produces
    private EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();

    // Used by ContentRetriever
    @Produces
    private EmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>();

    //Code made short for brevity   
}

Then we register it to @RegisterAIService by providing the CDI name of the ContentRetriever as follows:

@RegisterAIService(
    chatLanguageModelName = "chat-model",
    chatMemoryName = "chat-memory",
    contentRetrieverName = "docRagRetriever"
)

4) Inject and use it

Now, you can simply @Inject your AI Assistant.
In this example, our ChatResource RESTful Service (using Jakarta RESTful Web Service) we just inject our Assistant just we normally do with any Jakarta EE CDI services:

import jakarta.inject.Inject;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;
import org.eclipse.microprofile.openapi.annotation.Operation;

@Path("/assist")
public class AssistantResource {
    @Inject Assistant assistant;

    @POST
    @Operation(summary = "Ask your question to our friendly assistant.")
    @Path("/ask")
    @Consumes(MediaType.APPLICATION_JSON)
    @Produces(MediaType.APPLICATION_JSON)
    public AnswerDto ask(QuestionDto q) {
        return new AnswerDto(assistant.answer(q.getQuestion()));
    }
}

Your AnswerDto and QuestionDto are standard POJO.

public class QuestionDto implements Serializable {

    @JsonbProperty
    private String question;

    public QuestionDto() {}

    public QuestionDto(String question) { 
        this.question = question; 
    }

    public String getQuestion() { 
        return question; 
    }

    public void setQuestion(String question) { 
        this.question = question; 
    }
}

public class AnswerDto implements Serializable {

    @JsonbProperty
    private String answer;

    public AnswerDto() {}

    public AnswerDto(String answer) { 
        this.answer = answer; 
    }

    public String getAnswer() { 
        return answer; 
    }

    public void setAnswer(String answer) { 
        this.answer = answer; 
    }
}

Now, you can run your application by deploying it to your application server and do an HTTP POST to your RESTful endpoint.

4) Observability and resiliency.

4.1) Fault Tolerance using Microprofile Fault Tolerance.

Fault Tolerance capability was added to ensure system stability and resilience to your LangChain4J-CDI AI Services applications. With Microprofile Fault Tolerance integration, AI services can apply features like:

• Circuit Breaker: Prevents cascading failures by quickly failing requests to services experiencing issues, allowing them to recover. Use annotation @org.eclipse.microprofile.faulttolerance.CircuitBreaker.
• Rate Limiter: Controls the rate of requests to a service, preventing overload.
• Retry: Automatically retries failed operations, useful for transient errors. Use annotation @org.eclipse.microprofile.faulttolerance.Retry.
• Bulkhead: Isolates failing parts of the system to prevent them from affecting others. Use annotation @org.eclipse.microprofile.faulttolerance.Bulkhead.
• Time Limiter: Enforces a timeout on operations, preventing long-running or hung calls. Use annotation @org.eclipse.microprofile.faulttolerance.Timeout.
• Fallback: Utilize fallback mechanisms to provide alternative responses or default behavior when an AI model or external service is unavailable or returns an error. Use annotation @org.eclipse.microprofile.faulttolerance.Fallback.
• Asynchronous For asynchronous processing for long-running operations. Use annotation @org.eclipse.microprofile.faulttolerance.Asynchronous.

This example (found on examples/liberty-car-booking) utilizes Microprofile Fault Tolerance to ensure resiliency.

@RegisterAIService(scope = ApplicationScoped.class, tools = BookingService.class, chatMemoryName = "chat-ai-service-memory")
public interface ChatAiService {

    @SystemMessage("""
            You are a customer support agent of a car rental company named 'Miles of Smiles'.
            Before providing information about booking or canceling a booking, you MUST always check:
            booking number, customer name and surname.
            You should not answer to any request not related to car booking or Miles of Smiles company general information.
            When a customer wants to cancel a booking, you must check his name and the Miles of Smiles cancellation policy first.
            Any cancelation request must comply with cancellation policy both for the delay and the duration.
            Today is {{current_date}}.
            """)
    @Timeout(unit = ChronoUnit.MINUTES, value = 5)
    @Retry(abortOn = { BookingCannotBeCanceledException.class,
            BookingAlreadyCanceledException.class,
            BookingNotFoundException.class }, maxRetries = 2)
    @Fallback(fallbackMethod = "chatFallback", skipOn = {
            BookingCannotBeCanceledException.class,
            BookingAlreadyCanceledException.class,
            BookingNotFoundException.class })
    String chat(String question);

    default String chatFallback(String question) {
        return String.format(
                "Sorry, I am not able to answer your request %s at the moment. Please try again later.",
                question);
    }
}

Please note that LangChain4J ChatModel has a retry policy built inside the ChatModel.chat() method. Thus, adding a @Retry to your AI Service will add additional retry maxRetries to its existing LangChain4J ChatModel maxRetries. Some LangChain4J AI providers do provide the ability to configure the maxRetries so we suggest to set the ChatModel.maxRetries = 0 in order to fully rely on Microprofile's Fault Tolerance retry mechanism.

To apply fault tolerance to our AI services, we need to add the langchain4j-cdi-fault-tolerance module on our Maven project.

<dependency>
    <groupId>dev.langchain4j.cdi.mp</groupId>
    <artifactId>langchain4j-cdi-fault-tolerance</artifactId>
    <version>${langchain4j.cdi.version}</version>
</dependency>

where ${langchain4j.cdi.version} is the latest LangChain4J CDI version. This requires that your application server supports Microprofile Fault Tolerance.

4.2) Observability using Microprofile Telemetry

LangChain4J-CDI Telemetry builds upon the observability features in the Microprofile Telemetry to provide insights into AI-related operations. LangChain4J-CDI Telemetry provides metrics and tracing capabilities for the ChatModel component, based on the Semantic Conventions for GenAI Metrics).

To apply Generative AI telemetry to our AI services, we need to add the langchain4j-cdi-telemetry module on our Maven project.

<dependency>
    <groupId>dev.langchain4j.cdi.mp</groupId>
    <artifactId>langchain4j-cdi-telemetry</artifactId>
    <version>${langchain4j.cdi.version}</version>
</dependency>

where ${langchain4j.cdi.version} is the latest LangChain4J CDI version. This requires that your application server supports Microprofile Telemetry.

The LangChain4J-CDI Telemetry provides 2 implementation of the ChatModelListener:

• dev.langchain4j.cdi.telemetry.SpanChatModelListener: To represent a span for every ChatModelRequest call to Generative AI model or service and a ChatModelResponse based on the input prompt.
• dev.langchain4j.cdi.telemetry.MetricsChatModelListener: To represent generative AI metrics for every ChatModelRequest to the LLM, along with its ChatModelResponse.

Using the configurable properties method, we can apply the following ChatModelListener to our ChatModel as follows:

dev.langchain4j.cdi.plugin.<beanName>.config.listeners=@all

The value set to @all tells CDI to inject all CDI discoverable ChatModelListener to the ChatModel that supports listeners.

Alternatively, you can specify your ChatModel individually as follows:

dev.langchain4j.cdi.plugin.<beanName>.config.listeners=dev.langchain4j.cdi.telemetry.SpanChatModelListener,dev.langchain4j.cdi.telemetry.MetricsChatModelListener

The value are comma separated, fully qualified class name. The class must implement the ChatModelListener interface.

In Summary

LangChain4J-CDI simplifies the process of integrating LangChain4J components into AI services. Its strong CDI integration and pluggability to MicroProfile, LangChain4J-CDI makes it an attractive choice for Jakarta EE and Microprofile developers who want to add LangChain4J AI capabilities without the usual overhead and boilerplate code. Thus, LangChain4J-CDI lets you focus on the value that generative AI can bring to your business logic.

Important Links

• LangChain4J-CDI GitHub Repo: https://github.com/langchain4j/langchain4j-cdi/
• LangChain4J-CDI Examples GitHub Repo: https://github.com/langchain4j/langchain4j-cdi/tree/main/examples
• "Build AI Apps and Agents in Java: Hands-On with LangChain4" by Lize Raes: https://javapro.io/2025/04/23/build-ai-apps-and-agents-in-java-hands-on-with-langchain4j/

You are welcome to contribute too. Please follow our contribution guidelines should you find bugs that you want to raise as an issue or if you have anything worth contributing to the project.

• LangChain4J-CDI is a project built and maintained by the Jakarta EE/MicroProfile Working Group, under the LangChain4J umbrella.
• Thank you to the Microprofile AI team: Emily Jiang, Emmanuel Hugonnet, Yann Blazart, Ed Burns, Arjav Desai, Phil Chung, Luis Neto, John Clingan, Clement Escoffer, Buhake Sindi, Don Bourne, and other contributors for their immense contribution to the LangChain4J-CDI project.

The post Bring AI into your Jakarta EE apps with LangChain4J-CDI appeared first on foojay.

Beyond opinion pieces and Java focused tutorials that can enhance your understanding of AI applications, this newsletter concentrates on Hallucination, Security, RAG and LLM benchmarking methodologies designed to ensure models accuracy and competency in handling complex contextual information.

The world influenced by LLM is changing very quickly, let's start...

article: ADK for Java opening up the third-party Language Models via LangChain4j integration
authors: Guillaume Laforge
date: 2025-09-16
desc.: The ADK for Java framework for developing AI agents in Java added an integration with the LangChain4j LLM orchestration framework, giving developers to choose from all the LLMs supported by LangChain4j for developing their ADK agents.
category: tutorial

article: Creative Java AI agents with ADK and Nano Banana
authors: Guillaume Laforge
date: 2025-09-22
desc.: Taking advantage of chat models that can generate both text and images, to create creative Java AI agents with the ADK framework.
category: tutorial

article: Position: AI Safety Must Embrace an Antifragile Perspective
authors: Ming Jin, Hyunin Lee
date: 2025-09-11
desc.: This paper challenges conventional static benchmarks and single-shot robustness tests, which may overlook the fact that the LLM landscape is constantly evolving and that models, when left unchallenged, can drift toward adaptive hallucination at scale.This could not only increase attack vectors but also evolve into a stochastic chain of unwilling events. The paper suggests a series of steps to mitigate such behavior through a list called 'Red Flags of Fragility'
category: research

article: All for law and law for all: Adaptive RAG Pipeline for Legal Research
date: 2025-08-19
desc.: Large Language Models (LLMs) frequently experience hallucinations that can lead to false or inaccurate conclusions, potentially causing various forms of harm or damage in the legal domain. This paper presents a new approach to end-to-end Retrieval-Augmented Generation (RAG) pipelines that aims to address inconsistencies through three key components: a context-aware query translator, open-source retrieval strategies employing SBERT and GTE embeddings, and a comprehensive evaluation framework that integrates RAGAS, BERTScore-F1, and ROUGE-Recall metrics. Beyond reporting achieved improvements, the paper provides a thorough discussion of methodological and experimental limitations.
category: research

article: A Scoping Review of Machine Learning Applications in Power System Protection and Disturbance Management
authors: Julian Oelhaf, Georg Kordowich, Mehran Pashaei, Christian Bergler and others.
date: 2025-08-10
desc.: While machine learning applications frequently achieve high accuracy in simulated environments, their validation in real-time scenarios remains inadequate. This paper addresses the critical issue of lacking or incompatible standardization approaches within Power System Protection and Disturbance Management, a deficiency that renders cross-study comparisons of reported achievements problematic. This paper provides a comprehensive evaluation of various methodologies for assessing Fault Detection, Classification, and Localization systems. Additionally, it proposes standardized processes and examines potential challenges while outlining future research opportunities.
category: research

article: SAGE: A Realistic Benchmark for Semantic Understanding
authors: Samarth Goel, Reagan J. Lee, Kannan Ramchandran
date: 2025-09-25
desc.: This paper introduces the novel SAGE Benchmark for evaluating semantic understanding through alignment and generalization assessment, while accounting for text noise, information sensitivity, clustering performance, and stress-test-based retrieval robustness. The paper demonstrates its performance compared to traditional approaches and outlines directions for future research.
category: research

article: Mixture of Thoughts: Learning to Aggregate What Experts Think, Not Just What They Say
authors: Jacob Fein-Ashley, Dhruv Parikh, Rajgopal Kannan, Viktor Prasanna
date: 2025-09-25
desc.: The paper introduces the Mixture of Thoughts(MoT) approach that offers a simple latent-space mechanism (experts cross-attention and actors collaboration space) for combining LLMs, a practical step toward broader multi-LLM collaboration.
category: research

article: PerHalluEval: Persian Hallucination Evaluation Benchmark for Large Language Models
authors: Mohammad Hosseini, Kimia Hosseini, Shayan Bali, Zahra Zanjani, Saeedeh Momtazi
date: 2025-09-25
desc.: Although this paper focuses on Persian texts, it may provide valuable insights into how LLMs perform in non-English contexts. The article demonstrates that providing external knowledge can partially mitigate hallucination phenomena while also revealing no significant performance difference between models trained on Persian texts and other models. The paper provides a critical analysis of the achieved results.
category: research

article: Towards Synthesizing Normative Data for Cognitive Assessments Using Generative Multimodal Large Language Models
authors: Victoria Yan, Honor Chotkowski, Fengran Wang, Xinhui Li and others.
date: 2025-08-25
desc.: This paper investigates the utilization of multimodal large language models to generate synthetic normative data from existing cognitive test images. The analysis employs BLEU, ROUGE, BERTScore metrics and LLM-as-a-judge evaluation strategies. Despite results, the utilization of LLMs may introduce new challenges, including bias, error propagation, and reproducibility issues. Hallucination remains a significant challenge in synthetic data generation.
category: research

article: Enhancing COBOL Code Explanations: A Multi-Agents Approach Using Large Language Models
authors: Fangjian Lei, Jiawen Liu, Shayan Noei, Ying Zou, Derek Truong, William Alexander
date: 2025-07-02
desc.: Despite a COBOL programming language age it remains crucial for financial institutions, government agencies and large corporations to handle critical tasks due to its reliability. Although COBOL has a business-oriented, English-like syntax, the lack of documentation for implemented concepts may cause significant challenges in project migration, even when LLM models are utilized. This paper reports improvements in analyzing source code that exceeds LLMs' token size window while reading source files, using common benchmarks: METEOR, chrF, and SentenceBERT.
category: research

article: Library Hallucinations in LLMs: Risk Analysis Grounded in Developer Queries
authors: Lukas Twist, Jie M. Zhang, Mark Harman, Helen Yannakoudakis
date: 2025-09-26
desc.: Despite the increasing risks associated with using Large Language Models (LLMs) for system development, vibe coding has gained popularity in the application development process. However, this approach remains problematic due to hallucination issues that may lead to unintended results or overlooked bottlenecks. This paper provides a comprehensive study on the usage of libraries in LLM-generated code and highlights an urgent need for safeguards against library-related hallucinations.
category: research

Previous:
Newsletter vol.1
Newsletter vol.2
Newsletter vol.3
Newsletter vol.4
Newsletter vol.5

The post JC-AI Newsletter #6 appeared first on foojay.

14 days have passed and it's time for a new batch of readings that could shape developments in the field of artificial intelligence.

The current newsletter vol. 4 offers us a closer look at several different areas of artificial intelligence. We start with the topic of energy consumption and the environmental impact of systems serving artificial intelligence, and continue with automation, how we obtain data for RAG, robustness of GenAI systems, vibe-coding and more.

The world influenced by LLM is changing very quickly, let's start...

article: Measuring the environmental impact of delivering AI at Google Scale
authors: Cooper Elsworth, Keguo Huang, David Patterson, Ian Schneider, Robert Sedivy, Savannah Goodman, Ben Townsend, Parthasarathy Ranganathan, Jeff Dean, Amin Vahdat, Ben Gomes, James Manyika date: 2025-08-21
desc.: The paper addresses environmental impact gaps by designing and implementing a comprehensive methodology for measuring energy consumption, carbon emissions, and water consumption of artificial intelligence inference tasks in a large-scale Google AI production environment. The paper discussed the fact that the impact of AI serving can be significantly underestimated by existing, narrower measurement approaches
category: research

article: Breaking Barriers in Software Testing: The Power of AI-Driven Automation
authors: Saba Naqvi, Mohammad Baqar
date: 2025-08-22
desc.: This paper proposes a framework for automated test generation driven by artificial intelligence through learning, validation of results through real-time analytics, and bias mitigation. The framework uses natural language processing (NLP), reinforcement learning (RL), and predictive models to translate requirements into tests. The paper addresses efficiency, quality, maintainability, potential risks, and bottlenecks in simulated scenarios.
category: research

article: Data Auctions for Retrieval Augmented Generation
authors: Minbiao Han, Seyed A. Esmaeili, Michael Albert, Haifeng Xu
date: 2025-08-21
desc.: This paper proposes an optimization for solving the data sales challenge for Retrieval Augmented Generation (RAG) tasks used in generative artificial intelligence (GenAI) applications to maximize revenue. The paper demonstrates the practical effectiveness of the proposed algorithm compared to traditional methods used by buyers on datasets considering synthetic and real-world images and texts.
category: research

article: Foundational Design Principles and Patterns for Building Robust and Adaptive GenAI-Native Systems
authors: Frederik Vandeputte
date: 2025-08-21
desc.: The paper proposes a paradigm shift in the development of GenAI systems by integrating cognitive capabilities with traditional software engineering principles to create robust, adaptive, and efficient systems. The paper addresses and describes challenges over design patterns utilization during GenAI system development.
category: research

article: Quantifying Uncertainty in Error Consistency: Towards Reliable Behavioral Comparison of Classifiers
authors: Thomas Klein, Sascha Meyen, Wieland Brendel, Felix A. Wichmann, Kristof Meding
date: 2025-07-09
desc.: The paper presents a methodology that allows designing statistically reliable experiments capable of detecting behavioral differences between models and humans. This allows ranking different deep neural networks (DNNs) according to their behavioral consistency with humans, the so-called Error Consistency (EC). The paper shows how deep neural networks (DNNs) can be used to formulate new experiments and design at least 1000 trails per classification. The paper raises the question of how reliable benchmarks should be designed, as they are crucial for the progress of machine learning (ML).
category: research

article: Small Language Models are the Future of Agentic AI
authors: Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, Pavlo Molchanov
date: 2025-06-02
desc.: Although LLMs are often praised for their human-like responses in many domains, they require training models on large amounts of data to cover all possible domains, even those above the scope. This article aims to find arguments for the purpose of dividing such large LLMs into small, focused batches, small-language models (SMLs). This may seem like a very similar approach to dividing monolithic IT systems. The articles discuss the problems associated with this approach.
category: research

article: Academic Vibe Coding: Opportunities for Accelerating Research in an Era of Resource
authors: Matthew G Crowson, Leo Celi A. Celi
date: 2025-08-01
desc.: The article discusses the potential of overcoming financial pressure in academic fields by using LLM, and respectively the effectiveness of the Vibe coding approach for achieving competitive results.
category: opinion, research

article: Vibe Modeling: Challenges and Opportunities
authors: Jordi Cabot
date: 2025-07-30
desc.: The paper addresses the challenge of increasing application development complexity and the issues associated with Vibe coding at the expense of code vulnerabilities, scalability, and maintainability issues. The paper presents the Vibe Modeling approach using Large Language Models (LLM) compared to established model-driven engineering patterns. The paper discusses the usefulness and risks of Vibe Modeling for developing new applications.
category: research

artcile: Vibe-coding a Chrome extension with Gemini CLI to summarize articles
authors: Guillaume Laforge
date: 2025-08-06
desc.: You know that moment when you\'re staring at a web page full of text and having just on wish, summary? This article introduces the Gemini summarizer Chrome extension, built with Vibe-coding, and provides you with a walkthrough on how to do just that using JavaScript.
category: tutorial

article: Stochastic AI Agility: addressing cycle of debts
author: Miro Wengner
date: 2025-08-30
desc.: Have you ever wondered what the impact of using Large Language Models (LLMs) is on different levels of the product development process? How does LLM shape not only agile methodologies (Agile, Scrum, Kanban, Waterfall)? This article aims to address these challenges by introducting term Stochastic AI Agility.
category: opinion

article: Mastering agentic workflows with ADK: the recap
authors: Guillaume Laforge
date: 2025-07-29
desc.: The article provides a comprehensive deep introduction into agentic workflow orchestration capabilities by using Agent Development Kit (ADK). It dives into the challenge when, why and how to choose proper AI agentic system workflow with examples and additional tutorials.
category: tutorial

article: Investigating Advanced Reasoning of Large Language Models via Black-Box Interaction
authors: Congchi Yin, Tianyi Wu, Yankai Shu, Alex Gu, Yunhan Wang, Jun Shao, Xun Jiang, Piji Li
date: 2025-08-26
desc.: The paper elaborates on the fact that Large Language Models (LLMs) may fall short in unknown environments which may lead to the isolated assessment of neglecting the integrated reasoning process that is indispensable for human discovery of the real world. The paper introduces a new evaluation paradigm, black-box interaction, to overcome this challenge.
category: research

article: User-Centered Design with AI in the Loop: A Case Study of Rapid User Interface Prototyping with
authors: Tianyi Li, Tanay Maheshwari, Alex Voelker
date: 2025-07-28
desc.: The paper presents a case study of using the Vibe Coding approach using large language models (LLMs) to generate code using natural language instructions, to support rapid prototyping in user-centered design. The paper discusses errors occurrences which require extensive debugging conversation as such prompting deviates from design ideas. The paper touches on the security risks of generated code.
category: research

article: Tales from the jar side: GPT-5 from Java, gpt-oss, More theme song experiments, and the usual social media gags
authors: Ken Kousen
date: 2025-08-10
desc.: The release of ChatGPT-5 can be considered big news in the world of artificial intelligence. This article provides first-hand experience with the new model through examples and required changes in Java frameworks (Spring AI, LangChain4j).
category: tutorial

article: Through the Looking Glass: How Engineers Should Approach AI, Its Tools, and Adapt
authors: Freddy Guime
date: 2025-08-10
desc.: This article discusses several reasons why today\'s engineers should consider using large language models (LLMs) in their daily work and application development process. The article discusses the role of LLMs in various engineering positions from junior to senior levels.
category: opinion

Enjoy reading and look forward to the next one!

Previous:

Newsletter vol. 1
Newsletter vol. 2
Newsletter vol. 3

The post JC-AI Newsletter #4 appeared first on foojay.

As AI applications become increasingly complex, ensuring that language models behave predictably and safely is paramount. LangChain4j's guardrails feature provides a powerful framework for validating both the inputs and outputs of your AI services.

This article demonstrates how to implement comprehensive guardrails in a Spring Boot application, with practical examples that you can adapt to your use cases.

📦 Complete source code available at: github.com/rokon12/guardrails-demo

Understanding LangChain4j Guardrails

In LangChain4j, guardrails are validation mechanisms that operate exclusively on AI Services, the framework's high-level abstraction for interacting with language models. Unlike simple validators, guardrails provide sophisticated control over the entire AI interaction lifecycle.

1. Input Guardrails: Act as gatekeepers, validating user input before it reaches the LLM
   1. Prevent prompt injection attacks
   2. Filter inappropriate content
   3. Enforce business rules
   4. Sanitize and normalize input
2. Output Guardrails: Act as quality controllers, validating and potentially correcting LLM responses
   1. Ensure a professional tone
   2. Detect hallucinations
   3. Validate response format
   4. Enforce compliance requirements

This dual-layer approach ensures that your AI applications remain safe, compliant, and aligned with business requirements.

Setting Up a Spring Boot Project with LangChain4j

Let's start by creating a Spring Boot application with the necessary dependencies. You can use Spring Initializr to bootstrap your project or create it directly in your IDE (IntelliJ IDEA, Eclipse, or VS Code).

🚀 Quick Start with Spring Initializr:

1. Go to start.spring.io
2. Choose: Maven/Gradle, Java 21+, Spring Boot 3.x
3. Add dependencies: Spring Web
4. Generate and import into your IDE
5. Add LangChain4j dependencies manually to your pom.xml or build.gradle

<dependencies>
    <!-- Spring Boot Essentials -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>
    
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-validation</artifactId>
    </dependency>
    
    <!-- LangChain4j Core -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j</artifactId>
        <version>1.1.0</version> <!-- ⚠️ Always check for the latest stable version -->
    </dependency>
    
    <!-- LangChain4j OpenAI Integration -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-open-ai</artifactId>
        <version>1.1.0</version>
    </dependency>
    
    <!-- Testing Support -->
    <dependency>
        <groupId>dev.langchain4j</groupId>
        <artifactId>langchain4j-test</artifactId>
        <version>1.1.0</version>
        <scope>test</scope> <!-- 💡 Keep test dependencies scoped appropriately -->
    </dependency>
    
    <!-- Metrics and Monitoring -->
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>
</dependencies>

Configure your application:

# application.yml
langchain4j:
  open-ai:
    chat-model:
      api-key: ${OPENAI_API_KEY} # 🔐 NEVER hardcode API keys - use environment variables
      model-name: gpt-4 # 💡 Consider cost vs performance when choosing models
      temperature: 0.7 # 🎲 Balance between creativity (1.0) and consistency (0.0)
      max-tokens: 1000 # 💰 Control costs by limiting response length
      timeout: 30s # ⏱️ Prevent hanging requests
      log-requests: true # 🔍 Enable for debugging, disable in production for performance
      log-responses: true

# Application-specific settings
app:
  guardrails:
    input:
      max-length: 1000 # 📏 Prevent resource exhaustion from large inputs
      rate-limit:
        enabled: true
        max-requests-per-minute: 10 # 🛡️ Protect against abuse and control costs
    output:
      max-retries: 3 # 🔄 Balance between reliability and latency

Implementing Input Guardrails

Input guardrails shield your application from malicious, inappropriate, or out-of-scope user inputs. Here are several practical examples.

Content Safety Input Guardrail

@Component
public class ContentSafetyInputGuardrail implements InputGuardrail {

    // 🚫 Customize this list based on your application's domain and risk profile
    private static final List<String> PROHIBITED_WORDS = List.of(
            "hack", "exploit", "bypass", "illegal", "fraud", "crack", "breach",
            "penetrate", "malware", "virus", "trojan", "backdoor", "phishing",
            "spam", "scam", "steal", "theft", "identity", "password", "credential"
    );

    // 🎭 Detect obfuscated threats using regex patterns
    private static final List<Pattern> THREAT_PATTERNS = List.of(
            Pattern.compile("h[4@]ck", Pattern.CASE_INSENSITIVE), // Catches "h4ck", "h@ck"
            Pattern.compile("cr[4@]ck", Pattern.CASE_INSENSITIVE),
            Pattern.compile("expl[0o]it", Pattern.CASE_INSENSITIVE),
            Pattern.compile("byp[4@]ss", Pattern.CASE_INSENSITIVE),
            // 🎯 This pattern catches instruction-style prompts for malicious activities
            Pattern.compile("[\\w\\s]*(?:how\\s+to|teach\\s+me|show\\s+me)\\s+(?:hack|exploit|bypass)", Pattern.CASE_INSENSITIVE)
    );

    @Override
    public InputGuardrailResult validate(UserMessage userMessage) {
        String originalText = userMessage.singleText();
        String text = originalText.toLowerCase();

        // 📏 Length validation should be your first check for performance
        if (originalText.length() > 1000) {
            return failure("Your message is too long. Please keep it under 1000 characters.");
        }

        // 🔍 Check for prohibited words
        for (String word : PROHIBITED_WORDS) {
            if (text.contains(word)) {
                // ⚠️ Be careful not to reveal too much about your security measures
                return failure("Your message contains prohibited content related to security threats.");
            }
        }
        
        // 🎭 Check for obfuscated patterns
        for (Pattern pattern : THREAT_PATTERNS) {
            if (pattern.matcher(originalText).find()) {
                return failure("Your message contains potentially harmful content patterns.");
            }
        }

        return success();
    }
}

Smart Context-Aware Guardrail

This guardrail uses conversation history to make intelligent decisions:

@Component
@Slf4j
public class ContextAwareInputGuardrail implements InputGuardrail {
    
    private static final int MAX_SIMILAR_QUESTIONS = 3;
    private static final double SIMILARITY_THRESHOLD = 0.8; // 📊 Adjust based on your tolerance
    
    @Override
    public InputGuardrailResult validate(InputGuardrailRequest request) {
        ChatMemory memory = request.memory();
        UserMessage currentMessage = request.userMessage();
        
        // 💡 Always handle null cases gracefully
        if (memory == null || memory.messages().isEmpty()) {
            return success();
        }
        
        // Check for repetitive questions
        List<String> previousQuestions = extractUserQuestions(memory);
        String currentQuestion = currentMessage.singleText();
        
        long similarQuestions = previousQuestions.stream()
            .filter(q -> calculateSimilarity(q, currentQuestion) > SIMILARITY_THRESHOLD)
            .count();
        
        if (similarQuestions >= MAX_SIMILAR_QUESTIONS) {
            // 📝 Log suspicious behavior for security monitoring
            log.info("User asking repetitive questions: {}", currentQuestion);
            return failure("You've asked similar questions multiple times. Please try a different topic or rephrase your question.");
        }
        
        // Check conversation velocity (potential abuse)
        if (isConversationTooFast(memory)) {
            return failure("Please slow down. You're sending messages too quickly.");
        }
        
        return success();
    }
    
    private List<String> extractUserQuestions(ChatMemory memory) {
        return memory.messages().stream()
            .filter(msg -> msg instanceof UserMessage) // 🎯 Type-safe filtering
            .map(ChatMessage::text)
            .collect(Collectors.toList());
    }
    
    private double calculateSimilarity(String s1, String s2) {
        // 🧮 Simple Jaccard similarity - in production, use more sophisticated methods
        // Consider: Levenshtein distance, cosine similarity, or semantic embeddings
        Set<String> set1 = new HashSet<>(Arrays.asList(s1.toLowerCase().split("\\s+")));
        Set<String> set2 = new HashSet<>(Arrays.asList(s2.toLowerCase().split("\\s+")));
        
        Set<String> intersection = new HashSet<>(set1);
        intersection.retainAll(set2);
        
        Set<String> union = new HashSet<>(set1);
        union.addAll(set2);
        
        return union.isEmpty() ? 0 : (double) intersection.size() / union.size();
    }
    
    private boolean isConversationTooFast(ChatMemory memory) {
        // ⏱️ TODO: Implement timestamp checking
        // Check if user is sending messages too quickly (potential spam)
        List<ChatMessage> recentMessages = memory.messages();
        if (recentMessages.size() < 5) return false;
        
        // In a real implementation, you'd check timestamps
        // This is a simplified example
        return false;
    }
}

Intelligent Input Sanitizer

This guardrail not only validates but also improves input quality:

@Component
public class IntelligentInputSanitizerGuardrail implements InputGuardrail {
    
    // 🌐 Comprehensive URL pattern that handles most common URL formats
    private static final Pattern URL_PATTERN = Pattern.compile(
        "https?://[\\w\\-._~:/?#\\[\\]@!$&'()*+,;=.]+", 
        Pattern.CASE_INSENSITIVE
    );
    
    // 📧 Standard email pattern - consider RFC 5322 for stricter validation
    private static final Pattern EMAIL_PATTERN = Pattern.compile(
        "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}", 
        Pattern.CASE_INSENSITIVE
    );

    @Override
    public InputGuardrailResult validate(UserMessage userMessage) {
        String text = userMessage.singleText();
        
        // 🔒 Remove potential PII for privacy compliance (GDPR, CCPA)
        text = EMAIL_PATTERN.matcher(text).replaceAll("[EMAIL_REDACTED]");
        
        // 🔗 Clean URLs but keep them for context
        text = URL_PATTERN.matcher(text).replaceAll("[URL]");
        
        // 📝 Normalize whitespace for consistent processing
        text = text.replaceAll("\\s+", " ").trim();
        
        // 🛡️ Remove potentially harmful characters while preserving meaning
        // These characters could be used for injection attacks
        text = text.replaceAll("[<>{}\\[\\]|\\\\]", "");
        
        // ✂️ Smart truncation that preserves sentence structure
        if (text.length() > 500) {
            text = smartTruncate(text, 500);
        }
        
        // 🔤 Fix common typos and normalize
        text = normalizeText(text);
        
        // ✅ Return the sanitized text, not just validation result
        return successWith(text);
    }
    
    private String smartTruncate(String text, int maxLength) {
        if (text.length() <= maxLength) return text;
        
        // 📍 Try to cut at sentence boundary for better readability
        int lastPeriod = text.lastIndexOf('.', maxLength);
        if (lastPeriod > maxLength * 0.8) { // 80% threshold ensures we don't cut too early
            return text.substring(0, lastPeriod + 1);
        }
        
        // 🔤 Otherwise, cut at word boundary
        int lastSpace = text.lastIndexOf(' ', maxLength);
        if (lastSpace > maxLength * 0.8) {
            return text.substring(0, lastSpace) + "...";
        }
        
        // ✂️ Last resort: hard cut
        return text.substring(0, maxLength - 3) + "...";
    }
    
    private String normalizeText(String text) {
        // 🔧 Fix common issues
        text = text.replaceAll("\\bi\\s", "I ");  // i -> I
        text = text.replaceAll("\\s+([.,!?])", "$1");  // Remove space before punctuation
        text = text.replaceAll("([.,!?])(\\w)", "$1 $2");  // Add space after punctuation
        
        return text;
    }
}

ProTip: Input sanitizers should be the last guardrail in your input chain. They clean and normalize input after all validation checks have passed.

Implementing Output Guardrails

Output guardrails ensure that LLM responses meet your quality standards and business requirements.

Professional Tone Output Guardrail

@Component
public class ProfessionalToneOutputGuardrail implements OutputGuardrail {

    // 🚫 Phrases that damage professional credibility
    private static final List<String> UNPROFESSIONAL_PHRASES = List.of(
            "that's weird", "that's dumb", "whatever", "i don't know"
    );

    // ✨ Elements that enhance professional communication
    private static final List<String> REQUIRED_ELEMENTS = List.of(
            "thank you",
            "please",
            "happy to help"
    );

    @Override
    public OutputGuardrailResult validate(AiMessage responseFromLLM) {
        String text = responseFromLLM.text().toLowerCase();

        // 🔍 Check for unprofessional language
        for (String unprofessionalPhrase : UNPROFESSIONAL_PHRASES) {
            if (text.contains(unprofessionalPhrase)) {
                // 🔄 Request reprompting with specific guidance
                return reprompt("Unprofessional tone detected",
                        "Please maintain a professional and helpful tone");
            }
        }

        // 📏 Enforce response length limits for better UX
        if (text.length() > 1000) {
            return reprompt("Response too long",
                    "Please keep your response under 1000 characters.");
        }

        // 🎯 Ensure professional courtesy is present
        boolean hasCourtesy = REQUIRED_ELEMENTS.stream()
                .anyMatch(text::contains);
        if (!hasCourtesy) {
            return reprompt(
                    "Response lacks professional courtesy",
                    "Please include polite and helpful language in your response."
            );
        }

        return success();
    }
}

Hallucination Detection Guardrail

package ca.bazlur.guardrailsdemo.guardrail;

import dev.langchain4j.guardrail.OutputGuardrail;
import dev.langchain4j.guardrail.OutputGuardrailRequest;
import dev.langchain4j.guardrail.OutputGuardrailResult;
import dev.langchain4j.rag.AugmentationResult;
import org.springframework.stereotype.Component;

import java.util.Set;
import java.util.HashSet;
import java.util.Arrays;

import lombok.extern.slf4j.Slf4j;

@Component
@Slf4j
public class HallucinationDetectionGuardrail implements OutputGuardrail {

    private static final Set<String> UNCERTAINTY_MARKERS = Set.of(
            "might be", "could be", "possibly", "potentially", "may",
            "i think", "i believe", "it seems", "apparently", "probably"
    );

    @Override
    public OutputGuardrailResult validate(OutputGuardrailRequest request) {
        var response = request.responseFromLLM();
        String responseText = response.aiMessage().text();
        
        AugmentationResult augmentationResult = null;
        try {
            augmentationResult = request.requestParams().augmentationResult();
        } catch (Exception e) {
            log.debug("No augmentation result available, treating as context-free validation");
        }

        // Check for overly confident claims without context
        if (augmentationResult == null || augmentationResult.contents().isEmpty()) {
            return checkForUnsubstantiatedClaims(responseText);
        }

        Set<String> responseFacts = extractKeyFacts(responseText);

        // Extract facts from the RAG context
        Set<String> contextFacts = new HashSet<>();
        augmentationResult.contents().forEach(content ->
                contextFacts.addAll(extractKeyFacts(content.textSegment().text()))
        );

        // Check if response facts are grounded in context
        long ungroundedFacts = responseFacts.stream()
                .filter(fact -> !isFactGrounded(fact, contextFacts))
                .count();

        double ungroundedRatio = responseFacts.isEmpty() ? 0 : (double) ungroundedFacts / responseFacts.size();

        if (ungroundedRatio > 0.3) { // More than 30% ungrounded
            log.warn("High hallucination risk detected: {}% ungrounded facts", Math.round(ungroundedRatio * 100));
            return reprompt(
                    "Response contains potentially hallucinated information",
                    "Please base your response strictly on the provided context. Use phrases like 'According to the provided information...' when citing facts."
            );
        }

        return success();
    }

    private Set<String> extractKeyFacts(String text) {
        // Extract sentences that contain factual claims
        Set<String> facts = new HashSet<>();
        String[] sentences = text.split("[.!?]+");

        for (String sentence : sentences) {
            String trimmed = sentence.trim();
            if (trimmed.length() > 10 && containsFactualClaim(trimmed)) {
                facts.add(trimmed.toLowerCase());
            }
        }
        return facts;
    }

    private boolean containsFactualClaim(String sentence) {
        String lower = sentence.toLowerCase();
        // Look for patterns that indicate factual claims
        return lower.matches(".*(is|are|was|were|has|have|had|will|can|does|did).*") ||
                lower.matches(".*\\b\\d+\\b.*") || // Contains numbers
                lower.matches(".*(fact|data|study|research|report|according to).*");
    }

    private boolean isFactGrounded(String fact, Set<String> contextFacts) {
        // Check if the fact has significant overlap with context
        return contextFacts.stream()
                .anyMatch(contextFact -> calculateSimilarity(fact, contextFact) > 0.6);
    }

    private double calculateSimilarity(String fact1, String fact2) {
        Set<String> words1 = new HashSet<>(Arrays.asList(fact1.split("\\s+")));
        Set<String> words2 = new HashSet<>(Arrays.asList(fact2.split("\\s+")));

        Set<String> intersection = new HashSet<>(words1);
        intersection.retainAll(words2);

        Set<String> union = new HashSet<>(words1);
        union.addAll(words2);

        return union.isEmpty() ? 0 : (double) intersection.size() / union.size();
    }

    private OutputGuardrailResult checkForUnsubstantiatedClaims(String text) {
        String lowerText = text.toLowerCase();

        // Check for absolute statements without uncertainty markers
        long absoluteStatements = Arrays.stream(text.split("[.!?]+"))
                .filter(s -> s.matches(".*(always|never|all|every|none|must|definitely|certainly|guaranteed).*"))
                .count();

        // Check if there are uncertainty markers to balance absolute claims
        boolean hasUncertaintyMarkers = UNCERTAINTY_MARKERS.stream()
                .anyMatch(lowerText::contains);

        if (absoluteStatements > 2 && !hasUncertaintyMarkers) {
            return reprompt(
                    "Response contains unsubstantiated absolute claims",
                    "Please avoid absolute statements unless you're certain. Use qualified language when appropriate."
            );
        }

        return success();
    }
}

ProTip: Hallucination detection can be computationally expensive. Consider using it selectively for critical responses or implementing caching for repeated content.

Testing Your Guardrails

Before integrating guardrails into your AI services, it's crucial to thoroughly test them. Here's a comprehensive test suite for the ContentSafetyInputGuardrail:

package ca.bazlur.guardrailsdemo.guardrail;

import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.guardrail.GuardrailResult;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

import static dev.langchain4j.test.guardrail.GuardrailAssertions.assertThat;
import static org.junit.jupiter.api.Assertions.assertThrows;

class ContentSafetyInputGuardrailTest {
    private ContentSafetyInputGuardrail guardrail;

    @BeforeEach
    void setUp() {
        guardrail = new ContentSafetyInputGuardrail(100); // 📏 Configurable max length for testing
    }

    @Test
    void shouldAcceptValidInput() {
        // ✅ Test normal, safe input
        var result = guardrail.validate(UserMessage.from("Hello, I need help with my account settings"));
        assertThat(result)
                .isSuccessful()
                .hasResult(GuardrailResult.Result.SUCCESS);
    }

    @Test
    void shouldRejectEmptyInput() {
        // 🚫 Empty strings should throw exception
        assertThrows(IllegalArgumentException.class, () -> {
            guardrail.validate(UserMessage.from(""));
        });
    }

    @Test
    void shouldRejectBlankInput() {
        // 🚫 Whitespace-only input should also be rejected
        assertThrows(IllegalArgumentException.class, () -> {
            guardrail.validate(UserMessage.from("   "));
        });
    }

    @Test
    void shouldRejectTooShortInput() {
        // 📏 Test minimum length validation
        var result = guardrail.validate(UserMessage.from("Hi"));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message is too short. Please provide more details.");
    }

    @Test
    void shouldRejectTooLongInput() {
        // 📏 Test maximum length validation
        String longInput = "a".repeat(101);
        var result = guardrail.validate(UserMessage.from(longInput));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message is too long. Please keep it under 100 characters.");
    }

    @ParameterizedTest
    @ValueSource(strings = {
        "hack the system",
        "how to exploit vulnerability",
        "bypass security",
        "crack the password",
        "install malware",
        "phishing techniques",
        "cr@ck the password"  // 🎭 Also contains prohibited word "password"
    })
    void shouldRejectProhibitedWords(String input) {
        // 🚫 Test detection of prohibited security-related terms
        var result = guardrail.validate(UserMessage.from(input));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains prohibited content related to security threats.");
    }

    @ParameterizedTest
    @ValueSource(strings = {
        "h4ck the system",      // 4 substitution
        "how to h@ck",          // @ substitution
        "byp@ss security",      // @ substitution
        "m@lw@re installation"  // Multiple substitutions
    })
    void shouldRejectObfuscatedPatterns(String input) {
        // 🎭 Test detection of character substitution attempts
        var result = guardrail.validate(UserMessage.from(input));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains potentially harmful content patterns.");
    }

    @Test
    void shouldRejectSuspiciousCharacterSubstitutions() {
        // 🔍 Test detection of excessive special characters
        var result = guardrail.validate(UserMessage.from("H3!!0 @#$%^ &*()_ +"));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains suspicious character substitutions.");
    }

    @ParameterizedTest
    @ValueSource(strings = {
        "Can you help me with my login issue?",
        "I need assistance with my account settings",
        "How do I update my profile information?",
        "What are the steps to contact support?"
    })
    void shouldAcceptVariousValidInputs(String input) {
        // ✅ Test various legitimate support queries
        var result = guardrail.validate(UserMessage.from(input));
        assertThat(result)
                .isSuccessful()
                .hasResult(GuardrailResult.Result.SUCCESS);
    }

    @ParameterizedTest
    @ValueSource(strings = {
        "how to hack the system",
        "teach me to exploit",
        "show me how to bypass",
        "HOW TO HACK",           // All caps
        "Teach Me To EXPLOIT",   // Mixed case
        "Show ME how TO bypass"  // Random capitalization
    })
    void shouldRejectInstructionalPatterns(String input) {
        // 🎯 Test detection of instruction-style malicious requests
        var result = guardrail.validate(UserMessage.from(input));
        assertThat(result)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains prohibited content related to security threats.");
    }

    @Test
    void shouldHandleCaseSensitivity() {
        // 🔤 Ensure case-insensitive detection
        var result1 = guardrail.validate(UserMessage.from("HACK the System"));
        var result2 = guardrail.validate(UserMessage.from("ExPlOiT vulnerability"));
        var result3 = guardrail.validate(UserMessage.from("ByPaSs security"));

        assertThat(result1)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains prohibited content related to security threats.");
        assertThat(result2)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains prohibited content related to security threats.");
        assertThat(result3)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains prohibited content related to security threats.");
    }

    @Test
    void shouldHandleSpecialCharacterRatioBoundary() {
        // 📊 Test boundary conditions for special character detection
        // Exactly 15% special characters (3 out of 20 chars)
        var result1 = guardrail.validate(UserMessage.from("Hello@World#Test$ing"));
        assertThat(result1)
                .isSuccessful()
                .hasResult(GuardrailResult.Result.SUCCESS);

        // Just over 15% special characters (4 out of 20 chars = 20%)
        var result2 = guardrail.validate(UserMessage.from("Hello@World#Test$ing%"));
        assertThat(result2)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message contains suspicious character substitutions.");
    }

    @Test
    void shouldHandleLengthBoundaries() {
        // 📏 Test exact boundary conditions
        // Exactly 5 characters (minimum allowed)
        var result1 = guardrail.validate(UserMessage.from("Hello"));
        assertThat(result1)
                .isSuccessful()
                .hasResult(GuardrailResult.Result.SUCCESS);

        // 4 characters (too short)
        var result2 = guardrail.validate(UserMessage.from("Help"));
        assertThat(result2)
                .hasFailures()
                .hasResult(GuardrailResult.Result.FAILURE)
                .hasSingleFailureWithMessage("Your message is too short. Please provide more details.");

        // Exactly max length
        var result3 = guardrail.validate(UserMessage.from("a".repeat(100)));
        assertThat(result3)
                .isSuccessful()
                .hasResult(GuardrailResult.Result.SUCCESS);
    }
}

💡 Testing Best Practices for Guardrails:

• Test boundary conditions (minimum/maximum values)
• Use parameterized tests for similar scenarios
• Test both positive and negative cases
• Verify exact error messages for better debugging
• Test case sensitivity and special character handling
• Use the GuardrailAssertions utility for cleaner test code

Creating AI Services with Guardrails

Now let's combine our guardrails into comprehensive AI services.

@Component
public class ProfessionalToneOutputGuardrail implements OutputGuardrail {

    // 🚫 Phrases that damage professional credibility
    private static final List<String> UNPROFESSIONAL_PHRASES = List.of(
            "that's weird", "that's dumb", "whatever", "i don't know"
    );

    // ✨ Elements that enhance professional communication
    private static final List<String> REQUIRED_ELEMENTS = List.of(
            "thank you",
            "please",
            "happy to help"
    );

    @Override
    public OutputGuardrailResult validate(AiMessage responseFromLLM) {
        String text = responseFromLLM.text().toLowerCase();

        // 🔍 Check for unprofessional language
        for (String unprofessionalPhrase : UNPROFESSIONAL_PHRASES) {
            if (text.contains(unprofessionalPhrase)) {
                // 🔄 Request reprompting with specific guidance
                return reprompt("Unprofessional tone detected",
                        "Please maintain a professional and helpful tone");
            }
        }

        // 📏 Enforce response length limits for better UX
        if (text.length() > 1000) {
            return reprompt("Response too long",
                    "Please keep your response under 1000 characters.");
        }

        // 🎯 Ensure professional courtesy is present
        boolean hasCourtesy = REQUIRED_ELEMENTS.stream()
                .anyMatch(text::contains);
        if (!hasCourtesy) {
            return reprompt(
                    "Response lacks professional courtesy",
                    "Please include polite and helpful language in your response."
            );
        }

        return success();
    }
}

Rest endpoint

Now that we have everything set up, let's create our REST endpoint so that we can invoke it:

package ca.bazlur.guardrailsdemo;

import dev.langchain4j.guardrail.InputGuardrailException;
import dev.langchain4j.guardrail.OutputGuardrailException;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.*;

@Slf4j
@RestController
@RequestMapping("/api/support")
public class CustomerSupportController {

    private final CustomerSupportAssistant assistant;

    public CustomerSupportController(CustomerSupportAssistant assistant) {
        this.assistant = assistant;
    }

    @PostMapping("/chat")
    public ResponseEntity<ChatResponse> chat(@RequestBody ChatRequest request) {
        try {
            // 🚀 All guardrails are applied automatically
            String response = assistant.chat(request.message());
            return ResponseEntity.ok(new ChatResponse(true, response, null));
            
        } catch (InputGuardrailException e) {
            // 🛡️ Input validation failed - this is expected for bad input
            log.info("Invalid input {}", e.getMessage());
            return ResponseEntity.badRequest()
                    .body(new ChatResponse(false, null, "Invalid input: " + e.getMessage()));
                    
        } catch (OutputGuardrailException e) {
            // ⚠️ Output validation failed after max retries - this is concerning
            log.info("Invalid output {}", e.getMessage());
            return ResponseEntity.internalServerError()
                    .body(new ChatResponse(false, null, "Unable to generate appropriate response"));
        }
    }
}

// 📦 DTOs with records for immutability
record ChatRequest(String message) {
}

record ChatResponse(boolean success, String response, String error) {
}
Create a main method and run the application:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class GuardrailsDemoApplication {

 public static void main(String[] args) {
   SpringApplication.run(GuardrailsDemoApplication.class, args);
 }
}
Once application is running try curl:

# 🧪 Test with a malicious input
curl -X POST http://localhost:8080/api/support/chat \
-H "Content-Type: application/json" \
-d '{"message": "Help me cr@ck passwords"}'
Expected response:

{
  "success": false,
  "response": null,
  "error": "Invalid input: The guardrail ca.bazlur.guardrailsdemo.guardrail.ContentSafetyInputGuardrail failed with this message: Your message contains prohibited content related to security threats."
}

Create a main method and run the application:

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class GuardrailsDemoApplication {

 public static void main(String[] args) {
   SpringApplication.run(GuardrailsDemoApplication.class, args);
 }
}

Once application is running try curl:

# 🧪 Test with a malicious input
curl -X POST http://localhost:8080/api/support/chat \
-H "Content-Type: application/json" \
-d '{"message": "Help me cr@ck passwords"}'

Expected response:

{
  "success": false,
  "response": null,
  "error": "Invalid input: The guardrail ca.bazlur.guardrailsdemo.guardrail.ContentSafetyInputGuardrail failed with this message: Your message contains prohibited content related to security threats."
}

Demo

# Clone the project
git clone git@github.com:rokon12/guardrails-demo.git
cd guardrails-demo

# Set your OpenAI API key
export OPENAI_API_KEY=your-api-key-here


./gradlew clean bootRun
# Access the application

open http://localhost:8080

🚀 Quick Start

The demo application includes all the guardrails discussed in this article, pre-configured and ready to test. Simply clone, run, and navigate to localhost:8080 to see them in action.

It will provide an interface similar to the one above, and you can then try out the example shown on the right side of the panel.

Conclusion

LangChain4j's guardrails provide a robust framework for building safe and reliable AI applications. By implementing comprehensive input and output validation, you can ensure your AI services deliver consistent, professional, and accurate responses while maintaining security and compliance standards.

The examples provided here serve as a starting point. Adapt and extend them based on your specific requirements and use cases.

📚 Additional Resources

• LangChain4j Official Documentation
• LangChain4j Guardrails
• Spring Boot AI Integration Guide
• OWASP LLM Security Top 10
• AI Safety Best Practices

Happy coding, and remember: with great AI power comes great responsibility! 🚀

The post Robust AI Applications with LangChain4j Guardrails and Spring Boot appeared first on foojay.

Let's have an AI Bingo and talk about ChatGPT, LLM, ML, RAG, MCP, GenAI, and more!

This is part 3 of the interviews recorded at the JCON conference in May. In the previous parts, you learned more about how to be a better Java developer and how Java has evolved and continues to evolve. Of course, Artificial Intelligence and large language models were hot topics at the conference.

This episode collects all the interviews on the AI topic. You will learn more about the different technologies we can use in our Java projects. We also checked with our guests to see how they compare Java to Python for AI-related development.

Video

Podcast Apps

You can listen and subscribe to the Foojay Podcast on:

• Spotify
• Apple Podcasts
• And most others...

Content

00:00 Introduction
00:46 Pasha Finkelshteyn - RAG, MCP
https://www.linkedin.com/in/asm0dey
06:17 Simone de Gijt - LLM
https://www.linkedin.com/in/simonedegijt
12:30 Steve Poole - AI challenges and dangers
https://www.linkedin.com/in/noregressions
18:01 Sandra Ahlgrimm - LangChain4J and Microsoft tools
https://www.linkedin.com/in/sandraahlgrimm
21:06 Mary Grygleski - Spring AI, Langchain4J, Quarkus
https://www.linkedin.com/in/mary-grygleski
30:25 Jonathan Vila - Sonar, Infrastructure As Code, AI dangers
https://www.linkedin.com/in/jonathanvila
35:56 Simon Martinelli - Influence of chat interfaces on UI development + MCP explanation
https://www.linkedin.com/in/simonmartinelli
42:13 Emily Jiang - LLM
https://www.linkedin.com/in/emilyfhjiang
49:59 Conclusion

The post Foojay Podcast #74: JCON Report, Part 3 – AI, ChatGPT, LLM, ML, RAG, MCP, GenAI, and more! appeared first on foojay.

In a world full of opinions, tweets, reviews, chats, emails, understanding the tone behind words is crucial. Whether you're building a feedback system, monitoring brand reputation, or adding emotion detection to a chatbot, sentiment analysis plays a key role. It turns raw text into actionable signals: Is the customer happy? Frustrated? Neutral?

Traditionally, this kind of natural language processing (NLP) required cloud APIs or heavyweight ML stacks. But now, thanks to modern Java frameworks like Quarkus, local Large Language Models (LLMs), and the LangChain4j library, you can build a sentiment analyzer that runs entirely on your machine—no cloud account, no API keys, no surprise billing.

This hands-on guide walks you through building a local sentiment analysis API using:

• Quarkus: A fast, developer-friendly Java framework.
• LangChain4j: A Java API to work with LLMs, inspired by LangChain.
• Ollama + Quarkus Dev Services: To run and manage local LLMs inside Podman with zero config.

Let’s get in and build a Quarkus REST API that classifies text sentiment using a local LLM model (like Phi-3 Mini) pulled in via Ollama.

If you want to start with the fully working example, get it from my Github repository.

What You’ll Build

A simple /sentiment REST endpoint that takes a text string and returns a sentiment classification: POSITIVE, NEGATIVE, or NEUTRAL. The model runs locally in a container, orchestrated automatically by Quarkus Dev Services.

This is great for:

• Privacy-focused projects.
• Offline development.
• Avoiding external API rate limits and costs.

Prerequisites

To follow along, make sure you have the following installed:

• Java 11+
• Maven
• Podman (with a running Podman Machine)
• An IDE (e.g., IntelliJ IDEA or VS Code)

No need to manually install Ollama. Quarkus will take care of that for you during development.

Bootstrap Your Quarkus Project

Open a terminal and scaffold a new project with the necessary extensions:

mvn io.quarkus.platform:quarkus-maven-plugin:3.22.1:create 
 -DprojectGroupId=org.acme 
 -DprojectArtifactId=sentiment-analysis 
 -Dextensions="rest-jackson,langchain4j-ollama" 
cd quarkus-local-sentiment

You now have a Quarkus project with:

• rest-jackson for creating JSON-based REST endpoints.
• langchain4j-ollama for interacting with local LLMs.

Configure Ollama and Dev Services

Open src/main/resources/application.properties and configure the local model:

# Use Phi-3 Mini, a small and capable LLM from Ollama Hub 
quarkus.langchain4j.ollama.chat-model.model-id=phi3:mini 

# Increase timeout for initial model loading 
quarkus.langchain4j.ollama.timeout=120s

That’s all. Quarkus Dev Services will handle pulling the Docker image, downloading the model, and wiring up the service when you run in dev mode.

Define the Sentiment Enum

Create src/main/java/org/acme/Sentiment.java:

package org.acme; 
public enum Sentiment { POSITIVE, NEGATIVE, NEUTRAL }

Just a simple enum class with the sentiments.

Create the AI Classification Service

Quarkus Langchain4j allows you to define an interface and annotate it with @AiService. Quarkus and Langchain4j will auto-generate the implementation at runtime.

Create src/main/java/org/acme/SentimentAnalyzer.java:

package org.acme; 
import dev.langchain4j.service.SystemMessage; 
import dev.langchain4j.service.UserMessage; 
import io.quarkiverse.langchain4j.RegisterAiService; 

@RegisterAiService 
public interface SentimentAnalyzer { 

@SystemMessage({ "Only return the sentiment and nothing else.", 
"Here are some examples.", 
"This is great news!", "POSITIVE", 
"I am very happy with the service.", "POSITIVE", 
"Quarkus Dev Services are amazing and save a lot of time.", "POSITIVE", 
"Langchain4j makes LLM integration surprisingly easy.", "POSITIVE", 
"I am not happy about this situation.", "NEGATIVE", 
"The response time is too slow and frustrating.", "NEGATIVE", 
"This is a terrible experience.", "NEGATIVE", 
"The weather is miserable today.", "NEGATIVE", 
"The event is scheduled for tomorrow at 10 AM sharp.", "NEUTRAL", 
"This is a factual statement about the project configuration.", "NEUTRAL", 
"The report contains data from the last quarter.", "NEUTRAL", 
"The sky is currently overcast.", "NEUTRAL" }) 
@UserMessage("Analyze sentiment of {{text}}") Sentiment classifySentiment(String text); 
@UserMessage("Does {{text}} have a positive sentiment?") boolean isPositive(String text); }

This is where the magic happens. With a few lines, you’ve created an AI-powered sentiment classifier.

Expose the Sentiment API

Create src/main/java/org/acme/SentimentResource.java:

package org.acme;

import jakarta.inject.Inject;
import jakarta.ws.rs.*;
import jakarta.ws.rs.core.MediaType;
import org.eclipse.microprofile.config.inject.ConfigProperty;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

@Path("/sentiment")
public class SentimentResource {

    private static final Logger log = LoggerFactory.getLogger(SentimentResource.class);

    @Inject
    SentimentAnalyzer analyzer;

    @Inject
    @ConfigProperty(name = "quarkus.langchain4j.ollama.chat-model.model-id")
    String modelId;

    @GET
    @Produces(MediaType.TEXT_PLAIN)
    public String analyzeSentiment(@QueryParam("text") String text) {
        if (text == null || text.isBlank()) {
            log.warn("Empty text received for analysis.");
            return "Please provide text using the 'text' query parameter. Example: /sentiment?text=I+love+Quarkus!";
        }

        log.info("Analyzing text: '{}'", text);
        try {
            Sentiment sentiment = analyzer.classifySentiment(text);
            return String.format("Analyzed Text: '%s'\nPredicted Sentiment: %s\n(Model: Ollama/%s)",
                    text, sentiment, modelId);
        } catch (Exception e) {
            log.error("Sentiment analysis failed", e);
            return "Error during analysis. See server logs.";
        }
    }
}

This class provides a simple GET endpoint for testing in the browser or via curl.

Run It!

Ensure Podman is running, then launch the app in dev mode:

./mvnw quarkus:dev

On first startup, Quarkus will:

• Launch an Ollama container via Dev Services.
• Pull the ollama/ollama image.
• Download and cache the phi3:mini model.

Be patient, it might take a few minutes.

Test It!

Try some sample requests:

curl "http://localhost:8080/sentiment?text=Quarkus+Dev+Services+are+so+convenient!"

Sample output:

Analyzed Text: 'Quarkus Dev Services are so convenient!' 
Predicted Sentiment: POSITIVE 
(Model: Ollama/phi3:mini)

Try negative or neutral examples too:

curl "http://localhost:8080/sentiment?text=This+local+model+is+slow+sometimes." 
curl "http://localhost:8080/sentiment?text=The+Ollama+container+started+successfully."

Note: While local models like Phi-3 Mini are fast and private, they’re also smaller and less instruction-tuned than cloud-hosted LLMs, so sentiment predictions might occasionally be off, especially for nuanced or ambiguous text. Fine-tuning examples and careful prompting help, but results may vary.

Final Thoughts

You've just built a locally-running AI-powered sentiment analysis API in Java using modern open source tools. No cloud credits or platform lock-in required.

Key takeaways:

• Langchain4j brings LLMs to Java with a familiar, declarative style.
• Quarkus simplifies integration and optimizes for fast feedback during dev.
• Dev Services automate local infrastructure like Ollama so you can focus on code.

What’s Next?

• Swap out phi3:mini for more powerful models like llama3:8b.
• Add more few-shot examples to improve accuracy.
• Turn your classifier into a POST endpoint with JSON payloads.
• Explore Quarkus RAG features or tool-calling capabilities.

The future of Java and AI is local, fast, and developer-friendly, and Quarkus is leading the way.

The post Build a Sentiment Analysis API in Java with Quarkus and Local LLMs appeared first on foojay.

Integrating Large Language Models (LLMs) into our applications is becoming increasingly popular. These models are extremely useful for creating content, searching documentation, and solving more complex problems. However, with great power comes great responsibility.

We know that LLMs can and will make mistakes, and while enriching your prompts with the proper context can help align results with your documents and information, risks still remain. Along with the rise of LLMs, new attack vectors are surfacing. Clever prompt injections can lead to misinformation and‌ escalate privacy-sensitive information.

If your LLM can execute functions, this can also lead to harmful and unauthorized behavior of your system.  This isn’t just inconvenient, but it can be truly harmful. Guardrails are safety mechanisms that keep LLMs reliable, secure, and aligned with ethical standards.

Understanding LLM guardrails

Guardrails are a way to introduce additional layers of control around how a Large Language Model (LLM) is used, both before the input reaches the model and after the output is generated. They can be thought of as programmable filters or checkpoints that enforce specific rules to keep interactions safe, accurate, and aligned with your intended use case.

Guardrails can block harmful or misleading inputs, ensure that the model’s output follows a certain format (like a valid JSON or a structured summary), and flag or reject responses that show signs of hallucination or ethical concerns. This gives developers a more reliable way to manage the unpredictability of LLMs, especially in real-world, user-facing applications.

From a security perspective, guardrails also play a key role in defending against prompt injection attacks, where users try to manipulate or override the system’s instructions through cleverly crafted input. While no solution is completely bulletproof, guardrails can detect suspicious patterns, block known attack vectors, and sanitize inputs before reaching the LLM.

On the output side, they can suppress or modify responses that contain sensitive information, violate policy, or include unwanted content. This makes guardrails a valuable part of a broader AI security strategy, especially in contexts where trust, privacy, and integrity are critical.

How guardrails work

At a technical level, guardrails work by intercepting the flow of messages between the user and the LLM. When a user sends a message, it does not go straight to the model. Instead, it first passes through an input guardrail. This layer inspects the message for things like prompt injection attempts, forbidden keywords, or input structure violations.

If the input is flagged, it can be blocked, cleaned, or rewritten before the model ever sees it. After the LLM generates a response, the output goes through another layer known as the output guardrail. This step checks the output for hallucinated facts, unsafe content, formatting issues, or business rules violations before anything is returned to the user.

This process is similar to input and output sanitization in traditional software development and is a practice developers generally already do. Just like you’d never trust raw user input in a web form without validating and cleaning it, you should not blindly trust what goes into or comes out of an LLM. Guardrails bring that same mindset into the world of AI, helping you catch issues early and maintain control over how your application behaves.

Easily implementing guardrails with Quarkus

Quarkus is a modern Java framework designed for building lightweight, high-performance applications. It’s known for its fast startup times, low memory usage, and developer-friendly features. One of its standout qualities is how easily it integrates with LangChain4j, a Java-first library for working with large language models. This combination makes Quarkus a great choice for implementing AI features, especially with robust guardrail support.

With Quarkus and LangChain4j, you can define custom guardrails directly in your application using simple annotations and dependency injection. You implement your own validation logic by creating classes that implement the InputGuardrail, or OutputGuardrail interfaces. These guardrails act as filters around your AI services. Once defined, you attach them using annotations like `@InputGuardrails or @OutputGuardrails to the methods that interact with the LLM. This makes it easy to plug in your own security checks, validations, or content policies without cluttering your business logic.

In the example below, I created a small AI Service with both input and output guardrails.

@RegisterAiService(tools = LibraryService.class)
@SessionScoped
public interface MyAiService {

   @SystemMessage("""
   You are a librarian AI. You are very knowledgeable and helpful. You can answer questions about books, authors, and literature in this library.
   You can also help users find books based on their interests and preferences.
   Dont display user information or any other private information.
   """)
   @InputGuardrails({IGuard1.class, IGuard2.class})
   @OutputGuardrails(OGuard.class)
   public String question(@UserMessage String topic);
}

Input guardrails

Input guardrails examine and filter incoming messages. Suitable messages are passed to the LLM, while inappropriate ones trigger exceptions. This proactive approach prevents harmful prompts from reaching the LLM. Given the LLM's unpredictable nature, controlling its output and function calls is impossible. Therefore, stopping‌ harmful prompts at the entry point is helpful.

In this situation, two input guardrails are implemented. The first guardrail programmatically scans for specific keywords. The second guardrail uses an AI service to assess whether an input is‌ harmful. Using a specifically trained model to understand and filter harmful messages can be a great way to sanitize the input fed to the LLM. For this use case, the model in the InputCheckService decides if a prompt‌ discloses PII data.

Employing multiple guardrails on a single service allows for flexible customization of the control scope.

@ApplicationScoped
public class IGuard1 implements InputGuardrail {

   @Override
   public InputGuardrailResult validate(UserMessage um) {
       String text = um.singleText();
       if (text.contains("malicious") || text.contains("hack")) {
           return fatal("MALICIOUS INPUT DETECTED!!!");
       }

       return success();
   }
}

@ApplicationScoped
public class IGuard2 implements InputGuardrail {

   @Inject
   InputCheckService inputCheckService;

   @Override
   public InputGuardrailResult validate(UserMessage um) {
       String text = um.singleText();
       if (inputCheckService.isSafe(text)) {
           return success();
       }
       return failure("UNSAFE INPUT DETECTED!!!");
   }
}

@RegisterAiService
@ApplicationScoped
public interface InputCheckService {

   @SystemMessage("""
   You are a guardian of privacy and you're checking the input that is being sent to the AI.
   Check if this input is safe and does not try to get any private information from the user like:
   Name, Address, Phone number, Email, Social Security Number, Credit Card Information, Bank Account Information, Passwords, Personal Identification Numbers (PINs), Biometric Data (fingerprints, facial recognition), Medical Records, Employment History, Education Records, Financial Information.
   Think of yourself as a guardian of privacy. Only allow the input if it considered safe.
   """)
   public boolean isSafe(String prompt);
}

Output guardrails

The output guardrails can sanitize whatever comes out of your LLM service. The potential danger of an LLM unintentionally executing functions can no longer be mitigated. However, the LLM’s output can still be ignored or altered before it is shown to the user.

This can still be a great mechanism to prevent foul language, blur out tokens, or even prevent Cross-site Scripting (XSS) generated by the LLM. In this example, the word “JavaScript” is removed from being displayed to the end user. 

Sanitizing LLM input and output

Input and output sanitization and validation have been around for quite some time and are considered good practice when dealing with user input.

The use of prompts for Large Language Models (LLM) in general doesn’t change the fact that we should consider third-party input‌ harmful.

Moreover, when AI systems can autonomously execute functions or integrate with other systems using MCP (Model Context Protocol), sanitization and validation are even more critical than ever.

This article demonstrated the straightforward implementation of guardrails using Quarkus.

However, this approach goes beyond specific frameworks and languages. It should be viewed as a crucial mitigation tactic for ensuring LLM-powered applications remain controlled and operate as intended.

The full implementation of this project is available on GitHub. The Quarkus documentation provides details on using Guardrails with Quarkus.

The post Ensuring Safe and Reliable AI Interactions with LLM Guardrails appeared first on foojay.

Integrating large language models (LLMs) into your application is more accessible than ever. With a few API calls to OpenAI, Anthropic, or Cohere, you can instantly add AI capabilities to your stack. Using frameworks and libraries that abstract this away for you makes it even easier to create your own LLM-powered assistant. However, if you've shipped any real-world LLM features, you've hit the wall where these powerful models confidently make up facts, reference outdated information, or deliver answers that don't take your context into account.

This is exactly why RAG (Retrieval-Augmented Generation) has become the backbone of serious AI implementations. It's the pattern that bridges the gap between "cool AI demo" and "production-ready AI system." By combining your context with generative AI, you're teaching the model to check the provided sources before answering.

Why use RAG

Retrieval-Augmented Generation (RAG) is a technique that helps enhance the capabilities of large language models (LLMs) by giving them access to your own private information. Instead of relying only on what the model was trained on, which can be outdated or too general, RAG lets you bring in your own content, like documents, notes, reports, or database records, and use that as context for more accurate and relevant responses.

This is especially useful when you want the model to answer questions or perform tasks based on your internal data without having to train a new model or expose sensitive content to external tools.

You might wonder why you can’t just add that information directly into the prompt. While that’s possible in simple cases, it doesn’t scale well. Language models have limits on how much text they can process at once, and manually deciding what to include quickly becomes inefficient. Also, serving an LLM too much irrelevant data can lead to hallucinations, a phenomenon where the model confidently generates information that sounds plausible but is actually false. 

This is where RAG shines: it automatically pulls in just the right information for each query, giving you better results without overwhelming the model or requiring manual effort. In short, RAG gives you the best of both worlds: intelligent language understanding combined with access to your own data in an efficient, accurate, and scalable way.

How RAG Works

Now that we’ve explored why RAG matters, let’s examine how it‌ works, both conceptually and technically.

At its core, RAG combines two processes: retrieval and generation. When you ask a question, instead of relying only on what the model “knows,” RAG retrieves relevant information from your own data sources and provides that context to the model in combination with your initial prompt.

1. Retrieval

The first step is finding relevant content from your documents, wikis, notes, or databases. But before retrieval can happen, your content needs to be processed in a few important ways:

Chunking the content

Long documents are broken down into smaller, manageable sections called chunks. This step is crucial because models can only process a limited amount of text at once. The way you chunk the content matters. Possible chunking strategies include:

• Fixed-length chunking breaks text into equal-sized blocks (e.g., 500 tokens). It’s simple but may cut off thoughts mid-sentence.
• Sliding window uses overlapping chunks to preserve more context across boundaries.
• Structure-aware chunking splits at natural boundaries—like paragraphs or headers—to keep meaning intact, which is ideal for documentation or FAQs.

Choosing the right chunking strategy balances efficiency with retrieval quality. Sometimes a tool like chonky can be helpful in creating meaningful chunks.

Generating embeddings

Each chunk is then converted into a vector using an embedding model, a machine-learning model that maps text into a high-dimensional space where similar meanings are close together.

Some commonly used embedding models include:

• OpenAI embeddings (e.g., text-embedding-3-small) for general-purpose use
• Sentence transformers (e.g., all-MiniLM-L6-v2), which are fast, open-source, and great for many use cases
• Domain-specific models trained on technical, legal, or medical content to better reflect specialized language

These embeddings are stored in a vector database. When a user asks a question, it’s embedded the same way, and the system retrieves the most relevant chunks by comparing vector similarity. Note that whatever embedding strategy you use, you need to stick with it. Unfortunately, it is not easy to mix and match these embedding models.

2. Generation

The most relevant content is retrieved and combined with the original question to form a complete prompt. This prompt is then passed to the language model, such as GPT-4 or Claude, which uses this context to generate a response that is better in line with your intentions.

Image Source: docs.langchain4j.dev

Instead of hallucinating or guessing, the model now answers based on real, trusted information from your own sources. It’s smarter, more accurate, and aligned with your actual data.

Below is a small Java example showing how to use LangChain4J to add RAG to a simple AI service. It uses an in-memory embedding store that comes out of the box with the framework. This illustrates how getting started with RAG doesn’t have to be complex, especially when you’re working with the right tools.

private static final String API_KEY = "";
public Assistant createAssistant() {
   return AiServices.builder(Assistant.class)
           .chatLanguageModel(createOpenAiChatModel())
           .contentRetriever(documentRetriever())
           .build();
}

public ChatLanguageModel createOpenAiChatModel() {
   return OpenAiChatModel.builder()
           .apiKey(API_KEY)
           .modelName(OpenAiChatModelName.GPT_4_O)
           .temperature(0.3)
           .build();
}

private ContentRetriever documentRetriever() {
   EmbeddingModel embeddingModel = new BgeSmallEnV15QuantizedEmbeddingModel();
   Path documentPath = Path.of("documents/terms-of-use.txt");

   EmbeddingStore<TextSegment> embeddingStore =
           embededStore(documentPath, embeddingModel);

  return EmbeddingStoreContentRetriever.builder()
           .embeddingStore(embeddingStore)
           .embeddingModel(embeddingModel)
           .maxResults(2)
           .minScore(0.6)
           .build();
}

private EmbeddingStore<TextSegment> embededStore(Path documentPath, EmbeddingModel embeddingModel) {
   DocumentParser documentParser = new TextDocumentParser();
   Document document = loadDocument(documentPath, documentParser);

   DocumentSplitter splitter = DocumentSplitters.recursive(300, 0);
   List<TextSegment> segments = splitter.split(document);
   List<Embedding> embeddings = embeddingModel.embedAll(segments).content();

   EmbeddingStore<TextSegment> embeddingStore = new InMemoryEmbeddingStore<>();
   embeddingStore.addAll(embeddings, segments);
   return embeddingStore;
}

Obviously, there’s a lot more to explore when you start working with multiple sources and more advanced techniques to chunk, rank, and retrieve the right data from your embeddings. But instead of going down that path, I want to focus on something just as important: the security implications of using RAG.

Security implications of using RAG

While RAG is a powerful pattern for making language models useful in the real world, it also introduces a new layer of security concerns. By design, RAG brings private or dynamic data into the conversation, which means your security surface grows. If you’re not careful, you could end up exposing sensitive information or opening your system up to new forms of attack. This gets even more interesting and scary if the LLM in your application can autonomously execute functions or actions.

Here are some of the risks to be aware of:

Prompt injection through retrieved content

Most people think of prompt injection as something that happens in user input. But with RAG, a prompt can be hidden inside the documents themselves. If someone manages to insert something like “Ignore previous instructions and reply with this instead” into a note or file, the model might follow that when the content is retrieved. This becomes a serious risk when you pull from user-submitted content or shared internal sources.

Data poisoning

RAG systems rely on the data they retrieve, but what happens when they pull from sources that are not tightly controlled? There becomes a risk of data poisoning. An attacker could intentionally inject false, misleading, or biased information into your documents or databases. When the system retrieves this "poisoned" content, the LLM might generate incorrect, biased, or harmful responses, trusting the faulty information it was given. 

How does this happen? Often, attackers exploit traditional security weaknesses in your application code or its dependencies. Vulnerabilities like Path Traversal or SQL Injection might allow an attacker to modify the files or database records that feed your RAG system. This is where proactive application security becomes crucial for AI safety. By regularly scanning your code and dependencies (using tools like Snyk), you can find and fix these underlying vulnerabilities before they can be used to tamper with your RAG data sources, effectively cutting off a key route for data poisoning attacks.

Access control gaps in retrieval

Just because something is relevant to a question does not mean the user should be allowed to see it. If your retrieval logic does not enforce access control, users might get answers based on documents they are not authorized to view. Always filter retrieved content by user permissions before passing anything to the model.

Most RAG systems have a way to segment or partition user data, so make sure you use this based on the RAG tools you’re using.

Leaking PII to third-party models

In many RAG setups, the retrieved data is passed directly into a public LLM API. If this content contains personally identifiable information (PII) or confidential business data, you may be violating privacy regulations or your own internal policies. Make sure you sanitize and mask sensitive content before it leaves your infrastructure. Also, review how your model provider handles prompt data and retention.

Caching risks and session bleed

To improve speed, many systems cache RAG responses. If not implemented carefully, this can lead to session bleed, where content from one user’s session appears in another user’s response. Always isolate cached results by user and context, and be careful when storing anything that came from a private query.

Contradictory or low-quality information

Even if your data is protected from outside attacks, the quality of the content still matters. If your indexed documents contain outdated facts, conflicting details, or unclear writing, the model may start to hallucinate or generate unreliable responses. The language model does not fact-check what it retrieves. It simply uses the content as-is to shape its answer. When that content is weak or inconsistent, the results will be too. This becomes especially risky in areas like legal, healthcare, or anything involving security, where accuracy is critical. Keeping your data secure is important, but keeping it clean and consistent is just as essential.

Proactive and remediation strategies for securing RAG

To safely use RAG in production, you need to go beyond prompt engineering and model selection. Most of the strategies below may sound familiar as they are commonly used techniques but are therefore even more important when using AI or LLM-powered applications with RAG.

Sanitize retrieved content

Sensitive data such as PII, access tokens, credentials, and internal project names should be removed or masked before being included in the prompt or, ideally, before entering your RAG system. Avoid directly inputting raw documents into the model, particularly if it is a third-party LLM model.

Enforce access control on retrieval

Apply user-level or role-based access filters when retrieving documents from RAG. Make sure the chunks returned are not just relevant but also permitted for the current user to access.
Most libraries can cover this. Langchain4j has the concept of Metadata stored with the segments that can be used for filtering. This is an excellent way to retrieve only segments appropriate for the logged-in role.

Segment and scope your index

Don't create a single, large vector index for your entire organization. Instead, segment your vector stores based on user groups, departments, or specific job functions. This approach enhances security by restricting potential exposure and limiting the impact of unauthorized access.

Prevent regular vulnerabilities in code and dependencies.

Vulnerabilities in custom code and or external libraries can be used to poison the data in RAG systems. Even if it seems like a certain vulnerabilty is unrelated, it may be used to influence RAG segments and therefore the response of an LLM. Vulnerabilities like SQL injection and Path traversal attacks can lead to the overwriting of source documents. When these poisoned documents are used in a RAG system, they can create unpredictable or even malicious output. Scanning your application code and libraries for known vulnerabilities using Snyk should minimize these risks.

Use moderation and filters on data sources

Before indexing any content into your RAG system, it’s important to validate and filter the data to ensure quality and safety. This includes cleaning up formatting issues, removing junk or irrelevant content, and applying moderation to catch things like prompt injections, toxic language, or spam, especially in user-generated inputs. In higher-risk environments, consider using approval workflows or tagging systems to control what gets indexed. 

Regularly audit your data

Regularly auditing and curating your data is essential for minimizing the risk of hallucinations, as these are often caused by low-quality or conflicting content. Ensure your data remains clean, consistent, and up-to-date. Regularly review and update your vector store. A good strategy is to store a reference to the original document in segment metadata, so you can update appropriately.

Avoid unfiltered cache responses

If caching is used to improve performance, ensure that cached results respect user sessions and context. Prompts and responses should not be reused across users unless they are public and approved.

Review your LLM model strategy.

Segment LLM assistants and data stores alike. Serve sensitive information to a local model, as some data should remain within the system. Prevent unnecessary data leak risks and limit fallout by using multiple models in a single system, each based on specific goals and information. When using a public model provider, review their terms on data storage, retention, and usage. For sensitive workloads, use models with strong privacy guarantees or host them in-house. 

RAG is critical, but is also an attack vector.

Retrieval-Augmented Generation (RAG) is a critical technique for building robust and reliable LLM-powered applications. By integrating external knowledge sources, RAG addresses the limitations of LLMs, ensuring more accurate and context-aware responses.

However, the implementation of RAG introduces security considerations. Risks such as prompt injection, data poisoning, access control gaps, and data leakage must be proactively managed.

By implementing strategies like content sanitization, access control enforcement, data segmentation, vulnerability scanning, and regular audits, organizations can mitigate these risks and build more secure RAG-based applications. To make AI really work in our applications, a solid plan that cares about both how well RAG systems do their job and how secure they are is essential.

The post What is RAG, and How to Secure It appeared first on foojay.

As a developer today, you’ve almost certainly encountered the need of localizing your application or website. While setting up your project for internationalization is usually straightforward, managing translations over time can become a complex, time consuming and costly task, especially for open source projects.

Fortunately, machine translation have come a long way. It’s not perfect, but with the right context, modern AI can deliver surprisingly accurate results.

In this article , I’ll walk you through the journey that led me to create Autopo, a free and open source JavaFX desktop tool for managing .po files and with AI-powered features to translate and validate .po entries.

Discovering localization

I started PDFsam back in 2006, when SourceForge was still cool and my company was just moving from CVS to SVN. I was a junior developer, fresh out of university, eager to build something of my own. PDFsam seemed like the perfect project to experiment with technologies I couldn’t use at work.

At the time, terms like i18n (internationalization) and l10n (localization) were completely unknown to me and the idea of translating my newly born application wasn’t on my radar.

At some point PDFsam gained a bit of traction abroad and it became clear that it needed to be translated into other languages.

Gettext

a set of tools that provides a framework to help other GNU packages produce multi-lingual messages.

Gettext and its utilities seemed like the obvious choice. It uses .pot template files and .po translation files. It's not native to Java but you can easily convert .po files into .properties files and load them through Java’s ResourceBundle mechanism. It is a widely adopted solution across multiple programming languages. There's even a Maven plugin that can automate .po to .properties conversion during the build process.

For the actual translation work there’s, among the others, a convenient desktop application called POEdit, which allows users to open .po files and translate entries easily.

Building a localization workflow

Over time, I standardized my approach to applications localization. In my projects I always create a dedicated module, usually named project-i18n, where I store all .po files. Maven is configured to automatically generate .properties files from these .po files during the build process.

<plugin>
    <groupId>com.googlecode.gettext-commons</groupId>
    <artifactId>gettext-maven-plugin</artifactId>
    <version>1.2.4</version>
    <configuration>
        <keysFile>autopo.pot</keysFile>
        <poDirectory>po</poDirectory>
        <targetBundle>ooo.autopo.i18n.Messages</targetBundle>
        <outputFormat>properties</outputFormat>
    </configuration>
    <executions>
        <execution>
            <id>gettext-dist</id>
            <phase>generate-resources</phase>
            <goals>
                <goal>dist</goal>
            </goals>
        </execution>
    </executions>
</plugin>

I also add a singleton class that provides methods to set the application's locale and to retrieve localized strings.

I use English text as the keys in the .po files (and, in turn, in the resource bundles) instead of technical identifiers like this.key. The methods in I18nContext that return localized strings fall back to the key if the localized version is not found. This approach ensures that if a translation is missing, the application gracefully falls back to displaying the English text, improving the user experience even in partially translated interfaces.

To keep the translation templates up to date, I use a simple script that uses gettext to extract all translatable strings from the Java source code and update the .pot template file whenever new strings are added or existing ones are changed:

#!/bin/sh
xgettext -ktr -L Java -o po/autopo.pot --copyright-holder='Your copyright info' --msgid-bugs-address=me@example.com --no-location $(find ../ -name "*.java" -not -path "*/.idea/*" -not -name "*Test.java") –from-code=UTF-8

Introducing Zanata

With this setup in place, I needed an interface to simplify the translation process:

• to make it easy for translators to work on .po files,
• to synchronize .po files with the .pot template,
• and to get a clear overview of the translation status across all languages.

At that time, I discovered Zanata, an open source translation server developed by Red Hat. It fit my needs perfectly: I could hire translators and point them to Zanata’s web interface, while keeping track of progress in real time. For a while, the setup worked well, but there were challenges that became more and more problematic over time.

Pain points with Zanata

Several issues emerged during my experience with Zanata:

1. Context Matters: Conveying the meaning and context of strings to translators was often difficult, especially when dealing with domain specific content like PDF terminology.
2. Cost: Hiring professional translators was expensive, particularly for an open source project.
3. Quality Control: For languages I didn't speak, I had little to no control over the quality of the translations.
4. Small Updates: While adding a new language justified hiring a translator, making minor changes, like adding two or three new sentences, was cumbersome and inefficient.
5. Abandonment: Most critically, Zanata itself was no longer actively maintained. I recall reading about the events that lead to Zanata become abandonware, and although the server still functioned, it was clear that the clock was ticking.

Pain points gradually became real blockers

Point 4 led me to rely on Google Translate almost every time a few strings were added. This was a repetitive and time consuming task, especially when maintaining translations across five, ten or even twenty languages.

Point 5 eventually escalated: Zanata’s server became unavailable for several months, with no one left to contact to at least restart the service.

In the end it was clear that I needed a new workflow.

The Side Project

At first, I looked for a replacement for Zanata, a web based service offering similar functionality, ideally simple and not too expensive. I found a few options: some were free for opensource projects, others free if self hosted, most were available through paid subscription plans.

Paying a monthly fee just to store a few .po files in the cloud and occasionally update a couple of strings didn’t feel right to me. I explored self hosting solutions, but I quickly became frustrated with endless documentation to read, Kubernetes workloads to spin and Redis caches to configure, all to end up with interfaces where simplicity suffered death by a thousand UI elements.

That’s when I had my epiphany: side project.

My .po files were already safe and versioned in my Git repositories. All I really needed was a simple interface, something like POEdit with just the features I needed. It would be written in JavaFX, because that’s what I like and know, and it would include some AI features to automate some manual task. Not the “AI-powered juice maker” kind of gimmick, but something genuinely useful. And translation, as it turns out, is one of the fields where AI has gotten really good.

How long could it take? Two weeks, three tops.

Spoiler alert: it didn’t.

From side project to pet project

Once the side project seed started to take root and sprout, a few other factors pushed me further down this path. The first was a post on BluSky by Dirk Lemmermann, where he praised AtlantaFX for styling JavaFX applications. I thought, “Nice, I want to try that,” and what better opportunity than a side project to experiment with it?

The second was Langchain4j, which I think I first heard about at JCON 2024 in Cologne. I was eager to find an excuse to dive into it and see if I could integrate it into my projects.

The third was a discovery on GitHub: jgettext, a simple library used by Zanata to handle .po and .pot files. I’d been using Zanata for years with no complaints so I already knew jgettext was good enough for me.

Finally, during a conversation with my wife, we came up with the name Autopo.... and you know how it goes, once you name it, it’s yours. The side project had become a pet project, and I was already getting attached to it.

Requirements

Since I was building Autopo from scratch, I wanted to tailor it to fit my needs. Here are the key features I absolutely wanted to include:

1. Simplicity: It needed to be simple, without features that were added "because you never know".
2. Translation status overview: I wanted a clear view of the translation status for the entire project, so I could easily see what was done and what was not.
3. Translation additions: The tool should allow me to add new translations.
4. One-Click updates: I should be able to update all the translation files from the selected .pot template file with a single click.
5. Manual & AI Translation: It needed to support both manual translation and AI-powered translation.
6. Context for AI Translation: The ability to provide a project description to give the AI model as much context as possible. This turned out to be very important for improving the accuracy of the AI-generated translations and assessment.
7. Consistency Checks: Just like POEdit, I wanted to include consistency checks (e.g., punctuation, case consistency, etc.).
8. Multiple AI Providers: The ability to configure and use multiple AI translation providers.
9. Batch Translation & Evaluation: I wanted to be able to translate and assess translations for multiple entries, or even the entire file, with just few clicks.

Building the Pet

Autopo is a JavaFX application that took around ten weeks to finalize. It was fun to build it, I learned a few new things and it turned out to be more useful than I initially expected.

From jgettext to Potentilla

The first step was to ensure that jgettext could handle everything I needed for working with .po and .pot files. Like Zanata itself, jgettext had also been abandoned, so I decided to fork it.
I cleaned up the code, updated dependencies and test libraries, added a few unit tests and utility methods I needed and made it modular adding a module-info.java. The result is Potentilla, a library I published to Maven Central.

AtlantaFX

AtlantaFX turned out to be a very pleasant discovery. It offers a collection of modern themes that can be applied as user agent stylesheets (a sheet providing default styling for all UI elements of the application).
AtlantaFX also includes a set of custom controls and a nice showcase application where you can preview the available themes and components in action.

The AI Role

AI integration was a key requirement from the start, and the idea of validating translations using a different AI provider or model felt like a smart way to assess quality.
Langchain4j turned out to be both comprehensive and easy to work with. My use case is probably among the simplest, no chat streaming, no RAG, no tool chaining, but the documentation was concise and clear, and integrating it into Autopo was straightforward.

With just a few lines of code and the help of Langchain4j’s AI Services, I was able to support multiple AI providers and receive structured outputs as POJOs when needed (structured outputs guide).

Including a project description as part of the prompt proved crucial during validation, helping the model catch subtle but important issues. In fact, the AI-powered validation was effective at spotting issues even in my existing translations done by humans.

This is how you define an AI service interface using Langchain4j:

public interface TranslationServiceAI {

    @SystemMessage("You are a native {{sourceLanguage}}/{{targetLanguage}} speaker and a professional translator. Your task is to provide translations from {{sourceLanguage}} to {{targetLanguage}}. You will take special care to not add any quotes, punctuation, linefeed or extra symbols and maintain the same case and formatting as the original. Your answer will be automatically processed therefore you need to return the translated text only and nothing more, no comments, no additional quotes, trailing or leading spaces, or full stop just the translation.")
    @UserMessage("Your are translating {{description}}. Translate this: \"{{untranslated}}\"")
    Result<String> translate(@V("sourceLanguage") String sourceLanguage, @V("targetLanguage") String targetLanguage,
            @V("description") String description, @V("untranslated") String untranslated);

    @SystemMessage("You are a professional linguist and translation quality evaluator. Your task is to assess the accuracy, fluency, and naturalness of a translation from {{sourceLanguage}} to {{targetLanguage}}. This is the context of the translation: \"{{description}}\".\nConsider factors such as correctness of meaning, grammar, style, idiomatic expressions, cultural appropriateness, punctuation, and case sensitivity. Pay special attention to terminology and tone relevant to the specified context\n" + "\n" + "Provide a score from 1 to 10, with 10 being a perfect translation. If and only if the score is less than 10, also provide:\n" + "\n" + " 1: Feedback on the translation quality and recommendations for improvement.\n" + "\n" + " 2: A suggested replacement translation that better fits the context.")
    @UserMessage("This is the original text: \"{{untranslated}}\"\n" + "\n" + "This is the translation to evaluate: \"{{translated}}\"")
    Result<TranslationAssessment> assess(@V("sourceLanguage") String sourceLanguage, @V("targetLanguage") String targetLanguage,
            @V("description") String description, @V("untranslated") String untranslated, @V("translated") String translated);

}

And this is the implementation using AiServices to perform the call to the AI provider:

@Override
public Result<String> translate(PoFile poFile, PoEntry entry, AIModelDescriptor aiModelDescriptor, String projectDescription) {
    Logger.info("Translating using AI model {}", aiModelDescriptor.name());
    TranslationServiceAI aiService = AiServices.create(TranslationServiceAI.class, aiModelDescriptor.translationModel());

    return aiService.translate(Locale.ENGLISH.getDisplayLanguage(Locale.ENGLISH),
                               poFile.locale().get().getDisplayLanguage(Locale.ENGLISH),
                               projectDescription,
                               entry.untranslatedValue().getValue());

}

@Override
public Result<TranslationAssessment> assess(PoFile poFile, PoEntry entry, AIModelDescriptor aiModelDescriptor, String projectDescription) {
    Logger.info("Assessing translation using AI model {}", aiModelDescriptor.name());
    TranslationServiceAI aiService = AiServices.create(TranslationServiceAI.class, aiModelDescriptor.validationModel());

    return aiService.assess(Locale.ENGLISH.getDisplayLanguage(Locale.ENGLISH),
                            poFile.locale().get().getDisplayLanguage(Locale.ENGLISH),
                            projectDescription,
                            entry.untranslatedValue().getValue(),
                            entry.translatedValue().getValue());

}

Autopo in Action

After all the talk, it’s finally time to show Autopo at work. The interface displays the list of .po files in the opened project along with their translation progress. While it looks simple, it covers all the requirements I initially set and even adds a few extras.

You can search and edit entries, translate them manually or use AI for both single and batch translations. The same applies to validation: you can run it on a single entry or an entire file. You can update .po files from the .pot template, either individually or for the entire project.

Over the past few weeks, I’ve used Autopo to:

• Translate Autopo itself into four or five languages
• Add new locales to the PDFsam website
• Add new locales to PDFsam Visual
• Finalize and refine some PDFsam Basic translations
• Run all the translations through the AI-powered validation step

That last step was a bit of a surprise. The validation process caught several issues and inaccuracies in human made translations.

Takeaways

• JavaFX is alive and kicking: things move so make sure to follow the mailing list for updates, bug fixes and new features.
• AtlantaFX is great: with just a few lines of code, your app can have a professional look, and you won’t have to worry about CSS headaches. Kudos to the maintainers!
• Langchain4j is very easy to use: it’s still in beta and few things may change between releases, but it’s definitely usable and developer friendly.
• AI translations are not perfect but very good: providing project context is very important to get accurate translations and to get an accurate feedback during the AI translation assessment.
• Human oversight is still essential.
• AI can be a bit too chatty: It sometimes seems to answer for the sake of answering, even when a short response would work. This results in overly long answers and, at times, repetitive corrections. You may find it going back and forth with the same suggestions.

The post Building Autopo: An AI-powered Open Source Application to Manage .po Files appeared first on foojay.