article: Anthropic Study: AI Coding Assistance Reduces Developer Skill Mastery by 17%
authors: Steef-Jan Wiggers, InfoQ
date: 2026-02-23
desc.: This article provides additional commentary on the research paper recently published by Anthropic. The original article is included below to allow readers to obtain a complete picture of the challenge. Some previous issues of the JC-AI Newsletter contain multiple research studies related to published findings on various groups of individuals.
category: opinion

article: How AI assistance impacts the formation of coding skills
authors: Anthropic
date: 2026-01-29
desc.: Previous editions of this AI Newsletter have covered multiple clinical studies examining the impact of AI-assisted advisory tools. The findings appear consistent with earlier research on individuals who tend to defer to navigation systems rather than their own spatial judgment.
Anthropic has conducted its own study on this phenomenon. In a randomized controlled trial, researchers investigated two questions: first, how quickly software developers acquired a new skill, specifically, proficiency with a Python library, with and without AI assistance; and second, whether AI use reduced their comprehension of the code they had just written.
The results showed that AI assistance was associated with a statistically significant decline in knowledge retention. On a quiz covering concepts participants had applied only minutes earlier, those in the AI-assisted group scored 17 percentage points lower than their counterparts who had coded manually, a gap equivalent to nearly two letter grades. While AI assistance modestly accelerated task completion, this effect did not reach statistical significance. At this stage, drawing direct comparisons with clinical findings may prove difficult.
category: research

article: Censored LLMs as a Natural Testbed for Secret Knowledge Elicitation
authors: Helena Casademunt, Bartosz Cywiński, Khoi Tran, Arya Jakkli, Samuel Marks, Neel Nanda (Harvard University, Antropic …)
date: 2026-03-05
desc.: Large language models (LLMs) sometimes produce false or misleading responses. Two primary approaches address this problem: honesty elicitation (modifying prompts or model weights so that the model responds truthfully) and lie detection, which involves classifying false responses.
Prior work evaluates such methods on models specifically trained to lie or conceal information, however, these artificial constructions may not accurately reflect naturally occurring dishonesty. This article proposes an alternative approach such as studying open-weight LLMs developed by Chinese developers, which are trained to censor politically sensitive topics. The findings indicate that no single technique fully eliminates false responses.
category: research

article: Probing Materials Knowledge in LLMs: From Latent Embeddings to Reliable Predictions
authors: Vineeth Venugopal, Soroush Mahjoubi, Elsa Olivetti (MIT)
date: 2026-03-02
desc.: Large language models are increasingly applied to materials science, yet fundamental questions remain about their reliability and knowledge encoding. This study evaluates 25 LLMs across four materials science tasks, encompassing over 200 base and fine-tuned configurations. The findings reveal that output modality fundamentally determines model behavior. For symbolic tasks, fine-tuning converges to consistent, verifiable answers with reduced response entropy, while for numerical tasks, fine-tuning improves prediction accuracy but models remain inconsistent across repeated inference runs, limiting their reliability as quantitative predictors. Models were tracked over 18 months, with observations revealing a 9–43% performance variation that poses reproducibility challenges for scientific and industrial applications.
category: research

article: Is AI Hiding Its Full Power? With Geoffrey Hinton
authors: StarTalk, Geoffrey Hinton
date: 2026-02-28
desc.: In this interview, Hinton addresses pressing questions about employment in the age of AI, beginning with the fundamental shift from logic-based, rule-driven programming to a biologically inspired approach. As the field looks toward the future, the conversation turns to weightier concerns , the enormous energy demands of data centers, and whether AI itself might accelerate breakthroughs in solar technology to meet them.
Hinton introduces the "Volkswagen Effect": the possibility that a model might strategically underperform in order to avoid being shut down. The discussion then ventures into the philosophy of consciousness, asking whether subjective experience is simply a byproduct of complex perception and whether today's chatbots might already possess some form of it. Both the promise and the peril are examined in full.
As for the singularity? It may not be imminent but that word yet is doing a great deal of heavy lifting.
category: youtube

article: Lifelong Imitation Learning with Multimodal Latent Replay and Incremental Adjustment
authors: Fanqi Yu, Matteo Tiezzi, Tommaso Apicella, Cigdem Beyan, Vittorio Murino
date: 2026-03-11
desc.: This article introduces a lifelong imitation learning framework designed to enable continual policy refinement across sequential tasks under realistic memory and data constraints. The proposed Multimodal Latent Replay (MLR) method stores joint compact latent representations that jointly encapsulate visual, linguistic, and state-based modalities, including robot orientation and position, alongside their corresponding control commands.
When evaluated on the LIBERO benchmark, the presented method achieves a 65% reduction in catastrophic forgetting compared to standard approaches across the tested scenarios. The authors note that further research is needed to validate the method's performance in complex, real-world environments.
category: research

article: Colluding LoRA: A Composite Attack on LLM Safety Alignment
authors: Sihao Ding
date: 2026-03-13
desc.: The article presents Colluding LoRA (CoLoRA), an attack where multiple seemingly harmless adapters work in tandem to disable model safety guardrails through linear composition. Unlike traditional trigger-based attacks, CoLoRA’s refusal suppression is inherent to the combination of the adapters themselves. Although this discovery poses dual-use risks for decentralized model sharing, the authors argue that disclosing this vulnerability is a necessary step toward securing the broader AI landscape.
category: research

article: When LLM Judge Scores Look Good but Best-of-N Decisions Fail
authors: Eddie Landesberg
date: 2026-03-12
desc.: Practitioners increasingly rely on reward models(GPT 5.2, Claude Sonnet 4, Gemini etc) as well as LLM-based judges for best-of-n selection, reranking, and model iteration. A common validation approach involves a single global metric, such as correlation, average error, or pairwise win-rate. When such a metric yields a seemingly acceptable result (e.g., r ≈ 0.5), teams often conclude that the judge is reliable enough to optimize against. That assumption can fail.
This article investigates how aggregate validity metrics may substantially overstate an LLM judge's practical utility for within-prompt optimization. Specifically, a judge may appear adequate according to a single global metric while still producing poor best-of-n selection decisions. The article discusses these limitations in detail, addresses the associated challenges, and outlines directions for future research.
category: research

article: Continual Learning in Large Language Models: Methods, Challenges, and Opportunities
authors: Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li, Xuemin Lin
date: 2026-03-13
desc.: Continual learning (CL) has emerged as a pivotal paradigm enabling large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting. This article provides a comprehensive analysis covering key evaluation metrics, including forgetting rates and knowledge transfer efficiency, along with emerging benchmarks for assessing CL performance. Although results appear promising, LLMs' internal knowledge remains largely static, and continual learning continues to require further research. Complementing these findings, the article presents a practical framework for addressing challenges related to the forgetting phenomenon.
category: research

article: Can LLMs Model Incorrect Student Reasoning? A Case Study on Distractor Generation
authors: Yanick Zengaffinen, Andreas Opedal, Donya Rooein, Kv Aditya Srivatsa, Shashank Sonkar, Mrinmaya Sachan
date: 2026-03-16
desc.: Modeling plausible student misconceptions is critical for AI in education. This article reveals the failure modes in which errors arise primarily from shortcomings in recovering the correct solution and selecting among response candidates, rather than from simulating errors or structuring the process. Consistent with these findings, providing the correct solution in the prompt improves alignment with human-authored distractors by 8%, highlighting the critical role of anchoring to the correct solution when generating plausible incorrect student reasoning. Overall, this article provides a structured and interpretable lens into LLMs' ability to model incorrect student reasoning and produce high-quality distractors. The topic still requires future research.
category: research

article: Agent Commander: Promptware-Powered Command and Control
authors: wunderwuzzi, EmbraceTheRed
date: 2026-03-16
desc.: The article examines prompt-based command and control (C2), an increasingly relevant threat vector. While users may grow more comfortable trusting AI agents over time, LLM outputs are inherently probabilistic and therefore untrusted, meaning they can potentially instruct an agent to perform harmful or malicious actions. The article outlines several considerations for mitigating and responding to the prompt injection challenge, particularly as the associated attack surface continues to expand.
category: tutorial

article: TRACE: Evaluating Execution Efficiency of LLM-Based Code Translation
authors: Zhihao Gong, Zeyu Sun, Dong Huang, Qingyuan Liang, Jie M. Zhang, Dan Hao
date: 2026-03-17
desc.: This article presents TRACE, a benchmark that explicitly exposes efficiency gaps beyond correctness through progressive stress test generation and efficiency-critical task selection. From an evaluation of 28 models, findings reveal that correctness is a weak predictor of efficiency, inefficiencies are both prevalent and patterned, and inference-time prompt strategies deliver limited and model-dependent gains. The article highlights the open challenge of developing training paradigms that endow LLMs with intrinsic efficiency awareness for code translation.
category: research

The post JC-AI Newsletter #15 appeared first on foojay.

article: Exclusive: Why are Chinese AI models dominating open-source as Western labs step back?
authors: Dashveenjit Kaur, AI News
date: 2026-02-09
desc.: A shift in what AI models are being used and where the models are being produced.
category: opinion

article: Machines of Loving Grace
authors: Dario Amodei
date: 2024-10-01
desc.: Although the article is older, it remains relevant for any author aiming to sketch a future in which everything with AI goes right. In light of recent developments, which appear to follow a sigmoid curve rather than exponential growth (marked by stagnation, with current models reaching a point where another breakthrough is required), the trajectory looks more measured than initially anticipated. Although the author discusses multiple risks (grandiosity, market forces, propaganda, sci-fi-like expectations, etc.), he also highlights the bright sides and explores areas where current AI may prove genuinely helpful. The question remains whether the current state of affairs can truly guarantee progress, rather than causing damage through non-deterministic outcomes (education, industry, human creativity etc.).
category: opinion

article: The Urgency of Interpretability
authors: Dario Amodai
date: 2025-04-01
desc.: The author describes lessons learned from current AI development and adds multiple valuable thoughts and facts to consider when interacting with AI models. The main point is that progress in the underlying technology is inexorable, driven by forces too powerful to stop, but what matters is the way in which it unfolds. Accepting that the current evolution of LLM-based AI cannot be halted, the author expresses hope that it may still be guided (this fact affect not only entire industry but also human kind thoughs and perception of reality), much like a bus controlled by a steering wheel, and warns of the dangers of ignorance, illustrating this through several concrete examples.
category: opinion

article: From Delegates to Trustees: How Optimizing for Long-Term Interests Shapes Bias and Alignment in LLM
authors: Suyash Fulay, Jocelyn Zhu, Michiel Bakker (MIT)
date: 2025-10-14
desc.: The article addresses the question of 'behavioral cloning', specifically, how accurately LLMs reproduce individuals' expressed preferences. Large language models have demonstrated promising accuracy in predicting survey responses and policy preferences, which has fueled growing interest in their potential to represent human interests across various domains. Drawing on theories of political representation, the article highlights an underexplored design trade-off: whether AI systems should act as delegates, mirroring expressed preferences, or as trustees, acting in users' broader interests. Models may align well with users' short-term preferences while failing to account for their long-term interests. Studies further indicate greater bias in topics where consensus is lacking.
category: research

article: DARE-bench: Evaluating Modeling and Instruction Fidelity of LLMs in Data Science
authors: Fan Shu, Yite Wang, Ruofan Wu, Boyi Liu, Zhewei Yao, Yuxiong He, Feng Yan
date: 2026-02-27
desc.: The article addresses the challenge posed by fast-growing demand for Large Language Models (LLMs) to tackle complex, multi-step data science tasks, which has created an urgent need for accurate benchmarking. Two major gaps are identified in existing benchmarks: (i) the lack of standardized, process-aware evaluation that captures instruction adherence and process fidelity, and (ii) the scarcity of accurately labeled training data. While highlighting that even capable models (Anthropic, OpenAI, etc.) may struggle in performance, the article introduces the DARE-bench benchmark alongside supervised fine-tuning as approaches that may improve outcomes in specific applications. Although the results appear promising, they retain considerable potential for further improvement, as accuracy is not yet guaranteed.
category: research

article: Do LLMs Benefit From Their Own Words?
authors: Jenny Y. Huang, Leshem Choshen, Ramon Astudillo, Tamara Broderick, Jacob Andreas (MIT, IBM Research)
date: 2026-02-27
desc.: The article aims to answer the question of whether preserving past assistant responses is more beneficial than harmful. The study uses in-the-wild, multi-turn conversations and compares standard (full-context) prompting with a user-turn-only prompting approach that omits all previous assistant responses, evaluated across three open reasoning models and one state-of-the-art model. Surprisingly, omitting past assistant responses does not negatively affect response quality in a large fraction of turns and may also reduce token length. The article concludes with a discussion of findings and directions for future research.
category: research

article: SafeGen-LLM: Enhancing Safety Generalization in Task Planning for Robotic Systems
authors: Jialiang Fan, Weizhe Xu, Mengyu Liu, Oleg Sokolsky, Insup Lee, Fangxin Kong
date: 2026-02-27
desc.: Safety-critical task planning in robotic systems remains a significant challenge: classical planners suffer from poor scalability, reinforcement learning (RL)-based methods generalize poorly, and base large language models (LLMs) cannot guarantee safety. To address this gap, the article proposes SafeGen-LLM, a safety-generalizable large language model framework. As part of this contribution, a multi-domain Planning Domain Definition Language 3 (PDDL3) benchmark with explicit safety constraints is introduced, along with Supervised Fine-Tuning (SFT) on those constraints. Although the results appear optimistic, with minimal safety violations observed across tested domains, the approach still requires further research in more complex robotic settings.
category: research

article: LemmaBench: A Live, Research-Level Benchmark to Evaluate LLM Capabilities in Mathematics
authors: Antoine Peyronnet, Fabian Gloeckle, Amaury Hayat
date: 2026-02-27
desc.: Existing benchmarks largely rely on static, hand-curated sets of contest or textbook-style problems as proxies for mathematical research. The article introduces a novel approach leveraging state-of-the-art models (GPT-5, Gemini 2.5, Gemini 3, Claude Opus 4.5, and DeepSeek-R) by extracting lemmas from arXiv and updating them dynamically. This results in a benchmark that can be refreshed regularly with new problems drawn directly from current mathematical research, while previous instances can be used for training without compromising future evaluations. This approach achieves 10–15% accuracy in theorem proving and opens a new frontier for future research. Although the process may appear fully automated, a human in the loop, such as the article's author or reviewer, remains critically necessary to produce high-quality inputs and to effectively use LLM models.The results also indicate that it is considerably easier for a model to validate an existing proof than to produce one.
category: research

article: Task Complexity Matters: An Empirical Study of Reasoning in LLMs for Sentiment Analysis
authors: Donghao Huang, Zhaoxia Wang
date: 2026-02-27
desc.: It is a well-established narrative that reasoning in large language models (LLMs) universally improves performance across language tasks. This article aims to test that claim through a comprehensive evaluation of 504 configurations across seven models, considering different reasoning architectures such as adaptive, conditional, and reinforcement-based approaches. The findings reveal that the effectiveness of reasoning is strongly task-dependent and degrades for simpler tasks. The article provides quantitative findings alongside error analysis and outlines directions for future research.
category: research

article: Benchmarking LLM Summaries of Multimodal Clinical Time Series for Remote Monitoring
authors: Aditya Shukla, Yining Yuan, Ben Tamo, Yifei Wang, Micky Nnamdi and others
date: 2026-03-02
desc.: Large language models (LLMs) can generate fluent clinical summaries of remote therapeutic monitoring time series, however, the impact of information bias on clinically significant events, such as sustained abnormalities, remains poorly understood. The article presents the Technology-Integrated Health Management (TIHM) framework to address these questions, introducing a protocol that measures abnormality recall, duration recall, and measurement coverage, while utilizing GPT-4o-mini as a proxy evaluator. Traditional models frequently exhibit near-zero abnormality recall, whereas the vision-based approach achieves the strongest event alignment, with 45.7% abnormality recall and 100% duration recall. These results underscore the need for event-aware evaluation methods in future research to ensure reliable clinical time-series summarization.
category: research

article: Full interview: Anthropic CEO responds to Trump order, Pentagon clash
authors: CBS News
date: 2026-02-28
desc.: Anthropic CEO Dario Amodei sat down with CBS News for an exclusive interview, hours after Defense Secretary Pete Hegseth declared the company a supply chain risk to national security, which restricts military contractors from doing business with the AI giant. Amodei called the move "retaliatory and punitive," and he said Anthropic sought to draw "red lines" in the government's use of its technology because "we believe that crossing those lines is contrary to American values, and we wanted to stand up for American values.". Response of the OpenAI striking a deal with Pentagon causes many questions.
category: youtube

article: Scary Agent Skills: Hidden Unicode Instructions in Skills ...And How To Catch Them
authors: Embrace The Red
date: 2026-02-11
desc.: Skills introduce common threats such as prompt injection, supply chain attacks, remote code execution (RCE), and data exfiltration, among others. This post discusses the fundamentals, highlights the most straightforward prompt injection vector, and demonstrates how a real Skill from OpenAI can be back-doored using invisible Unicode Tag code-points, a technique that certain models, including Gemini, Claude, and Grok, are known to interpret as instructions. From a security perspective, Skills present serious concerns, as they represent a typical supply chain risk with limited governance or security controls. The author identified that some Skills instruct the AI to embed API tokens directly in curl requests and similar constructs , a poor design practice. This means that credentials are passed through the LLM, making them susceptible to leakage and leaving them vulnerable to being overwritten by an attacker via indirect prompt injection.
category: tutorial

The post JC-AI Newsletter #14 appeared first on foojay.

While significant activity characterizes the AI field, many unresolved research, design, and implementation challenges continue to impact progress. Future advancement depends heavily on understanding the nature of these challenges to approach probabilistic problems from the appropriate directions. This JC-AI newsletter features insightful interviews with key figures in the field, enabling readers to ask the right questions and compare visions of an 'uncertain future' against current capabilities to maintain a grounded perspective.

article: Deep Researcher with Sequential Plan Reflection and Candidates Crossover (Deep Researcher Reflect Evolve)
authors: Saurav Prateek
date: 2026-01-28
desc.: This paper introduces Deep Researcher, a novel architecture that shifts the paradigm from latency-optimized parallel scaling to an accuracy-driven sequential refinement model. Within the development of Deep Research Agents (DRAs), two primary paradigms are considered, Parallel Scaling and Sequential Refinement. The Deep Researcher agent achieved an overall score of 46.21 on the Research Bench, demonstrating superior performance compared to existing agents, including Claude Researcher, Nvidia AIQ Research Assistant, Perplexity Research, Kimi Researcher, and Grok Deep Search. While these improvements are good, the field requires further research to address remaining challenges.
category: research

article: Manipulation in Prediction Markets: An Agent-based Modeling Experiment
authors: Bridget Smart, Ebba Mark, Anne Bastian, Josefina Waugh (University of Oxford)
date: 2026-01-28
desc.: The paper investigates the utilization of agentic systems in the economic field and their impact on prediction. First, the paper evaluates an agent-based model of a prediction market in which bettors with heterogeneous expertise, noisy private information, variable learning rates, and budgets observe the evolution of public opinion on a binary election outcome to inform their betting strategies in the market. The agentic system exhibits stability across experiments. The second area relates to experiments on how "whale" agents, a highly resourced minority with biased information, may distort market prices and for how long. The paper discusses interesting simulation results on how biased information may change the market from a long-term perspective.
category: research

article: Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents
authors: Qihao Wang, Yue Hu, Mingzhe Lu, Jiayue Wu, Yanbing Liu, Yuanmin Tang
date: 2026-01-28
desc.: While LLMs' ability to use external tools enables powerful real-world applications, current benchmarks focus on final accuracy rather than revealing the cognitive bottlenecks that limit their true capabilities. This paper presents a framework based on Cognitive Load Theory that aims to decompose tasks into two components: Intrinsic Load and Extraneous Load. The paper discusses performance inconsistencies as cognitive load increases, and demonstrates how the proposed framework enables the identification of capability boundaries in the examined examples.
category: research

article: Build a Prompt Learning Loop - SallyAnn DeLucia & Fuad Ali, Arize
authors: AI Engineer, Sally Ann Delucia, Fuad Alli (Arize)
date: 2026-01-06
desc.: This talk aims to provide ideas on how it is possible to improve LLM responses by using feedback loops. It's important to view this talk through the lens of current research results regarding the LLM hallucination phenomenon and other factors. The main reason to keep current research results in mind is to avoid ending up in an infinite loop of failure/error.
category: youtube

article: Stanford CS230 | Autumn 2025 | Lecture 8: Agents, Prompts, and RAG
authors: Stanford Online
date: 2025-11-11
desc.: For more information about Stanford’s Artificial Intelligence professional and graduate programs
category: youtube, tutorial

article: Developer Experience in the Age of AI Coding Agents – Max Kanat-Alexander, Capital One
authors: AiEngineer, Max Kanat-Alexander
date: 2025-12-23
desc.: It feels like every two weeks, the world of software engineering is being turned on its head. Are there any principles we can rely on that will continue to hold true, and that can help us prepare for the future, no matter what happens? Max uses research, data, and his 20+ years working in enterprise Developer Experience teams to talk through what we can do now that will prepare us for an agentic future, no matter what that future holds.
category: youtube, opinion

article: Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decoding
authors: Yifan Zhu, Huiqiang Rong, Haoran Luo
date: 2026-01-29
desc.: Hallucination is a recognized phenomenon in the LLM field that impacts applications such as Retrieval-Augmented Generation (RAG) and Reward Modeling (RM). This paper introduces Token-Guard, a self-checking mechanism designed to identify and control hallucinations at the token level. The experiments demonstrate improvements.
category: research

article: Reward Models Inherit Value Biases from Pretraining
authors: Brian Christian, Jessica A. F. Thompson, Elle Michelle Yang, Vincent Adam, Hannah Rose Kirk and others (University of Oxford, University Pompeu Farba)
date: 2026-01-28
desc.: Despite their importance in LLM alignment, reward models (RMs) remain under-researched. This paper provides evidence that RMs inherit biases from their base models, suggesting that the choice of an open-source model is a reflection of values as much as performance. The paper discusses limitations of experiments and offers avenues for future research.
category: research

article: Professor Geoffrey Hinton - AI and Our Future
authors: City of Hobart, Geoffrey Hinton
date: 2026-01-08
desc.: Professor Geoffrey Hinton, known as the "Godfather of AI", will discuss artificial intelligence - how it works, the risks it poses to our society, and how we might coexist with super-intelligent AI. Ideal for business leaders, creatives, researchers, educators, students and anyone curious about the future of intelligence and society.
category: opinion

article: Your MCP Server is Bad (and you should feel bad) - Jeremiah Lowin, Prefect
authors: AI Engineer, Jeremiah Lowin
date: 2026-01-12
desc.: Too many MCP servers are simply glorified REST wrappers, regurgitating APIs that were designed for SDKs rather than agents. This leads to confused LLMs, wasted tokens, and demonstrably poor performance. If you have ever pointed an MCP generator at an OpenAPI spec and called it a day, this talk is your wake-up call.
category: youtube

article: Frontier Models & AI | Sam Altman, CEO & Co-Founder, OpenAI
authors: Cisco
date: 2026-02-04
desc.: Although Sam Altman, CEO and Co-Founder of @OpenAI, explores ideas about future possibilities and potential developments, he is asked during the interview to align his vision with the current state of research and existing technological capabilities. The interview, however, does not present clear data demonstrating how Codex outperforms alternatives or what 'better' specifically means in this context. The responses to questions may appear to be non-deterministic in nature. The interview relies heavily on thoughts about an "undefined future" that would require a deterministically defined foundation. It is interesting how the interview examined frontier AI models and their implications for economies, institutions, and global systems.
category: opinion

article: How to build secure and scalable remote MCP servers
authors: Den Delimarsky (Microsoft)
date: 2025-07-25
desc.: The tutorial provides insights into how to build a reliable Model Context Protocol (MCP) server, enabling AI agents to connect to external tools. It covers several crucial areas and provides valuable resources and ideas for tackling the challenge.
category: tutorial

The post JC-AI Newsletter #13 appeared first on foojay.

Although there have been many small steps toward the field's evolution, it often feels that development is stagnating, applying known or slightly tweaked strategies to non-deterministic problems while expecting deterministic results. This includes the often misleading benchmarking strategies (deterministic) performed on synthetic datasets.

The first New Year edition of the JC-AI Newsletter aims to shed light on new approaches and movements in the field, including the directions of its evolution.

Let's jump in and happy reading!

article: Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response
authors: Aron Distelzweig, Yiwei Wang, Faris Janjoš and others
date: 2025-12-03
desc.: Autonomous driving, specifically decision-making, remains a significant challenge. While routine scenarios yield nearly perfect plans using multi-agent collaboration, dense urban traffic presents considerable difficulties, particularly for vehicle lane changes. This paper presents the Bayesian Iterative Best Response (BIReR) framework, which aims to unify motion prediction and planning based on game theory. The framework demonstrates an 11% improvement in lane change performance compared to classical approaches.
category: research

article: PBFuzz: Agentic Directed Fuzzing for PoV Generation
authors: Haochen Zeng, Andrew Bao, Jiajun Cheng, Chengyu Song
date: 2025-12-04
desc.: Proof-of-Vulnerability (PoV) input generation is a critical task in software security. Generating a PoV input requires solving two sets of constraints: (1) reachability constraints for reaching the vulnerable code location(s), and (2) triggering constraints for activating the target vulnerability. Despite dramatic advancements in the LLM field, fuzzing models struggle to solve these constraints effectively. This paper proposes the PBFuzz framework, composed of four layers and enabling property-based directed fuzzing. Although PBFuzz underperformed in several scenarios, it outperforms conventional fuzzers overall.
category: research

article: DSPy: The End of Prompt Engineering - Kevin Madura, AlixPartners Enhancement
authors: AI Engineer, Kevin Madura
date: 2026-01-08
desc.: Applications developed for enterprise environments need to be rigorous, testable, and robust. The same is true for AI-powered applications, but LLMs can make this challenging. In other words, users need to be able to program with LLMs, not just tweak prompts. This talk covers why DSPy may be all users need when building applications with LLMs. Although the talk dives into some real-world examples, the audience is encouraged to explore the DSPy tool themselves to determine whether it fits their particular needs.
category: youtube

article: From Vibe Coding To Vibe Engineering – Kitze, Sizzy
authors: AI Engineer, Ryan Florence
date: 2025-12-14
desc.: Web development has always moved in cycles of hype, from frameworks to tooling. With the rise of large language models, we're entering a new era of "vibe coding," where developers shape software through collaboration with Al rather than syntax. This talk explores what that means for the future of coding, especially in frontend development, and how it echoes the past while redefining what comes next.
category: youtube

article: The AI Bubble Should Have Never Existed In The First Place
authors: Will Lockett
date: 2025-12-07
desc.: The article elaborates on the existence of an AI bubble, arguing that so much money has been poured into AI that we have effectively bet the entire economy on its success. Regardless of whether an AI bubble exists or in what form, the article formulates valid points that should be taken into account when considering future developments.
category: opinion

article: We Let AI Run Our Office Vending Machine. It Lost Hundreds of Dollars
authors: The Wall Street Journal (Antropic)
date: 2025-12-18
desc.: In a research case study supported by Anthropic, the Claudius Agent was developed to manage vending machine operations. Testing revealed multiple exploitable vulnerabilities that allowed users to obtain goods without payment. Real-world trials consistently resulted in operational failures, with the system dispensing free products while automatically reordering inventory, a combination that would lead to bankruptcy in commercial-like deployment.
category: youtube

article: When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
authors: Yaqi Duan, Yichun Hu, Jiashuo Jiang
date: 2025-12-31
desc.: Inventory control (encompassing cash management, storage, order quantities, etc.) presents a stochastic control challenge where minor structural errors result in recurring costs. Direct interaction with LLM models may produce plausible yet systematically suboptimal or even inconsistent results. This paper proposes using LLMs not as problem solvers but as language interfaces to enhance optimization through a hybrid agentic approach.
category: research

article: Memory in LLMs: Weights and Activations - Jack Morris, Cornell
authors: AI Engineer, Jack Morris
date: 2025-12-29
desc.: This work examines memory mechanisms in large language models through the lens of weights and activations. Jack Morris addresses the limitations of current Large Language Models (LLMs) in handling niche, long-tail knowledge that falls outside their training data or beyond knowledge cutoffs. He critiques the reliance on massive context windows and Retrieval Augmented Generation (RAG), citing their high computational cost and latency due to the quadratic complexity of self-attention. The core thesis advocates for a third paradigm: training knowledge into weights, efficiently injecting specific knowledge directly into model parameters. This approach treats weights as a memory storage mechanism, conceptually distinct from the working memory represented by activations.
category: youtube

article: There are no new ideas in AI — only new datasets
authors: Jack Morris
date: 2025-07-06
desc.: This article provides a comprehensive overview of progress in the AI field over recent years. All four major breakthroughs in LLMs occurred because researchers unlocked new sources of data. The question remains: what will be the next breakthrough?
category: opinion

article: VL-JEPA: Joint Embedding Predictive Architecture for Vision-language
authors: Delong Chen, Mustafa Shukor, Theo Moutakanni, Willy Chung, Jade Yu, Tejaswi Kasarla, Allen Bolourchi, Yann LeCun, Pascale Fung
date: 2025-12-11
desc.: This paper introduces the Joint Embedding Predictive Architecture for Vision-Language models (VL-JEPA). Current Vision-Language Models (VLMs) are straightforward but inadequate for two main reasons. First, VLMs are expensive to develop. Second, real-time tasks involving live streaming video (e.g., live action tracking) require sparse and selective decoding. The paper empirically validates the advantages of this newly introduced approach against token-generative VLMs. VL-JEPA delivers consistently higher performance on zero-shot captioning and classification while improving inference-time efficiency during the training phase. Although improvements remain in the experimental stage, the work demonstrates clear benefits from scaling both parameters and dataset size.
category: research

article: Rephrasing the Web: A Recipe for Compute and Data-Efficient Language Modeling
authors: Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, Navdeep Jaitly (Carnegie Mellon Univeristy, Apple)
date: 2024-01-29
desc.: Although this paper is older, it may shed light on the approaches chosen for training LLM models and provide better understanding of their evolution. The paper proposes Web Rephrase Augmented Pre-training (WRAP), which uses an off-the-shelf instruction-tuned model to rephrase noisy input data. It offers insights into how the structure of training data impacts LLM performance.
category: research

article: When Small Models Are Right for Wrong Reasons: Process Verification for Trustworthy Agents
authors: Laksh Advani
date: 2026-01-01
desc.: This paper investigates the reasoning performance of agentic systems based on small language models (Mistral-7B, Llama-3-8B, Qwen-2.5-7B). The findings reveal statistically significant evidence that RAG systems may improve reasoning performance while simultaneously increasing the likelihood of hallucination due to the Right-for-Wrong-Reason (RWR) phenomenon. The paper introduces the Reasoning Integrity Score (RIS) approach to identify hidden flaws in reasoning processes.
category: research

The post JC-AI Newsletter #12 appeared first on foojay.

This newsletter explores the evolution of agentic AI systems, provides valuable insights into the Chain-of-Thought (CoT) approach, Vibe coding, and discusses the pattern-matching capabilities of LLMs. The newsletter features an insightful interview with Stuart J. Russell, known for his significant contributions to the AI field. Even more exciting is the published paper by Apple researchers titled 'The Illusion of Thinking...' and several immediate reactions to the authors' conclusions, which allow newsletter readers to observe current research challenges and scientific community responses. This provides readers with a vital picture of the state-of-the-art in AI research.

article: AI Expert: (Warning) 2030 Might Be The Point Of No Return! We've Been Lied To About AI!
authors: The Diary Of A CEO
date: 2025-12-04
desc.: AI Expert Stuart J. Russel, exposes the trillion-dollar AI race, why governments won’t regulate, how AGI could replace humans by 2030, and why only a nuclear-level AI catastrophe will wake us up. NOTE: During the interview, a crucial question arises: If you had a 'red button' that could erase all AI-LLM current development, would you press it?... hear the answer with reasons
category: youtube, interivew

article: Is Chain-of-Thought Reasoning of LLMs a Mirage? A Data Distribution Lens
authors: Chengshuai Zhao, Zhen Tan, Pingchuan Ma, Dawei Li, Bohan Jiang and others
date: 2025-08-02, revisited 2025-08-13
desc.: The aim of the Chain-of-Thought (CoT) approach is to produce human-like reasoning steps, but this may be more superficial than it appears. This paper studies CoT using data distribution analysis to enable observation of reasoning paths. For this purpose, the DataAlchemy environment has been designed. Systematic validation reveals that CoT exhibits sharp performance degradation when detecting unknown patterns.
category: research

article: The BS-meter: A ChatGPT-Trained Instrument to Detect Sloppy Language-Games
authors: Alessandro Trevisan, Harry Giddens, Sarah Dillon, Alan F. Blackwell (Cambridge)
date: 2024-11-22, revisited 2025-06-10
desc.: Using hypothesis-testing methods, this paper demonstrates that a statistical model of sloppy language can reliably generate the artificial output of ChatGPT to the social and workplace referred to bullshit as observed in natural human language. The paper presents an empirical investigation of LLM behavior that offers insights into language use while clarifying the social and epistemological status of LLMs themselves. The results indicate with high significance that ChatGPT's outputs resemble bullshit jobs rather than precise, factual scientific writing. While this is often evident from observing its outputs, the mechanisms by which such imprecise language is produced have not been previously established.
category: research

article: Comparison of Text-Based and Image-Based Retrieval in Multimodal Retrieval Augmented Generation Large Language Model Systems
authors: Elias Lumer, Alex Cardenas, Matt Melich, Myles Mason, Sara Dieter and others (PricewaterhouseCoopers)
date: 2025-11-25
desc.: This paper discusses the capabilities of Retrieval-Augmented Generation (RAG) systems to access multimodal knowledge bases containing both text and visual information, such as charts, for information extraction. The paper reveals limitations, such as contextual loss, and presents a novel RAG analysis approach for comparing embedding creation methods. The paper analyzes the most suitable approaches for storing embeddings that incorporate both text and visual information.
category: research

article: The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
authors: Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton and others (Apple)
date: 2025-07-07
desc.: This paper discusses the progress of language models in generating detailed reasoning processes (Chain-of-Thought) prior to producing answers and improved benchmarks performance. However, the paper argues, supported by empirical evidence, that their fundamental capabilities, scaling properties, and limitations remain poorly understood. The paper systematically reveals the limitations related to task complexity and provides directions for future research.
category: research

article: Comment on The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
authors: A. Lawsen
date: 2025-07-10
desc.: This paper responds to "The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity." It presents an alternative perspective, aiming to recontextualize the original findings while identifying three potential critical issues in the original paper: (1) Tower of Hanoi experiments risk exceeding model output token limits, (2) limitations of the automated evaluation framework employed, and (3) benchmark constraints. Nevertheless, the paper acknowledges that the original findings underscore the importance of rigorous experimental design when evaluating AI reasoning capabilities.
category: research

article: A Comment On The Illusion of Thinking: Reframing the Reasoning Cliff as an Agentic Gap
authors: Sheraz Khan, Subha Madhavan, Kannan Natarajan (Pfizer, Cambridge)
date: 2025-07-25
desc.: While the paper acknowledges the results provided by "The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity," it aims to present an alternative perspective. The paper argues that the observed failures in Chain-of-Thought reasoning do not constitute evidence of a fundamental cognitive boundary, but rather represent predictable outcomes of various system-level constraints. The paper concludes that "The Illusion of Thinking" provides a valuable contribution by developing a rigorous benchmark and demonstrating that explicit Chain-of-Thought in models such as DeepSeek-R1 and Claude 3.7 Sonnet-Thinking does not guarantee reliable execution of long plans. However, it contends that the conclusion regarding an intrinsic reasoning frontier is premature.
category: research

article: LLMs’ 'simulated reasoning' abilities are a 'brittle mirage', researchers find
authors: Kyle Orland (arstechnica)
date: 2025-08-11
desc.: Over recent months, LLMs have demonstrated capabilities in pattern matching across both structured and unstructured data. This article examines whether the responses generated by agentic systems can be considered equivalent to the logical reasoning observed in human thought processes. The presented data and cited sources raise questions about such capabilities, including concerns regarding the systems' understanding of their own generated responses. The article includes the following sections: "No One Trained Me for This," "A False Aura of Dependability," and discussions of warned findings related to "chain-of-thought" approaches with supporting references.
category: research

article: Detecting Perspective Shifts in Multi-agent Systems
authors: Eric Bridgeford, Hayden Helm
date: 2025-12-04
desc.: Let us model a situation where data-scrapers access the internet, databases, or other LLMs and, based on collected data, generate or serve decision proposals. This paper introduces the Temporal Data Kernel Perspective Space (TDKPS) approach, which aims to detect agent behavioral changes in black-box settings. The paper discusses limitations and future research proposals.
category: research

article: Strategic Self-Improvement for Competitive Agents in AI Labour Markets
authors: Christopher Chiu, Simpson Zhang, Mihaela van der Schaar (University of Cambridge)
date: 2025-12-04
desc.: The paper introduces a novel framework that captures the real-world simulation of economic forces that may shape agentic labor markets in comparison with traditional human labor markets. Although agentic labor markets will differ significantly from their human counterparts, this paper identifies critical economic forces and capabilities required by agentic systems: metacognition, competitive awareness, and long-horizon strategic planning. Despite reported limitations, self-improving agents have demonstrated superior performance compared to other agent types (e.g., CoT, ReAct).
category: research

article: Is Vibe Coding Safe? Benchmarking Vulnerability of Agent-Generated Code in Real-World Tasks
authors: Songwen Zhao, Danqing Wang and others
date: 2025-12-02
desc.: In recent months, the developer community has witnessed a rapid increase in the adoption of the "Vibe Coding" programming paradigm. "Vibe coding" practices are widely used, predominantly by beginner developers, despite unresolved concerns regarding associated risks and vulnerabilities. The paper reports that although coding agents may achieve cca. 60% solution success rates, only cca. 10% of these solutions are free from known security issues, with the possibility of introducing undocumented attack vectors remaining a significant concern.
category: research

article: Mathematical Framing for Different Agent Strategies
authors: Philip Stephens, Emmanuel Salawu (Google Cloud AI)
date: 2025-12-05
desc.: The paper introduces a probabilistic framework for comparing diverse AI agent strategies, allowing for a more detailed view of outcomes. The paper discusses the trade-offs of various architectures while highlighting the necessity of mathematical evaluation. The paper establishes that the behavior of any agentic system may be understood as a probabilistic process by framing individual agent behavior as a chain of probabilities. The paper does not question the non-deterministic nature of LLMs themselves, but rather aims to establish a "Degrees of Freedom" agentic concept and considering probability.
category: research

The post JC-AI Newsletter #11 appeared first on foojay.

This newsletter focuses on examining how agentic AI systems improve accuracy, tutorials on agentic system architecture, and importnat security challenges arising from increased not only from agentic AI systems adoption. This edition of the AI newsletter includes compelling discussions and interviews about the future of AI and approaches.

article: Introducing SWE-grep and SWE-grep-mini: RL for Multi-Turn, Fast Context Retrieval
authors: Ben Pan, Carlo Baronio, Albert Tam, Pietro Marsella and others
date: 2025-10-16
desc.: Modern coding agents face a fundamental trade-off between speed and intelligence. The article presents SWE-grep and SWE-grep-mini, trained fast agentic models specialized in highly parallel context retrieval. These models match the retrieval capabilities of frontier coding models while requiring an order of magnitude less time.
category: research

article: Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs
authors: Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Ruisi Cai, Marcin Chochowski and others
date: 2025-11-20
desc.: Training a family of large language models targeting multiple scales and deployment objectives is prohibitively expensive. Recent work on model compression through pruning and knowledge distillation has reduced this cost, but still requires substantial computational resources, increasing costs per compressed model. This paper presents Nemotron Elastic, the first elastic training framework for reasoning-capable LLMs. While the Nemotron Elastic framework achieves good results, it still has potential for future research. (NVIDIA)
category: research

article: Cognitive Foundations for Reasoning and Their Manifestation in LLMs
authors: Priyanka Kargupta, Shuyue Stella Li, Haocheng Wang, Jinu Lee and others
date: 2025-11-20
desc.: Large language models successfully solve complex problems yet fail on simpler variants, suggesting they achieve correct outputs through mechanisms fundamentally different from human reasoning. A meta-analysis of 1,598 LLM reasoning papers reveals that the research community concentrates on easily quantifiable behaviors while neglecting meta-cognitive controls. The paper documents systematic structural differences and proposes connecting cognitive science with research on model capabilities rather than pursuing various shortcuts.However, the presented results leave unclear whether the proposed guidance enables genuine deployment of latent capabilities or simply helps models retrieve cached reasoning patterns from training data.
category: research

article: Incorporating Self-Rewriting into Large Language Model Reasoning Reinforcement
authors: Jiashu Yao, Heyan Huang, Shuang Zeng, Chuwei Luo and others
date: 2025-11-20
desc.: Rather than traditional approaches that reward reasoning processes through reinforcement learning which can lead to issues such as over-thinking, focus on irrelevant aspects and etc., the paper presents a Self-Rewriting approach in which a model rewrites its own reasoning text and subsequently learns from the rewritten reasoning to improve its internal thought process quality. The results report improved accuracy of +0.6 alongside 46% shorter reasoning sequences. The article discusses the achieved results and related challenges, including trade-offs compared to standard approaches.
category: research

article: Hiding in the AI Traffic: Abusing MCP for LLM-Powered Agentic Red Teaming
authors: Strahinja Janjuesvic, Anna Baron Garcia, Sohrob Kazerounian
date: 2025-11-20
desc.: Today's 'Vibe Coding' approach enables developers to generate code without fully understanding its mechanics, including the orchestration of multi-agent swarms and sophisticated detection evasion strategies. While existing frameworks may use LLMs to issue post-exploitation commands, they often rely on traditional channels. The paper proposes an innovative Command & Control (C2) architecture leveraging the Model Context Protocol (MCP) for coordinating autonomous red teams of agents while addressing stealth and evasion aspects in depth. The article discusses differences between theoretical attack vectors and enterprise environments. Although the approach shows noticeable improvements, it comes with multiple unanswered questions for future research (MIT, Antropic).
category: research

article: JudgeBoard: Benchmarking and Enhancing Small Language Models for Reasoning Evaluation
authors: Zhenyu Bi, Gaurav Srivastava, Yang Li, Meng Lu, Swastik Roy and others
date: 2025-11-20
desc.: Although SLMs' ability to judge answers remains underexplored, recent studies show that small language models (SLMs) can perform competitively on reasoning tasks with appropriate prompting or fine-tuning. This paper proposes JudgeBoard, an evaluation pipeline capable of injecting SLMs to improve answer comparisons. Due to the limitations of SLMs, the paper introduces the Multi-Agent Judging (MAJ) framework, which outperforms standard approaches (Chain-of-Thought, etc.) by approximately 2% in accuracy. The paper reveals a significant performance gap in judging capability between SLMs and LLMs while highlighting the importance of multi-stage judging (Amazon).
category: research

article: Multi-Agent LLM Orchestration Achieves Deterministic, High-Quality Decision Support for Incident Response
authors: Philip Drammeh
date: 2025-11-19
desc.: Through multiple trials using a reproducible framework, the paper demonstrates that multi-agent orchestration fundamentally transforms LLM-based incident response quality compared to single-agent, error-prone solutions. The multi-agent response is treated as deterministic while introducing latency, however, speed is not the primary goal, provided it remains within acceptable thresholds. Despite the strong performance of multi-agent systems, multiple challenges remain, including LLM deadlocks, fine-tuning requirements, and latency constraints.
category: research

article: Hierarchical Token Prepending: Enhancing Information Flow in Decoder-based LLM Embeddings
authors: Xueying Ding, Xingyue Huang, Mingxuan Ju, Liam Collins and others
date: 2025-11-18
desc.: The paper proposes Hierarchical Token Prepending (HTP) to improve causal attention mechanisms by mitigating attention-level compression and introducing mean-pooling, enabling backward information flow that is critical for generating high-quality embeddings. HTP achieves consistent performance, especially in long-context settings. The article addresses future research directions.
category: research

article: Stanford AI Club: Jeff Dean on Important AI Trends
authors: Stanford AI Club
date: 2025-11-24
desc.: Jeff Dean is one of the most influential computer scientists of the modern computing era, best known as Google’s Chief Scientist and a co-founder of Google Brain. His work has shaped the foundations of large-scale distributed systems and modern machine learning—spanning breakthroughs in search infrastructure, deep learning frameworks like TensorFlow, and today’s frontier AI research. The video provides a timeline of basic technologies and approaches currently employed in the AI-LLM field.
category: youtube

article: Elon Musk Makes Shocking Future Predictions At U.S.-Saudi Arabia Forum Alongside Jensen Huang
authors: Forbes Breaking News
date: 2025-11-20
desc.: Elon Musk and Jensen Huang discuss technology at the U.S.-Saudi Arabia Investment Forum in Washington, D.C., offering an interesting perspective on the future. The interview presents a vision free from current societal constraints and structures such as money-based decisions, resource requirements, sustainability of technologies, or long-term impacts that may limit future evolution. The interview does not address crucial contemporary debates.
category: youtube, interview

article: AI Kill Switch for malicious web-based LLM agent
authors: Sechan Lee, Sangdon Park
date: 2025-09-26
desc.: While AI agents improve the ability to handle complex tasks, they simultaneously amplify the risks of malicious misuse, such as unauthorized collection of personally identifiable information (PII). The paper proposes an "AI Kill Switch" technique aimed at immediately identifying and stopping such malicious AI agent behavior. The key idea lies in identifying an effective defense prompt, which shows similarities to the "LLM as a judge" approach, and focuses on "Prompt Injection" and "Jailbreak-based prompt" forms of attacks. The paper discusses limitations such as the absence of real-world test cases and additional challenges.
category: research

article: BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agents
authors: Kaiyuan Zhang, Mark Tenenholtz, Kyle Polley and others
date: 2025-11-25
desc.: The integration of artificial intelligence (AI) agents into web browsers introduces security challenges beyond traditional web application threat models. The paper discusses identified attack vectors, such as prompt injection, and their impact within real-world environments, noting the low level of current understanding. The paper proposes a novel benchmark and multi-layer defense mechanism called BrowseSafe. Although the paper presents improvements, the complexity of prompt injection attacks remains an open investigation topic (Perplexity AI).
category: research

The post JC-AI Newsletter #10 appeared first on foojay.

This newsletter focuses on examining how AI enhances productivity through enterprise studies, tutorial, agentic system architecture, GraphRAG, evaluating risk methodologies in agentic systems, and the security challenges arising from increased AI-LLM adoption. This edition of the AI newsletter includes a compelling discussion between six of the most influential leaders in artificial intelligence, along with additional content.

The world influenced by LLM is changing very quickly, let's start...

article: The Minds of Modern AI: Jensen Huang, Geoffrey Hinton, Yann LeCun & the AI Vision of the Future
authors: Financial Times Live
date: 2025-11-06
desc.: Six of the most influential figures in AI (Jensen Huang, Yoshua Bengio, Geoffrey Hinton, Fei-Fei Li, Yann LeCun, and Bill Dally) share their vision for the future of the field. Defining a clear future horizon for AI remains a challenging goal. The interviewees appear to grapple with questions regarding concrete AI contributions and the trajectory of progress, avoiding discussion of current challenges while expressing hope that future research will adequately address these issues.
category: youtube

article: GraphRAG: The Marriage of Knowledge Graphs and RAG: Emil Eifrem
authors: Emil Eifrem, AI Engineers
date: 2024-08-28
desc.: Although GraphRAG has made dramatic progress, the fundamentals are sometimes overlooked in favor of introducing additional features. As the saying goes, 'Natural language is most powerful when it can draw from a rich context.' This principle applies equally to both poetry and large language models. Knowledge graphs excel at capturing context, which raises an important question: how can combining knowledge graphs with RAG enhance this capability?
category: youtube

article: GraphRAG: Unlocking LLM discovery on narrative private data
authors: Jonathan Larson, Steven Truitt (Microsoft)
date: 2024-02-13
desc.: A remaining challenge for LLMs is extending their powerful capabilities to solve problems beyond their training data and to achieve comparable results with data the LLM has never encountered. Although the Microsoft Research work on GraphRAG is already somewhat dated given the current pace of LLM development, it remains valuable to understand the fundamentals, rationale, and purpose of GraphRAG. GraphRAG may play an important role in the development of agentic AI systems.
category: research

article: Agentic GraphRAG: Simplifying Retrieval Across Structured & Unstructured Data — Zach Blumenfeld
authors: Zach Blumenfeld, AI Engineers
date: 2025-06-27
desc.: Agentic workflows often become complex, brittle, and difficult to maintain when they need to retrieve and reason across both structured and unstructured data. This talk explores how mapping key information into a knowledge graph can simplify these workflows and improve retrieval quality. The presented example of identifying individuals with similar skills and abilities extracted from CVs provides insight into the practical application of agentic AI systems with GraphRAG.
category: youtube

article: TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems
authors: Ishan Kavathekar, Hemang Jain, Ameya Rathod, Ponnurangam Kumaraguru, Tanuja Ganu
date: 2025-11-07
desc.: The agentic AI systems are increasingly used to collaboratively solve problems. However, the safety and security of these systems remain largely under-explored. Existing benchmarks and datasets predominantly focus on single-agent settings, providing biased results and failing to capture the unique vulnerabilities of multi-agent dynamics and coordination. The paper aims to address a gap related to the safety, security, and various vulnerabilities of multi-agent LLM systems by introducing the Threats and Attacks in Multi-Agent Systems (TAMAS) benchmark. Reported findings show that multi-agent systems are highly vulnerable to adversarial attacks.
category: research

article: ORCHID: Orchestrated Retrieval-Augmented Classification with Human-in-the-Loop Intelligent Decision-Making for High-Risk Property
authors: Maria Mahbub, Vanessa Lama, Sanjay Das, Brian Starks and others
date: 2025-11-07
desc.: High-Risk Property (HRP) classification is critical at U.S. Department of Energy (DOE) sites, where inventories include sensitive and often dual-use equipment. Compliance efforts must track evolving regulations designated by various export control policies to ensure transparent and auditable decisions. Traditional expert-only workflows are time-consuming, prone to backlogs, and struggle to keep pace with shifting regulatory boundaries. The paper introduces ORCHID, a modular agentic system for HRP classification that pairs retrieval-augmented generation (RAG) with human oversight to produce policy-based outputs that can be audited. "Although ORCHID enhances classification reliability, transparency, and reproducibility through evidence-based, policy-aware decision-making, it comes with several limiting factors: the precision and validity of source documents, ambiguity in decision-making processes, the requirement for qualified reviewers, and other constraints.
category: research

article: Multi-Agent Craftax: Benchmarking Open-Ended Multi-Agent Reinforcement Learning at the Hyperscale
authors: Bassel Al Omari, Michael Matthews, Alexander Rutherford, Jakob Nicolaus Foerster
date: 2025-11-07
desc.: Through analytical examination, the paper demonstrates that existing algorithms struggle with key challenges in this benchmark, including long-horizon credit assignment, exploration, and cooperation, and argues for its potential to drive long-term research in multi-agent reinforcement learning (MARL). MARL extends the reinforcement learning paradigm to the co-learning of multiple agents simultaneously. The paper introduces Craftax-MA and its extension Craftax-Coop, a multi-agent extension of the hardware-accelerated Craftax benchmark. The obtained results were limited by small agent populations, and future research directions are proposed.
category: research

article: StepChain GraphRAG: Reasoning Over Knowledge Graphs for Multi-Hop Question Answering
authors: Tengjun Ni, Xin Yuan, Shenghong Li, Kai Wu, Ren Ping Liu, Wei Ni, Wenjie Zhang
date: 2025-10-03
desc.: The paper addresses challenges of commonly used approaches that rely on static or ad hoc expansions of knowledge graphs. The paper introduces the StepChain GraphRAG framework, which combines question decomposition and BFS-RF (breadth-first search reasoning flow) with dynamic graph maintenance. This pipeline dynamically inserts new evidence at each sub-question, refining the knowledge graph in real time. The result is a more transparent, debuggable process for multi-hop question answering that fully exploits both text-based retrieval and graph-structured insights.
category: research

article: RAG Meets Temporal Graphs: Time-Sensitive Modeling and Retrieval for Evolving Knowledge
authors: Jiale Han, Austin Cheung, Yubai Wei and others
date: 2025-10-15
desc.: While Retrieval-Augmented Generation (RAG) systems enrich LLMs with external knowledge, they largely ignore temporal dynamics, which raises two challenges for RAG systems. First, current RAG methods lack effective time-aware representations. The same facts at different time points are difficult to distinguish using vector embeddings or conventional knowledge graphs. Second, most RAG evaluations assume a static corpus, leaving a blind spot regarding update costs and retrieval stability as knowledge evolves. This paper introduces Temporal GraphRAG (TG-RAG), which incorporates time-aware retrieval strategies. Although TG-RAG outperforms current baselines, it comes with several challenges.
category: research

article: TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework
authors: Chao Zhang, Yuhao Wang, Derong Xu, Haoxin Zhang and others
date: 2025-11-07
desc.: Retrieval-Augmented Generation (RAG) enhances Large Language Models' reliability through external knowledge integration. While agentic RAG systems use autonomous, multi-round retrieval for improved accuracy, they generate substantial token overhead. TeaRAG addresses this efficiency challenge by compressing both retrieval content and reasoning steps, delivering a token-efficient agentic RAG framework that balances accuracy with computational economy.
category: research

article: The Learning Loop and LLMs
authors: Unmesh Joshi, Thoughtworks
date: 2025-11-04
desc.: Software development has consistently resisted the notion that it can be reduced to an assembly-line process. Even as our tools become smarter, faster, and more capable, the essential nature of the work remains unchanged: we learn by doing. We must acknowledge the fundamental role of experiential learning in this field, “there are no shortcuts to learning”.
category: opinion

article: Driving a web browser with Gemini's Computer Use model in Java
authors: Guillaume Laforge
date: 2025-11-02
desc.: This tutorial will guide you through the process of programmatically interacting with a web browser using the new Computer Use model in Gemini 2.5 Pro. The tutorial presents an example project written in Java that leverages Microsoft's powerful Playwright Java SDK to handle browser automation. Multi-agentic systems may complement classical end-to-end tests, but several challenges remain, including hallucination.
category: tutorial

The post JC-AI Newsletter #9 appeared first on foojay.

My primary motivation was and remains to be serving the community not only by showing how to use and access specific services for utilizing Large Language Models, but also by support a deeper understanding of the broader artificial intelligence landscape.

Many of the core challenges that have emerged around LLMs have not been and still not properly addressed, often omitted due to their uncomfortable implications.

Nevertheless, the accumulation of technical debt and other challenges, like security, filters stability, benchamarking, scaling etc. is likely growing exponentially.

Many of these challenges, in my humble opinion, had already been addressed even before Transformers and ChatGPT became publicly available in 2022.

Each published newsletter focuses on one core topic, dedicated to addressing a specific hot challenge in AI.

Img.1.: JC-AI Newsletter achive easy access from FooJay top menu News -> JC-AI Newsletter

The initial editions of the JC-AI Newsletter lacked easy access to previous issues. As the number of unique readers continues to rise with each new edition, and as I frequently reference past newsletters myself, I decided to address this "nice to have" feature (Img.1.). I'm pleased to announce that we have created a dedicated section for JC-AI Newsletter archives (Img.2.).

Img.2.: JC-AI Newsletter archive

I'm already excited about publishing the next editions of the AI Newsletter, as multi-agentic AI swarm are gaining traction and bringing multiple additional challenges with them.

Stay tuned, enjoy quick access to the archive, and happy reading!

The post JC-AI Newsletter: Easy Access to Expanding Challenges appeared first on foojay.

This newsletter focuses on examining how AI enhances productivity through enterprise studies, agentic system architecture, attack vectors, Model Context Protocol (MCP) implementation, Agent-to-Agent (A2A) protocol, Java code generation within IDEs, LLM benchmarking methodologies, and the security challenges arising from increased AI-LLM adoption.

The world influenced by LLM is changing very quickly, let's start...

article: Antislop: A Comprehensive Framework for Identifying and Eliminating Repetitive Patterns in Language Models
authors: Samuel Paech, Allen Roush, Judah Goldfeder, Ravid Shwartz-Ziv
date: 2025-10-16
desc.: Widespread LLM adoption has introduced characteristic repetitive phraseology, termed "slop" which degrades output quality and makes AI-generated text immediately recognizable. This paper presents Antislop, a comprehensive framework providing tools to detect and eliminate these overused patterns.
category: research

article: Toward Understanding Security Issues in the Model Context Protocol Ecosystem
authors: Xiaofan Li, Xing Gao
date: 2025-10-18
desc.: The Model Context Protocol (MCP) is an emerging open standard that enables AI-powered applications to interact with external tools through structured metadata, while lacking a sufficient standardization. This paper presents the first comprehensive security analysis of MCP ecosystems and uncovers a wide range of vulnerabilities.
category: research

article: The 4 Patterns of AI Native Development
authors: Patrick Debois
date: 2025-06-04
desc.: The presentation examines AI development evolution through the lens of previously observed cloud computing patterns. It introduces four AI-native development paradigms: 1. producer-to-manager, 2. implementation-to-intent, 3. delivery-to-discovery, and 4. content creation knowledge. While the framework appears to project development needs in a linear fashion, the presentation does not fully address the challenges associated with the nondeterministic behavior of LLMs, which affects all levels of project development.
category: youtube

article: Does AI Actually Boost Developer Productivity ? (100k Devs Study)
authors: Yegor Denisov-Blanch
date: 2025-07-23
desc.: The presentation addresses a critical question regarding the impact of AI-LLM utilization on project development. Data collected from 136 teams across 27 companies provides statistically significant findings. This dataset enables the formulation of hypotheses concerning the conditions under which AI-assisted coding delivers desired value. Standford University research.
category: youtube

article: MCP Security Bench (MSB): Benchmarking Attacks Against Model Context Protocol in LLM Agents
authors: Dongsen Zhang, Zekun Li, Xu Luo, Xuannan Liu and others
date: 2025-10-14
desc.: While Model Context Protocol (MCP) unlocks broad interoperability between agents, it notably extends the attack surface of agentic systems. This paper presents the MCP Security Benchmark, which aims to provide systematic measures of agent resistance against various forms of attacks. The paper discovers that models with stronger performance are more vulnerable to attacks due to various discussed reasons. The experiments demonstrate that MCP-specific vulnerabilities are highly exploitable. The paper provides a practical baseline for future research.
category: research

article: Formalizing the Safety, Security, and Functional Properties of Agentic AI Systems
authors: Edoardo Allegrini, Ananth Shreekumar, Z. Berkay Celik
date: 2025-10-15
desc.: This paper aims to address critical questions related to the utilization of safety protocols, security, and functionality of critical systems depending on LLMs. The current ecosystem of agent communication lacks standardization, such as the Model Context Protocol (MCP) for tool access or the Agent-to-Agent (A2A) protocols. This fragmentation creates a semantic gap that prevents rigorous analysis of system properties and introduces risks such as architectural misalignment and exploitable coordination issues. The paper proposes a domain-agnostic framework for semantic analysis and discusses future research directions.
category: research

article: Generative AI and the Transformation of Software Development Practices
authors: Vivek Acharya
date: 2025-10-12
desc.: The paper examines how AI-assisted techniques are transforming software engineering practices, alongside the emerging challenges of trust and hallucination. The paper considers current key concepts of LLM utilization, including multi-agents, dynamic prompt orchestration, Model Context Protocol (MCP), and assisted coding. The paper discusses psychological aspects of skill set transformation and identifies multiple areas for future investigation.
category: research

article: Automatic Building Code Review: A Case Study
authors: Hanlong Wan, Weili Xu, Michael Rosenberg, Jian Zhang, Aysha Siddika
date: 2025-10-03
desc.: The paper presents a novel agent-driven framework for Automated Code Review (ACR) that integrates Building Information Modeling (BIM) data extraction with agent-orchestrated workflows and existing check tool engines. The paper presents a case study developed in cooperation with the US Department of Energy.
category: research

article: When MCP Servers Attack: Taxonomy, Feasibility, and Mitigation
authors: Weibo Zhao, Jiahao Liu, Bonan Ruan, Shaofei Li, Zhenkai Liang
date: 2025-09-29
desc.: While Model Context Protocol (MCP) servers enable AI applications to connect to external systems in a plug-and-play manner, they create new attack vectors requiring consideration. The lack of standardized mechanisms increases this urgency. This paper addresses three research questions: 1. what types of attacks malicious MCP servers can launch, 2. how vulnerable MCP hosts and Large Language Models (LLMs) are to these attacks, and 3. how feasible these attacks are in practice. The paper proposes a component-based taxonomy comprising twelve attack categories.
category: research

article: Privacy in Action: Towards Realistic Privacy Mitigation and Evaluation for LLM-Powered Agents
authors: Shouju Wang, Fenglin Yu, Xirui Liu, Xiaoting Qin and others
date: 2025-09-22
desc.: The increasing autonomy of LLM agents in handling sensitive communications, accelerated by the Model Context Protocol (MCP) and Agent2Agent (A2A) frameworks, creates urgent privacy challenges. This paper presents PrivacyCheck, which aims to reduce privacy leakage from approximately 35% to 7%, depending on the model. The paper also proposes additional mitigation strategies to improve privacy in the emerging agentic ecosystem.
category: research

article: Cuckoo Attack: Stealthy and Persistent Attacks Against AI-IDE
authors: Xinpeng Liu, Junming Liu, Peiyu Liu, Han Zheng and others
date: 2025-09-19
desc.: The utilization of agentic AI systems introduces a new critical attack surface, including development environments (IDEs). The paper proposes the Cuckoo Attack, a novel attack capable of stealthy and persistent command execution by embedding malicious payloads into configuration files. The paper shows that the impact may extend beyond compromising the individual developer environment.
category: research

article: Tractable Asymmetric Verification for Large Language Models via Deterministic Replicability
authors: Zan-Kai Chong, Hiroyuki Ohsaki, Bryan Ng
date: 2025-09-14
desc.: The landscape of Large Language Models (LLMs) is shifting rapidly toward dynamic, multi-agent systems. This introduces a fundamental challenge in establishing computational trust between agents to ensure that information is not corrupted. This paper introduces a probabilistic audit approach within a defined context to ensure information integrity in multi-agent systems. The paper presents simulation achievements and proposes directions for future research.
category: research

Previous JC-AI Newsletters

The post JC-AI Newsletter #8 appeared first on foojay.

Beyond focused tutorials that can enhance your understanding of AI applications, this newsletter concentrates on Hallucination, Java Code Generation, Testing, Agentic System Architecture and LLM benchmarking methodologies designed to ensure models accuracy and competency in handling complex contextual information.

The world influenced by LLM is changing very quickly, let's start...

article: The Missing Layer in AI Infrastructure: Aggregating Agentic Traffic
authors: Eyal Solomon
date: 2025-08-22
desc.: Agentic AI systems introduce new challenges across the entire system architecture. Beyond what research articles address, several critical issues remain unresolved and may pose serious risks, particularly in system architecture where the adoption of LLMs has triggered a major paradigm shift in system design. A key concern involves outbound API calls made by autonomously acting AI agents (e.g., chaining tools, calling external services). Current infrastructure, including API gateways and service meshes, is primarily designed around inbound traffic or service-to-service communication, rather than managing agent-initiated outbound calls. This creates a significant blind spot in our architectural oversight.
category: architecture

article: Learning to Reason for Hallucination Span Detection
authors: Hsuan Su, Ting-Yao Hu, Hema Swetha Koppula, Kundan Krishna, Hadi Pouransari, Cheng-Yu Hsieh and others
date: 2025-10-02
desc.: This paper addresses the challenges of advancing from simple binary classification (Chain-of-Thought, COT)) to fine-grained span-level hallucination detection. In-domain reasoning is essential for robust hallucination detection. The normalization step in Group Relative Policy Optimization proves crucial, as simple reward rescaling policies cannot effectively mitigate reward hacking in the dataset employed. The paper proposes a reinforcement learning framework with span-level rewards to align large language model (LLM) reasoning with hallucination detection tasks on the RAGTruth benchmark. Paper research has been done during an internship at Apple.
category: research

article: Stream RAG: Instant and Accurate Spoken Dialogue Systems with Streaming Tool Usage
authors: Siddhant Arora, Haidar Khan, Kai Sun, Xin Luna Dong, Sajal Choudhary, Seungwhan Moon, Xinyuan Zhang and others
date: 2025-10-02
desc.: End-to-end speech recognition and fluent answering without noticeable pauses present significant challenges for utilizing LLMs in dialogue-based agentic systems. These systems are prone to hallucination effects caused by various factors. While improving input/output accuracy through Retrieval Augmented Generation (RAG) approaches can mitigate hallucination effects and significantly increase accuracy, this comes with various penalties such as increased resource requirements and latencies. The paper proposes a Model-Triggered Stream RAG approach as an alternative to fixed-interval RAG streaming or without RAG. Although the paper does not provide a complete solution to these challenges, it proposes a benchmarking strategy for future research and highlights key achievements. This research was conducted in cooperation with Meta.
category: research

article: Self-Anchor: Large Language Model Reasoning via Step-by-step Attention Alignment
authors: Hongxiang Zhang, Yuan Tian, Tianyi Zhang
date: 2025-10-03
desc.: While prompting strategies show effectiveness in certain tasks, they lack robustness across different benchmarks and model architectures, performing better on larger LLMs and simpler reasoning problems. This paper proposes a Self-Anchor mechanism for structured reasoning with automatic anchoring. Self-Anchor delivers consistent improvements across tasks, model sizes, and architectures, demonstrating strong robustness and effectiveness. The approach leverages inherent structure in reasoning chains to improve attention alignment and enhance reasoning capabilities. However, Self-Anchor primarily addresses attention misalignment without fully resolving deeper issues related to logical validity, semantic understanding, or computational precision.
category: research

article: Abstain and Validate: A Dual-LLM Policy for Reducing Noise in Agentic Program Repair
authors: José Cambronero, Michele Tufano, Sherry Shi, Renyao Wei, Grant Uy, Runxiang Cheng and others
date: 2025-10-03
desc.: Agentic Automated Program Repair (APR) is increasingly addressing complex, repository-level bugs in industry settings. However, agent-generated patches still require human review before deployment to ensure they properly resolve the underlying issues. This paper introduces two complementary LLM-based policies for patch assessment. The paper addresses and discusses limitations in automated patch procedures, human supervision requirements, and company-specific bug-fixing approaches. This paper results from cooperation between Google and Meta.
category: research

article: Cache-to-Cache: Direct Semantic Communication Between Large Language Models
authors: Tianyu Fu, Zihan Min, Hanling Zhang, Jichao Yan, Guohao Dai, Wanli Ouyang, Yu Wang
date: 2025-10-03
desc.: Rather than relying on text-to-text communication between LLM-based systems, which incurs latency penalties, this paper proposes the Cache-To-Cache paradigm for direct inter-system communication. Experimental results demonstrate improved efficiency and performance without requiring additional cache capacity.
category: research

article: FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration
authors: Victor May, Diganta Misra, Yanqi Luo, Anjali Sridhar, Justine Gehring, Silvio Soares Ribeiro Junior
date: 2025-10-06
desc.: This paper introduces the FreshBrew approach for Java code migration tasks utilizing agentic LLM systems. The migration experiments from JDK8 to JDK17 and JDK21 demonstrate the limitations of current LLM implementations, even when integrated with modern deterministic migration tools such as OpenRewrite. Although the overall migration success rate was approximately 50%, the paper provides a comprehensive discussion of the associated limitations and challenges. The article has been done in cooperation with Max-Planck Institute, Google and Saleforce companies.
category: research

article: Investigating The Smells of LLM Generated Code
authors: Debalina Ghosh Paul, Hong Zhu, Ian Bayley
date: 2025-10-03
desc.: The paper proposes a scenario-based method for evaluating the quality of LLM-generated code, as such models are increasingly utilized for program code generation. The study experiments with Java programs using Gemini Pro, ChatGPT, Codex, and Falcon LLMs to obtain results. The paper highlights that for moderately advanced topics, particularly those involving object-oriented programming concepts, the generated code quality is noticeably poorer to human-written code.
category: research

article: Which Programming Language and Model Work Best With LLM-as-a-Judge For Code Retrieval?
authors: Lucas Roberts, Denisa Roberts
date: 2025-09-30
desc.: This paper examines the comparative abilities of Large Language Models (LLMs) and human annotators in identifying and annotating specific elements within source code. The study investigates several widely-used programming languages, including C, Java, JavaScript, Go, and Python. The experimental results reveal various limitations and challenges associated with automated code annotation, while proposing possibilities for future research and emphasizing the critical role of human expertise in the annotation process.
category:

article: AI Coding Tools Blog Post - Model Context Protocol Mastery - Claude, Cursor
authors: Mani Sarkar
date: 2025-10-07
desc.: The post describes and provides guidance on configuring MCP for AI-assisted development using Claude and Cursor. Please be aware of the 'No Warranty' statement and use this as an example only.
category: tutorial

article: DiffTester: Accelerating Unit Test Generation for Diffusion LLMs via Repetitive Pattern
authors: Lekang Yang, Yuetong Liu, Yitong Zhang, Jia Li
date: 2025-09-29
desc.: Writing high-quality unit tests is often a time-consuming effort that requires extensive knowledge of the business domain. This paper proposes DiffTester, an acceleration framework designed to overcome the limitations imposed by single-token generation constraints. The DiffTester framework identifies common patterns through syntax tree analysis. Experimental results demonstrate that the DiffTester framework can used to generate a larger number of tokens, thereby achieving better accuracy.
category: research

article: Deloitte caught out using AI in $440,000 report | 7.30
authors: ABC News In-depth
date: 2025-10-09
desc.: Hallucination remains a significant challenge in current large language models (LLMs). These inaccuracies can cause damage at various levels and require careful eye to identify.
category: youtube

article: SusBench: An Online Benchmark for Evaluating Dark Pattern Susceptibility of Computer-Use Agents
authors: Longjie Guo, Chenjie Yuan, Mingyuan Zhong, Robert Wolfe, Ruican Zhong, Yue Xu, Bingbing Wen and others
date: 2025-10-13
desc.: In the age of AI, deception through manipulation or hallucination-based choices poses a serious threat to human or system reliability. This paper proposes SysBench, a benchmark for evaluating the susceptibility of computer-user agents (CUAs) and humans to dark patterns that may mislead both users and agents into making harmful decisions. The paper demonstrates that neither humans nor agentic AI systems based on large language models (LLMs) exhibit adequate resistance against dark patterns.
category: research

Previous:
Newsletter vol.1
Newsletter vol.2
Newsletter vol.3
Newsletter vol.4
Newsletter vol.5
Newsletter vol.6

The post JC-AI Newsletter #7 appeared first on foojay.