Pezego-HITL: A policy-grounded large language model architecture for agricultural extension in Ghana

Introduction

The introduction discusses the critical role of agricultural extension services in addressing pest outbreaks and crop diseases in Ghana, highlighting the challenges faced by smallholder farmers due to limited access to timely agronomic advice and the potential of digital tools to enhance decision support.

Expert-Verified Case Memory (VCM)

This section describes the Expert-Verified Case Memory (VCM) as a low-latency cache for expert-approved crop protection recommendations, detailing how it improves query response times and integrates expert feedback to enhance decision support.

Digital Agricultural Extension and Decision Support Systems

The paper reviews the evolution of digital agricultural extension systems, emphasizing the limitations of traditional methods and the potential of ICT4D to improve access to agronomic advice for smallholder farmers.

Limitations of Image-Based Automated Diagnostics

This section critiques the reliance on image-based diagnostics in agriculture, arguing that they fail to provide comprehensive decision support and are often disconnected from regional agricultural policies.

P-EVAL Protocol

The P-EVAL protocol is introduced as a unified evaluation framework for assessing policy-grounded decision support systems, measuring key performance metrics such as Policy Alignment Rate (PAR) and Agronomic Utility Rate (AUR).

Figures Explained

The paper’s visual material highlights the workflow and the main system components.

• Figure 1 :: Figure 1: Decoupled policy-constrained decision-support architecture (Pezego-HITL). The proposed framework separates generative capabilities from regional policy databases and expert validator workflows. Input queries from mobile clients are processed through a two-stage routing mechanism: (1) queries matching cached, expert-verified advisory templates are immediately served via the Verified-Case Memory (VCM) caching layer, bypassing generation entirely to achieve sub-second execution; (2) cache misses are routed to a schema-aware SQL-grounded Structured Retrieval pipeline, audited against regional safety rules in a Constraint Auditing Layer, and aligned using Critique Multi-Agent loops before human review. Extension Services Officers (ESOs) validate all drafts, and expert corrections are compiled to dynamically update the VCM caching database and prompt structures.
• Figure 2 :: Figure 2: Integrated evaluation protocol workflow. The protocol comprises two primary streams: (A) Online Telemetry (green box) which logs simulated query telemetry to measure execution speed (P 95 latency), database memory reuse, and policy compliance; and (B) Human Feedback (orange box) which gathers survey data from 30 Extension Services Officers (ESOs) and 36 farmers during hands-on training workshops. These input streams feed into parallel evaluation levels: architecture-level validation uses an automated LLM-as-a-judge (calibrated offline against human expert verification decisions) to assess high-throughput compliance and efficiency, while user-level validation evaluates usability, clarity, and trust. The combined outcomes provide a comprehensive assessment of architectural performance (automated Quality and Efficiency) and empirical utility (usability, trustworthiness, and willingness to reuse).
• Figure 3: instantiated architecture (Pezego-HITL) against four baseline configurations representing major theoretical paradigms in retrieval, grounding, and safety auditing architectures. To demonstrate model generalisability, all baselines and our proposed architecture are evaluated using two different base models: a proprietary API (gpt-5.4-nano) and a locally deployable open-source model (Qwen3.5-9B-DeepSeek-V4-Flash):• B1: Unconstrained Generative Paradigm (Direct LLM): A single-pass text generation utilising the base language model without external retrieval or constraint auditing. This serves as the baseline for raw generative capabilities under zero-shot prompting.• B2: Unstructured Semantic Chunk Retrieval Paradigm (Standard Vector RAG): A standard RAG architecture utilising dense vector search over unstructured document passage chunks (representing Lewis et al. [2] ). This baseline introduces domain-specific reference facts but fails to model structured, multi-variable policy constraints.• B3: Structured Relational Retrieval Paradigm (Tool-Augmented RAG): A structured tooluse baseline that dynamically translates user queries into database API or SQL executions to query relational tables directly, without post-generation constraint auditing (representing Schick et al. [3] ).• B4: Critique-Guided Post-Hoc Alignment Paradigm (Multi-Agent RAG): A multi-agent configuration utilising a sequential validation and self-correction loop where critique agents audit generated drafts and instruct the synthesis agent to rewrite violating content, run without VCM caching or expert validation (representing self-refine architectures like Madaan et al. [4] ).
• While a sample of 30: respondents may appear modest for standard consumer application testing, it represents an exhaustive, census-like cohort in the context of professional public extension services. ESOs are highly specialised civil servants who undergo formal agricultural training and are stationed in specific operational districts by the Ministry of Food and Agriculture (MoFA). The pilot deployment of Pezego-HITL was targeted at two key agricultural zones: the Eastern and Ashanti regions. In these pilot districts, the entire active cohort of ESOs trained and equipped with smartphones for the digital extension pilot consists of 30 officers. Consequently, our target sample is not a low-powered random selection from a large population, but rather a near-total census of the active practitioner cohort directly interacting with the deployed architecture in the field. This high coverage rate ensures that the feedback captures authentic institutional and workflow realities.The survey was deployed digitally (via Google Forms) following hands-on training workshops. ESOs rated usability, AI quality, and communication features on a 5-point Likert scale (1: Strongly Disagree, 5: Strongly Agree). We evaluated the internal consistency and reliability of the Likert scales (Usability, AI Diagnostics, and Communication) using Cronbach’s alpha (α), with all scales exceeding the target threshold of α ≥ 0.70 (Usability: 0.967, AI Diagnostics: 0.885, and Communication: 0.959), indicating high measurement reliability.
• 1 . 2 . 3 . 4 . 5 . 6 .: crop protection and agricultural livelihoods, we administered a parallel survey study to Ghanaian smallholder farmers (N = 36 responses). The surveyed farmers are located in the Ashanti region of Ghana, where maize is grown as the primary staple crop. These smallholder farmers actively interacted with the Pezego mobile application in the field, submitting pest reports and receiving recommendations verified by local ESOs. The survey was deployed digitally (in English and local translations via Google Forms) following fieldtesting and app rollout sessions. Farmers rated app usability, AI trust, and extension officer connection features on a 5-point Likert scale (1: Strongly Disagree, 5: Strongly Agree). Similar to the ESO survey, we evaluated the internal reliability of the farmer Likert scales using Cronbach’s alpha (α), with all scales demonstrating high consistency (Usability Scale: α = 0.949; AI Trust & Officer Connection Scale: α = 0.944), well exceeding the standard target of α ≥ 0.70. The farmer questionnaire is structured into six sections: Informed Consent: Ethical compliance and voluntary participation agreement. Demographics: Captures farmer age, gender, farm location (region), main crop grown, total farm size, and comfort level with mobile apps. Traditional Pest Practices: Records traditional crop-protection actions and recommendation turnaround times before using Pezego. App Usability & Usefulness: Measures the user-friendliness, ease of navigation, and speed of obtaining pest management recommendations (7 statements). AI Trust & Officer Connection: Evaluates farmers’ trust in the AI-generated recommendations, and the value of HITL verification and communication features (6 statements). Final Impact & Feedback: Captures the perceived crop saving efficacy, avoided financial losses, and open-ended feature requests.