Journal of Technology Informatics and Engineering
Journal Title: Journal of Technology Informatics and Engineering Initials: JTIE Frequency: April, August, and December Print ISSN: 2961-9068 Online ISSN: 2961-8215 Editor in Chief: Dr. Ir. Agus Wibowo., M.Kom., M.Si., M.M. DOI: 10.51903 Publisher: University of Science and Computer Technology Journal of Technology Informatics and Engineering is a peer-reviewed scientific journal that publishes original research articles, scholarly studies, and applied research findings related to advances in technology and engineering. The journal is published in both electronic and print formats with e-ISSN: 2961-8215, and p-ISSN: 2961-9068. The scope of the journal encompasses, but is not limited to, Informatics, Mechanical Engineering, Electrical Engineering, Information Systems, and Industrial Engineering. The journal serves as an academic platform for researchers, academics, and practitioners to disseminate research outcomes and technological innovations. Journal of Technology Informatics and Engineering is published three times a year, in April, August, and December. Indexed by:
University of Science and Computer Technology
Clustering of Infectious and Non-Communicable Disease Distribution in Lhokseumawe City Using the Fuzzy Gustafson-Kessel Method
The high incidence of infectious and non-communicable diseases in Lhokseumawe City requires a data-driven approach for mapping distribution areas. This study designs a disease distribution clustering system using the Fuzzy Gustafson-Kessel (FGK) method and compares three fuzzy membership functions: linear, sigmoid, and Gaussian. FGK excels in handling non-uniform data distributions utilizing an adaptive covariance matrix based on Mahalanobis distance. The data comprised 38,063 cases from the Lhokseumawe City Health Office (2023–2024), aggregated into 4 sub-districts, and processed using z-score standardization. The clustering was implemented in a Python and MySQL web-based system and evaluated using the Xie-Beni validity index. The FGK method successfully grouped the sub-districts into 3 clusters: low, moderate, and high. For infectious diseases (converging at iteration 24), Blang Mangat formed the low cluster, Muara Dua and Muara Satu the moderate, and Banda Sakti the high. For non-communicable diseases (converging at iteration 9), Blang Mangat was low, Muara Satu moderate, while Banda Sakti and Muara Dua were high. The linear membership function proved optimal, yielding the lowest average Xie-Beni Index of 0.1316. In conclusion, the FGK method with a linear membership function effectively maps disease distribution, assisting the local Health Office in targeted policy planning and resource allocation.
Copyright (c) 2026 M. Rahmat Azhari, Eva Darnila, Yesy Afrillia https://creativecommons.org/licenses/by-sa/4.0
Psychology-Informed Live-Commerce Analytics: Structural Modeling, Predictive Validation, and Governed Script Scoring for TikTok Shopping
This paper examines the psychological factors associated with impulsive-purchase propensity in TikTok Live Commerce and translates those associations into an auditable analytics workflow. It presents a secondary analysis of a public survey workbook containing 415 Indonesian responses on Social Attraction, fear of missing out, Narrative Involvement, Telepresence, Parasocial Interaction, Social Presence, and Impulsive Purchase. The analysis combines measurement diagnostics, a bootstrapped composite structural model, repeated cross-validated classification and continuous prediction, calibration assessment, a coefficient-based illustrative scenario analysis, and external validation of an ethical microcopy screen. In the full sample, Narrative Involvement has the largest association with Parasocial Interaction (β = .516) and Social Presence (β = .496); Parasocial Interaction and Social Presence jointly account for 67.5% of the in-sample variance in Impulsive Purchase. The pattern remains in the quality-screened TikTok-viewer subset (n = 238; Impulsive Purchase R² = .647). For the quality-screened subset, the strongest full-feature classifier is Random Forest (ROC-AUC = .938 ± .031 across 50 held-out folds), while Ridge regression gives the largest mean continuous-outcome R² (.622 ± .114). On 2,356 ec-darkpattern strings, a page-grouped hybrid word/character classifier reaches F1 = .959 ± .011 and ROC-AUC = .989 ± .003. The scenario scores are deterministic implications of the estimated coefficients rather than observed purchase effects. The technical contribution is a governed decision-support architecture that keeps structural association, contemporaneous prediction, scenario prioritization, ethical screening, human review, and prospective testing distinct.
Copyright (c) 2026 Xiaochen Li, Yinchen Shi, Jason Zhang https://creativecommons.org/licenses/by-sa/4.0
Explainable GLCM–Decision Tree Framework for Pulmonary Tuberculosis Classification from Chest X-Rays
Pulmonary tuberculosis (TB) remains a major global health challenge, while chest X-ray interpretation is still influenced by radiologist expertise and inter-observer variability. This study develops an explainable machine learning framework for pulmonary TB classification from chest X-ray images by integrating Gray-Level Co-occurrence Matrix (GLCM) texture features, statistical image features, and a Decision Tree classifier implemented in RapidMiner. The dataset comprised 1,000 chest X-ray images representing TB-positive and normal classes. Image preprocessing included resizing, grayscale conversion, and intensity normalization, followed by extraction of four GLCM features—contrast, energy, homogeneity, and correlation—and four statistical features comprising mean, variance, standard deviation, and entropy. Model performance was evaluated using stratified 10-fold cross-validation. The Decision Tree achieved an average accuracy of 85.4%, precision of 87.1%, recall of 83.6%, and F1-score of 85.3%. Beyond predictive performance, the model generated interpretable IF–THEN rules that provide transparent reasoning for classification outcomes. Analysis of false-negative cases further identified clinically important misclassification patterns that require additional diagnostic confirmation. The proposed framework demonstrates that interpretable machine learning can provide a practical balance between classification performance, transparency, computational efficiency, and reproducibility, supporting its potential use as a clinical decision-support tool for TB screening in resource-constrained healthcare environments.
Advanced Data Interpretation, Machine Learning, and Artificial Intelligence
Copyright (c) 2026 Wiwid Wahyudi, Fitro Nur Hakim, Nur Rokhman, Irdha Yunianto https://creativecommons.org/licenses/by-sa/4.0
LLM-Inspired Ontology-Based Semantic Enrichment for FinTech M&A Intelligence in SEC Structured Disclosures
This study develops an ontology-based event-intelligence framework for FinTech merger-and-acquisition evidence in U.S. Securities and Exchange Commission (SEC) structured disclosures. Five quarterly SEC Financial Statement Data Sets from 2025Q1 through 2026Q1 contain 32,254 filings, 7,335 registrants, and 18,312,494 numerical XBRL facts; validation adds a 1,800-filing full-text Form 8-K sample and 192 FDIC events. A deterministic ontology maps XBRL tag names, labels, and documentation to acquisition, disposition, valuation, integration, risk, and payment concepts. The method is therefore LLM-inspired semantic enrichment rather than direct LLM extraction. Logistic regression, decision tree, and random forest classifiers are evaluated in three expanding forward-quarter tests with training-only preprocessing and threshold selection. A proximal protocol retains semantically close predictors, whereas a strict protocol excludes label-generating variables and deterministic descendants. Across the rolling tests, proximal logistic-regression M&A detection attains mean F1 = 0.981941, ROC-AUC = 0.999517, and average precision = 0.998315; strict performance falls to F1 = 0.735640, ROC-AUC = 0.930870, and average precision = 0.824107. FinTech M&A F1 declines from 0.921198 to 0.449503. Strict random forests yield F1 = 0.798129 for integration risk and 0.753987 for valuation signals. In independent full text, strict main-text-plus-exhibit F1 is 0.297482; among 24 automatically linked FDIC events in rolling test quarters, 9 are detected. Near-perfect scores therefore describe ontology reconstruction, not transaction-level accuracy.
Copyright (c) 2026 Guanzheng Zhao, Dingyuan Zhang, Sisi Meng https://creativecommons.org/licenses/by-sa/4.0
Token-Burst-Aware Capacity Planning for LLM Inference Services: Request Arrival, Token Demand, and Failure Risk Modeling from BurstGPT Traces
Large language model (LLM) inference services convert request arrivals into coupled input-token prefill and output-token decoding workloads. Capacity planning therefore depends on token volume, temporal bursts, service mix, queueing behavior, and reliability signals rather than request counts alone. This study evaluates an integrated planning pipeline on BurstGPT v1.1, comprising 5,288,173 raw requests over 121 trace days and 5,188,507 completed requests. Strictly chronological experiments aggregate demand, forecast hourly completed tokens, detect minute-level burst pressure, estimate zero-response risk, simulate capacity policies, and replay representative test hours in Vidur. Random forest, selected on the validation interval, achieved 64.73% weighted absolute percentage error (WAPE) on the locked test interval; XGBoost achieved the lowest test WAPE (64.57%), while the last-hour baseline reached 67.42%, indicating limited forecastability under a pronounced level shift. A seasonal-residual burst detector achieved F1 = 0.653, although burst prevalence was sensitive to rolling-horizon and quantile settings. For minute-level zero-response risk, raw XGBoost achieved ROC-AUC = 0.813 and average precision = 0.116; isotonic calibration improved the Brier score (0.0230–0.0204) and 10-bin expected calibration error (0.0262–0.0122), despite low F1. Static P90/P95 capacity eliminated under-provisioned test hours at cost indices of 13.10 and 18.09. More economical dynamic baselines achieved 12.65% under-provisioned hours at a cost index of 2.37 and 13.63% at 2.43. The validation-selected random-forest policy was cheaper but less reliable (34.31% at 1.23). Vidur replay linked normalized demand to A100/H100 GPU counts, latency, batching, and memory pressure. The results support conservative interpretation of point forecasts and validation of reserve rules under distribution shift, rare-event calibration, and serving-stack constraints.
Copyright (c) 2026 Jiayi Nie, Yinchen Shi, Lucas Zhao https://creativecommons.org/licenses/by-sa/4.0
Cross-Dataset Parcel Workload Priors for Last-Mile Capacity Forecasting: Integrating Package Segmentation with Delivery Operation Signals
This paper evaluates a cross-dataset framework for parcel segmentation, next-day last-mile operational-intensity forecasting, and capacity-oriented error analysis. The operational study uses 4,514,661 delivery tasks and 6,136,147 pickup tasks from LaDe-D and LaDe-P across five cities (May–October 2022), while the visual study uses 2,197 Package Segmentation images with 7,643 annotated package instances. A segmentation model estimates package count, foreground area, and instance-area dispersion to construct an additive workload descriptor. Because the images are not paired with operational records, city-day visual variables are represented as cross-dataset distributional proxies derived from empirical-rank mapping. Segmentation baselines are evaluated independently of forecasting. The random-forest pixel classifier achieves the best segmentation performance (IoU 0.5323, Dice 0.6450), outperforming YOLO11n-seg (IoU 0.3439, Dice 0.4591). Operational intensity is represented by the first principal component of delivery orders, active couriers, areas of interest, regions, and area types, explaining 89.19% of training variance. On a 230 city-day chronological holdout, ElasticNet Operational achieves the best forecasting accuracy (MAE 2.0521). Within XGBoost models, Vision-Prior records MAE 2.4988, slightly outperforming Delivery+Pickup (MAE 2.5154) and a shuffled-prior control (MAE 2.5973). However, the 0.0166 MAE improvement is not statistically significant under a paired moving-block bootstrap (95% CI: −0.0558 to 0.0319). A separate analysis of 6,112 Amazon routes and 1,457,175 packages similarly shows only a 0.24% MAE reduction from parcel-mix features. Overall, the proposed proxy provides only limited gains, while the operational ElasticNet remains the strongest baseline. Time- and location-paired visual observations are needed to establish practical operational benefits from computer vision.
Copyright (c) 2026 Ruiyan Ma, Long Zhang, Annie Bai https://creativecommons.org/licenses/by-sa/4.0
IntentRouter-QAC: Distilling LLM-Derived Intent Signals into Small Language Models for Context-Aware Autosuggest and Search Entry Routing
Search boxes increasingly function as routing interfaces: a partial query may require a completion, immediate submission, a context-sensitive suggestion, or a fallback action. This paper evaluates IntentRouter-QAC as a lightweight lexical routing framework and tests product retrieval independently. Five mutually exclusive operational outcomes are derived within the 20,000-row AmazonQAC test file and evaluated with a strict session-disjoint temporal split; product-query provenance is not used as a route class. Across five seeds, the combined word/character TF-IDF router reached 0.473 mean accuracy (SD 0.031) and 0.330 mean macro-F1 (SD 0.018), indicating uneven performance across routes. On the 951-row temporal test set, context-nearest completion achieved 0.033 Success@10 and 0.025 MRR@10, compared with 0.029 and 0.023 for confidence-gated routing. The paired MRR@10 difference was 0.002 (95% CI -0.007 to 0.010; Holm-adjusted p = 1.000). Product retrieval on 480 WANDS queries and 42,994 products produced a different pattern: a word/character lexical hybrid reached 0.690 nDCG@10 and 0.504 Exact-MRR@10, significantly exceeding the title-only baseline on both measures. Route-aware decision-making is therefore useful as an architectural separation of entry actions, but it does not automatically improve exact autocomplete ranking over a strong session-context baseline.
Copyright (c) 2026 Haowei Tu, Yinchen Shi, Sophia Chen https://creativecommons.org/licenses/by-sa/4.0
Performance Evaluation and Validation of an AI-Driven Hyperspectral Remote Sensing Framework for Locust Surveillance Using Ground-Truth Data
Desert locust outbreaks continue to threaten agricultural productivity and food security across East Africa, creating an urgent need for surveillance systems that are accurate, scalable, and capable of supporting early intervention. This study evaluates and validates an AI-driven hyperspectral remote sensing framework for locust surveillance in South Nyanza, Kenya, using ground-truth observations and conventional field-scouting records as benchmarks. The study employed 150 georeferenced spatial units, with 70% used for model training and 30% reserved for independent validation. Random Forest and Logistic Regression were applied to hyperspectral and environmental indicators, while model performance was assessed using confusion matrix metrics, Out-of-Bag error, and Receiver Operating Characteristic analysis. On the 45-unit validation set, the AI-driven framework correctly classified 37 locations, achieving an accuracy of 82.22%, precision of 83.33%, recall of 83.33%, and an F1-score of 83.33%, with an Out-of-Bag error rate of 18.20%. The Random Forest model achieved an ROC-AUC of 0.844, substantially higher than the 0.585 obtained from the conventional field-scouting baseline. The framework also reduced false-negative detections from 10 to 4 cases. These findings demonstrate that integrating machine learning with hyperspectral remote sensing can strengthen locust surveillance by improving classification reliability, reducing missed infestations, and supporting evidence-based early warning and intervention planning.
Copyright (c) 2026 Emmanuel Ochako Manyange, Juliana Kamaghe, Lilian Mutalemwa https://creativecommons.org/licenses/by-sa/4.0
Design of an AI-Driven Analytical Framework Integrating Machine Learning and Hyperspectral Remote Sensing for Detection and Classification of Locust-Prone Areas in South Nyanza Kenya
Desert locust outbreaks pose a persistent threat to agricultural productivity and food security in East Africa, while conventional surveillance approaches remain limited by delayed reporting, restricted spatial coverage, and weak predictive capability. This study develops and evaluates an artificial intelligence-driven analytical framework that integrates machine learning with hyperspectral remote sensing for detecting and classifying locust-prone areas in South Nyanza, Kenya. An empirical quantitative experimental design was applied to 150 georeferenced spatial observation units using spectral, vegetation, and bioclimatic indicators derived from Sentinel-2, CHIRPS, and MODIS data. The analytical framework incorporated automated feature engineering, Random Forest classification, Logistic Regression, and geospatial hazard visualization. The Random Forest model, configured with 500 decision trees and an mtry value of 3, achieved an overall classification accuracy of 81.33%, precision of 81.58%, recall of 81.58%, F1-score of 81.58%, and an Out-of-Bag error rate of 18.67%. The validated ROC-AUC reached 0.835, indicating good discrimination capability. Variable importance analysis identified precipitation, soil moisture represented by SAVI, and vegetation greenness represented by NDVI as the most influential predictors. Logistic Regression showed a positive association between hyperspectral indicators and locust-prone classification, although the predictors were not statistically significant at the 5% level. The findings demonstrate that integrating machine learning with remotely sensed environmental indicators provides a scalable approach for strengthening locust surveillance and supporting evidence-based early warning systems.
Copyright (c) 2026 Emmanuel Ochako Manyange, Juliana Kamaghe, lilian Mutalemwa https://creativecommons.org/licenses/by-sa/4.0
Adaptive Scalability Optimization for Blockchain-Powered Academic Credential Repositories Using Intelligent Caching and Metadata-Aware Sharding
Academic credential verification remains difficult for institutions because manual checks are slow, fragmented, and vulnerable to fraud. Blockchain can improve trust by anchoring credential proofs, but repeated verification requests and growing off-chain repositories can still create performance bottlenecks. This study presents an adaptive blockchain-powered academic credential repository that combines off-chain MySQL storage, Solidity-based hash anchoring, Redis verification caching, and metadata-aware sharding. Full academic records are not stored on-chain or in Redis; only credential hashes, verification responses, and related metadata are used for trust validation and performance optimization. A CodeIgniter 4 prototype was evaluated using synthetic academic credential records and controlled workloads of 1,000, 5,000, and 10,000 verification requests under fresh, mixed, and repeated access patterns. The results show that Redis caching substantially reduced repeated blockchain queries, especially under mixed and repeated workloads, while metadata-aware sharding improved repository organization and supported more targeted credential retrieval. Sepolia testnet validation confirmed smart-contract feasibility, including issuance, verification, revocation, gas use, confirmation time, and event evidence, but was treated separately from scalability testing. The findings indicate that combining blockchain trust anchoring with cache-aware verification and metadata-based repository partitioning can improve the scalability of academic credential repositories, provided that cache consistency, revocation handling, and deployment limitations are carefully managed.
Copyright (c) 2026 Blessing Emmanuel Oladele, Adekunle Olugbenga Ejidokun, Chukwuemeka Odi Agwu https://creativecommons.org/licenses/by-sa/4.0
Detection of Locust Breeding and Feeding Zones Using Hyperspectral Signatures and Environmental Variables in South Nyanza Region, Kenya
Desert locust infestations continue to threaten agricultural productivity, food security, and environmental sustainability across East Africa. Conventional locust surveillance approaches remain largely dependent on field-based monitoring systems that are reactive, labor-intensive, and constrained by limited spatial coverage. This study investigated the hyperspectral signatures and environmental variables associated with locust breeding and feeding zones in South Nyanza, Kenya, using an Artificial Intelligence (AI)-driven hyperspectral remote sensing framework. Environmental variables including vegetation indices, soil moisture, rainfall, temperature, vegetation stress indicators, and hyperspectral bands were analyzed using correlation analysis, binary logistic regression, Random Forest classification, and variable importance ranking. Correlation analysis revealed weak relationships between individual environmental indicators and historical locust occurrence, with all predictors demonstrating statistically insignificant relationships (p > 0.05). Logistic regression findings similarly indicated that no individual environmental variable independently predicted locust occurrence. However, Random Forest classification achieved a classification accuracy of 75.33% with an Out-of-Bag error rate of 24.67%, suggesting moderate predictive capability when multiple variables were integrated. Variable importance analysis identified rainfall, soil moisture, temperature, and vegetation-related indicators as dominant predictors. The findings suggest that locust breeding and feeding habitats are influenced by complex multidimensional interactions rather than isolated environmental indicators. The study demonstrates the practical potential of AI-driven hyperspectral frameworks for strengthening locust surveillance and early warning systems within emerging high-risk agricultural regions.
Development and Validation of a Pyrolyzer Model for Ethylene Production from Natural Gas Liquids Using Mass and Energy Balance Approaches
Ethylene is one of the most important petrochemical feedstocks used in the production of plastics, synthetic fibers, solvents, and numerous industrial chemicals. The increasing global demand for ethylene has necessitated the development of efficient and sustainable production technologies. Natural gas liquids (NGLs), particularly ethane, have emerged as attractive feedstocks for ethylene production because of their high hydrogen-to-carbon ratio, availability, and favorable cracking characteristics. This study developed a pyrolyzer model for ethylene production from natural gas liquids using fundamental mass and energy balance approaches. The model was implemented in MATLAB to simulate the performance of a tubular pyrolysis reactor under varying operating conditions. Reactor performance was evaluated using fractional conversion, reactor temperature, reactor volume, pressure drop, space time, and space velocity as key performance indicators. The simulation results showed that reactor temperature increased from 894 K to 1063 K as conversion increased from 10% to 90%, while reactor volume, pressure drop, and space time exhibited corresponding increases. Conversely, space velocity decreased with increasing conversion due to longer residence time requirements. Model validation was conducted through comparison with published benchmark literature data. The validation results showed prediction deviations below 5% for major reactor performance parameters, indicating good agreement between model outputs and reported industrial operating conditions. The developed model provides a reliable preliminary engineering tool for pyrolysis reactor analysis, process evaluation, and future optimization studies involving ethylene production from natural gas liquids
Copyright (c) 2026 Godloves Tondie Nonju, Braye Oritom https://creativecommons.org/licenses/by-sa/4.0
Enhancing Conversion Of Flared Natural Gas Into Liquefied Petroleum Gas For Industrial And Domestic Energy Use
Gas flaring remains a major environmental and economic challenge in Nigeria, resulting in the loss of valuable hydrocarbon resources and contributing to greenhouse gas emissions. This study evaluates the technical and economic feasibility of converting flared natural gas into Liquefied Petroleum Gas (LPG) for domestic and industrial energy utilization in the Niger Delta region of Nigeria. Process simulation was carried out using Aspen HYSYS Version 3.2 with the Peng–Robinson Equation of State to model gas dehydration, cryogenic Natural Gas Liquids (NGL) recovery, and hydrocarbon fractionation processes. A flare gas feed stream of 34.83 million standard cubic feet per day (MMScfd) was subjected to Triethylene Glycol (TEG) dehydration, cryogenic cooling, and separation through demethanizer, deethanizer, and depropanizer columns. Simulation results indicated that the proposed process could recover approximately 3.87 MMScfd of Liquefied Petroleum Gas, 4.01 MMScfd of ethane, and 25.10 MMScfd of methane. The recovered LPG contained propane and butane predominantly and corresponded to an annual production of approximately 74.96 million kilograms. Economic analysis showed an estimated annual revenue of ₦18.74 billion at an LPG selling price of ₦250 per kilogram. The results demonstrate that flare gas can be effectively converted into commercially valuable products while reducing environmental pollution associated with routine gas flaring. Although profitability depends on assumptions regarding market conditions and operating costs, the study confirms that LPG recovery from flared gas represents a technically feasible and economically attractive option for enhancing energy utilization and supporting sustainable development in Nigeria.
Copyright (c) 2026 GODLOVES TONDIE NONJU https://creativecommons.org/licenses/by-sa/4.0
Intelligent IoT-based Alert System for Monitoring Drip of Intravenous Fluids (IV) and Reverse Blood Flow
Intravenous (IV) therapy is widely used in hospitals for administering fluids, medications, and nutrients, yet conventional manual monitoring may lead to delayed intervention, fluid depletion, reverse blood flow, and other patient-safety risks. This study proposes an Intelligent IoT-Based IV Drip System (IBIVDS) for real-time monitoring of IV fluid levels and reverse blood flow. The system integrates a NodeMCU ESP8266 microcontroller, load-cell weight sensor, LCD, buzzer, and Node-RED platform, supported by Arduino IDE, FreeRTOS, Python, HTML, JavaScript, and Django. The system continuously monitors IV fluid status and automatically sends alerts to healthcare personnel when predefined thresholds are reached. Notifications are generated at 50% remaining fluid and again at 20%, with the latter accompanied by an audible buzzer alert. Experimental evaluation showed that the proposed system achieved an average notification delay of approximately 0.01 min, substantially lower than the approximately 1 min delay reported for the baseline Smart Infusion Pump System under comparable conditions. The system also provides room and patient identifiers to facilitate rapid clinical response. The novelty of this study lies in integrating real-time IV level monitoring, reverse blood-flow prevention, multi-threshold alerting, and remote IoT-based notification within a single low-cost monitoring framework. The findings indicate that IBIVDS can improve the responsiveness, reliability, and consistency of IV therapy monitoring while reducing dependence on continuous manual observation.
Copyright (c) 2026 Douglas Omwenga Nyabuga, Nadia Iradukunda https://creativecommons.org/licenses/by-sa/4.0
Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck
Climate misinformation often combines a factual proposition with a recognizable narrative, such as denying observed warming, rejecting human causation, minimizing impacts, attacking mitigation, or casting doubt on climate science. This paper presents a lightweight narrative-aware scientific claim verification agent for the official ClimateCheck setting. The revised evaluation uses the official annotated ClimateCheck training data, the official publications corpus of 394,269 abstracts, and a claim-level validation split of the annotated data. The public ClimateCheck test file is treated as a blind claim list because its public fields do not contain verification or narrative labels. The system combines hashed BM25, TF-IDF retrieval, latent semantic analysis, narrative-family probabilities, and a logistic-regression verifier. Full-corpus retrieval shows that BM25 remains the strongest first-stage retriever, with Recall@10 = 0.466, while the narrative-aware hybrid obtains Recall@10 = 0.444. In the judged candidate reranking setting, the narrative-aware ranker obtains the highest Candidate Recall@1 = 0.789 and MAP = 0.848, compared with 0.759 and 0.843 for TF-IDF. End-to-end verification remains difficult: the BM25 top-1 pipeline reaches Macro-F1 = 0.408, while the narrative-aware pipeline reaches Macro-F1 = 0.355. Claim-level narrative evaluation no longer produces a perfect score; single-label top-family Macro-F1 is 0.422, and fine-grained multi-label CARDS-code Macro-F1 is 0.098. These results show that narrative information is useful for reranking already plausible evidence candidates, but it does not replace strong lexical retrieval and does not by itself solve claim verification.
Copyright (c) 2026 Wenhao Su, Siyu Chen, Ethan Qian https://creativecommons.org/licenses/by-sa/4.0
Power-Aware Inventory Planning for AI Infrastructure Using Job-Level Forecasting and LLM Workload Explanations
AI infrastructure planning is commonly expressed as a GPU-count problem, yet operational risk is created by the electric and thermal envelope that accompanies each accelerator. This paper evaluates a power-aware planning method on Dataset A, using the B200 eight-GPU Llama-8B training trace with 45,000 raw 20 ms telemetry rows and 8,940 reproducible supervised decision records after a 100 ms decision stride. The forecasting task predicts total eight-GPU power one second ahead from job-level counters, autoregressive lags, and rolling statistics. The planning task converts forecasts into a peak-aware admission rule and a circuit-inventory simulation for 32 concurrent jobs. XGBoost produced the strongest mean forecast, with MAE 273.26 W, RMSE 636.74 W, and R2 0.923. A calibrated high-quantile forecast produced lower peak-error behavior, reducing the scheduling violation rate from 5.31% under GPU-count-only admission to 0.18% while admitting 61.63% of decision points. In the inventory simulation, XGBoost mean forecasting used 21.00 mean circuits with 1.80% violation risk, whereas the calibrated p95 plan used 22.70 circuits and eliminated observed violations in 1,000 trials. The results show that capacity plans based only on GPU count hide measurable electrical risk. A combined GPU-capacity, power-envelope, and workload-explanation view produces a reproducible basis for AI data center purchasing, placement, and sustainability decisions.
Copyright (c) 2026 Shilu He https://creativecommons.org/licenses/by-sa/4.0
A Therapist-Facing Session Copilot for Live Counseling Support: Reasoning-Guided Retrieval and Ranking from Multi-Turn Counseling Dialogues
This study develops and evaluates a retrieval-based therapist-facing session copilot for live counseling support using the public bilingual Psy-Insight dataset. Rather than providing autonomous psychotherapy or relying on a generative large language model (LLM), the system assists human therapists by ranking historical responses, retrieving interpretable rationales, and providing conservative contextual support. The reproducible pipeline combines TF-IDF representations, class-balanced LinearSVC routers, nearest-neighbor rationale retrieval, and label-aware response ranking without LLM fine-tuning. Experiments use all 520 English and 431 Chinese sessions (6,208 and 5,776 turns, respectively) with session-level train/dev/test splits. Psychotherapy routing achieves strong macro-F1 scores of 0.897 in English and 0.757 in Chinese, whereas strategy routing remains weak (0.253 and 0.268). Label-aware rationale retrieval improves ROUGE-L from 0.145 to 0.152 in English and from 0.142 to 0.151 in Chinese. The best response-ranking approach presents retrieved reasoning in parallel rather than through reasoning-fused reranking, increasing MRR from 0.498 to 0.541 in English and from 0.519 to 0.523 in Chinese while maintaining low latency (6.98–14.64 ms/query). These results demonstrate computational feasibility but do not establish therapeutic safety or clinical effectiveness.
Copyright (c) 2025 Yifan Zhang, Hailey Zhang https://creativecommons.org/licenses/by-sa/4.0
Few-Shot Cold-Start Workload Forecasting for New AI Inference Tenants with Time-Series Foundation Models
This paper presents a reproducible empirical study of few-shot cold-start workload forecasting for new AI inference tenants using the Alibaba GPU-disaggregated DLRM serving trace. Instance lifecycles are transformed into hourly active-demand series, and resource reservations are normalized into capacity units to evaluate 24-hour forecasting under zero-shot, 5-shot, 10-shot, and full-history settings. Seven forecasting methods are compared: archetype mean prior, persistence, moving average, linear trend, seasonal naive, global residual ridge, and CT-TSFM, a compact cross-tenant time-series foundation model. The cold-start evaluation uses 46 held-out tenants, with 110 source tenants for pretraining and calibration. Results show that hourly demand is strongly persistence-dominated. Zero-shot forecasting yields a mean absolute error (MAE) of 326.26 normalized capacity units, whereas only five observations reduce MAE to 4.00 for persistence, global residual ridge, and CT-TSFM. Validation consistently selects a residual gate of 0.0 for CT-TSFM, indicating that retaining the persistence prior and rejecting cross-tenant residual transfer is the most reliable strategy. Calibration intervals achieve approximately 85–87% coverage against a 90% target. The findings demonstrate that a few recent observations substantially improve cold-start forecasting, while source-tenant metadata alone provides limited zero-shot planning capability.
Copyright (c) 2025 Shilu He, Andrew Qian https://creativecommons.org/licenses/by-sa/4.0
Profit-Aware Spot GPU Admission Control with Cost-Sensitive Loss and Evidence-Grounded Policy Memos for AI Workload Supply-Demand Matching
AI clusters increasingly operate with heterogeneous GPU resources where production workloads and opportunistic spot jobs compete for limited accelerator capacity. This study presents a trace-driven admission-control framework using the Alibaba cluster-trace-v2026-spot-gpu dataset, consisting of 466,867 job records and 4,278 GPU-node records. The experiment evaluates GPU demand forecasting, profit-aware spot admission control, and evidence-grounded operational policy generation using chronological training, validation, and test splits. Hourly spot GPU demand forecasting was evaluated across six GPU models, where Ridge regression achieved the best test performance with an RMSE of 38.50 requested GPUs per hour, improving over both last-hour and seasonal naive baselines. The admission-control evaluation compared FIFO, greedy packing, classifier-based acceptance, utility ranking, and the proposed cost-sensitive policy. The proposed approach achieved a test profit of 67,278.96, improving 1.97% over the accuracy-oriented classifier while increasing spot success rate and reducing costly false acceptances by 13.17%. Sensitivity analysis showed that the optimal policy depends on the protection cost assigned to high-priority workloads. A deterministic evidence-grounded explanation layer generated 500 policy memos and passed numeric, policy, and evidence consistency checks. The findings suggest that profit-aware admission control can serve as a practical scheduling guardrail before detailed GPU placement and resource allocation decisions.
Copyright (c) 2026 Siming Zhao, Yuxuan Ren, Xiaohan Chang https://creativecommons.org/licenses/by-sa/4.0
Accounting-Aware Evidence-Constrained Agents for Disclosure, Settlement, and Secondary-Market Risk Monitoring in Tokenized
Tokenized real-world asset (RWA) infrastructure exposes platform operators, investors, and reporting teams to a combined settlement, disclosure, liquidity, and accounting-quality monitoring problem. A tokenized claim can continue to trade while the underlying issuer releases new financial statements, securities fail to deliver in the reference market, or protocol-level liquidity changes in RWA venues. This paper develops an accounting-aware, evidence-constrained agent workflow for risk alerting and source-grounded report generation. The revised experiment replaces the earlier rule-generated monitoring sandbox with external datasets: SEC fails-to-deliver observations, SEC EDGAR XBRL company facts, SEC submissions metadata, Financial PhraseBank sentiment labels, and DefiLlama RWA protocol TVL. The issuer-day panel contains 2,648 surveillance tasks for eight large U.S. issuers from 2024-12-01 through 2026-03-31. Observed settlement stress is defined from external SEC FTD balances rather than from the agent's own rule. Accounting risk is computed from XBRL-derived liquidity, leverage, accrual, and cash-flow indicators. A stronger market-plus-accounting logistic baseline is added alongside single-source baselines and the proposed fusion agent. The machine-learning baseline achieves the strongest F1 score for settlement-stress detection (0.909), while the proposed fusion agent achieves the highest report faithfulness and tool-use correctness (1.000 each) and high recall (0.849). The results support a governance-oriented interpretation: an evidence-constrained agent is most useful not as an opaque high-accuracy classifier, but as an auditable layer that connects settlement evidence, filing metadata, accounting fundamentals, independent sentiment calibration, and RWA protocol liquidity into a reproducible monitoring record.
Copyright (c) 2026 Sihan Zhou, Yuanzheng Chen, Kenny Lee https://creativecommons.org/licenses/by-sa/4.0
Budgeted Multi-Hop Retrieval Agent for Compositional Question Answering: A Retrieval-Policy Evaluation on the Official MultiHop-RAG Benchmark
Multi-hop question answering requires a retrieval system to assemble several complementary evidence documents before an answer module can reason reliably. Single-shot retrieval is efficient, but it often misses later-hop evidence when a question combines source, time, comparison, and entity constraints. This paper evaluates a budgeted multi-hop retrieval agent for compositional question answering on the official MultiHop-RAG benchmark. The benchmark contains 2,556 queries and 609 news-article corpus documents, with answerable evidence distributed across two to four documents. Four retrieval policies are compared under the same sparse lexical scorer: fixed top-k retrieval, iterative retrieval, query decomposition, and the proposed budgeted retrieval agent. The revised evaluation frames the task as retrieval-policy evaluation rather than as a full free-form generative QA system: retrieval-conditioned EM/F1 are reported together with evidence recall, MRR, retrieval rounds, selected documents, and context-token cost. On the official data, the budgeted agent achieves the strongest overall retrieval-conditioned EM/F1 at 62.75% and the highest final evidence recall at 74.67%, using 3.011 average retrieval calls and 509.7 average context tokens. Query decomposition improves over fixed top-k and iterative retrieval but is less stable across question types. Fixed top-k is cheapest but incomplete on longer chains. The four-hop results remain difficult for every policy, showing that a fixed 620-token controller should be extended with hop-aware or dynamic budget allocation. The findings support a moderated contribution claim: explicit budget control is useful for auditable multi-hop retrieval, but it should be evaluated as a cost-accuracy trade-off rather than as a universally dominant RAG architecture.
Copyright (c) 2025 Wenhao Su, Siyu Chen, Chloe Zhao https://creativecommons.org/licenses/by-sa/4.0
Accounting-Aware Evidence Retrieval for Institutional Due Diligence of Tokenized Trade Receivable RWA
Institutional investors evaluating tokenized real-world asset (RWA) transactions need retrieval systems that can answer short, ambiguous, and legally loaded due-diligence questions with traceable evidence. Trade receivable pools are especially difficult because the same question may require accounting policy, financial metrics, footnote disclosure, legal covenants, insurance language, servicer reporting, or waterfall mechanics. This study implements and evaluates an accounting-aware evidence-retrieval pipeline for tokenized trade receivable RWA due diligence. The main experiment uses the official FinDER benchmark with 5,703 query-evidence-answer triples, 6,121 annotated evidence references, and 5,830 deduplicated evidence passages derived from financial disclosures. The pipeline compares vanilla sparse retrieval, accounting-aware query rewriting, feature reranking, section-aware evidence selection, and calibrated abstention. On the official FinDER evaluation, query rewriting increased Recall@10 from 28.25% to 28.62%, reranking increased Recall@10 to 33.86% and answer-support accuracy to 24.57%, and section-aware evidence selection achieved 34.44% Recall@10, 24.04% nDCG@10, 8.32% EvidencePrecision@3, and 25.23% answer-support accuracy. The accounting-relevant subset, defined as Accounting, Financials, and Footnotes, achieved 37.10% Recall@10 and 26.54% answer-support accuracy. A supplementary stress check using a public receivables purchase agreement and SEC 2026-04 financial statement notes showed that the same retrieval logic can surface schedule, lock-box, GAAP, receivable, and note-disclosure evidence, while also highlighting the need for table extraction and field-level numerical validation. The findings support a narrower deployment claim: accounting-aware RAG can improve evidence discovery and analyst review, but it is not yet suitable for autonomous investment or accounting decision-making
Copyright (c) 2025 Yuanzheng Chen, Sihan Zhou, Emma Lin https://creativecommons.org/licenses/by-sa/4.0
Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data
This paper presents a compact reproducible benchmark for evaluating numerical-reasoning guardrails in a quant research assistant. The revised experiment uses a fixed 2026 source snapshot derived from the SEC 2026 Q1 Financial Statement Data Sets and FRED CSV series for VIXCLS, DGS10, DGS3MO and T10Y3M. The benchmark contains 460 tasks: 300 SEC financial-ratio tasks over 50 issuer-period records, 120 FRED VIX and Treasury-rate change tasks, and 40 macro-regime classification tasks. Each answer is evaluated by five programmatic guardrails: numeric consistency, unit correctness, time-window correctness, formula correctness and citation/source consistency. Four controlled response profiles are tested: Naive-RAG, Calculator-Only, Prompted-Checklist and Guarded-Quant. These profiles are deterministic failure-mode controls rather than performance claims about any particular deployed LLM. The empirical results show that arithmetic alone is not sufficient for financial safety: Calculator-Only reaches 79.78% numeric accuracy but only 0.43% all-guardrails pass rate because source, unit, formula and window fields often fail. Guarded-Quant achieves an 88.48% all-guardrails pass rate, 97.17% numeric accuracy, 100.00% unit pass rate, 96.30% window pass rate, 98.26% formula pass rate and 96.30% citation pass rate. The findings support a modest claim: a compact benchmark can make numerical audit failures visible, but it should not be read as evidence of broad quant-assistant reliability without broader data, live model outputs and operational stress tests.
Copyright (c) 2026 Zeyi Li, Kai Zhang, Annie Wong https://creativecommons.org/licenses/by-sa/4.0
News-Based Uncertainty and Macro-Market Fusion for VIX Direction Forecasting: Evidence from 2015-2024 FRED Panel
This paper evaluates news-based uncertainty and macro-market fusion for one-trading-day VIX direction forecasting using a 2015-2024 daily FRED panel. The dependent variable equals one when the next trading-day VIX close exceeds the current close. The final panel contains 2,512 processed observations from 2015-02-04 to 2024-12-30, with 1,997 training observations from 2015-2022, 257 validation observations in 2023, and 258 holdout observations in 2024. The feature set combines VIX market-state variables, the daily newspaper-based Economic Policy Uncertainty index, the effective federal funds rate, 10-year and 2-year Treasury yields, lagged CPI inflation, lagged unemployment, and interaction terms. Expanding cross-validation on the 2015-2022 training sample gives the highest average ROC AUC to Fusion Random Forest (0.5620). The 2023 validation window selects a weighted fusion ensemble with ROC AUC 0.6005; its weights are fixed before the 2024 test. In the 2024 holdout window, VIX-only Logistic achieves the highest one-day ROC AUC (0.5869) and F1 (0.5448), while the weighted fusion ensemble reaches ROC AUC 0.5638 and F1 0.4786. Event-window diagnostics show that EPU shocks following calm VIX states have a next-day VIX-up rate of 0.5672, compared with 0.4445 on other trading days. The findings support a cautious interpretation: news-based uncertainty contains conditional information, but one-day practical forecasting reliability remains modest and VIX state variables remain the strongest 2024 holdout benchmark.
Copyright (c) 2025 Hailin Zhou, Kai Zhang https://creativecommons.org/licenses/by-sa/4.0
Trajectory Reliability Prediction for Generalist AI Agents: Tool-Use Failure Analysis and Success Forecasting on ZClawBench
Generalist AI agents increasingly perform complex tasks through planning, tool execution, action revision, and artifact generation rather than isolated response generation. This study empirically investigates trajectory reliability prediction on ZClawBench, a public OpenClaw-style agent benchmark containing 696 model-task trajectories across 116 tasks, six model families, and six scenario categories. The study evaluates whether operational trajectory signals—including tool-call volume, replanning behavior, tool errors, invalid-action ratio, recovery patterns, looping indicators, trajectory length, and task category—can predict task success before manual evaluation. Since the dataset provides trajectories and identifiers but lacks explicit per-instance success labels, the binary target is reconstructed from official model-by-category score distributions and treated as a modeling assumption. Five-fold cross-validation with task-level splitting was applied to prevent task leakage. Logistic Regression, Random Forest, XGBoost, sequence TF-IDF classification, and a rubric-based trajectory judge were compared. Logistic Regression achieved the strongest calibrated performance, obtaining ROC-AUC of 0.970, F1-score of 0.913, Brier score of 0.066, and expected calibration error of 0.018. Feature analysis indicated that task difficulty, no-progress behavior, response size, tool errors, and invalid actions contributed most to reliability prediction. The findings suggest that lightweight trajectory diagnostics can support agent monitoring, failure triage, and routing decisions, while further validation with direct case-level evaluation labels is required for deployment claims.
Copyright (c) 2026 Ziliang Samuel Zhong, Chenyu Li, Hengning Rao https://creativecommons.org/licenses/by-sa/4.0
Cost-Aware LLM-Style Routing for AIOps Log Analysis: Log Parsing, Anomaly Detection, Fault Diagnosis, and Incident Summarization on LogEval Task Files
This study investigates a local and cost-aware routing framework for AIOps log analysis using the LogEval benchmark. The evaluation covers four tasks: log parsing, anomaly detection, fault diagnosis, and incident summarization. Instead of relying on external large language model APIs, the experiment implements deterministic local policies that simulate zero-shot and few-shot LLM-style inference under controlled token-cost and latency assumptions. Six approaches were compared: regex normalization, TF-IDF with machine learning, a local character-based classifier, zero-shot policy, few-shot retrieval policy, and a routing cascade. At a risk threshold of 0.20, the router directed only 12.9% of queries to the few-shot retrieval policy while achieving parsing accuracy of 0.991, anomaly F1-score of 1.000, diagnosis accuracy of 1.000, ROUGE-L of 0.743, and BLEU-1 of 0.814. The routing strategy reduced simulated token cost by 80.1% compared with always using few-shot retrieval. Additional unseen-template evaluation revealed limited generalization for closed-label classifiers and retrieval methods when encountering unseen patterns. The findings indicate that routing can effectively reduce AIOps inference costs, while further validation with real LLMs and stronger generalization testing are required before production deployment.
Copyright (c) 2026 Chenyu Li, Ge Liu, Zoe Zhao https://creativecommons.org/licenses/by-sa/4.0
LLM-Inspired Offline Reranking for Financial Search: Query Rewriting, Hybrid Retrieval, and Listwise Relevance Ranking on FiQA
Financial search has high practical value because investors and retail users often ask natural-language questions whose wording differs from relevant financial passages. This paper evaluates a multi-stage retrieval pipeline on FiQA, a financial question-answering retrieval collection in BEIR. The systems include BM25, Dense LSA, BM25-LSA hybrid retrieval, reciprocal-rank fusion, a compact linear reranker, fixed pointwise and listwise relevance rubrics inspired by LLM reranking, query rewriting, and the proposed query rewriting plus hybrid retrieval plus listwise reranking pipeline. The evaluation used the full 57,638-document FiQA corpus, 6,648 available queries, and the 648-query BEIR FiQA test qrels with 1,706 binary relevance judgments. BM25 was the best-performing system, with nDCG@10 = 0.2285, MAP = 0.1863, MRR = 0.2994, and Recall@100 = 0.5207. The proposed full pipeline underperformed BM25. The listwise rubric ranked second on nDCG@10 (0.2228) and improved over the pointwise rubric, suggesting that candidate-list normalization can be useful in this setting. The rubric rerankers are fixed local scoring rules, so these results should be read as an evaluation of LLM-inspired ranking logic rather than as a benchmark of an actual prompt-based LLM reranker. Dense LSA retrieval alone was weak (nDCG@10 = 0.0287), which shows the limitation of a conservative non-neural dense baseline in financial semantic matching. Query rewriting reduced average effectiveness. The findings recommend strong lexical baselines, conservative rewrite gating, and careful evaluation before adopting prompt-based or model-based LLM rerankers in financial search.
Copyright (c) 2026 Siquan Meng, Jing Chen, Isa Zheng https://creativecommons.org/licenses/by-sa/4.0
Evidence-Calibrated RAG for Unanswerable Question Answering: Retrieval Coverage, Abstention Calibration, and Hallucination-Proxy Analysis on SQuAD 2.0
This paper presents a controlled and reproducible empirical study of evidence-calibrated retrieval-augmented question answering (RAG) for answerable and unanswerable reading-comprehension tasks using the SQuAD 2.0 benchmark. The study focuses on whether a system should abstain when retrieved evidence is insufficient rather than always producing an answer. Six lightweight architectures were evaluated on the full validation set of 11,873 questions, including closed-book, BM25, dense, hybrid, reranked, and a proposed evidence-calibrated hybrid RAG model. The proposed approach combines hybrid top-25 retrieval, lexical reranking, deterministic extractive answering, and evidence sufficiency calibration trained on 43,482 examples. On the validation set, it achieved 31.65% exact match, 34.74% F1, 53.01% answerability accuracy, 53.71% refusal F1, and a 37.49% hallucination-proxy rate. Although overall QA performance remains modest, calibrated evidence sufficiency substantially reduced unsupported answers compared with a forced-answer hybrid reranker, lowering the hallucination-proxy rate from 77.80% while improving F1. However, evidence calibration itself remained weak (AUROC 0.5475, ECE 0.1144). The findings demonstrate that retrieval coverage alone is insufficient to prevent hallucinations and highlight the need for stronger evidence calibration in trustworthy RAG systems.
Copyright (c) 2025 Ziliang Samuel Zhong, Jing Chen, Eric Zhong, Xinzhuo Sun https://creativecommons.org/licenses/by-sa/4.0
Evidence-Chain Reliable RAG: Hallucination Detection, Source Attribution, and Deterministic Provenance Explanations
Retrieval-augmented generation (RAG) reduces unsupported generation by grounding answers in source content, but retrieval alone does not guarantee that every output claim is attributable to evidence. This paper presents Evidence-Chain Reliable RAG, an empirical hallucination-detection and provenance framework that scores whether generated response sentences are supported by the corresponding RAG source record. The evaluation uses the complete RAGTruth JSONL data available for this study: 2,965 source records, 17,790 assistant responses, and 14,289 exact-offset hallucination spans across Data2Text, QA, and summarization. The experiment converts word-level spans into response-level, sentence-level, and character-span targets; extracts lexical, BM25, TF-IDF, unsupported-number, unsupported-entity, refusal, and Evidence-Chain Score features; and compares seven methods. On the official held-out test split of 2,700 responses, RandomForest achieved the best case-level F1 of 0.626 and PR-AUC of 0.553. The proposed ECS-Span model achieved case-level F1 of 0.614, ROC-AUC of 0.742, and PR-AUC of 0.536 while also producing deterministic provenance explanations. At sentence level, RandomForest again achieved the highest F1 of 0.321; the proposed method obtained F1 of 0.312, ROC-AUC of 0.777, and PR-AUC of 0.245. Exact character-span localization remained difficult, with character-level F1 of 0.197 because sentence-level predictions often include supported text around shorter hallucinated spans. The findings indicate that evidence-chain features are useful for interpretable RAG auditing, but precise span extraction requires token-level sequence labeling or a comparable fine-grained model.
Copyright (c) 2025 Jiaying Jin https://creativecommons.org/licenses/by-sa/4.0
LLM-Style DevOps Copilot for Cloud-Native Troubleshooting: Retrieval-Augmented Runbook Generation and Command-Safety Evaluation
Cloud-native incident response requires engineers to connect symptoms, observability signals, infrastructure state, and safe remediation commands under time pressure. Large language models can draft runbooks, but an ungrounded assistant can invent commands, recommend the wrong diagnostic path, or reproduce destructive operational shortcuts. This paper evaluates an LLM-style, retrieval-augmented DevOps Copilot simulation for cloud-native troubleshooting on the canonical Szaid3680/Devops Arrow export. The experiment indexes all 42,819 rows with the public Response, Instruction, and Prompt schema and evaluates a deterministic 400-query subset with TF-IDF, BM25, a compact dense-semantic baseline, RAG-style answer construction, reranking, command-safety checking, and the combined reranker-plus-checker pipeline. No live LLM inference is used in the executed experiment; the generation and checking components are deterministic so that the safety effects can be reproduced exactly. Results show that retrieval improves answer grounding but does not by itself guarantee safe automation: RAG-only reaches 0.2966 semantic similarity and emits matched unsafe command text at a rate of 0.0324. The command-safety checker reduces the matched unsafe command rate to 0.0000 for the declared rule set and keeps command validity at 0.9922. The full pipeline obtains 0.3051 semantic similarity, 0.4225 root-cause accuracy, 0.5825 root-category accuracy, and 0.0078 hallucinated-command rate. The findings support treating DevOps copilots as retrieval-grounded and policy-checked workflow systems rather than free-form chat agents.
Copyright (c) 2026 Boning Zhang, Xinzhuo Sun, Ge Liu, Binghua Zhou https://creativecommons.org/licenses/by-sa/4.0