Lightweight Image–Text Triage for Rice Leaf Diseases with Clarify-or-Abstain Decisions and Nepal-Context Advisory Support

Lumeng  Han; Derek  Zhou

doi:10.51903/jtie.v4i1.593

Authors

Lumeng Han International Agricultural Development, University of California, Davis, CA, USA
Derek Zhou Computer Science, University of California, Berkeley, Berkeley, CA, USA

DOI:

https://doi.org/10.51903/jtie.v4i1.593

Keywords:

Rice Leaf Disease, Lightweight Computer Vision, Open-Set Recognition, Agricultural Advisory, Nepal context

Abstract

Image-based rice disease screening requires assessment of unfamiliar conditions, degraded photographs, and the evidence used for advice. This study evaluates a modular image–text workflow with separately trained visual and retrieval components. A rotation- and reflection-aware audit of 5,932 rice leaf images identified 343 similarity groups for leakage-reduced training, validation, and testing. Seven lightweight classifiers and a 24,724-parameter CNN were evaluated alongside open-set rejection, image-quality triage, simulated recapture, and external testing on 10,333 exact-unique Paddy Doctor images. Color-texture logistic regression achieved 78.41% grouped-test accuracy and the CNN achieved 81.03%; their accuracies on 4,232 external images from the four known classes fell to 13.33% and 12.55%, respectively. External unknown-detection AUROCs were 0.512 for Mahalanobis scoring and 0.486 for CNN nonconfidence. Across 7,745 scoped CROP records, reranked retrieval recovered the paired answer within five candidates for 934/1,168 test queries (79.97%). Evidence-assisted benchmark accuracy was 65.21% on 365 questions and 41.34% on 179 without an exact question match in the retrieval collection. Four advisory gates enforced disease, location, treatment, and numeric-rate rules, with a trade-off between answer delivery and reference-answer retention. Nepal-context retrieval remained preliminary: the reranker succeeded on 7/8 queries (87.50%; Wilson 95% CI, 52.91–97.76%). Classification, triage, and advisory retrieval provide separate levels of evidence. The findings support further investigation of the workflow, while the external failures preclude claims of reliable field deployment.

References

Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101

Bendale, A., & Boult, T. E. (2016). Towards open set deep networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 1563–1572). https://doi.org/10.1109/CVPR.2016.173

Cormack, G. V., Clarke, C. L. A., & Buettcher, S. (2009). Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (pp. 758–759). https://doi.org/10.1145/1571941.1572114

Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20, 273–297. https://doi.org/10.1007/BF00994018

Dalal, N., & Triggs, B. (2005). Histograms of oriented gradients for human detection. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 1, pp. 886–893). https://doi.org/10.1109/CVPR.2005.177

Davis, J., & Goadrich, M. (2006). The relationship between precision-recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning (pp. 233–240). https://doi.org/10.1145/1143844.1143874

Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K., & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science, 41(6), 391–407. https://doi.org/10.1002/(SICI)1097-4571(199009)41:6<391::AID-ASI1>3.0.CO;2-9

Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognition Letters, 27(8), 861–874. https://doi.org/10.1016/j.patrec.2005.10.010

Ferentinos, K. P. (2018). Deep learning models for plant disease detection and diagnosis. Computers and Electronics in Agriculture, 145, 311–318. https://doi.org/10.1016/j.compag.2018.01.009

Geifman, Y., & El-Yaniv, R. (2017). Selective classification for deep neural networks. In Advances in Neural Information Processing Systems (Vol. 30). https://proceedings.neurips.cc/paper/2017/hash/4a8423d5e91fda00bb7e46540e2b0cf1-Abstract.html

Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63(1), 3–42. https://doi.org/10.1007/s10994-006-6226-1

Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 1321–1330). PMLR. https://proceedings.mlr.press/v70/guo17a.html

Hendrycks, D., & Dietterich, T. (2019). Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations. https://arxiv.org/abs/1903.12261

Hendrycks, D., & Gimpel, K. (2017). A baseline for detecting misclassified and out-of-distribution examples in neural networks. In International Conference on Learning Representations. https://arxiv.org/abs/1610.02136

Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M., Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, Q. V., & Adam, H. (2019). Searching for MobileNetV3. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 1314–1324). https://doi.org/10.1109/ICCV.2019.00140

Lee, K., Lee, K., Lee, H., & Shin, J. (2018). A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems (Vol. 31, pp. 7167–7177). https://proceedings.neurips.cc/paper_files/paper/2018/hash/abdeb6f575ac5c6676b747bca8d09cc2-Abstract.html

Li, Y. (2024). Findable then explainable: Retrieval–summary integration for code intelligence on a lightweight CodeSearchNet subset. Journal of Advanced Computing Systems, 4(7), 65–82. https://doi.org/10.69987/JACS.2024.40706

Liu, W., Wang, X., Owens, J., & Li, Y. (2020). Energy-based out-of-distribution detection. In Advances in Neural Information Processing Systems (Vol. 33, pp. 21464–21475). https://proceedings.neurips.cc/paper_files/paper/2020/hash/f5496252609c43eb8a3d147ab9b9c006-Abstract.html

Mohanty, S. P., Hughes, D. P., & Salathé, M. (2016). Using deep learning for image-based plant disease detection. Frontiers in Plant Science, 7, Article 1419. https://doi.org/10.3389/fpls.2016.01419

Nie, J., & Zheng, D. (2023). Ambiguity-aware HDFS log anomaly detection with retrieval-augmented failure narratives and selective refusal. Journal of Advanced Computing Systems, 3(1), 66–80. https://doi.org/10.69987/JACS.2023.30105

Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F., El-Nouby, A., Assran, M., Ballas, N., Galuba, W., Howes, R., Huang, P.-Y., Li, S.-W., Misra, I., Rabbat, M., Sharma, V., . . . Bojanowski, P. (2023). DINOv2: Learning robust visual features without supervision. arXiv. https://doi.org/10.48550/arXiv.2304.07193

Petchiammal, Kiruba, B., Murugan, & Arjunan, P. (2023). Paddy Doctor: A visual image dataset for automated paddy disease classification and benchmarking. In Proceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD) (pp. 203–207). Association for Computing Machinery. https://doi.org/10.1145/3570991.3570994

Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., & Sutskever, I. (2021). Learning transferable visual models from natural language supervision. In Proceedings of the 38th International Conference on Machine Learning (Vol. 139, pp. 8748–8763). PMLR. https://proceedings.mlr.press/v139/radford21a.html

Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4), 333–389. https://doi.org/10.1561/1500000019

Romano, Y., Sesia, M., & Candès, E. (2020). Classification with valid and adaptive coverage. In Advances in Neural Information Processing Systems (Vol. 33, pp. 3581–3591). https://proceedings.neurips.cc/paper_files/paper/2020/hash/244edd7e85dc81602b7615cd705545f5-Abstract.html

Salton, G., & Buckley, C. (1988). Term-weighting approaches in automatic text retrieval. Information Processing & Management, 24(5), 513–523. https://doi.org/10.1016/0306-4573(88)90021-0

Scheirer, W. J., de Rezende Rocha, A., Sapkota, A., & Boult, T. E. (2013). Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(7), 1757–1772. https://doi.org/10.1109/TPAMI.2012.256

Sethy, P. K. (2024). Rice leaf disease image samples (Version 2) [Data set]. Mendeley Data. https://doi.org/10.17632/fwcj7stb8r.2

Sethy, P. K., Barpanda, N. K., Rath, A. K., & Behera, S. K. (2020). Deep feature based rice leaf disease identification using support vector machine. Computers and Electronics in Agriculture, 175, Article 105527. https://doi.org/10.1016/j.compag.2020.105527

Tan, M., & Le, Q. V. (2019). EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the 36th International Conference on Machine Learning (Vol. 97, pp. 6105–6114). PMLR. https://proceedings.mlr.press/v97/tan19a.html

Too, E. C., Yujian, L., Njuki, S., & Yingchun, L. (2019). A comparative study of fine-tuning deep learning models for plant disease identification. Computers and Electronics in Agriculture, 161, 272–279. https://doi.org/10.1016/j.compag.2018.03.032

Zhai, X., Mustafa, B., Kolesnikov, A., & Beyer, L. (2023). Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF International Conference on Computer Vision (pp. 11975–11986). https://openaccess.thecvf.com/content/ICCV2023/html/Zhai_Sigmoid_Loss_for_Language_Image_Pre-Training_ICCV_2023_paper.html

Zhang, H., Sun, J., Chen, R., Liu, W., Yuan, Z., Zheng, X., Wang, Z., Yang, Z., Yan, H., Zhong, H., Wang, X., Ouyang, W., Yang, F., & Dong, N. (2024). Empowering and assessing the utility of large language models in crop science. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, & C. Zhang (Eds.), Advances in Neural Information Processing Systems (Vol. 37, pp. 52670–52722). Curran Associates. https://doi.org/10.52202/079017-1669

Zhong, Z. S., Ma, R., & Zhao, H. (2023). Human-uncertainty distillation for calibrated vision models on CIFAR-10H. Journal of Advanced Computing Systems, 3(2), 77–89. https://doi.org/10.69987/JACS.2023.30206