Domain-Shift-Aware Climate-Smart Crop Portfolio Advisory for Nepal with Self-Supervised Tabular Learning and Conformal Shortlists

Lumeng  Han; Derek  Zhou

doi:10.51903/jtie.v4i2.592

Authors

Lumeng Han International Agricultural Development, University of California, Davis, CA, USA
Derek Zhou Computer Science, University of California, Berkeley, Berkeley, CA, USA

DOI:

https://doi.org/10.51903/jtie.v4i2.592

Keywords:

Climate-Smart Agriculture, Nepal, Self-Supervised Tabular Learning, Conformal Prediction, Evidence Consistency

Abstract

Crop advisory models can perform well on balanced benchmarks while failing under unfamiliar environmental or household conditions. This study evaluates two separate research tracks: crop-label classification using 2,200 agronomic benchmark records and observed crop-portfolio prediction using 43,753 village-census households and 981 TAFSSA households in Nepal. Eight crop classifiers were compared within a training pool, with independent temperature-calibration, conformal-calibration, and test subsets. Household models were evaluated under random and geographic holdouts, including 30 paired community-level ablations. An external explanation-consistency experiment used 900 held-out, human-annotated RAGTruth responses. Random forest attained the highest cross-validated macro-F1 (0.992), while LightGBM had the lowest conventional-model log loss (0.059). Temperature scaling produced negligible change in held-out log loss. At 90% nominal coverage, adaptive prediction sets included the benchmark label in 438 of 440 cases, with a mean of 1.09 crops. Environmental-cluster holdouts reduced macro-F1 to 0.000–0.656, partly because some test crops were absent from training. Village-census logistic-regression macro average precision declined from 0.680 under random partitioning to 0.578 under district holdout for the same test households. TAFSSA cross-district values were 0.332 and 0.445. A fixed-split contextual gain did not persist across community holdouts: full context reduced macro average precision by 0.037 relative to farm-only predictors. On RAGTruth, numeric checks detected 132 of 579 erroneous responses; a source-linked lexical screen detected 432 but also rejected 102 of 321 valid responses. The findings support cautious interpretation of benchmark shortlists and household associations. The components remain a research prototype without end-to-end or field validation of agricultural decisions in Nepal.

References

Angelopoulos, A. N., & Bates, S. (2023). Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4), 494–591. https://doi.org/10.1561/2200000101

Breiman, L. (2001). Random forests. Machine Learning, 45, 5–32. https://doi.org/10.1023/A:1010933404324

Central Bureau of Statistics. (2014). Statistical pocket book Nepal 2014. Government of Nepal, National Planning Commission Secretariat. https://giwmscdnone.gov.np/media/app/public/36/posts/1695537893_62.pdf

Chen, S., He, S., & Sun, E. (2024). Risk-bounded GPU resource oversubscription via conformal demand envelopes in production AI clusters. Journal of Advanced Computing Systems, 4(5), 119–134. https://doi.org/10.69987/JACS.2024.40509

Food and Agriculture Organization of the United Nations. (2017). Climate-smart agriculture sourcebook (2nd ed.). https://www.fao.org/climate-smart-agriculture-sourcebook/

Geifman, Y., & El-Yaniv, R. (2019). SelectiveNet: A deep neural network with an integrated reject option. In K. Chaudhuri & R. Salakhutdinov (Eds.), Proceedings of the 36th International Conference on Machine Learning (Vol. 97, pp. 2151–2159). PMLR. https://proceedings.mlr.press/v97/geifman19a.html

Geurts, P., Ernst, D., & Wehenkel, L. (2006). Extremely randomized trees. Machine Learning, 63, 3–42. https://doi.org/10.1007/s10994-006-6226-1

Grinsztajn, L., Oyallon, E., & Varoquaux, G. (2022). Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems, 35, 507–520. https://arxiv.org/abs/2207.08815

Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017). On calibration of modern neural networks. In D. Precup & Y. W. Teh (Eds.), Proceedings of the 34th International Conference on Machine Learning (Vol. 70, pp. 1321–1330). PMLR. https://proceedings.mlr.press/v70/guo17a.html

Hollmann, N., Müller, S., Eggensperger, K., & Hutter, F. (2023). TabPFN: A transformer that solves small tabular classification problems in a second. In The Eleventh International Conference on Learning Representations. https://openreview.net/forum?id=cp5PvcI6w8_

Ingle, A. (2020). Crop recommendation dataset (Version 1) [Data set]. Kaggle. https://www.kaggle.com/datasets/atharvaingle/crop-recommendation-dataset

Intergovernmental Panel on Climate Change. (2022). Climate change 2022: Impacts, adaptation and vulnerability. Cambridge University Press. https://doi.org/10.1017/9781009325844

International Food Policy Research Institute. (2024a). TAFSSA district agrifood systems assessment in Nepal 2023: Community (Version 1) [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/PZMNBU

International Food Policy Research Institute. (2024b). TAFSSA district agrifood systems assessment in Nepal 2023: Male [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/Z0RF6L

International Rice Research Institute, & International Food Policy Research Institute. (2024). TAFSSA district agrifood systems assessment in Nepal 2023: Market survey (Version 1) [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/POVL6D

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023). Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12), Article 248. https://doi.org/10.1145/3571730

Kamilaris, A., & Prenafeta-Boldú, F. X. (2018). Deep learning in agriculture: A survey. Computers and Electronics in Agriculture, 147, 70–90. https://doi.org/10.1016/j.compag.2018.02.016

Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., Ye, Q., & Liu, T.-Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30, 3146–3154. https://proceedings.neurips.cc/paper/2017/hash/6449f44a102fde848669bdd9eb6b76fa-Abstract.html

Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., Lee, T., David, E., Stavness, I., Guo, W., Earnshaw, B., Haque, I., Beery, S. M., Leskovec, J., Kundaje, A., . . . Liang, P. (2021). WILDS: A benchmark of in-the-wild distribution shifts. Proceedings of Machine Learning Research, 139, 5637–5664. https://proceedings.mlr.press/v139/koh21a.html

Krishna, V., & Khed, V. (2024). Farmer census: Adoption of sustainable intensification technologies in Nepal Terai region (Version 1) [Data set]. CIMMYT Research Data & Software Repository Network. https://hdl.handle.net/11529/10548922

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in Neural Information Processing Systems, 33, 9459–9474. https://proceedings.neurips.cc/paper/2020/hash/6b493230205f780e1bc26945df7481e5-Abstract.html

Li, Y. (2024). Findable then explainable: Retrieval–summary integration for code intelligence on a lightweight CodeSearchNet subset. Journal of Advanced Computing Systems, 4(7), 65–82. https://doi.org/10.69987/JACS.2024.40706

Liakos, K. G., Busato, P., Moshou, D., Pearson, S., & Bochtis, D. (2018). Machine learning in agriculture: A review. Sensors, 18(8), Article 2674. https://doi.org/10.3390/s18082674

Lipper, L., Thornton, P., Campbell, B. M., Baedeker, T., Braimoh, A., Bwalya, M., Caron, P., Cattaneo, A., Garrity, D., Henry, K., Hottle, R., Jackson, L., Jarvis, A., Kossam, F., Mann, W., McCarthy, N., Meybeck, A., Neufeldt, H., Remington, T., . . . Torquebiau, E. F. (2014). Climate-smart agriculture for food security. Nature Climate Change, 4, 1068–1072. https://doi.org/10.1038/nclimate2437

Niculescu-Mizil, A., & Caruana, R. (2005). Predicting good probabilities with supervised learning. In Proceedings of the 22nd International Conference on Machine Learning (pp. 625–632). Association for Computing Machinery. https://doi.org/10.1145/1102351.1102430

Nie, J., & Zheng, D. (2023). Ambiguity-aware HDFS log anomaly detection with retrieval-augmented failure narratives and selective refusal. Journal of Advanced Computing Systems, 3(1), 66–80. https://doi.org/10.69987/JACS.2023.30105

Niu, C., Wu, Y., Zhu, J., Xu, S., Shum, K., Zhong, R., Song, J., & Zhang, T. (2024). RAGTruth: A hallucination corpus for developing trustworthy retrieval-augmented language models. In L.-W. Ku, A. Martins, & V. Srikumar (Eds.), Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 10862–10878). Association for Computational Linguistics. https://doi.org/10.18653/v1/2024.acl-long.585

Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019). Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift. Advances in Neural Information Processing Systems, 32, 13991–14002. https://proceedings.neurips.cc/paper/2019/hash/8558cb408c1d76621371888657d2eb1d-Abstract.html

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., & Duchesnay, E. (2011). Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12, 2825–2830. https://jmlr.org/papers/v12/pedregosa11a.html

Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V., & Gulin, A. (2018). CatBoost: Unbiased boosting with categorical features. Advances in Neural Information Processing Systems, 31, 6638–6648. https://proceedings.neurips.cc/paper/2018/hash/14491b756b3a51daac41c24863285549-Abstract.html

Roberts, D. R., Bahn, V., Ciuti, S., Boyce, M. S., Elith, J., Guillera-Arroita, G., Hauenstein, S., Lahoz-Monfort, J. J., Schröder, B., Thuiller, W., Warton, D. I., Wintle, B. A., Hartig, F., & Dormann, C. F. (2017). Cross-validation strategies for data with temporal, spatial, hierarchical, or phylogenetic structure. Ecography, 40(8), 913–929. https://doi.org/10.1111/ecog.02881

Romano, Y., Sesia, M., & Candès, E. J. (2020). Classification with valid and adaptive coverage. Advances in Neural Information Processing Systems, 33, 3581–3591. https://proceedings.neurips.cc/paper/2020/hash/244edd7e85dc81602b7615cd705545f5-Abstract.html

Saito, T., & Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3), Article e0118432. https://doi.org/10.1371/journal.pone.0118432

Sugiyama, M., Krauledat, M., & Müller, K.-R. (2007). Covariate shift adaptation by importance weighted cross validation. Journal of Machine Learning Research, 8, 985–1005. https://jmlr.org/papers/v8/sugiyama07a.html

Vovk, V., Gammerman, A., & Shafer, G. (2005). Algorithmic learning in a random world. Springer. https://doi.org/10.1007/b106715

Wolfert, S., Ge, L., Verdouw, C., & Bogaardt, M.-J. (2017). Big data in smart farming: A review. Agricultural Systems, 153, 69–80. https://doi.org/10.1016/j.agsy.2017.01.023

Zhang, H. (2023). DriftGuard: Multi-signal drift early warning and safe re-training/rollback for CTR/CVR models. Journal of Advanced Computing Systems, 3(7), 24–40. https://doi.org/10.69987/JACS.2023.30703

Zhong, Z. S., Ma, R., & Zhao, H. (2023). Human-uncertainty distillation for calibrated vision models on CIFAR-10H. Journal of Advanced Computing Systems, 3(2), 77–89. https://doi.org/10.69987/JACS.2023.30206

Zhou, S., Li, Z., & Wang, E. (2023). Evidence-grounded RAG for tokenized trade receivable disclosure QA under U.S. capital market standards. Journal of Advanced Computing Systems, 3(7), 41–57. https://doi.org/10.69987/JACS.2023.30704