Beyond capability: a limiting-axis framework for assessing the field readiness of agricultural intelligent agents
DOI:
https://doi.org/10.63646/MQEP8827Keywords:
Field Readiness; Dependability Evidence; Deployment Assessment; Autonomous Agricultural Machinery; Evaluation Protocol; Limiting Factor; Agricultural RoboticsAbstract
Intelligent agents for agriculture have advanced rapidly in perception, decision-making and execution, yet almost none operate unsupervised in commercial production. The usual explanation is that the technology remains immature. This paper advances a different diagnosis: what is immature is the evidence apparatus used to decide when an agent may leave the experimental plot. Agricultural agent research has inherited the benchmark culture of computer vision and reinforcement learning, in which a system is judged by aggregate accuracy on nominal cases, without inheriting the dependability culture of agricultural machinery safety, in which a system is judged by what it does when something fails. We develop a six-axis field readiness assessment that scores the evidence supporting a deployment claim rather than the capability claimed: ecological validity, temporal adequacy, distributional stress, degradation behavior, accountability traceability and operational burden fit. Axis scores are combined by limiting-axis aggregation, a weighted geometric mean motivated by the agronomic law of the minimum, so that strength on one axis cannot offset absence of evidence on another, and the resulting Field Readiness Index is mapped onto a five-band deployment ladder with an explicit safety veto. Applying the instrument to eight system archetypes shows that degradation behavior is the lowest-scoring axis and the binding constraint in five of eight cases, that compensatory averaging overstates readiness by up to 0.30 scale points, and that the overstatement is strongly associated with profile imbalance. We provide a per-axis minimum reporting checklist intended for authors, reviewers and editors, and argue that readiness in agriculture is limited by what is not reported rather than by what is not yet possible.
References
Wooldridge, M., Jennings, N.R., Intelligent agents: theory and practice, Knowledge Engineering Review 10 (1995) 115–152. https://doi.org/10.1017/S0269888900008122
[2] Dorri, A., Kanhere, S.S., Jurdak, R., Multi-agent systems: a survey, IEEE Access 6 (2018) 28573–28593. https://doi.org/10.1109/ACCESS.2018.2831228
[3] Jennings, N.R., An agent-based approach for building complex software systems, Communications of the ACM 44 (2001) 35–41. https://doi.org/10.1145/367211.367250
[4] Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A., Deep reinforcement learning: a brief survey, IEEE Signal Processing Magazine 34 (2017) 26–38. https://doi.org/10.1109/MSP.2017.2743240
[5] Vaswani, A., et al., Attention is all you need, in: Advances in Neural Information Processing Systems 30, 2017, pp. 5998–6008. https://doi.org/10.48550/arXiv.1706.03762
[6] Lewis, P., et al., Retrieval-augmented generation for knowledge-intensive NLP tasks, in: Advances in Neural Information Processing Systems 33, 2020, pp. 9459–9474. https://doi.org/10.48550/arXiv.2005.11401
[7] Park, J.S., O'Brien, J.C., Cai, C.J., Morris, M.R., Liang, P., Bernstein, M.S., Generative agents: interactive simulacra of human behavior, in: Proceedings of the 36th ACM Symposium on User Interface Software and Technology, San Francisco, USA, 2023. https://doi.org/10.1145/3586183.3606763
[8] Wolfert, S., Ge, L., Verdouw, C., Bogaardt, M.-J., Big Data in smart farming – a review, Agricultural Systems 153 (2017) 69–80. https://doi.org/10.1016/j.agsy.2017.01.023
[9] Kamilaris, A., Prenafeta-Boldú, F.X., Deep learning in agriculture: a survey, Computers and Electronics in Agriculture 147 (2018) 70–90. https://doi.org/10.1016/j.compag.2018.02.016
[10] Fountas, S., Mylonas, N., Malounas, I., Rodias, E., Hellmann Santos, C., Pekkeriet, E., Agricultural robotics for field operations, Sensors 20 (2020) 2672. https://doi.org/10.3390/s20092672
[11] Bechar, A., Vigneault, C., Agricultural robots for field operations: concepts and components, Biosystems Engineering 149 (2016) 94–111. https://doi.org/10.1016/j.biosystemseng.2016.06.014
[12] Duckett, T., et al., Agricultural Robotics: The Future of Robotic Agriculture, UK-RAS White Paper, UK-RAS Network, 2018. https://doi.org/10.31256/wp2018.2
[13] Shamshiri, R.R., et al., Research and development in agricultural robotics: a perspective of digital farming, International Journal of Agricultural and Biological Engineering 11 (2018) 1–14. https://doi.org/10.25165/j.ijabe.20181104.4278
[14] Klerkx, L., Jakku, E., Labarthe, P., A review of social science on digital agriculture, smart farming and agriculture 4.0, NJAS – Wageningen Journal of Life Sciences 90–91 (2019) 100315. https://doi.org/10.1016/j.njas.2019.100315
[15] International Electrotechnical Commission, Functional safety of electrical/electronic/programmable electronic safety-related systems, IEC 61508, IEC, Geneva.
[16] International Organization for Standardization, Tractors and machinery for agriculture and forestry — Safety-related parts of control systems, ISO 25119, ISO, Geneva.
[17] International Organization for Standardization, Agricultural machinery and tractors — Safety of highly automated agricultural machines, ISO 18497, ISO, Geneva.
[18] von Liebig, J., Die organische Chemie in ihrer Anwendung auf Agricultur und Physiologie, Vieweg, Braunschweig, 1840. https://doi.org/10.5962/bhl.title.42117
[19] Mankins, J.C., Technology Readiness Levels: A White Paper, Advanced Concepts Office, NASA, Washington, DC, 1995.
[20] Breck, E., Cai, S., Nielsen, E., Salib, M., Sculley, D., The ML test score: a rubric for ML production readiness and technical debt reduction, in: Proceedings of the IEEE International Conference on Big Data, Boston, USA, 2017, pp. 1123–1132. https://doi.org/10.1109/BigData.2017.8258038
[21] Paleyes, A., Urma, R.-G., Lawrence, N.D., Challenges in deploying machine learning: a survey of case studies, ACM Computing Surveys 55 (2022) 114. https://doi.org/10.1145/3533378
[22] International Organization for Standardization, Tractors and machinery for agriculture and forestry — Serial control and communications data network (ISOBUS), ISO 11783, ISO, Geneva.
[23] Leveson, N.G., A new accident model for engineering safer systems, Safety Science 42 (2004) 237–270. https://doi.org/10.1016/S0925-7535(03)00047-X
[24] Rasmussen, J., Risk management in a dynamic society: a modelling problem, Safety Science 27 (1997) 183–213. https://doi.org/10.1016/S0925-7535(97)00052-0
[25] Varshney, K.R., Alemzadeh, H., On the safety of machine learning: cyber-physical systems, decision sciences, and data products, Big Data 5 (2017) 246–255. https://doi.org/10.1089/big.2016.0051
[26] Dietterich, T.G., Steps toward robust artificial intelligence, AI Magazine 38 (2017) 3–24. https://doi.org/10.1609/aimag.v38i3.2756
[27] Mitchell, M., et al., Model cards for model reporting, in: Proceedings of the Conference on Fairness, Accountability, and Transparency, Atlanta, USA, 2019, pp. 220–229. https://doi.org/10.1145/3287560.3287596
[28] Gebru, T., et al., Datasheets for datasets, Communications of the ACM 64 (2021) 86–92. https://doi.org/10.1145/3458723
[29] Moreno-Torres, J.G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N.V., Herrera, F., A unifying view on dataset shift in classification, Pattern Recognition 45 (2012) 521–530. https://doi.org/10.1016/j.patcog.2011.06.019
[30] Hendrycks, D., Gimpel, K., A baseline for detecting misclassified and out-of-distribution examples in neural networks, in: Proceedings of the International Conference on Learning Representations, Toulon, France, 2017. https://doi.org/10.48550/arXiv.1610.02136
[31] Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q., On calibration of modern neural networks, in: Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia, 2017, pp. 1321–1330. https://doi.org/10.48550/arXiv.1706.04599
[32] Recht, B., Roelofs, R., Schmidt, L., Shankar, V., Do ImageNet classifiers generalize to ImageNet?, in: Proceedings of the 36th International Conference on Machine Learning, Long Beach, USA, 2019, pp. 5389–5400. https://doi.org/10.48550/arXiv.1902.10811
[33] Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D., Deep reinforcement learning that matters, in: Proceedings of the AAAI Conference on Artificial Intelligence, New Orleans, USA, 2018, pp. 3207–3214. https://doi.org/10.1609/aaai.v32i1.11694
[34] Tobin, J., Fong, R., Ray, A., Schneider, J., Zaremba, W., Abbeel, P., Domain randomization for transferring deep neural networks from simulation to the real world, in: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, Vancouver, Canada, 2017, pp. 23–30. https://doi.org/10.1109/IROS.2017.8202133
[35] Sünderhauf, N., et al., The limits and potentials of deep learning for robotics, International Journal of Robotics Research 37 (2018) 405–420. https://doi.org/10.1177/0278364918770733
[36] Kirkpatrick, J., et al., Overcoming catastrophic forgetting in neural networks, Proceedings of the National Academy of Sciences 114 (2017) 3521–3526. https://doi.org/10.1073/pnas.1611835114
[37] European Parliament and Council of the European Union, Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence, Official Journal of the European Union, 2024.
[38] Rudin, C., Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nature Machine Intelligence 1 (2019) 206–215. https://doi.org/10.1038/s42256-019-0048-x
[39] Lipton, Z.C., The mythos of model interpretability, Communications of the ACM 61 (2018) 36–43. https://doi.org/10.1145/3233231
[40] Shneiderman, B., Human-centered artificial intelligence: reliable, safe and trustworthy, International Journal of Human–Computer Interaction 36 (2020) 495–504. https://doi.org/10.1080/10447318.2020.1741118
[41] Page, M.J., et al., The PRISMA 2020 statement: an updated guideline for reporting systematic reviews, BMJ 372 (2021) n71. https://doi.org/10.1136/bmj.n71
