Closed-Loop Deformation Control of Soft Manipulators for Fruit Harvesting Fusing Tactile and Depth Vision

Abstract

The automation of agricultural harvesting presents a distinct set of challenges, primarily driven by the unstructured nature of the environment and the biological variability of the produce. Soft robotics has emerged as a promising avenue for handling delicate crops such as tomatoes, apples, and strawberries due to the inherent compliance of the materials used. However, the non-linear deformation characteristics of soft actuators make precise control difficult, often leading to either insufficient grasping force or damage to the fruit. This paper introduces a novel closed-loop deformation control framework that fuses depth vision with distributed tactile sensing to enhance the grasping efficacy of pneumatically actuated soft manipulators. By utilizing a depth camera to generate a preliminary volumetric estimation of the target and integrating real-time feedback from capacitive tactile sensors embedded within the gripper fingers, the system dynamically adjusts the internal pneumatic pressure to optimize the contact profile. We employ a sensor fusion algorithm that mitigates the limitations of individual modalities, such as visual occlusion and tactile sparsity. Experimental validation demonstrates that this multi-modal approach significantly reduces surface bruising on harvested fruits while maintaining a high success rate in detachment. The results indicate that fusing tactile and visual data allows for robust compensation of mechanical hysteresis and external disturbances, marking a substantial advancement in autonomous harvesting technologies.