Advances in Production Engineering & Management
Volume 20 | Number 1 | March 2025 | pp 5–17
https://doi.org/10.14743/apem2025.1.523
Reinforcement learning for robot manipulation tasks in human-robot collaboration using the CQL/SAC algorithms
Husaković, A.; Banjanović-Mehmedović, L.; Gurdić-Ribić, A.; Prljača, N.; Karabegović, I.
ABSTRACT AND REFERENCES (PDF) |
FULL ARTICLE TEXT (PDF)
A B S T R A C T
The integration of human-robot collaboration (HRC) into industrial and service environments demands efficient and adaptive robotic systems capable of executing diverse tasks, including pick-and-place operations. This paper investigates the application of Soft Actor-Critic (SAC) and Conservative Q-Learning (CQL)—two deep reinforcement learning (DRL) algorithms—for the learning and optimization of pick-and-place actions within HRC scenarios. By leveraging SAC’s capability to balance exploration and exploitation, the robot autonomously learns to perform pick-and-place tasks while adapting to dynamic environments and human interactions. Moreover, the integration of CQL ensures more stable learning by mitigating Q-value overestimation, which proves particularly advantageous in offline and suboptimal data scenarios. The combined use of CQL and SAC enhances policy robustness, facilitating safer and more efficient decision-making in continually evolving environments. The proposed framework combines simulation-based training with transfer learning techniques, enabling seamless deployment in real-world environments. The critical challenge of trajectory completion is addressed through a meticulously designed reward function that promotes efficiency, precision, and safety. Experimental validation demonstrates a 100 % success rate in simulation and an 80 % success rate on real hardware, confirming the practical viability of the proposed model. This work underscores the pivotal role of DRL in enhancing the functionality of collaborative robotic systems, illustrating its applicability across a range of industrial environments.
A R T I C L E I N F O
Keywords • Human-robot collaboration; Robot learning; Deep reinforcement learning; Soft actor-critic algorithm (SAC); Conservative Q-learning (CQL); Robot manipulation tasks
Corresponding author • Banjanović-Mehmedović, L.
Article history • Received 10 February 2025, Revised 2 March 2025, Accepted 7 March 2025
Published on-line • 29 April 2025
E X P O R T C I T A T I O N
» RIS format (EndNote, ProCite, RefWorks, and most other reference management software)
» BibTeX (JabRef, BibDesk, and other BibTeX-specific software)
» Plain text
