Shipping currently produces greenhouse gas emissions on par with aviation, and significant efforts are underway to reduce its environmental impact. One promising approach is Wind-Assisted Propulsion, which decreases the thrust required from propellers and, consequently, the load on internal combustion engines. However, the optimal control of these propulsion systems remains an open challenge. This project builds on our preliminary work using reinforcement learning to develop control strategies and aims to develop an advanced controller based on reinforcement learning that will optimise operation, aligned with financial and operational incentives.