As far as I can tell, VSCode's Remote - WSL service connects over port `41261`. So, we can allow it through the Firewall like this:

sudo ufw allow 41261 comment 'VSCODE'