Issue
The issue was indeed the wrong checksums on the outbound packets when the host OS was installed on the VMware ESX infrastructure. The VMware network interface was dropping the packets due to wrong checksums.
This issue was reported if the host OS is Red Hat/Centos or Debian/Ubuntu distribution.
Solution for Red Hat/Centos 7x
The solution was to disable checksum offloading. Using ethtool;
ethtool -K <interface> tx off
If you want this setting to be permanent/persistent, add below line to interface config file;
File located under /etc/sysconfig/network-scripts/ifcfg-xxxx
ETHTOOL_OPTS='-K ${DEVICE} tx off'
Solution for Red Hat/Centos 8.x/9x
Find interface which is used for docker swarm then edit configuration.
/etc/NetworkManager/system-connections/eth0.nmconnection
Add below lines
[ethtool] feature-tx=false
Solution for Debian/Ubuntu
The solution was to disable checksum offloading. Using ethtool;
ethtool -K <interface> tx off
If you want this setting to be permanent/persistent, add below line to interface config file. File located under /etc/netplan/00-eth0-conf.yaml
Original Config File
network:
version: 2
ethernets:
ens160:
addresses: [ip-address]
routes:
- to: default
via: gw
nameservers:
addresses: [dns1, dns2]
search: [domain.com]
You must add below lines to interface configuration;
transmit-checksum-offload: false tcp-segmentation-offload: false tcp6-segmentation-offload: false
After the editing of the file, it looks like below example;
network:
version: 2
ethernets:
ens160:
transmit-checksum-offload: false
tcp-segmentation-offload: false
tcp6-segmentation-offload: false
addresses: [ip-address]
routes:
- to: default
via: gw
nameservers:
addresses: [dns1, dns2]
search: [domain.com]