Re-resolve hostnames as fallback when all hosts are unreachable

[wprzytula]

If all nodes in the cluster change their IPs at one time, driver can no longer ever contact the cluster; the only solution is to restart the driver. A fallback should be added to the control connection reconnect logic so that when no known host is reachable, all hostnames among initial contact points are reresolved and control connection is attempted to be opened to any of them. If this succeeds, a metadata fetch is issued normally and the whole cluster is discovered with its new IPs.

An analogous feature has been merged to gocql lastly: apache/cassandra-gocql-driver#1708

[roydahan]

In scylla-4.x we should have the parameter called advanced.resolve-contact-points which should take care of that.
So, if it works this enhancement should be done only for scylla-3.x

[mykaul]

This needs to get some attention. It was solved in most drivers already, not in Java 3.x.

[dkropachev]

It is blocked by #333 and #334

[mykaul]

It is blocked by #333 and #334

#333 is now fixed, what about #334 ?

[dkropachev]

It is blocked by #333 and #334

#333 is now fixed, what about #334 ?

Both done, this issue is ublocked.

[avikivity]

Does the resolver respect the DNS entry TTL?

That's better than yet another special case.