MINOR: fix ssh problem in e2e

[unknowntpo]

This PR set soft links that points to actual Java tools location in
/usr/bin if it doesn't exist.

The Problem:

If we use eclipse-temurin:17-jdk-jammy to run e2e tests:

jdk_version="eclipse-temurin:17-jdk-jammy" \
image_name="ducker-ak:$(git rev-parse --short HEAD)" \
TC_PATHS="tests/kafkatest/tests/client/pluggable_test.py" \
/bin/bash tests/docker/run_tests.sh

we will get:

test_id:
kafkatest.tests.client.pluggable_test.PluggableConsumerTest.test_start_stop.metadata_quorum=ISOLATED_KRAFT
status:     FAIL
run time:   3.241 seconds

    RemoteCommandError({'ssh_config': {'host': 'ducker02', 'hostname':
'ducker02', 'user': 'ducker', 'port': 22, 'password': '',
'identityfile': '/home/ducker/.ssh/id_rsa', 'connecttimeout': None},
'hostname': 'ducker02', 'ssh_hostname': 'ducker02', 'user': 'ducker',
'externally_routable_ip': 'ducker02', '_logger': <Logger
kafkatest.tests.client.pluggable_test.PluggableConsumerTest.test_start_stop.metadata_quorum=ISOLATED_KRAFT-1
(DEBUG)>, 'os': 'linux', '_ssh_client': <paramiko.client.SSHClient
object at 0xffffb95fdae0>, '_sftp_client':
<paramiko.sftp_client.SFTPClient object at 0xffffb95feef0>,
'_custom_ssh_exception_checks': None}, 'java -version', 127, b'')
Traceback (most recent call last):
  File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py",
line 350, in _do_run
    self.setup_test()
  File
"/usr/local/lib/python3.10/dist-packages/ducktape/tests/runner_client.py",
line 402, in setup_test
    self.test.setup()
  File "/usr/local/lib/python3.10/dist-packages/ducktape/tests/test.py",
line 74, in setup
    self.setUp()
  File "/opt/kafka-dev/tests/kafkatest/tests/kafka_test.py", line 47, in
setUp
    self.kafka.start()
  File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line
651, in start
self.isolated_controller_quorum.start(nodes_to_skip=isolated_controllers_to_skip)
  File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line
653, in start
    Service.start(self, **kwargs)
  File
"/usr/local/lib/python3.10/dist-packages/ducktape/services/service.py",
line 265, in start
    self.start_node(node, **kwargs)
  File "/opt/kafka-dev/tests/kafkatest/services/kafka/kafka.py", line
854, in start_node
    self.security_config.setup_node(node)
  File
"/opt/kafka-dev/tests/kafkatest/services/security/security_config.py",
line 335, in setup_node
    if java_version(node) <= 11 and self.properties.get('tls.version')
== 'TLSv1.3':
  File "/opt/kafka-dev/tests/kafkatest/utils/remote_account.py", line
44, in java_version
    for line in node.account.ssh_capture("java -version"):
  File
"/usr/local/lib/python3.10/dist-packages/ducktape/cluster/remoteaccount.py",
line 697, in next
    return next(self.iter_obj)
  File
"/usr/local/lib/python3.10/dist-packages/ducktape/cluster/remoteaccount.py",
line 364, in output_generator
    raise RemoteCommandError(self, cmd, exit_status, stderr.read())
ducktape.cluster.remoteaccount.RemoteCommandError: ducker@ducker02:
Command 'java -version' returned non-zero exit status 127.

The reason why /home/ducker/.ssh/environment doesn't work :

When PATH is set in /home/ducker/.ssh/environment, it still gets
overwritten by /etc/environment during SSH command execution.

Explanation:

Steps to reproduce:

Start containers and run tests:

jdk_version="eclipse-temurin:17-jdk-jammy" \
image_name="ducker-ak:$(git rev-parse --short HEAD)" \
TC_PATHS="tests/kafkatest/tests/client/pluggable_test.py" \
/bin/bash tests/docker/run_tests.sh
# will get error described above.

Prepare SSH key:

mkdir -p ./.ssh
docker cp ducker01:/home/ducker/.ssh/id_rsa .ssh/ducker_id_rsa
docker cp ducker01:/home/ducker/.ssh/id_rsa.pub .ssh/ducker_id_rsa.pub

Test SSH into the container:

# Get container IP or use hostname
IP=$(docker exec ducker01 hostname -I)
ssh -i .ssh/ducker_id_rsa ducker@$IP 'echo $PATH'
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

See if we can find java command:

ssh -i .ssh/ducker_id_rsa ducker@$IP 'java -version'
bash: line 1: java: command not found

See which value of $PATH SSH non-interactive shell command can see:

ssh -i .ssh/ducker_id_rsa ducker@$IP 'echo $PATH'
PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
# (same as echo $PATH in ssh non-interactive shell command)

Value in /etc/environment:

PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
(Notice: no Java path)

Value in /home/ducker/.ssh/environment`:

PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/java/openjdk/bin

[chia7712]

It seems the user enviroment file has not been applied. The new image sets the link to /usr/bin/java, which is why it works even though the environment file is not applied

[unknowntpo]

Yeah, thats correct, and I manually set soft link for Java tools to make it work, see description for more information.