Skip to content

Commit 2d90564

Browse files
FaHaGitFaHaGit
committed
CCC: Add logging and debugging section (#36)
* ccc: Add logging and debugging section Signed-off-by: Fabian Hartung <[email protected]> * Add links to the new logging_and_debugging chapter Signed-off-by: Fabian Hartung <[email protected]> --------- Signed-off-by: Fabian Hartung <[email protected]>
1 parent ebc255a commit 2d90564

4 files changed

+581
-0
lines changed

‎development_debugging_and_logging.inc

+113
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
.. _logging_and_debugging:
2+
3+
Logging and Debugging
4+
=====================
5+
6+
There are different ways to analyze the EVerest charging software. In the following, different
7+
ways are described. In preparation for starting this chapter, it is necessary to be connected to the board
8+
via Ethernet. This is also necessary to copy logs and traces from the board to the local PC via a SFTP
9+
tool like `WinSCP <https://winscp.net/eng/download.php>`_ or `FileZilla <https://filezilla-project.org/>`_.
10+
11+
journalctl logs
12+
---------------
13+
As already described in the :ref:`start_charging_and_monitoring` section, the EVerest framework uses
14+
journalctl for logging. journalctl is a system service that collects and stores logging data.
15+
To view the logs, connect to the charge controller via SSH and run the following command:
16+
17+
.. code-block:: console
18+
19+
journalctl -f -n 100 -u everest
20+
21+
The -f flag is used to follow the logs live, and the -n flag is used to view the last 100 lines.
22+
The -u flag is used to filter the logs for the everest service. To stop following the logs, press :code:`Ctrl + C`.
23+
24+
The logs are stored by the systemd journal service in a binary format in
25+
"/var/log/journal". To open the binary in a human-readable format, use the following command:
26+
27+
.. code-block:: console
28+
29+
journalctl -u everest --file /path/to/your/file.journal
30+
31+
As shown in the :ref:`start_charging_and_monitoring` section, a log line consists, among other things,
32+
of a timestamp, a log level, a module id, a module name, and a message. The log level can be one of the following:
33+
34+
- "VERB" (Verbose)
35+
- "DEBG" (Debug)
36+
- "INFO" (Information)
37+
- "WARN" (Warning)
38+
- "ERRO" (Error)
39+
- "CRIT" (Critical)
40+
41+
The log level can be configured for each EVerest module separately in the logging configuration file.
42+
The logging configuration file is located at "/etc/everest/default_logging.cfg". The filter can be configured
43+
using the log level and the module identifier as defined in the EVerest configuration.
44+
Here is an example to configure the message filter to change the log level of the EvseManager module
45+
to DEBUG:
46+
47+
.. code-block:: console
48+
49+
# The filter expression consists of two parts:
50+
# 1. Logs with severity level INFO or higher (affects all EVerest modules).
51+
# 2. Logs from the EvseManager with module id "connector" with severity level DEBUG or higher.
52+
Filter="(%Severity% >= INFO) or (%Process% contains connector and %Severity% >= DEBG)"
53+
54+
pcap traces
55+
-----------
56+
pcap traces can be used to debug and analyze the high-level communication (HLC) between SECC and EVCC.
57+
In general, there are two ways to capture pcap traces:
58+
59+
#. Capture via tcpdump command
60+
61+
.. code-block:: console
62+
63+
tcpdump -i eth1 -w /srv/charge_com_01.pcap
64+
65+
This command captures all packets on the eth1 (PLC) interface and writes them to
66+
a pcap file. Stop the capture by pressing :code:`Ctrl + C`.
67+
68+
.. note::
69+
Please stop the trace before copying the file to a local machine. The file may be incomplete if
70+
copied while the tcpdump is still running.
71+
72+
#. Using the EVerest "PacketSniffer" module
73+
74+
The `PacketSniffer <https://github.com/EVerest/everest-core/tree/main/modules/PacketSniffer>`_ module
75+
is part of everest-core and can be used to capture pcap traces automatically. Please look into
76+
the module manifest to see how to configure the PacketSniffer module.
77+
78+
pcap traces with V2G communication can be analyzed using Wireshark with a V2G plugin like from `dSPACE <https://www.dspace.com/en/pub/home/news/wireshark-charging-plug-in.cfm>`_.
79+
80+
.. note::
81+
TLS encrypted communication cannot be analyzed with Wireshark without the TLS session key. Please
82+
look in the module manifest which is responsible for the TLS handshake and check if there is an option
83+
to export the TLS session key. E.g. the `IsoMux module <https://github.com/EVerest/everest-core/blob/main/modules/IsoMux/manifest.yaml>`_
84+
is able to export the TLS session key.
85+
86+
Session logging of the EVSEManager
87+
----------------------------------
88+
The EVerest "EVSEManager" can generate session log files with information about the current session state,
89+
configured CP state and duty cycle. It can also decode and log the content of the V2G messages.
90+
Please look into the module manifest to see how to configure the EVSEManager module to activate session logging.
91+
92+
.. note::
93+
The session logging is pretty performance intensive and should only be used for debugging purposes.
94+
95+
MQTT logs
96+
---------
97+
It is also possible to observe the internal MQTT communication between the EVerest modules.
98+
When developing new modules, this can be helpful to better understand the internal processes regarding
99+
the use of the interfaces.
100+
We recommend using the `MQTT Explorer <https://mqtt-explorer.com/>`_ tool to monitor and analyze the MQTT communication.
101+
It is also possible to use the following command to monitor the MQTT communication, but this is not so easy to read:
102+
103+
.. code-block:: console
104+
105+
mosquitto_sub -F "[@H:@M:@S.@N] %t %p" -t 'everest/#' -t 'everest_api/#' > /srv/charge_com_mqtt_01.log
106+
107+
Summary
108+
-------
109+
There are several ways to debug and analyze the EVerest framework. Which way to choose depends on the
110+
specific use case and the information needed. The most common way is using journalctl logs. If it is
111+
necessary to analyze the high-level communication between SECC and EVCC, pcap traces are the best choice.
112+
If it is not clear how the internal communication between the EVerest modules works, the MQTT logs can
113+
be helpful.

‎docs/source/development.rst

+113
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
.. _development.rst:
2+
3+
.. include:: ../../includes/development.inc
4+
5+
.. _cross_compiling:
6+
7+
Cross-compiling
8+
===============
9+
10+
Another way to integrate custom applications into the firmware image is to cross-compile the application
11+
for Tarragon and include it in the image. A pre-requisite for this is to have the latest firmware image
12+
as a developer build. Always keep in mind, if you want to build a new EVerest module it must be
13+
compatible to the EVerest release within the firmware. Please have a look at the official
14+
`EVerest documentation <https://everest.github.io/nightly/dev_tools/edm.html#setting-up-and-updating-a-workspace>`_,
15+
how to checkout a dedicated EVerest release.
16+
17+
#. On an Ubuntu or Debian-based Linux distribution, install the cross-compilers for Tarragon.
18+
19+
.. code-block:: console
20+
21+
sudo apt install build-essential libc6-armhf-cross libc6-dev-armhf-cross binutils-arm-linux-gnueabihf gcc-arm-linux-gnueabihf g++-arm-linux-gnueabihf
22+
23+
#. Download chargebyte's `digital certificate <https://chargebyte.com/controllers-and-modules/evse-controllers/charge-control-c#downloads>`_
24+
and use it to extract the root filesystem from the firmware image.
25+
26+
.. code-block:: console
27+
28+
rauc extract --keyring=<chargebyte_certificate>.crt <shipped_firmware>.image bundle-staging
29+
30+
.. note::
31+
Alternatively, if the above command does not work, you can use the following command:
32+
.. code-block:: console
33+
34+
unsquashfs -d bundle-staging <shipped_firmware>.image
35+
36+
But this will not verify the signature of the firmware image.
37+
38+
#. Mount the ext4 root filesystem image as a loop device.
39+
40+
.. code-block:: console
41+
42+
sudo mkdir -p /mnt/rootfs
43+
sudo mount bundle-staging/core-image-minimal-tarragon.ext4 /mnt/rootfs
44+
45+
#. Create a new directory in the folder where the new module was created (my-module) and create a new
46+
file called :code:`toolchain.cmake`. This file is used to set the toolchain for the cross-compilation.
47+
48+
.. code-block:: console
49+
50+
cd my-module
51+
mkdir toolchain
52+
cd toolchain
53+
touch toolchain.cmake
54+
55+
56+
#. Store the following lines in the :code:`toolchain.cmake` file:
57+
58+
.. literalinclude:: ../../includes/_static/files/toolchain.cmake
59+
60+
#. Create a new :code:`build` directory in "my-module" and navigate to it.
61+
62+
.. code-block:: console
63+
64+
mkdir build
65+
cd build
66+
67+
#. Run the following command inside to configure the build.
68+
69+
.. code-block:: console
70+
71+
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain/toolchain.cmake -DCMAKE_SYSROOT=/mnt/rootfs ..
72+
73+
#. When this ends successfully, start cross-compiling using :code:`make`:
74+
75+
.. code-block:: console
76+
77+
make install -j$(nproc)
78+
79+
#. Test that the resulting binaries are compiled for Tarragon as a target:
80+
81+
.. code-block:: console
82+
83+
file dist/libexec/everest/modules/MyModule/MyModule
84+
85+
The output should be something like:
86+
87+
.. code-block:: console
88+
89+
dist/libexec/everest/modules/MyModule/MyModule: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (GNU/Linux),
90+
dynamically linked, interpreter /lib/ld-linux-armhf.so.3, BuildID[sha1]=9f287c2dbdcacd9ecde770df4820de9218deb439, for GNU/Linux 3.2.0, not stripped
91+
92+
#. The resulting binary and manifest file can be copied to the previously mounted root filesystem.
93+
94+
.. code-block:: console
95+
96+
cp dist/libexec/everest/modules/MyModule /mnt/rootfs/usr/libexec/everest/modules/
97+
98+
#. umount the loop device.
99+
100+
.. code-block:: console
101+
102+
sudo umount /mnt/rootfs
103+
104+
#. Make sure that the customized filesystem is in a clean state.
105+
106+
.. code-block:: console
107+
108+
fsck.ext4 -f bundle-staging/core-image-minimal-tarragon.ext4
109+
110+
#. Follow the steps under the section :ref:`firmware_customization` to install your PKI certificate, pack
111+
the modified root filesystem image again into the firmware update image, and test the new firmware image.
112+
113+
.. include:: ../../includes/development_debugging_and_logging.inc

0 commit comments

Comments
 (0)