Skip to content

Commit d7fe2a4

Browse files
FaHaGitFaHaGit
authored
CCC: Add logging and debugging section (#36)
* ccc: Add logging and debugging section Signed-off-by: Fabian Hartung <[email protected]> * Add links to the new logging_and_debugging chapter Signed-off-by: Fabian Hartung <[email protected]> --------- Signed-off-by: Fabian Hartung <[email protected]>
1 parent 83fb92e commit d7fe2a4

4 files changed

+123
-1
lines changed

‎docs/source/development.rst

+2
Original file line numberDiff line numberDiff line change
@@ -109,3 +109,5 @@ how to checkout a dedicated EVerest release.
109109
110110
#. Follow the steps under the section :ref:`firmware_customization` to install your PKI certificate, pack
111111
the modified root filesystem image again into the firmware update image, and test the new firmware image.
112+
113+
.. include:: ../../includes/development_debugging_and_logging.inc

‎docs/source/getting_started.rst

+3-1
Original file line numberDiff line numberDiff line change
@@ -229,6 +229,7 @@ to restart the EVerest charging stack to apply the changes:
229229
start. Therefore, it is recommended to back up the original configuration file before making
230230
changes.
231231

232+
.. _start_charging_and_monitoring:
232233

233234
Starting and Monitoring the Charging Process
234235
--------------------------------------------
@@ -237,7 +238,8 @@ Before we start the first charging session, we shall open the EVerest log to mon
237238
process. The EVerest log is stored in the systemd journal and can be accessed via the journalctl
238239
command. The journalctl command provides a lot of options to filter the log messages.
239240
Now just type "journalctl -f -u everest -n 50" to see the last 50 log messages of the EVerest
240-
charging stack and to follow the charging process in real time.
241+
charging stack and to follow the charging process in real time. For more information about the
242+
EVerest log, see the :ref:`logging_and_debugging` chapter.
241243

242244
The EVerest log should look like this:
243245

‎includes/development_debugging_and_logging.inc

+113
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
.. _logging_and_debugging:
2+
3+
Logging and Debugging
4+
=====================
5+
6+
There are different ways to analyze the EVerest charging software. In the following, different
7+
ways are described. In preparation for starting this chapter, it is necessary to be connected to the board
8+
via Ethernet. This is also necessary to copy logs and traces from the board to the local PC via a SFTP
9+
tool like `WinSCP <https://winscp.net/eng/download.php>`_ or `FileZilla <https://filezilla-project.org/>`_.
10+
11+
journalctl logs
12+
---------------
13+
As already described in the :ref:`start_charging_and_monitoring` section, the EVerest framework uses
14+
journalctl for logging. journalctl is a system service that collects and stores logging data.
15+
To view the logs, connect to the charge controller via SSH and run the following command:
16+
17+
.. code-block:: console
18+
19+
journalctl -f -n 100 -u everest
20+
21+
The -f flag is used to follow the logs live, and the -n flag is used to view the last 100 lines.
22+
The -u flag is used to filter the logs for the everest service. To stop following the logs, press :code:`Ctrl + C`.
23+
24+
The logs are stored by the systemd journal service in a binary format in
25+
"/var/log/journal". To open the binary in a human-readable format, use the following command:
26+
27+
.. code-block:: console
28+
29+
journalctl -u everest --file /path/to/your/file.journal
30+
31+
As shown in the :ref:`start_charging_and_monitoring` section, a log line consists, among other things,
32+
of a timestamp, a log level, a module id, a module name, and a message. The log level can be one of the following:
33+
34+
- "VERB" (Verbose)
35+
- "DEBG" (Debug)
36+
- "INFO" (Information)
37+
- "WARN" (Warning)
38+
- "ERRO" (Error)
39+
- "CRIT" (Critical)
40+
41+
The log level can be configured for each EVerest module separately in the logging configuration file.
42+
The logging configuration file is located at "/etc/everest/default_logging.cfg". The filter can be configured
43+
using the log level and the module identifier as defined in the EVerest configuration.
44+
Here is an example to configure the message filter to change the log level of the EvseManager module
45+
to DEBUG:
46+
47+
.. code-block:: console
48+
49+
# The filter expression consists of two parts:
50+
# 1. Logs with severity level INFO or higher (affects all EVerest modules).
51+
# 2. Logs from the EvseManager with module id "connector" with severity level DEBUG or higher.
52+
Filter="(%Severity% >= INFO) or (%Process% contains connector and %Severity% >= DEBG)"
53+
54+
pcap traces
55+
-----------
56+
pcap traces can be used to debug and analyze the high-level communication (HLC) between SECC and EVCC.
57+
In general, there are two ways to capture pcap traces:
58+
59+
#. Capture via tcpdump command
60+
61+
.. code-block:: console
62+
63+
tcpdump -i eth1 -w /srv/charge_com_01.pcap
64+
65+
This command captures all packets on the eth1 (PLC) interface and writes them to
66+
a pcap file. Stop the capture by pressing :code:`Ctrl + C`.
67+
68+
.. note::
69+
Please stop the trace before copying the file to a local machine. The file may be incomplete if
70+
copied while the tcpdump is still running.
71+
72+
#. Using the EVerest "PacketSniffer" module
73+
74+
The `PacketSniffer <https://github.com/EVerest/everest-core/tree/main/modules/PacketSniffer>`_ module
75+
is part of everest-core and can be used to capture pcap traces automatically. Please look into
76+
the module manifest to see how to configure the PacketSniffer module.
77+
78+
pcap traces with V2G communication can be analyzed using Wireshark with a V2G plugin like from `dSPACE <https://www.dspace.com/en/pub/home/news/wireshark-charging-plug-in.cfm>`_.
79+
80+
.. note::
81+
TLS encrypted communication cannot be analyzed with Wireshark without the TLS session key. Please
82+
look in the module manifest which is responsible for the TLS handshake and check if there is an option
83+
to export the TLS session key. E.g. the `IsoMux module <https://github.com/EVerest/everest-core/blob/main/modules/IsoMux/manifest.yaml>`_
84+
is able to export the TLS session key.
85+
86+
Session logging of the EVSEManager
87+
----------------------------------
88+
The EVerest "EVSEManager" can generate session log files with information about the current session state,
89+
configured CP state and duty cycle. It can also decode and log the content of the V2G messages.
90+
Please look into the module manifest to see how to configure the EVSEManager module to activate session logging.
91+
92+
.. note::
93+
The session logging is pretty performance intensive and should only be used for debugging purposes.
94+
95+
MQTT logs
96+
---------
97+
It is also possible to observe the internal MQTT communication between the EVerest modules.
98+
When developing new modules, this can be helpful to better understand the internal processes regarding
99+
the use of the interfaces.
100+
We recommend using the `MQTT Explorer <https://mqtt-explorer.com/>`_ tool to monitor and analyze the MQTT communication.
101+
It is also possible to use the following command to monitor the MQTT communication, but this is not so easy to read:
102+
103+
.. code-block:: console
104+
105+
mosquitto_sub -F "[@H:@M:@S.@N] %t %p" -t 'everest/#' -t 'everest_api/#' > /srv/charge_com_mqtt_01.log
106+
107+
Summary
108+
-------
109+
There are several ways to debug and analyze the EVerest framework. Which way to choose depends on the
110+
specific use case and the information needed. The most common way is using journalctl logs. If it is
111+
necessary to analyze the high-level communication between SECC and EVCC, pcap traces are the best choice.
112+
If it is not clear how the internal communication between the EVerest modules works, the MQTT logs can
113+
be helpful.

‎includes/troubleshooting_contact.inc

+5
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,11 @@ Do you have general questions about EVerest, please use the EVerest community's
1414
Do you have questions about the chargebyte BSP (incl. Yocto), please use
1515
`our support desk <https://chargebyte.atlassian.net/servicedesk/customer/portal/13>`_.
1616

17+
.. note::
18+
Before submitting a support request, make sure that all relevant log files have been captured.
19+
The chapter :ref:`logging_and_debugging` describes the debugging options EVerest offers and shows
20+
which logs are relevant for the respective case.
21+
1722
Address
1823
^^^^^^^
1924

0 commit comments

Comments
 (0)