Handling of data_lists for signals with different number of columns

[jufr]

can_decoder/can_decoder/dataframe/DataFrameGenericDecoder.py

Line 113 in abfdd83

frame_data = np.array([a for a in data_lists], dtype=np.uint8)

When decoding, this line can sometimes lead to the following error: "ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. ...”

After looking into it, I believe the problem is, that for some signals not all lists (a) in in the data_list have the same number of columns. Having lines with different column numbers leads to the value error. The assignment "np.array(..)" cannot handle different numbers of columns.
e.g.: For one signal, the first "a" had 63 values, while the next "a" had only 19 values.

I solved it like this:
frame_data = pd.DataFrame([a for a in data_lists]).to_numpy(dtype=np.uint8)

This way, the the "missing columns" are replace with the default value of .to_numpy(dtype=np.uint8), which is "0".

Could you please verify this correction?

[MatinF]

Thanks for checking in - can you share a zip with a log file, DBC and script where this occurs?

Also, can you try utilizing the api-examples script below and see if the problem persists?
https://github.com/CSS-Electronics/api-examples/blob/master/examples/data-processing/process_data.py

If I understand your issue, it may be related to cases of differing data lengths across different frames. This needs to be handled by sorting the data by data length before decoding it, which is done in the higher layer function we've made in the utils.py file:
https://github.com/CSS-Electronics/api-examples/blob/master/examples/data-processing/process_data.py#L28