From 84d5e8a20b873d63b832c8e72c1a8665d4f08f78 Mon Sep 17 00:00:00 2001 From: Marcus Hughes Date: Tue, 20 Feb 2024 09:36:11 -0700 Subject: [PATCH 1/3] adds documentation on CSV loading --- docs/user-guide/fixedlength.rst | 10 +--- docs/user-guide/index.rst | 1 + docs/user-guide/loadfile.rst | 88 +++++++++++++++++++++++++++++++++ 3 files changed, 90 insertions(+), 9 deletions(-) create mode 100644 docs/user-guide/loadfile.rst diff --git a/docs/user-guide/fixedlength.rst b/docs/user-guide/fixedlength.rst index 32f5c36..40ba12a 100644 --- a/docs/user-guide/fixedlength.rst +++ b/docs/user-guide/fixedlength.rst @@ -33,16 +33,8 @@ The following code defines a simple fixed length packet ]) Note that the CCSDS header need not be included as it is included by default. -A packet need not be defined in code. -It can also be defined in a text file. -For example, -With this file, it is then possible to define the packet object with - -.. code-block:: python - - import ccsdspy - pkt = ccsdspy.FixedLength.from_file('packet_definition.csv') +Alternatively, fixed length packets can be :ref:`loaded from a CSV file `. Parsing a file ============== diff --git a/docs/user-guide/index.rst b/docs/user-guide/index.rst index 65a19c4..e7c5bbe 100644 --- a/docs/user-guide/index.rst +++ b/docs/user-guide/index.rst @@ -14,5 +14,6 @@ For more details checkout the :ref:`reference`. packetfields fixedlength variablelength + loadfile converters utils diff --git a/docs/user-guide/loadfile.rst b/docs/user-guide/loadfile.rst new file mode 100644 index 0000000..8e56de4 --- /dev/null +++ b/docs/user-guide/loadfile.rst @@ -0,0 +1,88 @@ +.. _loadfile: + +************************************** +Loading Packet Definitions from a File +************************************** + +Overview +========= + +:ref:`fixed` can be loaded from a CSV file: + +.. code-block:: python + + import ccsdspy + pkt = ccsdspy.FixedLength.from_file('packet_definition.csv') + +The only requirement is that the CSV is structured as either the :ref:`three column ` +or :ref:`four column ` format. + +At the moment, :ref:`variable` cannot be loaded from a CSV file. + + .. contents:: + :depth: 2 + +.. _threecolumn: + +Three column CSV +=================== + +The three column CSV format has columns for `name`, `data_type`, and `bit_length`. The first row of the CSV should be a +header line where the columns are named. Subsequent rows encode packet fields. This format is appropriate if the CSV +defines all the packets one after another without skipping any. The three column format automatically +calculates the bit offsets assuming that the packet order is correct. See the :ref:`fourcolumn` format +for more flexibility. + +.. csv-table:: Simple Three Column CSV + :file: ../../ccsdspy/tests/data/packet_def/simple_csv_3col.csv + :widths: 30, 30, 30 + :header-rows: 1 + +When the example above is loaded, five `~ccsdspy.PacketField` objects are defined +with varying names, data types, and bit lengths. To create a `~ccsdspy.PacketArray` instead, define the data type with +both the type and array shape. + +.. csv-table:: Three Column CSV with `~ccsdspy.PacketArray` + :file: ../../ccsdspy/tests/data/packet_def/simple_csv_3col_with_array.csv + :widths: 30, 30, 30 + :header-rows: 1 + +In the example above, `VOLTAGE` would instead be a `~ccsdspy.PacketArray` of type `int` with shape `(12, 24)`. + +.. _fourcolumn: + +Four column CSV +================== + +The four column CSV format has columns for `name`, `data_type`, `bit_length`, and `bit_offset`. +The first row of the CSV should be a header line where the columns are named. Subsequent rows encode packet fields. +This format allows more flexibility than the three column CSV format because bit offsets are explicitly defined instead +of automatically calculated. Due to this, some packet fields can be skipped +since the bit offset indicates exactly where the packet begins. + +.. csv-table:: Simple Four Column CSV + :file: ../../ccsdspy/tests/data/packet_def/simple_csv_4col.csv + :widths: 30, 30, 30, 30 + :header-rows: 1 + +When the example above is loaded, five `~ccsdspy.PacketField` objects are defined +with varying names, data types, and bit lengths. To create a `~ccsdspy.PacketArray` instead, define the data type with +both the type and array shape. + +.. csv-table:: Four Column CSV with `~ccsdspy.PacketArray` + :file: ../../ccsdspy/tests/data/packet_def/simple_csv_4col_with_array.csv + :widths: 30, 30, 30, 30 + :header-rows: 1 + +In the example above, `SHSCOARSE` would instead be a `~ccsdspy.PacketArray` of type `uint` with shape `(4)`. + +Limitations of the CSV format +============================= + +Not all features of `ccsdspy` are currently supported with the CSV format. + +For `~ccsdspy.PacketField` the byte order cannot be defined in the CSV. + +For `~ccsdspy.PacketArray` the array order and byte order cannot be defined in the CSV. + +Also, :ref:`variable` cannot currently be loaded from a CSV file. From d18dd5174507bab179dbf95facaf4892c94938b6 Mon Sep 17 00:00:00 2001 From: Marcus Hughes Date: Tue, 20 Feb 2024 09:43:20 -0700 Subject: [PATCH 2/3] fixes indentation of the contents directive --- docs/user-guide/loadfile.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user-guide/loadfile.rst b/docs/user-guide/loadfile.rst index 8e56de4..957d853 100644 --- a/docs/user-guide/loadfile.rst +++ b/docs/user-guide/loadfile.rst @@ -19,7 +19,7 @@ or :ref:`four column ` format. At the moment, :ref:`variable` cannot be loaded from a CSV file. - .. contents:: +.. contents:: :depth: 2 .. _threecolumn: From 7d11be54eb6eb164baa7de114458680892ed56d5 Mon Sep 17 00:00:00 2001 From: Marcus Hughes Date: Wed, 28 Feb 2024 11:48:03 -0600 Subject: [PATCH 3/3] updates with revisions from PR --- docs/user-guide/loadfile.rst | 48 +++++++++++++++++------------------- 1 file changed, 23 insertions(+), 25 deletions(-) diff --git a/docs/user-guide/loadfile.rst b/docs/user-guide/loadfile.rst index 957d853..8d61c88 100644 --- a/docs/user-guide/loadfile.rst +++ b/docs/user-guide/loadfile.rst @@ -1,39 +1,39 @@ .. _loadfile: -************************************** -Loading Packet Definitions from a File -************************************** +****************************************** +Loading Packet Definitions from a CSV File +****************************************** Overview ========= -:ref:`fixed` can be loaded from a CSV file: +:ref:`fixed` can be loaded from a CSV (comma separated value) file. +This is an alternative method for defining packet layouts which may be desirable to some users, +and is currently undergoing development. The syntax for loading from a CSV file is: .. code-block:: python import ccsdspy pkt = ccsdspy.FixedLength.from_file('packet_definition.csv') -The only requirement is that the CSV is structured as either the :ref:`three column ` -or :ref:`four column ` format. - -At the moment, :ref:`variable` cannot be loaded from a CSV file. +The only requirement is that the CSV is structured as either the :ref:`threecolumn` +or :ref:`fourcolumn`. At the moment, :ref:`variable` cannot be loaded from a CSV file. .. contents:: :depth: 2 .. _threecolumn: -Three column CSV -=================== +Basic Layout (Three Columns) +============================ -The three column CSV format has columns for `name`, `data_type`, and `bit_length`. The first row of the CSV should be a +The basic CSV layout has columns for `name`, `data_type`, and `bit_length`. The first row of the CSV should be a header line where the columns are named. Subsequent rows encode packet fields. This format is appropriate if the CSV defines all the packets one after another without skipping any. The three column format automatically calculates the bit offsets assuming that the packet order is correct. See the :ref:`fourcolumn` format for more flexibility. -.. csv-table:: Simple Three Column CSV +.. csv-table:: Basic Layout CSV :file: ../../ccsdspy/tests/data/packet_def/simple_csv_3col.csv :widths: 30, 30, 30 :header-rows: 1 @@ -42,7 +42,7 @@ When the example above is loaded, five `~ccsdspy.PacketField` objects are define with varying names, data types, and bit lengths. To create a `~ccsdspy.PacketArray` instead, define the data type with both the type and array shape. -.. csv-table:: Three Column CSV with `~ccsdspy.PacketArray` +.. csv-table:: Basic Layout CSV with `~ccsdspy.PacketArray` :file: ../../ccsdspy/tests/data/packet_def/simple_csv_3col_with_array.csv :widths: 30, 30, 30 :header-rows: 1 @@ -51,16 +51,16 @@ In the example above, `VOLTAGE` would instead be a `~ccsdspy.PacketArray` of typ .. _fourcolumn: -Four column CSV -================== +Extended Layout (Four Columns) +============================== -The four column CSV format has columns for `name`, `data_type`, `bit_length`, and `bit_offset`. +The extended CSV layout has columns for `name`, `data_type`, `bit_length`, and `bit_offset`. The first row of the CSV should be a header line where the columns are named. Subsequent rows encode packet fields. -This format allows more flexibility than the three column CSV format because bit offsets are explicitly defined instead +This format allows more flexibility than the basic layout because bit offsets are explicitly defined instead of automatically calculated. Due to this, some packet fields can be skipped since the bit offset indicates exactly where the packet begins. -.. csv-table:: Simple Four Column CSV +.. csv-table:: Extended Layout CSV :file: ../../ccsdspy/tests/data/packet_def/simple_csv_4col.csv :widths: 30, 30, 30, 30 :header-rows: 1 @@ -69,7 +69,7 @@ When the example above is loaded, five `~ccsdspy.PacketField` objects are define with varying names, data types, and bit lengths. To create a `~ccsdspy.PacketArray` instead, define the data type with both the type and array shape. -.. csv-table:: Four Column CSV with `~ccsdspy.PacketArray` +.. csv-table:: Extended Layout CSV with `~ccsdspy.PacketArray` :file: ../../ccsdspy/tests/data/packet_def/simple_csv_4col_with_array.csv :widths: 30, 30, 30, 30 :header-rows: 1 @@ -79,10 +79,8 @@ In the example above, `SHSCOARSE` would instead be a `~ccsdspy.PacketArray` of t Limitations of the CSV format ============================= -Not all features of `ccsdspy` are currently supported with the CSV format. - -For `~ccsdspy.PacketField` the byte order cannot be defined in the CSV. - -For `~ccsdspy.PacketArray` the array order and byte order cannot be defined in the CSV. +The CSV format is in development and is currently limited. The limitations are: -Also, :ref:`variable` cannot currently be loaded from a CSV file. +* the byte order cannot be defined in the CSV. +* the array order and byte order cannot be defined in the CSV. +* :ref:`variable` cannot currently be loaded from a CSV file.