Merge branch 'master' into maint-3.8

Author: daniestevez

CHANGELOG.md

@@ -6,6 +6,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Unreleased
 
+## [3.7.0] - 2021-02-19
+
+### Added
+- Read sample rate from WAV files automatically
+- PDU Length Filter block
+- Example decoder for NEXUS 19k2 pi/4-DQPSK telemetry
+- Support for AALTO-1 CC1125 mode
+- Support for CAPE-3
+- Support for DELFI-C3
+- Support for EXOCUBE-2
+- Support for Grizu-263A
+- Support for IDEASSat
+- Support for MiTEE-1
+- Support for SOMP 2b
+- Support for UVSQ-SAT
+- Support for YUSAT-1
+
+### Changed
+- Add filter before quadrature demod for IQ input FSK demodulator
+
+### Fixed
+- Runtime error when the baudrate is too high for the sample rate
+- Errors about wrong Reed-Solomon message size with OPS-SAT deframer
+- Bug in SIDS submit URL encoding
+- Bug with SatYAML files in platforms not using UTF-8
+- Build problems with clang
+
 ## [3.6.0] - 2020-12-04
 
 ### Added
@@ -318,7 +345,8 @@ Large refactor release bringing new functionality and improvements. This is an o
 ## [1.0.0] - 2018-08-02
 First gr-satellites release using semantic versioning
 
-[Unreleased]: https://github.com/daniestevez/gr-satellites/compare/v3.6.0...master
+[Unreleased]: https://github.com/daniestevez/gr-satellites/compare/v3.7.0...master
+[3.7.0]: https://github.com/daniestevez/gr-satellites/compare/v3.6.0...v3.7.0
 [3.6.0]: https://github.com/daniestevez/gr-satellites/compare/v3.5.2...v3.6.0
 [3.5.2]: https://github.com/daniestevez/gr-satellites/compare/v3.5.1...v3.5.2
 [3.5.1]: https://github.com/daniestevez/gr-satellites/compare/v3.5.0...v3.5.1

CMakeLists.txt

@@ -43,7 +43,7 @@ list(INSERT CMAKE_MODULE_PATH 0 ${CMAKE_SOURCE_DIR}/cmake/Modules)
 
 # Set the version information here
 set(VERSION_MAJOR 3)
-set(VERSION_API 6)
+set(VERSION_API 7)
 set(VERSION_ABI 0)
 
 cmake_policy(SET CMP0011 NEW)

README.md

@@ -53,16 +53,18 @@ gr-satellites:
 The repository is organized in the following branches:
 
 * `master` is where the active development happens. From time to time, features
-  will be frozen in a new release.
+  will be frozen in a new release. This branch is compatible with GNU Radio 3.8.
 
 * `maint-3.8` is the branch where releases in the current `v3.x.y` line are
   published. This branch is compatible with GNU Radio 3.8.
 
-* `maint-3.8-v2` is the branch where releases in the `v2.x.y` line are
-  published. This branch is compatible with GNU Radio 3.8.
+* `maint-3.8-v2` is the branch where releases in the `v2.x.y` line were
+  published. This branch is compatible with GNU Radio 3.8. No changes
+  happen in this branch any longer.
 
-* `maint-3.7` is the branch where releases in the `v1.x.y` line are
-  published. This branch is compatible with GNU Radio 3.7.
+* `maint-3.7` is the branch where releases in the `v1.x.y` line were
+  published. This branch is compatible with GNU Radio 3.7. No changes happen
+  in this branch any longer.
 
 In general, pull requests should be submitted to `master`.
 

apps/gr_satellites

@@ -44,6 +44,7 @@ def argument_parser():
 
     p.add_argument('--version', action = 'version', version = version)
     p.add_argument('--list_satellites', action = ListSatellites, nargs = 0, help = 'list supported satellites and exit')
+    p.add_argument('--ignore_unknown_args', action = 'store_true', help = 'Treat unknown arguments as warning')
 
     p_input = p.add_argument_group('input')
 
@@ -59,6 +60,7 @@ def argument_parser():
     p_input.add_argument('--udp_ip', default = '::', help = 'UDP input listen IP [default=%(default)r]')
     p_input.add_argument('--udp_port', default = '7355', type = int, help = 'UDP input listen port [default=%(default)r]')
     p_input.add_argument('--iq', action = 'store_true', help = 'Use IQ input')
+    p_input.add_argument('--udp_raw', action = 'store_true', help = 'Use RAW UDP input (float32 or complex64)')
     p_input.add_argument('--input_gain', type = float, default = 1, help = 'Input gain (can be negative to invert signal) [default=%(default)r]')
     p_input.add_argument('--start_time', type = str, default = '', help = 'Recording start timestamp')
     p_input.add_argument('--throttle', action = 'store_true', help = 'Throttle recording input to 1x speed')
@@ -74,8 +76,8 @@ def argument_parser():
     return p
 
 def check_options(options, parser):
-    if options.kiss_in is None and options.samp_rate is None:
-        print('Need to specify --samp_rate unless --kiss_in is used', file = sys.stderr)
+    if options.kiss_in is None and options.wavfile is None and options.samp_rate is None:
+        print('Need to specify --samp_rate unless --wavfile or --kiss_in is used', file = sys.stderr)
         parser.print_usage(file = sys.stderr)
         sys.exit(1)
 
@@ -91,8 +93,24 @@ class gr_satellites_top_block(gr.top_block):
     def __init__(self, parser):
         gr.top_block.__init__(self, 'gr-satellites top block')
         sat = parse_satellite(sys.argv[1])
-        satellites.core.gr_satellites_flowgraph.add_options(parser, **sat)
-        options = parser.parse_args(sys.argv[2:])
+        try:
+            satellites.core.gr_satellites_flowgraph.add_options(parser, **sat)
+        except ValueError as e:
+            if e.args[0] == 'satellite not found':
+                print(f'Satellite {sys.argv[1]} not found')
+                exit(1)
+            else:
+                raise e
+        except FileNotFoundError as e:
+            print('Could not open SatYAML file:')
+            print(e)
+            exit(1)
+        if '--ignore_unknown_args' in sys.argv[2:]:
+            options, unknown = parser.parse_known_args(sys.argv[2:])
+            if len(unknown) > 0:
+                print('Warning: unknown arguments passed {}'.format(unknown), file = sys.stderr)
+        else:
+            options = parser.parse_args(sys.argv[2:])
         if options.list_satellites:
             list_satellites()
             sys.exit(0)
@@ -103,11 +121,25 @@ class gr_satellites_top_block(gr.top_block):
         pdu_in = options.kiss_in is not None
 
         self.config = open_config()
-        
-        self.flowgraph = satellites.core.gr_satellites_flowgraph(samp_rate = options.samp_rate, iq = self.options.iq,
-                                                                 pdu_in = pdu_in, options = options, config = self.config,
-                                                                 dump_path = options.dump_path, **sat)
+
         self.setup_input()
+
+        # set appropriate sample rate
+        if options.samp_rate:
+            samp_rate = options.samp_rate
+        elif options.wavfile:
+            samp_rate = self.wavfile_source.sample_rate()
+        else:
+            raise ValueError('sample rate not set')
+            
+        self.flowgraph = satellites.core.gr_satellites_flowgraph(samp_rate = samp_rate,
+                                                                 iq = self.options.iq,
+                                                                 pdu_in = pdu_in,
+                                                                 options = options,
+                                                                 config = self.config,
+                                                                 dump_path = options.dump_path,
+                                                                 **sat)
+
         if pdu_in:
             self.msg_connect((self.input, 'out'), (self.flowgraph, 'in'))
         else:
@@ -166,8 +198,12 @@ class gr_satellites_top_block(gr.top_block):
             self.input = self.audio_source
 
     def setup_udp_input(self):
-        self.input_int16 = blocks.udp_source(gr.sizeof_short, self.options.udp_ip, self.options.udp_port, 1472, False)
-        self.setup_input_int16()
+        if self.options.udp_raw:
+            size = gr.sizeof_gr_complex if self.options.iq else gr.sizeof_float
+            self.input = blocks.udp_source(size, self.options.udp_ip, self.options.udp_port, 1472, False)
+        else:
+            self.input_int16 = blocks.udp_source(gr.sizeof_short, self.options.udp_ip, self.options.udp_port, 1472, False)
+            self.setup_input_int16()
 
     def setup_input_int16(self):
         self.short_to_float = blocks.short_to_float(1, 32767)

debian/copyright

@@ -52,6 +52,10 @@ License: LGPL-2.1
  License (LGPL) version 2.1 can be found in the file
  '/usr/share/common-licenses/LGPL-2.1'.
 
+Files: lib/viterbi/*
+Copyright: Min Xu <[email protected]>
+License: Apache-2.0
+
 Files: python/ccsds/*
 Copyright: 2019 Athanasios Theocharis <[email protected]>
 License: GPL-3+

docs/source/command_line.rst

@@ -18,9 +18,10 @@ basic information about the arguments it allows.
 
    $ gr_satellites
    usage: gr_satellites satellite [-h] [--version] [--list_satellites]
+                               [--ignore_unknown_args]
                                (--wavfile WAVFILE | --rawfile RAWFILE | --rawint16 RAWINT16 | --audio [DEVICE] | --udp | --kiss_in KISS_IN)
                                [--samp_rate SAMP_RATE] [--udp_ip UDP_IP]
-                               [--udp_port UDP_PORT] [--iq]
+                               [--udp_port UDP_PORT] [--iq] [--udp_raw]
                                [--input_gain INPUT_GAIN]
                                [--start_time START_TIME] [--throttle]
                                [--kiss_out KISS_OUT] [--kiss_append]
@@ -91,7 +92,8 @@ Besides specifying the satellite to use for decoding, it is mandatory to specify
 the input source by using exactly one of the following options:
 
 * ``--wavfile`` can be used to read a recording in WAV format. The sample rate
-  of the recording needs to be specified with the ``--samp_rate`` argument.
+  is obtained from the WAV header, but it can be overriden using the the
+  ``--samp_rate`` argument if necessary.
 
   By default, the WAV file is interpreted as a one-channel file containing real
   RF samples. To read a two-channel file containing IQ RF samples, the ``--iq``
@@ -106,7 +108,7 @@ the input source by using exactly one of the following options:
 
      .. code-block:: console
 
-        $ gr_satellites FUNcube-1 --wavfile satellite-recordings/ao73.wav --samp_rate 48e3
+        $ gr_satellites FUNcube-1 --wavfile satellite-recordings/ao73.wav
   
 * ``--rawfile`` can be used to read a recording in ``complex64`` or ``float32``
   format (depending on whether the ``--iq`` argument is used or not). The sample rate
@@ -156,6 +158,7 @@ the input source by using exactly one of the following options:
   The streaming format is the same as for the ``--rawint16`` and both real
   samples (by default) and IQ samples (using the ``--iq`` argument) are
   supported.
+  If the ``--udp_raw`` is used the format will be the same as for ``--rawfile``.
 
   By default, ``gr_satellites`` will listen on the IP address ``::`` (all
   addresses) and the UDP port 7355. A different IP address or port can be
@@ -225,6 +228,7 @@ For example, this shows all the options allowed by the FUNcube-1 decoder:
      -h, --help            show this help message and exit
      --version             show program's version number and exit
      --list_satellites     list supported satellites and exit
+     --ignore_unknown_args Treat unknown arguments as warning
 
    input:
      --wavfile WAVFILE     WAV input file
@@ -331,7 +335,7 @@ An example of the use of this option can be seen here:
 .. code-block:: console
 
     $ gr_satellites FUNcube-1 --wavfile ~/gr-satellites/satellite-recordings/ao73.wav \
-             --samp_rate 48e3 --hexdump
+             --hexdump
     * MESSAGE DEBUG PRINT PDU VERBOSE *
     ()
     pdu_length = 256
@@ -445,7 +449,7 @@ internal signals produced by decoding a sample recording of AU02.
 
     $ mkdir -p /tmp/fsk
     $ gr_satellites AU02 --wavfile satellite-recordings/au02.wav \
-         --samp_rate 48e3 --dump_path /tmp/fsk
+         --dump_path /tmp/fsk
 
 We see that we do not get any decoded packets. Then, we can plot the FSK symbols
 with the following Python code:
@@ -496,7 +500,9 @@ certain satellite projects:
 * `Harbin Institute of Technology`_, which connects to the telemetry proxy included in
   `gr-lilacsat`_ and `gr-dslwp`_.
 
-* Any custom server using the SIDS protocol.
+* Any custom server using the SIDS protocol. The `SIDS protocol`_ is an HTTP-based protocol
+  that was first developed by the ESTCube team and later used by the UWE-3 team. It is the
+  basis of the SatNOGS DB server and other telemetry servers.
 
 To enable telemetry submission, it is necessary to edit some parameters in
 ``gr_satellites``'s config file, which is located in
@@ -736,6 +742,15 @@ limit playback speed to 1x and use the ``--start_time`` parameter followed by th
 timestamp in ISO 8601 format (``YYYY-MM-DDTHH:MM:SS``) to indicate the start time
 of the recording.
 
+Treating unknown args as warning
+""""""""""""""""""""""""""""""""
+
+Using the argument ``--ignore_unknown_args`` will change the behaviour on unknown
+arguments to a warning instead of exiting with an error. This can be useful when
+running in automated scripts and some options may not be available on that satellite.
+For example the ``--f_offset`` and ``--use_agc``
+
+
 .. _GQRX: https://gqrx.dk/
 .. _gr-frontends: https://github.com/daniestevez/gr-frontends
 .. _Message Debug: https://wiki.gnuradio.org/index.php/Message_Debug
@@ -755,3 +770,4 @@ of the recording.
 .. _gr-dslwp: https://github.com/bg2bhc/gr-dslwp
 .. _pyqt5 branch of gr-lilacsat: https://github.com/daniestevez/gr-lilacsat/tree/pyqt5
 .. _Audio Source: https://wiki.gnuradio.org/index.php/Audio_Source
+.. _SIDS protocol: https://github.com/janvgils/sids

docs/source/conf.py

@@ -26,7 +26,7 @@
 # The short X.Y version
 version = ''
 # The full version, including alpha/beta/rc tags
-release = '3.6.0'
+release = '3.7.0'
 
 
 # -- General configuration ---------------------------------------------------

docs/source/installation.rst

@@ -97,11 +97,14 @@ After running ``make``, you can run the tests by doing ``make test`` in the
 ``build/`` directory.
 
 .. note::
-   There are systems where the AO-73 and similar decoders fail if
+   There are systems where the AO-73 and similar decoders fail to decode correctly if
    ``volk_profile`` has not been run ever in the system. This seems to be caused
    by the Viterbi decoder chosen by Volk by default when there is no
    ``~/.volk/volk_config`` file. If problems with these decoders are seen, it
-   is recommended to run ``volk_profile`` to see if it fixes the problems.
+   is recommended to run ``volk_profile`` to see if it fixes the problems. Additionally,
+   it is recommended to check if in ``~/.volk/volk_config`` there is a line that
+   contains ``volk_8u_x4_conv_k7_r2_8u avx2 avx2`` and replace both occurences
+   of ``avx2`` by either ``spiral`` or ``generic``.
 
 
 PYTHONPATH

docs/source/satyaml.rst

@@ -203,6 +203,23 @@ The framings allowed in the ``framing`` field are the following:
 * ``FOSSATSAT``, a custom protocol used by FOSSASAT
 
 * ``AISTECHSAT-2``, a custom CCSDS-like protocol used by AISTECHSAT-2
+
+* ``AALTO-1``, custom framing used by AALTO-1. It uses a
+  Texas Intruments CC1125 transceiver with a PN9 scrambler and a CRC-16 CCITT
+  (as in AX.25)
+
+* ``Grizu-263A``, custom framing used by Grizu-263A. It uses a Semtech SX1268
+  with a PN9 scrambler and CRC-16.
+
+* ``IDEASSat``, custom framing used by IDEASSat. It uses NRZI encoding,
+  an 1N8 UART-like encoding with MSB-bit-ordering,
+  and HDLC ``0x7e`` flags to mark the frame boundaries.
+
+* ``YUSAT``, custom framing used by YUSAT-1. It is like AX.25 but without
+  bit stuffing, LSB byte endianness, and NRZ-I.
+
+* ``AX5043``, FEC framing used by the AX5043 transceiver IC. This uses a convolutional
+  code, a 4x4 interleaver, and HDLC framing with the CRC16-USB.
 
 Some framings, such as the CCSDS protocols need the additional field
 ``frame size`` to indicate the frame size.