Updates from PyHSS

.dockerignore

@@ -0,0 +1,17 @@
+.github
+.git
+.gitignore
+.idea
+
+
+docs
+README.md
+systemd
+tests
+tools
+config.yaml
+
+venv
+
+docker/Dockerfile
+docker/docker-compose.yaml

.github/workflows/debian-package.yaml

@@ -0,0 +1,56 @@
+name: Debian package
+
+on:
+  push:
+    branches: [ "master" ]
+    tags:
+      - "*"
+  pull_request:
+    branches: [ "master" ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v3
+
+    - uses: jtdor/build-deb-action@v1
+      env:
+        DEB_BUILD_OPTIONS: noautodbgsym
+      with:
+        buildpackage-opts: --build=binary --no-sign
+
+    - name: Archive build artifacts
+      uses: actions/upload-artifact@v4
+      with:
+        name: debian-package
+        path: "debian/artifacts/*.deb"
+
+  release:
+    name: Upload Debian package to GitHub Release
+    runs-on: ubuntu-latest
+    permissions:
+      contents: write
+    needs: build
+    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags')
+
+    steps:
+      - uses: actions/checkout@v3
+      - name: Debug GitHub context
+        run: |
+          echo "repository=${{ github.repository }}"
+          echo "actor=${{ github.actor }}"
+          echo "ref=${{ github.ref }}"
+      - name: Download Debian package artifact
+        uses: actions/download-artifact@v4
+        with:
+          name: debian-package
+          path: debian/artifacts
+      - name: Upload .deb to GitHub Release
+        uses: ncipollo/release-action@v1
+        with:
+          tag: ${{ github.ref_name }}
+          artifacts: debian/artifacts/*.deb
+          bodyFile: 'CHANGELOG.md'
+          token: '${{ secrets.GITHUB_TOKEN }}'

.github/workflows/docker-image.yml

@@ -15,4 +15,4 @@ jobs:
     steps:
     - uses: actions/checkout@v3
     - name: Build the Docker image
-      run: cd docker && docker build . --file Dockerfile --tag my-image-name:$(date +%s)
+      run:  docker build . --file docker/Dockerfile --tag pyhss-imagebuild-test:$(date +%s)

.github/workflows/docker-publish.yaml

@@ -30,7 +30,7 @@ jobs:
       - uses: actions/checkout@v2
 
       - name: Build image
-        run: cd docker && docker build . --file Dockerfile --tag $IMAGE_NAME
+        run:  docker build . --file docker/Dockerfile --tag $IMAGE_NAME
 
       - name: Log into registry
         run: echo "${{ secrets.GITHUB_TOKEN }}" | docker login docker.pkg.github.com -u ${{ github.actor }} --password-stdin

.github/workflows/test.yaml

@@ -0,0 +1,34 @@
+name: Run tests
+
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version:
+          - "3.9"  # Debian 10
+          - "3.13"  # Debian 13
+
+    steps:
+    - uses: actions/checkout@v6
+
+    - name: Set up Python ${{ matrix.python-version }}
+      uses: actions/setup-python@v6
+      with:
+        python-version: ${{ matrix.python-version }}
+
+    - name: Install dependencies (apt)
+      run: |
+        sudo apt-get -q install -y redis libsctp-dev
+
+    - name: Install dependencies (pip)
+      run: |
+        python -m pip install --upgrade pip
+        pip install -r requirements.txt -r requirements-test.txt
+
+    - name: Run PyHSS tests
+      run: |
+        export PYTEST_ADDOPTS="--color=yes"
+        pytest -xvv

.gitignore

@@ -28,3 +28,9 @@ lib/__pycache__/milenage.cpython-36.pyc
 .gitignore
 .vscode/settings.json
 .cspell/custom-dictionary-workspace.txt
+pyhss.egg-info
+
+!tests/config.yaml
+tests/.pyhss.db
+.venv
+hss.db

CHANGELOG.md

@@ -5,6 +5,34 @@ All notable changes to PyHSS are documented in this file, beginning from [Servic
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+### Added
+
+- 2G / 3G support via Osmocom GSUP.
+- Support for running PyHSS services in Docker containers and provide official Docker images.
+- Database types postgresql and sqlite.
+- Config loading from `/etc/pyhss/config.yaml`, `/usr/share/pyhss/config.yaml`,
+  the `PYHSS_CONFIG` env var, or (old behavior) `config.yaml` at the top of the
+  source tree, depending on which is available.
+- Running services outside of the source tree.
+- Building PyHSS with `python3 -m build` and as debian package.
+- RAT restriction checking for subscribers.
+
+### Changed
+
+- Set the default database backend to SQLite.
+
+### Removed
+
+- Unused options from config.yaml.
+- Debug prints in API service.
+
+### Fixed
+
+- Fix unit tests and run them with pytest in CI.
+- Let services/apiService return HTTP status code 500 on errors instead of 200.
+
 ## [1.0.2] - 2024-07-03
 
 ### Added
@@ -23,8 +51,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [1.0.1] - 2024-01-23
 
-
 ### Removed
+
  - Assert on missing "IMS Services" for AAA/Audio Request
 
 ### Changed
@@ -97,4 +125,4 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 [1.0.0]: https://github.com/nickvsnetworking/pyhss/releases/tag/1.0.0
 [1.0.1]: https://github.com/nickvsnetworking/pyhss/releases/tag/1.0.1
-[1.0.2]: https://github.com/nickvsnetworking/pyhss/releases/tag/1.0.2
+[1.0.2]: https://github.com/nickvsnetworking/pyhss/releases/tag/1.0.2

README.md

@@ -14,7 +14,7 @@ This includes support for acting as:
  - Generate BSF Credentials
  - Gateway Mobile Location Centre 
 
-Supported database backends include `MySQL`, `Postgresql`, `SQLite`, `Orcale`, `MS-SQL` and more, all provisioned through a Swagger based [RESTful API](docs/API.md) for easy, safe CRUD operations on the subscriber data.
+Supported database backends are `MySQL`, `PostgreSQL` and `SQLite`, all provisioned through a Swagger based [RESTful API](docs/API.md) for easy, safe CRUD operations on the subscriber data.
 
 The software supports full monitoring through `Prometheus`, and has been tested to over 1m subscribers.
 
@@ -37,7 +37,11 @@ The underlying library - ``diameter.py`` can be easily worked with to add suppor
 
 ## Usage
 
-Basic configuration is set in the ``config.yaml`` file,
+Basic configuration is set in the `config.yaml` file, which gets loaded from:
+* The path in the `PYHSS_CONFIG` environment variable (if set)
+* `/etc/pyhss/config.yaml`
+* `/usr/share/pyhss/config.yaml`
+* The same directory as this `README.md`
 
 You will need to set the IP address to bind to (IPv4 or IPv6), the Diameter hostname, realm, your PLMN and transport type to use (SCTP or TCP).
 
@@ -52,6 +56,63 @@ If you're provisioning the HSS for the first time, you'll also want to run:
 
 The rest of the services aren't strictly necessary, however your own configuration will dictate whether or not they are required.
 
+### Using Docker (compose)
+
+A docker-compose file is provided to make spinning up PyHSS quick and easy. For development purposes, you can simply run:
+
+```shell
+cd docker && docker compose up --build -d 
+```
+
+This will start the following services, bound to 127.0.0.1 to avoid exposing them to the network by default:
+
+ - PyHSS Diameter Service (Port 3868/tcp)
+ - PyHSS HSS Service
+ - PyHSS API Service (Port 8080/tcp)
+ - PyHSS GSUP Service (Port 4222/tcp)
+ - Redis (Port 6379/tcp)
+ - MySQL (Port 3306/tcp)
+
+For production, just pull the following image:
+
+```shell
+docker pull ghcr.io/nickvsnetworking/pyhss/pyhss:latest
+```
+
+The `latest` tag is automatically built from current master.  For configuration, please reference `docker/config.yaml`.
+Every option in the configuration can be changed through environment variables, which are documented in the `docker/.env` file.
+
+Each container needs a `CONTAINER_ROLE` variable as well with one of the following values:
+
+ - diameter
+ - hss
+ - api
+ - geored
+ - logs
+ - metrics
+ - gsup
+ - database
+
+See the `docker/docker-compose.yaml` file for an example of how to set this up. This variable determines which service the container will run.
+
+### For developers: Configuring your IDE to run / debug inside the docker container
+
+Should you desire to run / debug PyHSS through your IDE like PyCharm to Run / Debug services through docker, you can.
+
+1. Create a Python interpreter on docker (not docker compose) that uses the image `ghcr.io/nickvsnetworking/pyhss/pyhss:development`
+2. Prepare the docker image by running `cd docker && docker compose build`. You'll need to rebuild only if requirements.txt changes.
+3. Create a Run configuration for each script:
+   * Select the docker interpreter as runtime
+   * Be sure to configure the env vars to use the `docker/.env` file
+   * Setup these additional env vars: `CONFIG_TEMPLATE=/opt/pyhss/docker/config.yaml;PYHSS_CONFIG=/tmp/config.yaml;PYTHONUNBUFFERED=1`
+   * Be sure that the container run options look like this: `--entrypoint=/opt/pyhss/docker/launch-container.sh -v /home/YOUR_USER/git/pyhss:/opt/pyhss -p 127.0.0.1:4222:4222 --network docker_default --rm`
+   * NOTE: The port example is for the GSUP daemon. Refer to the compose file for the relevant ports of your daemon
+   * NOTE: the path `/home/YOUR_USER/git/pyhss` is meant to point to your local source tree
+   * NOTE: The `--network docker_default` might need to be replaced with the network that is used by compose
+4. Start the entire environment with `docker compose up -d`
+5. Stop the service you'd like to manually run / debug. For instance: `docker compose stop pyhss_api`
+6. Use your IDE run configuration to launch the desired service
+
 ## Structure
 
 PyHSS uses a queued microservices model. Each service performs a specific set of tasks, and uses redis messages to communicate with other services.
@@ -73,6 +134,14 @@ The [RESTful API](docs/API.md) allows for easy, safe CRUD operations on the subs
 If REST isn't your jam and you instead want to interact directly with Python, `database.py` can be imported into your project and contains all the same hooks as the API.
 
 ## Installation
+
+Create and activate a virtual environment:
+
+```shell
+python3 -m venv .venv
+source .venv/bin/activate
+```
+
 Dependencies can be installed using Pip3:
 
 ```shell
@@ -99,6 +168,16 @@ Historically stats were collected through Redis counters and exposed via an SNMP
 
 More info about the legacy monitoring the system is available in [SNMP Readme](docs/monitoring.md).
 
+## Running tests
+
+Activate the virtual environment (see [installation](#installation), then
+install test dependencies and execute the tests as follows:
+
+```shell
+pip3 install -r requirements-test.txt
+pytest
+```
+
 ## About
 
 This software was written to address the limited options for lightweight HSS platforms out there, particularly those implementing IMS HSS functionality.
@@ -108,3 +187,6 @@ It is now deployed by several mid-tier operators, private LTE networks and lab n
 Any contributions are welcome, just submit a PR or contact me.
 
 You can contact me at nick (at) nickvsnetworking.com or via my blog at [nickvsnetworking.com](https://nickvsnetworking.com)
+
+A matrix room for discussing PyHSS is available at:
+[#pyhss:matrix.org](https://matrix.to/#/#pyhss:matrix.org)