Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make KBox repeat all NMEA/NMEA2000 messages to USB port #128

Merged
merged 5 commits into from
Apr 23, 2018
Merged

Make KBox repeat all NMEA/NMEA2000 messages to USB port #128

merged 5 commits into from
Apr 23, 2018

Conversation

sarfata
Copy link
Owner

@sarfata sarfata commented Feb 17, 2018 •
edited
Loading

@sarfata
Make KBox repeat all NMEA/NMEA2000 messages to USB port
af99c37 
 - New USB mode active when the connection speed is set to 38400 bauds
 - In this mode all NMEA and NMEA2000 messages are repeated to USB
 - Removed KMessage/KMessageNMEAVisitor - they are not needed anymore
 - Make sure we read data from serial port in that mode to avoid the
   serial port locking up on Raspberry Pi (closes #68)
 - Also convert SerialK data to NMEA and output it (this provides an
   output for internal sensors but may result in duplicated data)
@coveralls
Copy link

coveralls commented Feb 17, 2018 •
edited
Loading

Coverage Status

Coverage decreased (-0.2%) to 67.566% when pulling 8dbd45d on feature/usb-interface into 19bc52b on master.

@ronzeiller
Copy link
Contributor

ronzeiller commented Feb 17, 2018 •
edited
Loading

@sarfata
nice PR 👍

New USB mode active when the connection speed is set to 38400 bauds

Would be nice to have it by config settings?
- Enable/Disable Debug msg output
- Enable Data output (configured for different sentences like Serial)
- Set USB bauds
- Switch #build_flags = -DDebugSerial=Serial3 in platformio.ini to config setting

edit 24/2/18:
Sorry, a closer look (and test) thought me, that it is done in a very genious way by just switching the baud rate of USB connection!!!

Also convert SignalK data to NMEA and output it (this provides an
output for internal sensors but may result in duplicated data)

In "my KBox" I made a config to enable/disable:

  • single N2k PGNs – if not enabled they get dropped before they are parsed and added to SKHub
  • enable own Talker-ID for internal sensor NMEA0183 sentences (to differentiate sources, if e.g. HdG is coming from N2k AND internal sensor)
  • same for Performance calculations, made an own SKSource for it to differentiate output

(But too many PR's pending, so I wait with pushing.)

@CaptainRon47
Copy link

Need to calibrate IMU before it starts showing output on wifi or usb
calibration with long push on button It does not show calibrated on the kbox display
using kbox.py
need to install png pip install pypng
use tools/kbox.py --help to see options and arguements
ex: tools/kbox.py --p COM3 fread kbox-config.json - prints out the file to the std output

@ronzeiller
Copy link
Contributor

@sarfata

in WiFiService
in line 116: k.appendNullTerminatedString(pcdin);
pcdin is missing \r\n

ditto in line 101: k.appendNullTerminatedString(sentence.c_str());

@ronzeiller
Copy link
Contributor

ronzeiller commented Mar 4, 2018 •
edited
Loading

Also convert SignalK data to NMEA and output it (this provides an
output for internal sensors but may result in duplicated data)

as all datas from NMEA2000 and Serial are "repeated" AND coming from the Hub as SKUpdates aren´t they all duplicated?

@ronzeiller
Copy link
Contributor

ronzeiller commented Mar 4, 2018 •
edited
Loading

@sarfata
Today I made a drawing, trying to understand the dataflow in KBox.
The SKHub to subscribe to is a very nice idea.
But, concerning NMEA0183 for each Serial Output, WiFi and also SD-CardService: at the moment the same conversion from SKUpdate to NMEA0183 takes place up to 4 times!
(in my box I have 4 serials and there it is even more)

I could imagine to extend the SKHub to do one central conversion for each format needed there.
Then Services could subscribe directly to NMEA0183 or PCDIN (Seasmart) or N2kMsg, whatever is needed.
Then it would also be easier for a SerialService to subscribe to high frequency raw datas (e.g. for Autopilot), or filtered (damped) datas for display and navigation and/or calculated/corrected datas like calibrated boat speed, true wind speed and angle, leeway, performance datas etc.

In principle all high frequency datas could run through a filter.
(Some) Output tasks (for displays and navigation) could then be interval services (like pages) just grabbing the actual value of what they need.

I plan to move Performance calculations to SKHub too, as this is the central object where all datas come together. For sentences where mixed SKUpdates are necessary I think it is the right class too, to have variables there for storing.

Especially for the WiFiService it will be positive to subscribe to "display" datas, because at the moment when you have NMEA2000 coming in, together with an external (10Hz) NMEA0183 IMU-Sensor plus internal barometer plus voltage the WiFi is freaking out. The same I am afraid will happen to logging too, as I cannot believe that all those datas could be written to SD-Card.

@CaptainRon47
Copy link

CaptainRon47 commented Mar 5, 2018 •
edited
Loading

@sarfata @ronzeiller
Ron I applaud you for being able to picture the data flow in KBox, I can't (only snippets here and there).
There is no need (for me) to have all the options active all the time. I don't have any n2K instruments, I never envisioned the KBox would host SKdata (I think it is great if it does, but I will probably (at least for now) connect to an Rpi running SK and OpenCPN, I don't need NMEA0183 data transmitted over both USB and WiFi at the same time (I will connect to the Rpi either over WiFi {which I am having trouble doing right now} or USB - The instruments on my boat are already interconnected they don't depend on KBox, but I would like to be able to get waypoints from OpenCPN thru KBox to my GPS/Autopilot, so I don't need "ALL' data repeated at the NMEA outputs. I would like 'ALL' NMEA data written to the log, but it does not have to be at the same rate as all other transmissions once every few minutes is plenty, it is only historical data for me. I would like to be able to access the logs using tools/kbox.py - I don't think I have been able to do that. Currently SK does not deal with 'XDR' statements, so if there are 'native' SK sentences that would allow the transmission of our current 'XDR' statements then those would be the ones I would enable. I understand that all this creates a puzzle as how to configure and implement the data flow in KBox, but I think with the 'right' selection of options we could manage not to over tax the hardware.

@CaptainRon47
Copy link

@sarfata @ronzeiller
One thing I don't understand (well that is an understatement - anyway) The IMU display shows that it is uncalibrated for heading, but the pitch and roll seem to be working with valid numbers. Why doesn't the Kbox generate pitch and roll output (like what is on the display) when the IMU is uncalibrated (red)? The numbers on the display for pitch and roll seem ok and they have been zeroed ok by a long push. But I get no XDR output for pitch and roll unless the IMU (heading) display is white and not red. I can live without the IMU heading (I have two other digital compasses on the boat) but I would like the pitch and roll data to be sent to signalk and OpenCPN. - Ron

@ronzeiller
Copy link
Contributor

@CaptainRon47

Hi Ron,

the IMU sensor has (internal) calibration values for accel, gyro, magnetic and an over all "system"

  • In my opinion heel (=roll) and pitch are quite reliable, even if all cal-values are low
    - Hdg and heel/pitch are sent independent from each other
    - To set offset by "long push" is always possible, calibration will only be stored if cal-values are "full calibrated" (all values 3)

But values will only be sent (e.g. NMEA to serial, if enabled) if following conditions are true:


bool isMagCalibrated() {
      return _magCalib == 3 && _sysCalib > 0;
    };

    bool isRollAndPitchCalibrated() {
      return _accelCalib >= 2 && _gyroCalib >= 2;
    };

Just made an update to PR #124
Even if it is not merged to master, you could give it a try.

Now you should see on the IMU page if values are sent or not.

@ronzeiller
Copy link
Contributor

@CaptainRon47 @sarfata

Yeah, the great thing with Kbox firmware is, that you can configure it as a simple logging device (without any sensors or display), as NMEA0183 Multiplexer (which I believe is more or less your demand) or as performance sailing computer.
Some of it is working already, some of it is work in progress.

There where so many great changes in the last month, but no time at all to do a proper documentation, data flow diagrams, "how to" - descriptions etc.

@CaptainRon47
Copy link

@sarfata @ronzeiller
Well my problem is poor short term memory it seems!! In fact you don't need to SET the USB serial baud rate to 38400 to get only NMEA data, you ONLY need to CONNECT to it at that speed!! So no configuration change is necessary. ALSO, it seems I DO get pitch and roll NMEA output when the IMU is not calibrated - I was not seeing it on the SIGNALK-Server because it does not handle XDR statements!
IF I were to attempt to write-up some documentation for these changes - where would you want that posted or sent to? I am thinking the Configuring Kbox page, but right now all I have are a bunch of NOTES. which if I don't read often I forget!

@ronzeiller
Copy link
Contributor

ronzeiller commented Mar 7, 2018 •
edited
Loading

Yeah, meanwhile I have a whole book of notes too :-)
As you might know, I have rebuild a "KBox" with a Teensy 3.6 and some of the components found in the original KBox.
I try to keep it as close as possible to KBox firmware, but I need things which might be of no use for "the common" KBox user, so I had to split development.
(Should be fully compatible with any KBox just by switching to Board Teensy 3.1 in platformio.ini)
I try to comment the/my code as much as possible as I find it very hard to read the code.
(promise to to it just in english in the future :-) )
Here is the scheme I was talking about, hoping it is correct and usefully for somebody:
https://github.com/ronzeiller/kbox-firmware/blob/Sailmax-CU_V2_5/Sailmax-CU_V2_5_classes-scheme.pdf

Remark:
The IMU-sensor is not in the scheme, as I kicked the sensor out, working with an external NMEA0183 NXP-sensor on serial input now.
On the image of my Sailmax-CU unit you can see a green plug with a mouse symbol as I have some PS/2 connectors for serial devices including 5V and 12V power supply. Stripped some old mice and keyboards to get the nice cables ;-)

Remark to documentation:
As the whole thing is work in progress it is hard to stay in tune with some kind of user manual, but take a look at the wiki, as many of the new things are described there already.

@CaptainRon47
Copy link

RonZ,
Where are you with Kbox as a performance monitor? I have a 35' sailboat ( 1989 HOLBY Clearwater 35) that has a 'swing keel,' I draw 1'10" with the keel completely retracted and 6' with it down. It will sail with the keel in any position but I sacrifice leeway, so I would love to have something that told me how much leeway I am making. I have not tried your latest firmware yet, but I will look at it as soon as I get a chance.
RonS

This was referenced Mar 17, 2018
Open
Open
@sarfata
Copy link
Owner Author

sarfata commented Apr 23, 2018

Hey guys, Thomas here back from sailing for a little bit ;)

Thanks for the great feedback on this. I understand that it was not clear that you just need to connect at 38400 bauds but now that you have both understood that, I think you like this and I will merge it.

@ronzeiller Thanks for putting that diagram together. That is super interesting. You are right that right now the conversion of SignalK to NMEA can happen multiple times. If we think that repeating the same thing to multiple serial ports (especially in your case where you have more) is going to become a use-case we can come up with a better solution.

In "my KBox" I made a config to enable/disable:

  • single N2k PGNs – if not enabled they get dropped before they are parsed and added to SKHub

Love that. Was it painful to write? Did you find a flexible way to do it so that we do not have to write each possible PGN in the config parsing code?

  • enable own Talker-ID for internal sensor NMEA0183 sentences (to differentiate sources, if e.g. HdG is coming from N2k AND internal sensor)
    -same for Performance calculations, made an own SKSource for it to differentiate output

I think creating different sources is a great solution. This is what B&G does on NMEA2000 network too. For example the Vulcan 7 appears as "Vulcan 7 GPS", "Vulcan 7 Navigator", "Vulcan 7 Autopilot Controller". The ZG100 appears as a GPS and as a compass separately.

need to install png pip install pypng

I have added that to the requirements.txt file. That is the proper way to list our dependencies.

I will merge this PR now and continue work on KBox.

@sarfata sarfata merged commit c041cb3 into master Apr 23, 2018
@sarfata sarfata deleted the feature/usb-interface branch April 23, 2018 23:51
@CaptainRon47
Copy link

CaptainRon47 commented Apr 24, 2018 via email

@ronzeiller
Copy link
Contributor

@sarfata
Hello Thomas, welcome back!!
Hope you had a great time.

In the meantime I sailed my regatta and made lot of experiences with KBox. I will try to put these into concerning issues the next days (or weeks) to discuss it little by little.

To your question:

In "my KBox" I made a config to enable/disable:

single N2k PGNs – if not enabled they get dropped before they are parsed and added to SKHub

Love that. Was it painful to write? Did you find a flexible way to do it so that we do not have to write each possible PGN in the config parsing code?

No, sorry did not find a flexible way.

One thing I did, was to put config "more global" which means I can look for config values more flexible wherever I need it in the code. As the config is growing, and so many things are connected, I believe it is a better way then to pass small pieces to classes and functions.

And I put lot of stuff into SKHub, which is a central piece of dealing, construction and distribution of different SKUpdates. SKUpdates as they come in, SKUpdates from different inputs merged together (as needed for some NMEA0183 sentences), calculated values, hi-frequency and (filtered) low-frequency (for displays and navigation programs) SKUpdates. So you can "choose" which SKUpdate style you subscribe to.
May be it is an interesting concept, worth working in that direction.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@sarfata @coveralls @ronzeiller @CaptainRon47