WSJT-X UDP protocol reporting support#97
Conversation
|
This is an interesting idea - it certainly would be nice to have JS8Call compatibility with GridTracker and the other programs. @chrbayer84 Thanks for submitting the idea. We will need to test and verify that it does not interfere in any way with other network functionality. 73, |
|
I did some initial testing from your repository, and find that indeed GridTracker2 works fine with it. I also was able to simultaneously connect to JS8Call via the other UDP port with another application and that worked fine. At least in terms of basic functionality, it seems good. 73, |
|
@chrbayer84 Well, there is a bug, either in the data being sent to GridTracker, or in GridTracker itself. If I start afresh and a station sends a HB, it will plot in the GridTracker map. Then, if another station sends a HB, his grid square will plot, and his callsign shows in that grid plot. However, if I look at the first station's grid plot, it shows the SECOND station's callsign. As this contiues, ALL grid plots show the last station callsign to plot. I'm testing on Linux Mint 22.2, JS8Call 2.5.0, your repository, branch origin/cbayer-wsjtx-udp-reporting, GridTracker2 v2.250914.1, installed from the .deb installer from the GridTracker web site. More work needed somewhere... 73, |
|
Interesting observation, this must be a bug with how the data is packaged into the wsjtx packages.. Do you happen to have live JS8Call traffic anywhere? I'm hearing zilch rn. I was lucky to receive 3 stations over the night on 40 last night but not too much traffic these days.. If you do have traffic, could you by any chance run https://github.com/bmo/py-wsjtx/blob/master/samples/dump_wsjtx_packets.py and send me the output for what it shows for a couple decode cycles? |
|
This certainly looks good to me. Just a comment that's by no means a show-stopper; the js8call.pro file is not used here anymore, and is obsolete. It just hangs around in the source tree, won't even build with Qt Creator anymore. In fact, it still defines the fortran compiler, which is also not used here anymore. When I get time I'm going to start cleaning up the source tree, sort out and move some of these classes to folders. And since our code has diverged so far from WSJT-x, rename some of these classes and variables from WSJTX to JS8. |
Here is a capture using dump_wsjtx_packets.py which has a few stations who transmitted HBs. Let me know if this file is useful to you. 73, |
|
While watching 40m for a bit I got a few HB packages. None of them contained the grid. Maybe there is some gap in GT that parses the free text messages and assigns a grid somewhere. I'll keep monitoring, maybe I can observe this in the wild. |
|
@Joe-K0OG I looked at your dump file. It looks like this is for JS8Calls protocol. What does your reporting settings screen look like? I left the JS8Call reporting on default settings (port 2242 I believe) and the wsjtx reporting on 2237. I need the dump for 2237. |
@chrbayer84 Sorry about that - I grabbed the wrong port! I'll do it again this morning with the WSJT-X port. 73, |
|
I'd love to see it added. Winter Field Day is coming up and JS8Call is going to be the main app I'll use. Thanks for all your great work! |
|
Ok, there is an opportunity here as my boss would say when there is trouble. The opportunity is an architectural decision... So the WSJT-X protocol for decodes mirrors the limited message size. so it really just supports grid, CQ/RRR/73 and SNR. The first callsign has a colon appended.. GT put that colon next to the callsign and as such didn't know what it was. Also, the SNR is represented with the SNR prefix. |
@chrbayer84 Here is a dump file from this morning on 40m. There are several HBs there. It seems that if I query
Only received text that contains a grid string could be parsed to plot on the map. Another alternative is to keep a database of callsigns and their grids either collected from on-air transmissions as does js8map (from http://github.com/lcgreenwald/js8map), or some downloaded database like from the FCC in the U.S. 73, |
|
Yeah, my earlier point.. I think we should do some parsing on the JS8Call side and maintain WSJT-X protocol compatibility. |
|
Ok, talked with the other Gridtracker folks. Scratch my latest comments, we will provide parsing support for the JS8Call messages in GT based on the 'mode' field being set to 'JS8'. This makes parsing easy and should resolve thje grid placement issues above. @Joe-K0OG |
|
Branch to add support for this in GridTracker: https://gitlab.com/gridtracker.org/gridtracker2/-/merge_requests/18 |
|
@Joe-K0OG any chance we can get branch build artifacts for this? people in the GT discord seem to be pretty excited to test this out and I can't build a Windows binary, which is where most people are who want to test this :) |
@chrbayer84 Sure, I can build a Windows installer for you. Give me a few minutes and I'll post it here. 73, |
|
@chrbayer84 Here is the Windows installer, set up so it will install into Download from: 73, P.S. This was built from |
|
Thanks for the builds! The build is mostly intended for folks who can't wait for this PR to be merged and artifacts available.. Speaking of merging - are there any open issues with this branch left? I saw the nit for js8call.pro.. I guess I could remove my changes to that file or the file altogether if you want me to. Other than that, when do you guys think this could be merged? |
|
I would say if it meets @Joe-K0OG testing requirements we could merge it anytime. The js8call.pro file is not a big deal. Just leave it as-is for now, that's a future code cleanup issue. If you'd want to help with the startup of that code cleanup you could put your new classes in a UDP_Protocol folder or some such. |
|
Ah, I was wondering my I couldn't get qlog to work, doh! Thank you! |
|
Qlog is still not working for me with the artifact Ubuntu build, I haven't had time to do any troubleshooting as to why that may be yet. |
If that's the case, then continue testing from this PR until we're sure it's ready for final merge. We can merge anytime since master is not being forked to a release branch yet. But I'll rely on the discretion of @chrbayer84 and the testers as to whether or not it is considered fully functional. I don't personally use this feature so I'm not testing it, as I use RUMLogNG on Mac which worked fine before this. |
It won't until QLog implements support for parsing JS8Call style messages. |
As Chris said, I don't use this feature, but am glad to test it, and have been hoping for several years that GridTracker would support JS8Call, so will probably use it when it's up & running! 73, |
|
CI build fails: |
shoot, committed not just the doxygen stuff but also my hacks.. reverting |
|
Builds locally now. |
What is the status of modifications to GridTracker? Is there a build to test with on Windows & Linux? 73, |
|
GT branch is here: https://gitlab.com/gridtracker.org/gridtracker2/-/merge_requests/18 We have issues with Windows signing in gitlab right now which is why there are no artifacts right now.. but you can just run it from the branch if you want:
|
I'll wait until you get a Linux and/or Windows build done, then I'll do testing on both platforms. 73, |
|
Somebody please slap the Ready to merge label on this one when all testing is done and it's ready to go. |
|
@chrbayer84 what is the status of this? This is the oldest outstanding mergable PR, and I would like to merge this if you can fix the conflicts and get it ready for merge. I have a general code cleanup PR that is going to seriously break this one, since it moves all the files in the source tree to class group folders and gets rid of all the loose source and header files in the root of the tree. For instance, instead of the So I'll leave this up to you. If this is ready for merge, but just not functional until we hear from the Grid Tracker guys, fix the conflicts and we'll merge it now and worry about the Grid Tracker compatibility later. Which puts the workload on my shoulders to fix the conflicts in the code cleanup PR. Or, if you would rather wait, it will require more work to rebase this one to fit the new code tree. The choice is yours. |
|
I'll just comment that in my testing, it appeared that the JS8Call component was working, but the GridTracker software needed to fix some parsing problems in the data sent to it. @chrbayer84 is that a correct understanding? Is the JS8Call component most likely good as it is? Yes, @Chris-AC9KH it would be good to start preparing for a 2.5 release soon. There is a lot to offer that the community would appreciate. 73, |
|
@chrbayer84 as a reference to how bad this would be, this is what the new source tree looks like after the general code cleanup is done. So it is a pretty significant change, but something we've been wanting to get done for a very long time.
|
|
@Joe-K0OG yes, before a release/2.5.0 branch is created, I'd like to have the codebase cleaned up for 2.6.0 development so the code is more understandable. We're still going to break the mainwindow monster down into separate classes, and this PR adds more stuff yet to that mainwindow monster. But that is destined for 2.6.0 development to break that down before the monster grows two heads. |
This will be GREAT! I'm not a code-jockey and when I look at the code as it is my head spins to near explosive angular velocities! It's probably not quite as bad for you guys, but I would like to be able to look at it and have it make sense. Your source tree looks SOOO much better! Thanks for the excellent work @Chris-AC9KH! 73, |
|
We (the gridtracker team) are working on getting GT ready to parse a whole litany of JS8 messages. However none of that work seems to require Any backtracking in the js8call code base. In other words this can be merged. Unfortunately the build in gitlab is screwed up so we can't provide installable artifacts for you guys to test against.. |
|
I vote to merge after CI finishes. |
|
Excellent. As soon as it passes CI, @wmiler could you please merge this? There's another outstanding one from @mgochoa57 as well that can be merged. Then I can rebase the big one and fix it up. The outstanding one from @aknrdureegaesr is pretty basic, he indicated he had a different idea anyway. So that one should be pretty trivial to handle later. |

JS8Call used to support the WSJT-X reporting protocol that emits current band and mode, decoded stations as well as QSO logging events. That protocol was removed from mainline JS8Call at some point, cutting off loggers and programs like Gridtracker from received data from JS8Call. This PR adds back support for that protocol. It's a backport of the code from WSJT-X-improved 3.0.0 with hooks added back. Multicast is supported.
Tested with Gridtracker 25.0914.1 and QLog 0.35.1 on Linux and MacOS.