locked Decoding UDP messages from WSJT-X


Roger Sjölin
 

Searching for source code in visual basic to decode the messages from WSJT-X UDP server.
I manage to connect to the server and get the messages but they are in this
format and I don´t know how to decode it
????         WSJT-X - Slice-A??`   ?         ?   ~   US5NFC EA5SE R-20.


Bill Somerville
 

On 17/03/2019 11:54, Roger Sjölin wrote:
Searching for source code in visual basic to decode the messages from WSJT-X UDP server.
I manage to connect to the server and get the messages but they are in this
format and I don´t know how to decode it
????         WSJT-X - Slice-A??`   ?         ?   ~ US5NFC EA5SE R-20.
Hi Roger,

the format of the WSJT-X UDP messages are described in this document:

https://sourceforge.net/p/wsjt/wsjtx/ci/master/tree/NetworkMessage.hpp

73
Bill
G4WJS.


Bob
 

What flavor of VB?  If you want it in VB6, send me a 'real' eMail address and I'll put together a crude, but functional, test project for you. There's lot of voodoo (including secret numbers) in the message structure, and lot of overhead for a handful of data.  SeventyThree(s).

On March 17, 2019 at 7:54 AM Roger Sjölin <rogersjolin@...> wrote:

Searching for source code in visual basic to decode the messages from WSJT-X UDP server.
I manage to connect to the server and get the messages but they are in this
format and I don´t know how to decode it
????         WSJT-X - Slice-A ??`   ?         ?   ~   US5NFC EA5SE R-20. -=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2719): https://groups.io/g/WSJTX/message/2719
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits<br />(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [k4cy@...]
-=-=-=-=-=-=-=-=-=-=-=-


Roger Sjölin
 
Edited

Hi Bob!
I would be very happy if You could help me with a VB6 code for this decoding. My email is
rogersjolin(AT)gmail.com.
Thanks in advance
Roger
www.sa0and.com


Andy Talbot
 

and to complement that,  here is source code for decoding UDP messages written in PowerBasic
http://www.g4jnt.com/DropF/monudp.bas    is a routine that monitors UDP messages broadcast from WSJT-X



On Sun, 17 Mar 2019 at 13:53, Roger Sjölin <rogersjolin@...> wrote:

[Edited Message Follows]

Hi Bob!
I would be very happy if You could help me with a VB6 code for this decoding. My email is
rogersjolin@....
Thanks in advance
Roger
www.sa0and.com -=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2725): https://groups.io/g/WSJTX/message/2725
Mute This Topic: https://groups.io/mt/30462433/393564
-=-=-
Moderated by The Two Brits<br />(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3816196/191523242/xyzzy  [andy.g4jnt@...]
-=-=-=-=-=-=-=-=-=-=-=-


Bob
 

Problem is, last time I tried, gmail doesn't accept zipped attachments. You have a different address?  Should take about 30 minutes to put all the pieces together. SeventyThree(s).

On March 17, 2019 at 9:52 AM Roger Sjölin <rogersjolin@...> wrote:

Hi Bob!
I would be very happy if You could help me with a VB6 code for this decoding. My email is rogersjolin@....
Thanks in advance
Roger
www.sa0and.com -=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2725): https://groups.io/g/WSJTX/message/2725
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits<br />(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [k4cy@...]
-=-=-=-=-=-=-=-=-=-=-=-


Ron Herring <w7hd@...>
 

simple workaround for zip files, just rename the extension from zip to piz.  Then it will go through gmail just fine.  When receiving it, change the extension name back to zip.

Ron W7HD

On 3/17/19 7:12 AM, Bob wrote:

Problem is, last time I tried, gmail doesn't accept zipped attachments. You have a different address?  Should take about 30 minutes to put all the pieces together. SeventyThree(s).



-- 
Ron W7HD - NAQCC#7587 OMISS#9898 KX3#6966 LinuxUser#415320


Roger Sjölin
 

Ok

You can try sa0and@...

/Roger


Bill Somerville
 

On 17/03/2019 13:35, Bob wrote:
There's lot of voodoo (including secret numbers) in the message structure, and lot of overhead for a handful of data.
Bob,

examples of unnecessary overhead please?

What secret numbers are you referring to?

73
Bill
G4WJS.


Andy Talbot
 

I found decoding it very straightforward after reading the documentation referred to.
It really is a useful facility, allowing anyone to write another layer of code to do anything they want to with decoded messages.



On Sun, 17 Mar 2019 at 17:48, Bill Somerville <g4wjs@...> wrote:
On 17/03/2019 13:35, Bob wrote:
> There's lot of voodoo (including secret numbers) in the message
> structure, and lot of overhead for a handful of data.

Bob,

examples of unnecessary overhead please?

What secret numbers are you referring to?

73
Bill
G4WJS.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2732): https://groups.io/g/WSJTX/message/2732
Mute This Topic: https://groups.io/mt/30462433/393564
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3816196
-=-=-
Moderated by The Two Brits
(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3816196/191523242/xyzzy  [andy.g4jnt@...]
-=-=-=-=-=-=-=-=-=-=-=-


Bob
 

Without sitting down and counting then, let's say a typical UDP message is 60 something bytes. The first 16 (or so) bytes of each message is this information:


Public Type HdrDataInfo
MagicNum As Double
Schema As Integer
MsgType As Integer
Id As String
End Type



On March 17, 2019 at 1:48 PM Bill Somerville <g4wjs@...> wrote:


On 17/03/2019 13:35, Bob wrote:
There's lot of voodoo (including secret numbers) in the message
structure, and lot of overhead for a handful of data.

Bob,

examples of unnecessary overhead please?

What secret numbers are you referring to?

73
Bill
G4WJS.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2732): https://groups.io/g/WSJTX/message/2732
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits
(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [k4cy@...]


Bob
 

Bill, Let me try that again without fat fingering the 'send' shortcut ...

A UDP message is typicall in the order of 60 bytes. The first 16 (or so are duplicated in each and every message. Here is the message header structure

Public Type HdrDataInfo
MagicNum As Double
Schema As Integer
MsgType As Integer
Id As String
End Type

The only information if value (everything else is in the periodic heartbeat message) is the message type.

As to the secret number. Fist four bytes of the header are a number derived from the calculation 16^6*byte(0) + 16^4*byte(1) + 16^2*byte(3) + byte(4).

Again, all this fancy fluff is repeated in every message. SeventyThree(s).

On March 17, 2019 at 3:39 PM Bob <k4cy@...> wrote:


Without sitting down and counting then, let's say a typical UDP message is 60 something bytes. The first 16 (or so) bytes of each message is this information:


Public Type HdrDataInfo
MagicNum As Double
Schema As Integer
MsgType As Integer
Id As String
End Type


On March 17, 2019 at 1:48 PM Bill Somerville < g4wjs@...> wrote:


On 17/03/2019 13:35, Bob wrote:
There's lot of voodoo (including secret numbers) in the message
structure, and lot of overhead for a handful of data.
Bob,

examples of unnecessary overhead please?

What secret numbers are you referring to?

73
Bill
G4WJS.

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2732): https://groups.io/g/WSJTX/message/2732
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits
(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [ k4cy@...]

---

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2734): https://groups.io/g/WSJTX/message/2734
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [k4cy@...]
-=-=-=-=-=-=-=-=-=-=-=-


Bill Somerville
 

Bob,

I am not sure if you are citing the following as unnecessary or as containing secret numbers. Below are some comments on the content you attempt to list.

On 17/03/2019 19:39, Bob wrote:

Public Type HdrDataInfo

you should not define a fixed length type here as one field is in fact variable length.

MagicNum As Double

This field is a 32-bit unsigned number and is announces the datagram as belonging to the WSJT message protocol. You may or may not understand this to mean that other protocols may use this channel simultaneously with WSJT-X clients and servers that may be interested in it.

Schema As Integer

This field is necessary so that network nodes can collaborate without all being built to support exactly the same underlying data formats. For example if the way date-time fields were to be encoded on the wire then recipients of messages need to know which vintage of the underlying data formats the sender was using. Remember that this protocol is designed to be used by diverse applications developed by different authors independently yet all must interoperate in an ad hoc many-to-many mesh arrangement in perpetuity.

MsgType As Integer

Well obviously the type selector of the message must be included, how else would recipients know what they have? Clearly this field is not the same for all messages.

Id As String

This is simply the sender of the message, most important when there are multiple WSJT-X instances running concurrently. This field is a variable length qtf-8 string with a prepended 32-bit unsigned number length. Clearly the value of this field is not the same for all messages.

End Type


So in summary, the header if you wish to call it such is two 32-bit unsigned numbers, i.e. a total of 64-bits, the rest of the message is payload whose content is dependent on the specific message type.

73
Bill
G4WJS.


Bill Somerville
 

On 17/03/2019 19:53, Bob wrote:
As to the secret number.  Fist four bytes of the header are a number derived from the calculation 16^6*byte(0) + 16^4*byte(1) + 16^2*byte(3) + byte(4).

Again, all this fancy fluff is repeated in every message.

Bob,

there is no secret here. The "magic number" field is documented here:

https://sourceforge.net/p/wsjt/wsjtx/ci/master/tree/NetworkMessage.hpp#l17

the value is simply a unique byte string with the hex value 0xadbccbda. A commonly used technique to verify that the following payload is in fact belonging to the WSJT UDP message protocol. Note again that there is no requirement for the channel to be used just for WSJT protocol messages, recipients should simply ignore any datagram that does not start with this byte sequence, or more generally with a byte string that they do not recognize.

Your characterization of "fancy fluff" is showing that you do not understand the full intent of the WSJT message protocol and that it is needed for the channel to be flexible and extendible.

73
Bill
G4WJS.


Bob
 

Bill, to you it may well be a 'Picasso'. To me it's just a childish painting. SeventyThree(s).

On March 17, 2019 at 4:10 PM Bill Somerville <g4wjs@...> wrote:

On 17/03/2019 19:53, Bob wrote:
As to the secret number.  Fist four bytes of the header are a number derived from the calculation 16^6*byte(0) + 16^4*byte(1) + 16^2*byte(3) + byte(4).

Again, all this fancy fluff is repeated in every message.

Bob,

there is no secret here. The "magic number" field is documented here:

https://sourceforge.net/p/wsjt/wsjtx/ci/master/tree/NetworkMessage.hpp#l17

the value is simply a unique byte string with the hex value 0xadbccbda. A commonly used technique to verify that the following payload is in fact belonging to the WSJT UDP message protocol. Note again that there is no requirement for the channel to be used just for WSJT protocol messages, recipients should simply ignore any datagram that does not start with this byte sequence, or more generally with a byte string that they do not recognize.

Your characterization of "fancy fluff" is showing that you do not understand the full intent of the WSJT message protocol and that it is needed for the channel to be flexible and extendible.

73
Bill
G4WJS.


 

-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2737): https://groups.io/g/WSJTX/message/2737
Mute This Topic: https://groups.io/mt/30462433/849411
Mute #wsjtx: https://groups.io/mk?hashtag=wsjtx&subid=3528040
-=-=-
Moderated by The Two Brits<br />(Andy K3UK and Roger G4HZA)
-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/3528040/852945762/xyzzy [k4cy@...]
-=-=-=-=-=-=-=-=-=-=-=-


 


Greg Beam, KI7MT <ki7mt01@...>
 

Thanks for the example Andy. I was trying to deconstruct the messages without checking the id first .. Dooh!

What I looked at doing (some time back) was implementing a simple service for logging QSO's / decodes to a custom data table and remote monitoring of decodes cross-platform.

Your example gave me the clues I was missing :-)

73's
Greg, KI7MT

On 3/17/19 7:57 AM, Andy TALBOT wrote:
and to complement that,  here is source code for decoding UDP messages written in PowerBasic
http://www.g4jnt.com/DropF/monudp.bas    is a routine that monitors UDP messages broadcast from WSJT-X
Andy
www.g4jnt.com <http://www.g4jnt.com>


Roger Sjölin
 
Edited

Now I'm almost done. The only thing left is to decode Delta time in type 2 message.
Can't figure out how to do this with these data
63 211 51 51 64 00 00 00


Michael Black
 

8 bytes of what you show is a double format

I believe you said you're coding in Visual Basic, right?


de Mike W9MDB




On Thursday, March 21, 2019, 10:12:27 AM CDT, Roger Sjölin <rogersjolin@...> wrote:


[Edited Message Follows]

Now I'm almost done. The only thing left is to decode Delta time in type 2 message.
Can't figure out how to do this with these data
63 211 51 51 64 00 00 00 00
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.

View/Reply Online (#2755): https://groups.io/g/WSJTX/message/2755
Mute This Topic: https://groups.io/mt/30462433/212453
-=-=-
Moderated by The Two Brits<br />(Andy K3UK and Roger G4HZA)

-=-=-
Group Owner: WSJTX+owner@groups.io
Unsubscribe: https://groups.io/g/WSJTX/leave/484761/1473993136/xyzzy [mdblack98@...]
-=-=-=-=-=-=-=-=-=-=-=-


Roger Sjölin
 

Yes, in visual basic 2107


Bill Somerville
 

On 21/03/2019 15:37, Michael Black via Groups.Io wrote:
8 bytes of what you show is a double format

I believe you said you're coding in Visual Basic, right?

https://stackoverflow.com/questions/15782705/convert-8-bytes-array-into-double

de Mike W9MDB
Hi Mike,

close but not quite right. The data is in network byte order so on Intel an endian swap is required.

73
Bill
G4WJS.