According to the manuals for the 817 and 817ND the cat commands are identical.
Could you please run wsjt-x using hamlib net rigctl and run rigctld-wsjtx with your rig, then send the output around the time this error occurs?
de Mike W9MDB
From: "dl8mmr@... [WSJTX]" <WSJTX@...>
Sent: Wednesday, November 23, 2016 1:50 PM
Subject: Re: [WSJTX] yaesu ft-817nd
I'm using the current version 1.7.x and i was also try the the version 1.6.x the os is mac os x sierra.
if i'm select the rig ft-817 it happen that the application ask me every time to check the rig configuration.
and i'get the info that that command vfoA change to vfoB not work. i'think this is the command to test
the cat interface. this command work for the old ft-817 but not for the ft-817nd.
the rigctl i'dont use. i'think it need only a configuration file for the ft-817nd like in fldigi, there exist
also a file ft-817.xml and ft-817nd.xml
macloggerdx and fldigi, flrig and some other software work very fine.