Maybe you should go over the baud etc settings in Device Manager?
This is a program (trialware) that monitors Windows serial ports. You can use this to capture any data sent and received through any Windows COM port. If DosBOX is sending something to serial ports then it has to go through the Windows serial driver and the data will be captured by this program.