Target is an ATmega32U4 (Arduino Micro) over JTAG, with an Atmel-ICE. I don't know whether this also occurs with JTAGICE3.

avarice -d -C -4 -B 2M localhost:6000 exits with an error upon first contact by GDB, but avarice -d -4 -B 2M localhost:6000 seems to work.

I've traced this down to a failure to use CMD_AVR8_ATTACH (CMD3_START_DEBUG in jtag3.h) in capture mode. I have preliminary patches to fix this. The EDBG protocol reference can be quite helpful, even if it does seem inaccurate or incomplete in places.

jtag3::initJtagOnChipDebugging does use that attach command, but only on DebugWire or for XMEGA. Oddly enough, it turns out to be unnecessary there, because leaving progmode during a debug session implicitly does a reset and attach (if OCD is enabled). (On DebugWire, the attempt to enable OCD bypasses progmode, because DebugWire doesn't do that, but DebugWire implicitly attaches on activation of the physical interface, which I've determined experimentally, but isn't documented.)

GDB: <g>

GDB: (Registers)Read 32 bytes from 0x800000
jtagRead
command "read memory" [0x12, 0x21]
0E 00 08 00 12 21 00 20 00 00 00 00 20 00 00 00
Received 0x81 0x11 0x00 0x07 0x0e 0x08
read:  0e 08 00 12 a0 00 32

Got message seqno 8 (command_sequence == 8)
response: 12 A0 00 32

0x32 is AVR8_FAILURE_ILLEGAL_STATE, which avarice handles by attempting a stop command, which also fails:

command "stop" [0x12, 0x31]
0E 00 09 00 12 31 00 01
Received 0x81 0x11 0x00 0x07 0x0e 0x09
read:  0e 09 00 12 a0 00 23

Got message seqno 9 (command_sequence == 9)
response: 12 A0 00 23

0x23 is AVR8_FAILURE_NOT_ATTACHED.

Full log:

jtag-c-32u4.txt