Protocol negotiation isn't correctly determining protocol version for at least some Cassandra versions

[absurdfarce]

Originally noted when this sample client:

package main

import (
	"fmt"

	gocql "github.com/apache/cassandra-gocql-driver/v2"
)

func main() {

	cluster := gocql.NewCluster("127.0.0.1")
	cluster.Logger = gocql.NewLogger(gocql.LogLevelDebug)
	//cluster.ProtoVersion = 4
	session, err := cluster.CreateSession()
	if err != nil {
		fmt.Println(err)
	}

	var dse_version string
	query := session.Query(`select dse_version from system.local`).Consistency(gocql.One)
	if err = query.Scan(&dse_version); err != nil {
		fmt.Println(err)
	}

	fmt.Println("Version is: ", dse_version)
}

built against v2.0.0-rc1 fails against DSE 6.9.0 with the following error:

$ ./version 
2025/09/24 15:15:39 DBG gocql: Discovered protocol version using host. protocol_version=5 host_addr=127.0.0.1 host_id=
2025/09/24 15:15:39 INF gocql: Discovered protocol version. protocol_version=5
2025/09/24 15:15:39 WRN gocql: Control connection error. host_addr=127.0.0.1 host_id= err=gocql: crc24 mismatch in frame header, computed: 7052973, got: 0
2025/09/24 15:15:39 DBG gocql: Reconnecting the control connection. 
2025/09/24 15:15:39 ERR gocql: Unable to connect to any ring node, control connection falling back to initial contact points. err=
2025/09/24 15:15:39 INF gocql: Control connection setup failed after connecting to host. host_addr=127.0.0.1 port=9042 host_id= err=could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 7052973, got: 0
gocql: unable to create session: unable to connect to initial hosts: could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 7052973, got: 0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x201c94]

goroutine 1 [running]:
github.com/apache/cassandra-gocql-driver/v2.(*Query).defaultsFromSession(...)
        /home/mersault/go/pkg/mod/github.com/apache/cassandra-gocql-driver/v2@v2.0.0-rc1/session.go:1021
github.com/apache/cassandra-gocql-driver/v2.(*Session).Query(0x0, {0x2f772f, 0x24}, {0x0, 0x0, 0x0})
        /home/mersault/go/pkg/mod/github.com/apache/cassandra-gocql-driver/v2@v2.0.0-rc1/session.go:436 +0x94
main.main()
        /home/mersault/work/scratch/gocql-dse/version.go:20 +0x1ec

What's happening here is that the driver is trying to use some session-level values as defaults for the query struct and is failing to do so because those values are nil. Those values are nil because the session setup never happens, and that fails because we're trying to apply protocol v5 frame decoding rules to a DSE server which (at best) speaks protocol DSEv2.

This begs the question though... why did the client apply v5 framing rules in this case? Clearly it did so because it detected protocol v5 was agreeable to both parties (see the log messages in the snippet above)... but that's clearly false. So what's going on here? The protocol detection logic (it's not really a negotiation as such) lives here. When v5 support was added we wound up modifying the session-level connection config to v5 as part of the detection logic (and we don't reset it if detection fails) which isn't great (since we're modifying session-level state in order to try and perform detection) but that isn't the big problem.

Much more significant is the fact that we consider a protocol workable if we're able to dial the remote host without errors using that protocol version. The thing is that "dial the remote host" here means establish a socket connection and exchange STARTUP messages (and responses). And according to the protocol v5 spec these messages are always unframed. That means we'll always be able to handle those responses for any server... and since we're starting with protocol v5 we'll always wind up returning v5 as the protocol version.

To confirm I tried the exact same code against Cassandra 3.11.9 which does not support protocol v5. The results were basically the same:

$ ./version 
2025/09/24 15:31:57 DBG gocql: Discovered protocol version using host. protocol_version=5 host_addr=127.0.0.1 host_id=
2025/09/24 15:31:57 INF gocql: Discovered protocol version. protocol_version=5
2025/09/24 15:31:57 WRN gocql: Control connection error. host_addr=127.0.0.1 host_id= err=gocql: crc24 mismatch in frame header, computed: 6994124, got: 0
2025/09/24 15:31:57 DBG gocql: Reconnecting the control connection. 
2025/09/24 15:31:57 ERR gocql: Unable to connect to any ring node, control connection falling back to initial contact points. err=
2025/09/24 15:31:57 INF gocql: Control connection setup failed after connecting to host. host_addr=127.0.0.1 port=9042 host_id= err=could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 6994124, got: 0
gocql: unable to create session: unable to connect to initial hosts: could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 6994124, got: 0
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x201c94]

goroutine 1 [running]:
github.com/apache/cassandra-gocql-driver/v2.(*Query).defaultsFromSession(...)
        /home/mersault/go/pkg/mod/github.com/apache/cassandra-gocql-driver/v2@v2.0.0-rc1/session.go:1021
github.com/apache/cassandra-gocql-driver/v2.(*Session).Query(0x0, {0x2f772f, 0x24}, {0x0, 0x0, 0x0})
        /home/mersault/go/pkg/mod/github.com/apache/cassandra-gocql-driver/v2@v2.0.0-rc1/session.go:436 +0x94
main.main()
        /home/mersault/work/scratch/gocql-dse/version.go:20 +0x1ec

[joao-r-reis]

I believe this is fixed by #1906, we haven't been able to merge it yet but you can try it out on your sample code and see if it fixes it

[absurdfarce]

I've been looking at this for some time @joao-r-reis and to be honest I'm still not sure if it's exactly the same bug.

The really weird thing about this is that when I run my code against DSE 6.9.0 I receive no protocol error back from the server. Note the error output here:

$ ./version 
2025/09/25 00:16:44 DBG gocql: Discovered protocol version using host. protocol_version=5 host_addr=127.0.0.1 host_id=
2025/09/25 00:16:44 INF gocql: Discovered protocol version. protocol_version=5
2025/09/25 00:16:44 WRN gocql: Control connection error. host_addr=127.0.0.1 host_id= err=gocql: crc24 mismatch in frame header, computed: 7052973, got: 0
2025/09/25 00:16:44 DBG gocql: Reconnecting the control connection. 
2025/09/25 00:16:44 ERR gocql: Unable to connect to any ring node, control connection falling back to initial contact points. err=
2025/09/25 00:16:44 INF gocql: Control connection setup failed after connecting to host. host_addr=127.0.0.1 port=9042 host_id= err=could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 7052973, got: 0
gocql: unable to create session: unable to connect to initial hosts: could not retrieve control host info: gocql: crc24 mismatch in frame header, computed: 7052973, got: 0

When I try the exact same client code using your branch I see that v4 is set after parsing the protocol error... which is entirely consistent with what I would expect based on your changes:

$ ./version 
2025/09/25 00:18:09 DBG gocql: Request failed because context elapsed out on connection. host_id= addr=127.0.0.1 ctx_err=context canceled
2025/09/25 00:18:09 DBG gocql: Discovered protocol version using host after parsing protocol error. protocol_version=4 host_addr=127.0.0.1 host_id=
2025/09/25 00:18:09 INF gocql: Discovered protocol version. protocol_version=4
2025/09/25 00:18:09 DBG gocql: Added control host (session initialization). host_addr=127.0.0.1 host_id=402d725e-7e6c-4f79-b6ec-ba29ff55f33f
2025/09/25 00:18:09 INF gocql: Control connection connected to host. host_addr=127.0.0.1 host_id=402d725e-7e6c-4f79-b6ec-ba29ff55f33f
2025/09/25 00:18:09 INF gocql: Refreshed ring. ring=[127.0.0.1-402d725e-7e6c-4f79-b6ec-ba29ff55f33f:UP]
2025/09/25 00:18:09 INF gocql: Session initialized successfully. 
2025/09/25 00:18:09 DBG gocql: Pool connected to node. host_addr=127.0.0.1 port=9042 host_id=402d725e-7e6c-4f79-b6ec-ba29ff55f33f
2025/09/25 00:18:09 DBG gocql: Logging number of connections of pool after filling stopped. host_addr=127.0.0.1 host_id=402d725e-7e6c-4f79-b6ec-ba29ff55f33f count=2
Version is:  6.9.0

For sake of completeness here the client for both trials above is:

package main

import (
	"fmt"

	gocql "github.com/apache/cassandra-gocql-driver/v2"
)

func main() {

	cluster := gocql.NewCluster("127.0.0.1")
	cluster.Logger = gocql.NewLogger(gocql.LogLevelDebug)
	session, err := cluster.CreateSession()
	if err != nil {
		fmt.Println(err)
	}

	var dse_version string
	query := session.Query(`select dse_version from system.local`).Consistency(gocql.One)
	if err = query.Scan(&dse_version); err != nil {
		fmt.Println(err)
	}

	fmt.Println("Version is: ", dse_version)
}

The part I cannot understand is why your code gets to the protocol error handling at all. Your code is pretty clearly getting a non-nil err value in this block while my code is not... while my code is getting a nil err value in the equivalent block in v2.0.0-rc1.

I feel like I'm very much still missing something.

[absurdfarce]

Oh wait, maybe... that's a side effect of the beta protocol thing for v5? I'm not seeing an error coming off of the dial() op because my version is still including the beta flag for v5 (which DSE 6.9.0 expects... ?) and as a result it doesn't have a problem going through a STARTUP req/response cycle. Since yours doesn't include the beta flag the server freaks out and sends back the protocol error... which you now have handling for in your branch.

Okay, assuming all of that holds up for you then yeah, I think this is the same issue you reference in CASSGO-88 (just from a slightly different perspective).

[joao-r-reis]

The symptom is exactly the same so I believe it's the same issue, the driver sends v5 with beta flag, the server (DSE 6.9, Cassandra 3.11) replies with success because it does support v5 with beta flag but then things go bad because the driver is expecting the new frame format which isn't there in v5 beta (see the crc24 mismatch part of the error)

[worryg0d]

Hello.

I tried to reproduce this issue against the 2.0.0 gocql release and didn't get any panics. The driver successfully detected the latest supported protocol version as v4 for DSE 6.9 / Cassandra 3.11, so no v5 checksum mismatches there.

When I was implementing support for v5 protocol, I somehow missed this beta flag, so this is my bad. I actually expected this change to break something for older Cassandra versions because gocql supported a v5 beta, which is significantly different from v5 proto final version, but I assumed that it was fine since gocql didn't support older Cassandra versions, and it is not being tested against them.

Anyway, thanks @joao-r-reis for fixing it!

[joao-r-reis]

Fixed in #1906