Invalid device buffer to CPU pointer conversions are allowed

[maleadt]

Since the recent Memory rework, we keep device buffers around so that we can accurately call clSetKernelArgMemPointerINTEL:

julia> a = CLArray(rand(10))
10-element CLArray{Float64, 1, OpenCL.cl.UnifiedDeviceMemory}:
 0.5593082405441803
 0.8767556355465198
 0.2656337336514243
 0.832560239049259
 0.6293889437226778
 0.05840696344831575
 0.7670513212164926
 0.08970380371893028
 0.6208410671874172
 0.6153346169670701

julia> cl.unsafe_clconvert(Ptr{Float64}, cl.clconvert(Ptr{Float64}, a))
UnifiedDeviceMemory(80 bytes at 0x0000000108d04a00)

This is not a problem per se, but the subsequent (late) conversion doesn't implement the same semantics:

# what it should do
julia> Base.unsafe_convert(Ptr{Float64}, Base.cconvert(Ptr{Float64}, a))
ERROR: ArgumentError: cannot take the CPU address of a CLArray{Float64, 1, OpenCL.cl.UnifiedDeviceMemory}

# instead, calls are permitted
julia> clcall(mmul, Tuple{Int32, Int32, Int32, Ptr{Float32}, Ptr{Float32}, Ptr{Float32}},
              1, 1, 1, a, a, a)
OpenCL.Event(@0x00006000037d1950)