Various errors in SPARQL update

[doriantaylor]

Just registering this here as I'm in the middle of something, but I'm finding various errors when trying to use SPARQL update. Here's what I'm working with:

PREFIX rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dct:  <http://purl.org/dc/terms/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX ibis: <https://vocab.methodandstructure.com/ibis#>
PREFIX ci:   <https://vocab.methodandstructure.com/content-inventory#>
PREFIX cgto: <https://vocab.methodandstructure.com/graph-tool#>
PREFIX itcv: <https://vocab.methodandstructure.com/intertwingler#>

WITH <https://placeholder.ibis.makethingsmakesense.com/>
INSERT {
  ?s skos:inScheme <urn:uuid:cf16f32c-824a-47c2-b176-905c11f8978c> .
  # ?s ci:alias ?a .
}
WHERE {
  # BIND (?a AS (IRI(REPLACE(str(?s), "^urn:uuid:", "https://placeholder.ibis.makethingsmakesense.com/", "i"))))
  # ?s a ?o .
  # FILTER (?o IN (skos:Concept, ibis:Issue, ibis:Position, ibis:Argument))
  { ?s a skos:Concept } UNION { ?s a ibis:Issue } UNION { ?s a ibis:Position } UNION { ?s a ibis:Argument }
} ;

WITH <https://placeholder.ibis.makethingsmakesense.com/>
INSERT {
  ?s dct:creator <urn:uuid:ca637d4b-c11b-4152-be98-bde602e7abd4> .
}
WHERE {
  # { ?s a ?o } OPTIONAL { ?s dct:creator ?c }
  # FILTER (!BOUND(?c) && ?o IN (skos:Concept, ibis:Issue, ibis:Position, ibis:Argument))
  { ?s a skos:Concept } UNION { ?s a ibis:Issue } UNION { ?s a ibis:Position } UNION { ?s a ibis:Argument }
  OPTIONAL { ?s dct:creator ?c }
  FILTER (!bound(?c))
} ;

MOVE <https://placeholder.ibis.makethingsmakesense.com/> TO <dns:placeholder.methodandstructure.com>

This has been massaged into a non-breaking state but as you can see the commented-out bits are where the problems are:

1. the IN operator doesn't seem to include ?o in the algebra and thus evaluates to false (ie nothing selected and therefore nothing inserted)
2. that BIND just straight blows up during parsing with an error saying the operand is nil
3. I also had accidentally left off the dct: prefix declaration and it didn't complain about the unbound prefix
4. I guess the internal state of parsed queries get changed? because I can't seem to execute a parsed query twice without it raising
5. I also get a 409 error in open_url (???) if I try to run the query twice on the repo, even if I reparse it.

Anyway, again, just registering this now but I'll circle back when I have some more time to look into it.

[gkellogg]

The parser was changed a bit a go to be based on a packrat/PEG parser rather than LL(1), and it's possible that something that wasn't properly tested before got missed. You can use the LL(1) parser (if you're not using SPARQL 1.2 features) using the use11 option (or by using SPARQL::Grammar::Parser11 directly).

SPARQL.parse(query_string, use11: true)

Also, it's useful to serialize that to S-Expressions, which should generally match what you can get from the JENA ARQ parser.

I also had accidentally left off the dct: prefix declaration and it didn't complain about the unbound prefix

Looks like there's no check for that

sparql/lib/sparql/grammar/parser.rb

Lines 2926 to 2933 in 91c50c1

	def ns(prefix, suffix)
	base = prefix(prefix).to_s
	suffix = suffix.to_s.gsub(PN_LOCAL_ESC) {|esc| esc[1]} if
	suffix.to_s.match?(PN_LOCAL_ESC)
	suffix = suffix.to_s.sub(/^\#/, "") if base.index("#")
	debug {"ns(#{prefix.inspect}): base: '#{base}', suffix: '#{suffix}'"}
	iri(base + suffix.to_s)
	end

It should probably do something more like the Turtle parser:

##
# Expand a PNAME using string concatenation
def pname(prefix, suffix)
  # Prefixes must be defined, except special case for empty prefix being alias for current @base
  base = if prefix(prefix)
    prefix(prefix).to_s
  elsif prefix.to_s.empty? && !validate?
    base_uri.to_s
  else
    error("undefined prefix", production: :pname, token: prefix)
    ''
  end

...

I guess the internal state of parsed queries get changed? because I can't seem to execute a parsed query twice without it raising

A simple test case for this would be useful. The parsed query is executed, and that may add solutions to the internal state, which could persist between executions, but I'd need to look more closely to be sure.

I also get a 409 error in open_url (???) if I try to run the query twice on the repo, even if I reparse it.

Also, a simple test case for this would be helpful. Thanks!

[doriantaylor]

OK, one mystery solved: the 409 looks like it's actually intended behaviour (at least on this side; it's hitting my site and I make liberal use of that error code), although it would be valuable to be able to turn that off. It's happening because of the MOVE command is renaming the named graph and so there aren't any statements in it anymore, which triggers load.

(Actually, is that what the spec says is supposed to happen? Try to dereference the graph?)

Also, here is what the SSE of the first stanza looks like:

(prefix                         
 (
  (rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>)
  (dct: <http://purl.org/dc/terms/>)
  (skos: <http://www.w3.org/2004/02/skos/core#>)
  (ibis: <https://vocab.methodandstructure.com/ibis#>)
  (ci: <https://vocab.methodandstructure.com/content-inventory#>)
  (cgto: <https://vocab.methodandstructure.com/graph-tool#>)
  (itcv: <https://vocab.methodandstructure.com/intertwingler#>))
 (update
  (modify
   (with <https://placeholder.ibis.makethingsmakesense.com/>
    (filter (in skos:Concept ibis:Issue ibis:Position ibis:Argument) (bgp (triple ?s a ?o)))
    (insert ((triple ?s skos:inScheme <urn:uuid:cf16f32c-824a-47c2-b176-905c11f8978c>)))) )) )

If I understand correctly, that should be (in ?o skos:Concept…). Looks like ?o is just not getting added to the operands. This happens whether :use11 is set or not.

[doriantaylor]

Okay, I see what's happening with the IN operator:

sparql/lib/sparql/grammar/parser11.rb

Lines 1614 to 1627 in 91c50c1

	production(:RelationalExpression) do |input, data, callback|
	if data[:_Compare_Numeric]
	add_prod_datum(:Expression, SPARQL::Algebra::Expression.for(data[:_Compare_Numeric].insert(1, *data[:Expression])))
	elsif data[:in]
	expr = (data[:Expression] + data[:in]).reject {|v| v.eql?(RDF.nil)}
	add_prod_datum(:Expression, SPARQL::Algebra::Expression.for(expr.unshift(:in)))
	elsif data[:notin]
	expr = (data[:Expression] + data[:notin]).reject {|v| v.equal?(RDF.nil)}
	add_prod_datum(:Expression, SPARQL::Algebra::Expression.for(expr.unshift(:notin)))
	else
	# NumericExpression with no comparitor
	add_prod_datum(:Expression, data[:Expression])
	end
	end

The unbound RDF::Query::Variable.new(?o).eql? RDF.nil will return true, which will cause it to be removed from the operand list.

Also, is that :notin supposed to be equal? rather than eql?? Or the other way around? Or should they both be equal? (or eql?)?

(Is that what the spec says? that you aren't supposed to be able to test against rdf:nil?)

[doriantaylor]

OK another update: the BIND blowup is my fault: it should read BIND (IRI(REPLACE(str(?s), "^urn:uuid:", "https://placeholder.ibis.makethingsmakesense.com/", "i")) AS ?a), not the other way around. That said, a more helpful error message would be useful.

(Also note that while the BIND no longer blows up, ?a does not appear to be transmitted to the INSERT part of the statement.)

(EDITED TO ADD: it indeed works but only if you put it after the FILTER.)

[gkellogg]

The logic for IN and NOTIN parsing changed in the SPARQL 1.2 version of the parser, but you seem to have been testing against parser11, which uses the LL(1) version of the parser, where the internal logic is different. The comparison against RDF.nil is not present in this version (although similar affects could come out elsewhere).

If I take your (corrected) original query with the commented lines back in:

PREFIX rdf:  <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX dct:  <http://purl.org/dc/terms/>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX ibis: <https://vocab.methodandstructure.com/ibis#>
PREFIX ci:   <https://vocab.methodandstructure.com/content-inventory#>
PREFIX cgto: <https://vocab.methodandstructure.com/graph-tool#>
PREFIX itcv: <https://vocab.methodandstructure.com/intertwingler#>

WITH <https://placeholder.ibis.makethingsmakesense.com/>
INSERT {
  ?s skos:inScheme <urn:uuid:cf16f32c-824a-47c2-b176-905c11f8978c> .
  ?s ci:alias ?a .
}
WHERE {
  BIND (IRI(REPLACE(str(?s), "^urn:uuid:", "https://placeholder.ibis.makethingsmakesense.com/", "i")) AS ?a)
  ?s a ?o .
  FILTER (?o IN (skos:Concept, ibis:Issue, ibis:Position, ibis:Argument))
  { ?s a skos:Concept } UNION { ?s a ibis:Issue } UNION { ?s a ibis:Position } UNION { ?s a ibis:Argument }
} ;

WITH <https://placeholder.ibis.makethingsmakesense.com/>
INSERT {
  ?s dct:creator <urn:uuid:ca637d4b-c11b-4152-be98-bde602e7abd4> .
}
WHERE {
  { ?s a ?o } OPTIONAL { ?s dct:creator ?c }
  FILTER (!BOUND(?c) && ?o IN (skos:Concept, ibis:Issue, ibis:Position, ibis:Argument))
  { ?s a skos:Concept } UNION { ?s a ibis:Issue } UNION { ?s a ibis:Position } UNION { ?s a ibis:Argument }
  OPTIONAL { ?s dct:creator ?c }
  FILTER (!bound(?c))
} ;

MOVE <https://placeholder.ibis.makethingsmakesense.com/> TO <dns:placeholder.methodandstructure.com>

I get the following SSE using the default/latest version of the parser:

(prefix
 (
  (rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>)
  (dct: <http://purl.org/dc/terms/>)
  (skos: <http://www.w3.org/2004/02/skos/core#>)
  (ibis: <https://vocab.methodandstructure.com/ibis#>)
  (ci: <https://vocab.methodandstructure.com/content-inventory#>)
  (cgto: <https://vocab.methodandstructure.com/graph-tool#>)
  (itcv: <https://vocab.methodandstructure.com/intertwingler#>))
 (update
  (modify
   (with <https://placeholder.ibis.makethingsmakesense.com/>
    (filter
     (in ?o skos:Concept ibis:Issue ibis:Position ibis:Argument)
     (join
      (join
       (extend
        (
         (?a
          (iri
           (replace
            (str ?s) "^urn:uuid:"
            "https://placeholder.ibis.makethingsmakesense.com/" "i" )) ))
        (bgp))
       (bgp (triple ?s a ?o)))
      (union
       (union
        (union (bgp (triple ?s a skos:Concept)) (bgp (triple ?s a ibis:Issue)))
        (bgp (triple ?s a ibis:Position)))
       (bgp (triple ?s a ibis:Argument))) ))
    (insert
     (
      (triple ?s skos:inScheme <urn:uuid:cf16f32c-824a-47c2-b176-905c11f8978c>)
      (triple ?s ci:alias ?a)) )) )
  (modify
   (with <https://placeholder.ibis.makethingsmakesense.com/>
    (filter
     (exprlist
      (&& (! (bound ?c)) (in ?o skos:Concept ibis:Issue ibis:Position ibis:Argument))
      (! (bound ?c)))
     (leftjoin
      (join
       (leftjoin (bgp (triple ?s a ?o)) (bgp (triple ?s dct:creator ?c)))
       (union
        (union
         (union (bgp (triple ?s a skos:Concept)) (bgp (triple ?s a ibis:Issue)))
         (bgp (triple ?s a ibis:Position)))
        (bgp (triple ?s a ibis:Argument))) )
      (bgp (triple ?s dct:creator ?c))) )
    (insert ((triple ?s dct:creator <urn:uuid:ca637d4b-c11b-4152-be98-bde602e7abd4>)))) )
  (move <https://placeholder.ibis.makethingsmakesense.com/>
   <dns:placeholder.methodandstructure.com> )) )

Note that the in operator does include ?o: (in ?o skos:Concept ibis:Issue ibis:Position ibis:Argument).

Otherwise, I'm not clear on if there is a problem here, or just a misunderstanding.