Skip to content

Javadoc Comments #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: 3.5
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 3 additions & 5 deletions docs/asciidoc/_export.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,12 @@ Again, notice the URL enconding of the URI (the clean URI is `http://neo4j.org/i
</rdf:RDF>
----

Additionally, you can provide a graph URI to specify the context of the given resource using the `graphuri` parameter.
Here is how you can serialise as RDF the resource identified by URI `http://www.example.org/exampleDocument#Monica`
but only the statements in the named graph `http://www.example.org/exampleDocument#G1`. Normally such a model will
be the result of importing RDF Quads as described in the <<ImportQuadRDF>> section. Note that URIS are URL encoded:
Additionally, you can provide a graph URI to specify the context of the given resource using the `graphUri` parameter.
Here is how you can serialise the resource identified by URI `http://www.example.org/exampleDocument#Monica` as RDF in TriG format, but only the statements in the named graph `http://www.example.org/exampleDocument#G1` are included, which was imported in the <<ImportQuadRDF>> section.

[source,Cypher]
----
:GET /rdf/describe/uri/http%3A%2F%2Fwww.example.org%2FexampleDocument%23Monica?graphuri=http%3A%2F%2Fwww.example.org%2FexampleDocument%23G1&format=TriG
:GET /rdf/describe/uri/http%3A%2F%2Fwww.example.org%2FexampleDocument%23Monica?graphUri=http%3A%2F%2Fwww.example.org%2FexampleDocument%23G1&format=TriG
----

=== By Label + property value
Expand Down
1 change: 1 addition & 0 deletions docs/asciidoc/_import.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,7 @@ Language tags can also be used as a filter criteria. If we are only interested i
CALL semantics.importRDF("file:///Users/jesusbarrasa/Workspaces/neosemantics/docs/rdf/multilang.nt","Turtle", { languageFilter: 'es'})
----

[[CustomDataTypes]]
=== Handling custom data types

In RDF custom data types are annotated to literals after the seperator `^^` in form of an IRI.
Expand Down
9 changes: 8 additions & 1 deletion docs/asciidoc/_install.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -38,4 +38,11 @@ it will produce two jars
Please provide feedback and report bugs as GitHub issues or join the Neo4j Community Forum.

=== Acknowledgements
NSMNTX uses https://rdf4j.eclipse.org/[rdf4j] for parsing and serialising RDF. Eclipse rdf4j is a powerful Java framework for processing and handling RDF data.
NSMNTX uses https://rdf4j.eclipse.org/[rdf4j] for parsing and serialising RDF. Eclipse rdf4j is a powerful Java framework for processing and handling RDF data.

The following functionalities have been developed by https://github.com/ArkanEmre[Emre Arkan] in the context of his bachelor thesis in cooperation with https://www.eccenca.com/en/index.html[eccenca GmbH] and https://www.th-brandenburg.de[Brandenburg University of Applied Sciences]:

* Import (<<ImportQuadRDF>>) and Export of Quadruples
* Preservation of custom data types (<<CustomDataTypes>>)
* Deletion of formerly imported RDF data (<<DeleteRDF>>)

2 changes: 1 addition & 1 deletion src/main/java/semantics/ContextResource.java
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
* <p>
* It is used as Key for the Maps containing labels and properties.
*
* Created on 06.06.2019
* Created on 06/06/2019
*
* @author Emre Arkan
* @see RDFQuadDirectStatementLoader
Expand Down
94 changes: 61 additions & 33 deletions src/main/java/semantics/DirectStatementDeleter.java
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
package semantics;

import static semantics.RDFImport.RELATIONSHIP;
import static semantics.RDFParserConfig.URL_SHORTEN;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
Expand All @@ -10,7 +9,6 @@
import java.time.LocalDate;
import java.time.LocalDateTime;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
Expand All @@ -28,7 +26,7 @@
import org.neo4j.logging.Log;

/**
* This class implements an RDF handler to statement-wise delete imported RDF data
* This class implements an RDF handler to statement-wise delete imported RDF triples.
*
* Created on 03/06/2019.
*
Expand All @@ -37,42 +35,60 @@
class DirectStatementDeleter extends RDFToLPGStatementProcessor implements Callable<Integer> {

private static final Label RESOURCE = Label.label("Resource");

private final Cache<String, Node> nodeCache;

private long notDeletedStatementCount;
private long statementsWithBNodeCount;
private String bNodeInfo;
private String BNodeInfo;

DirectStatementDeleter(GraphDatabaseService db, RDFParserConfig conf, Log l) {

super(db, conf, l);
nodeCache = CacheBuilder.newBuilder()
.maximumSize(conf.getNodeCacheSize())
.build();
bNodeInfo = "";
BNodeInfo = "";
notDeletedStatementCount = 0;
statementsWithBNodeCount = 0;
}

/**
* Analog to endRDF in {@link DirectStatementLoader}, however modified for deletion.
*
* Executed at the end of each commit to inform the user of the current state of the deletion
* process.
*/
@Override
public void endRDF() throws RDFHandlerException {
Util.inTx(graphdb, this);
totalTriplesMapped += mappedTripleCounter;
if (parserConfig.getHandleVocabUris() == URL_SHORTEN) {
persistNamespaceNode();
}

log.info("Successful (last) partial commit of " + mappedTripleCounter + " triples. " +
"Total number of triples deleted is " + totalTriplesMapped + " out of "
+ totalTriplesParsed + " parsed.");
}

/**
* Analog to call in {@link DirectStatementLoader}, however strongly modified to delete nodes
* rather than creating or updating.
*
* {@link #resourceLabels}, {@link #resourceProps}, and {@link #statements}, which contain the
* statements to be deleted, are processed respectively. If a statement does not exist in the
* database, {@link #notDeletedStatementCount} is increased to inform the user of not deleted
* statement count.
*
* {@link #statementsWithBNodeCount} counts the number of statements, which could not be deleted
* due to containing a blank node.
*
* {@link #deleteNodeIfEmpty(Node)} is called for each {@code Node} processed, to check and delete
* it, if applicable.
*
* @return An obligatory return, which is always 0, since the overridden method must return an
* Integer
*/
@Override
public Integer call() throws Exception {

for (Map.Entry<String, Set<String>> entry : resourceLabels.entrySet()) {
if (entry.getKey().startsWith("genid")) {
//if the node represents a blank node
statementsWithBNodeCount += entry.getValue().size() + 1;
continue;
}
Expand All @@ -87,6 +103,7 @@ public Integer call() throws Exception {
node = tempNode;
entry.getValue().forEach(l -> {
if (node != null && node.hasLabel(Label.label(l))) {
//if node exist in the database and has the label to be deleted
node.removeLabel(Label.label(l));
} else {
notDeletedStatementCount++;
Expand Down Expand Up @@ -170,17 +187,15 @@ public Integer call() throws Exception {
}
Node fromNode = null;
try {
fromNode = nodeCache.get(st.getSubject().stringValue(), () -> { //throws AnyException
return graphdb.findNode(RESOURCE, "uri", st.getSubject().stringValue());
});
fromNode = nodeCache.get(st.getSubject().stringValue(),
() -> graphdb.findNode(RESOURCE, "uri", st.getSubject().stringValue()));
} catch (InvalidCacheLoadException icle) {
icle.printStackTrace();
}
Node toNode = null;
try {
toNode = nodeCache.get(st.getObject().stringValue(), () -> { //throws AnyException
return graphdb.findNode(RESOURCE, "uri", st.getObject().stringValue());
});
toNode = nodeCache.get(st.getObject().stringValue(),
() -> graphdb.findNode(RESOURCE, "uri", st.getObject().stringValue()));
} catch (InvalidCacheLoadException icle) {
icle.printStackTrace();
}
Expand Down Expand Up @@ -214,19 +229,23 @@ public Integer call() throws Exception {
deleteNodeIfEmpty(toNode);
deleteNodeIfEmpty(fromNode);
}

statements.clear();
resourceLabels.clear();
resourceProps.clear();
if (statementsWithBNodeCount > 0) {
setbNodeInfo(statementsWithBNodeCount
setBNodeInfo(statementsWithBNodeCount
+ " of the statements could not be deleted, due to containing a blank node.");
}

//TODO what to return here? number of nodes and rels?
return 0;
}

/**
* Analog to periodicOperation in {@link DirectStatementLoader}, however modified for the
* deletion
*
* After each partial commit, a short information about the current state of the deletion process
* is logged.
*/
@Override
protected void periodicOperation() {
Util.inTx(graphdb, this);
Expand All @@ -236,18 +255,33 @@ protected void periodicOperation() {
mappedTripleCounter = 0;
}

/**
* @return amount of not deleted statement count and statements with blank node count
*/
long getNotDeletedStatementCount() {
return notDeletedStatementCount + statementsWithBNodeCount;
}

String getbNodeInfo() {
return bNodeInfo;
/**
* @return information about statement not deleted due to containing a blank node
*/
String getBNodeInfo() {
return BNodeInfo;
}

private void setbNodeInfo(String bNodeInfo) {
this.bNodeInfo = bNodeInfo;
/*
* Called in {@link DirectStatementDeleter#call()} after the deletion process is done
* @param BNodeInfo information about statement not deleted due to containing a blank node
*/
private void setBNodeInfo(String BNodeInfo) {
this.BNodeInfo = BNodeInfo;
}

/**
* Deletes a given {@code node}, if all conditions are met. Call in the {@link #call()} method.
*
* @param node to be deleted
*/
private void deleteNodeIfEmpty(Node node) {
int nodePropertyCount = node.getAllProperties().size();
int labelCount = Iterators.size(node.getLabels().iterator());
Expand All @@ -259,13 +293,7 @@ private void deleteNodeIfEmpty(Node node) {
}
}

private void persistNamespaceNode() {
Map<String, Object> params = new HashMap<>();
params.put("props", namespaces);
graphdb.execute("MERGE (n:NamespacePrefixDefinition) SET n+={props}", params);
}

// Adapted from APOC :)
// Adapted from APOC
private Object toPropertyValue(Object value) {
if (value instanceof Iterable) {
Iterable it = (Iterable) value;
Expand Down
Loading