Migrate schema designer to filestoreapi part deux #3031
Conversation
|
Progress so far @gerlowskija .... |
| @@ -534,7 +534,6 @@ public void refresh(String path) { | |||
| @SuppressWarnings({"rawtypes"}) | |||
| List myFiles = list(path, s -> true); | |||
| for (Object f : l) { | |||
| // TODO: https://issues.apache.org/jira/browse/SOLR-15426 | |||
There was a problem hiding this comment.
@madrob you opened up this JIRA, and I think this TODO is incorrect or otherwise overcome by events. Regardless, it doesn't appear connected to SOLR-15426... Can you confirm?
There was a problem hiding this comment.
I think it was actually related to SOLR-15385, I must have made a copy/paste error here. Reading my comments and surface-level tracing the code suggests that this is still an issue.
| throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e); | ||
| } | ||
| String path = | ||
| "blob" + "/" + configSet |
There was a problem hiding this comment.
[0] Might be nice to make the "blob/" bit here and other similar path segments constants, that can be reused on both the storage and retrieval side of this helper code.
There was a problem hiding this comment.
Definitely. I don't even think we should use "blob" prefix. Maybe "schema_designer" since this is for this tool.
There was a problem hiding this comment.
If the format of this data is "javabin", lets name the file with that suffix.
|
Looks good to me so far. I notice this is still "Draft"; curious what else is remaining here? |
| * <p>TODO: Is fetchMissing actually used? I don't see it being used, but the IDE doesn't flag it | ||
| * not being used! |
There was a problem hiding this comment.
Lucene/Solr devs use a "nocommit" comment for something like this. Precommit will fail so we remember to address it. I also configure IntelliJ's "TODO" feature to consider "nocommit" and thus it shows in blood red (gets my attention)
| throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e); | ||
| } | ||
| String path = | ||
| "blob" + "/" + configSet |
There was a problem hiding this comment.
Definitely. I don't even think we should use "blob" prefix. Maybe "schema_designer" since this is for this tool.
| throw new SolrException(SolrException.ErrorCode.BAD_REQUEST, e); | ||
| } | ||
| String path = | ||
| "blob" + "/" + configSet |
There was a problem hiding this comment.
If the format of this data is "javabin", lets name the file with that suffix.
| } | ||
| }, | ||
| true); | ||
| } catch (java.io.FileNotFoundException e) { |
There was a problem hiding this comment.
not sure why fully qualified
| byte[] bytes = is.readAllBytes(); | ||
| if (bytes.length > 0) { | ||
| docs = (List<SolrInputDocument>) Utils.fromJavabin(bytes); | ||
| } |
There was a problem hiding this comment.
I suggest a little comment // TODO stream it; no byte array
| } finally { | ||
| httpPost.releaseConnection(); | ||
| } | ||
| FileStoreAPI.MetaData meta = ClusterFileStore._createJsonMetaData(bytes, null); |
There was a problem hiding this comment.
is the meta necessary in general for the FileStoreAPI -- 2 step? Sad if so.
https://issues.apache.org/jira/browse/SOLR-XXXXX
Description
Migrate Schema Designer calls from using Blob store to FileStore
Solution
Swap implementation methods.
Tests
So far, just using existing tests.