Skip to content

Log request details on error #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
clehene wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Log request details on error #7

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9aa0768
[DOC] - additional SyncFlag documentation
Aug 8, 2017
d11fc97
Output request details on errrors
Oct 21, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 16 additions & 8 deletions src/main/java/com/microsoft/azure/datalake/store/ADLFileInputStream.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -259,24 +259,32 @@ int readRemote(long position, byte[] b, int offset, int length, boolean speculat
while (inStream.read(junkbuffer, 0, junkbuffer.length)>=0); // read and consume rest of stream, if any remains
}
} catch (IOException ex) {
throw new ADLException("Error reading data from response stream in positioned read() for file " + filename, ex);
String requestDebugDetails = getRequestDebugDetails(opts, resp,
(System.nanoTime() - start) / 1000000, totalBytesRead, position);
throw new ADLException("Error reading data from response stream in positioned read() for file " +
filename + ". Details " + requestDebugDetails, ex);
} finally {
if (inStream != null) inStream.close();
long timeTaken=(System.nanoTime() - start)/1000000;
if (log.isDebugEnabled()) {
String logline ="HTTPRequestRead," + (resp.successful?"Succeeded":"Failed") +
",cReqId:" + opts.requestid +
",lat:" + Long.toString(resp.lastCallLatency+timeTaken) +
",Reqlen:" + totalBytesRead +
",sReqId:" + resp.requestId +
",path:" + filename +
",offset:" + position;
String logline = getRequestDebugDetails(opts, resp, timeTaken, totalBytesRead, position);
log.debug(logline);
}
}
return totalBytesRead;
}

private String getRequestDebugDetails(RequestOptions reqOpts, OperationResponse resp,
long timeTaken, int totalBytesRead, long position) {
return "HTTPRequestRead," + (resp.successful?"Succeeded":"Failed") +
",cReqId:" + reqOpts.requestid +
",lat:" + Long.toString(resp.lastCallLatency+timeTaken) +
",Reqlen:" + totalBytesRead +
",sReqId:" + resp.requestId +
",path:" + filename +
",offset:" + position;
}


/**
* Seek to given position in stream.
Expand Down
9 changes: 9 additions & 0 deletions src/main/java/com/microsoft/azure/datalake/store/SyncFlag.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,15 @@

/**
* Indicator flags to backend during append.
* Optionally indicates what to do after completion of the concurrent append.
* DATA indicates that more data will be sent immediately by the client, the file handle should
* remain open/locked, and file metadata (including file length, last modified time) should NOT
* get updated.
* METADATA indicates that more data will be sent immediately by the client, the file handle should
* remain open/locked, and file metadata should get updated. CLOSE indicates that the client is
* done sending data, the file handle should be closed/unlocked, and file metadata should get
* updated.
* Possible values include: 'DATA', 'METADATA', 'CLOSE'
*/
public enum SyncFlag {
/**
Expand Down