Is it possible to fetch a preview of a dataset from a cloud source? #960
Unanswered
EsaFazal72
asked this question in
Q&A
Replies: 1 comment
-
For remote access we use a Stream-based API and, AFAIK, there is no way to
retrieve a preview. If you had a way to stream the first n rows of the file
that would work. IDK off the top of my head how to do that.
For a local file, you could do something like this:
String fileName = "/Users/me/Desktop/file.txt";
try (Stream<String> stream = Files.lines(Paths.get(fileName))) {
stream.limit(10).forEach(System.out::println);
} catch (IOException e) {
e.printStackTrace();
}
…On Wed, Jul 14, 2021 at 9:39 AM EsaFazal72 ***@***.***> wrote:
I'm building a data connector to import datasets from cloud sources such
as AWS, Azure, GCP and etc. I would like to show the user a preview of the
data prior to importing that dataset. Currently, I believe Tablesaw
downloads the entire dataset temporarily and then fetches columns and the
required number of rows. This particularly takes a very long time to fetch
if the dataset is big. Is there a way around this to only fetch a preview
of the dataset instead of downloading the whole dataset. Maybe a Java-based
library that does this? Any help is highly appreciated. Thank you inadvance.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#960>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AA2FPAUFEFSWBMATMCNU6JLTXWHRRANCNFSM5ALPRXNA>
.
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm building a data connector to import datasets from cloud sources such as AWS, Azure, GCP and etc. I would like to show the user a preview of the data prior to importing that dataset. Currently, I believe Tablesaw downloads the entire dataset temporarily and then fetches columns and the required number of rows. This particularly takes a very long time to fetch if the dataset is big. Is there a way around this to only fetch a preview of the dataset instead of downloading the whole dataset. Maybe a Java-based library that does this? Any help is highly appreciated. Thank you inadvance.
Beta Was this translation helpful? Give feedback.
All reactions