Skip to content

Commit 64c6814

Browse files
author
Sumedh Wale
committed
revamped and enhanced the docs
- switched to mkdocs-material which is much better looking and functional - added custom stylesheet, footer etc for the new theme - updated the mkdocs.yml to use the new theme with customizations, and reorganized it - integrated the SQL functions doc in the top-level reference docs instead of linking to it - removed obsolete docs, resurrected VSD docs (with link to third-party tool) - moved and updated the docs for the new theme - updated and fixed references to Spark version, SnappyData releases and so on - tons of other changes to fix and improve the docs
1 parent d7e9ad6 commit 64c6814

File tree

215 files changed

+1312
-993
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

215 files changed

+1312
-993
lines changed

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ vm_*
1010
.lib/
1111
dist/*
1212
build-artifacts/
13+
site/
1314
lib_managed/
1415
src_managed/
1516
project/boot/

README.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -46,15 +46,15 @@ When speed is essential, applications can selectively copy the external data int
4646
In SnappyData, operational systems can feed data updates through Kafka to SnappyData. The incoming data can be CDC(Change-data-capture) events (insert, updates, or deletes) and can be easily ingested into in-memory tables with ease, consistency, and exactly-once semantics. The Application can apply custom logic to do sophisticated transformations and get the data ready for analytics. This incremental and continuous process is far more efficient than batch refreshes. Refer [Stream Processing with SnappyData](docs/howto/use_stream_processing_with_snappydata.md) </br>
4747

4848
* **Approximate Query Processing(AQP)** </br>
49-
When dealing with huge data sets, for example, IoT sensor streaming time-series data, it may not be possible to provision the data in-memory, and if left at the source (say Hadoop or S3) your analytic query processing can take too long. In SnappyData, you can create one or more stratified data samples on the full data set. The query engine automatically uses these samples for aggregation queries, and a nearly accurate answer returned to clients. This can be immensely valuable when visualizing a trend, plotting a graph or bar chart. Refer [AQP](docs/aqp.md)
49+
When dealing with huge data sets, for example, IoT sensor streaming time-series data, it may not be possible to provision the data in-memory, and if left at the source (say Hadoop or S3) your analytic query processing can take too long. In SnappyData, you can create one or more stratified data samples on the full data set. The query engine automatically uses these samples for aggregation queries, and a nearly accurate answer returned to clients. This can be immensely valuable when visualizing a trend, plotting a graph or bar chart. Refer [AQP](docs/sde/index.md).
5050

5151
* **Access from anywhere** </br>
5252
You can use JDBC, ODBC, REST, or any of the Apache Spark APIs. The product is fully compatible with Apache Spark 2.1.1. SnappyData natively supports modern visualization tools such as [TIBCO Spotfire](docs/howto/connecttibcospotfire.md), [Tableau](docs/howto/tableauconnect.md), and [Qlikview](docs/setting_up_jdbc_driver_qlikview.md). Refer
5353

5454

5555
## Downloading and Installing SnappyData
5656
You can download and install the latest version of SnappyData from [github](https://github.com/TIBCOSoftware/snappydata/releases).
57-
Refer to the [documentation](docs//install.md) for installation steps.
57+
Refer to the [documentation](docs/install/index.md) for installation steps.
5858

5959
## Getting Started
6060
Multiple options are provided to get started with SnappyData. Easiest way to get going with SnappyData is on your laptop. You can also use any of the following options:
@@ -66,7 +66,7 @@ Multiple options are provided to get started with SnappyData. Easiest way to get
6666
* Docker
6767
* Kubernetes
6868

69-
You can find more information on options for running SnappyData [here](docs/quickstart.md).
69+
You can find more information on options for running SnappyData [here](docs/quickstart/index.md).
7070

7171
## Quick Test to Measure Performance of SnappyData vs Apache Spark
7272

build.gradle

Lines changed: 1 addition & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1540,14 +1540,8 @@ task docs(type: ScalaDoc) {
15401540
destinationDir = file("${rootProject.buildDir}/docs")
15411541
}
15421542

1543-
task buildSqlFuncDocs(type: Exec) {
1544-
dependsOn product
1545-
//on linux
1546-
commandLine "${rootProject.projectDir}/spark/sql/create-docs.sh"
1547-
}
1548-
15491543
task publishDocs(type: Exec) {
1550-
dependsOn docs, buildSqlFuncDocs
1544+
dependsOn product, docs
15511545
//on linux
15521546
commandLine './publish-site.sh'
15531547
}

docs/GettingStarted.md

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -41,15 +41,15 @@ When speed is essential, applications can selectively copy the external data int
4141
In SnappyData, operational systems can feed data updates through Kafka to SnappyData. The incoming data can be CDC(Change-data-capture) events (insert, updates, or deletes) and can be easily ingested into in-memory tables with ease, consistency, and exactly-once semantics. The Application can apply custom logic to do sophisticated transformations and get the data ready for analytics. This incremental and continuous process is far more efficient than batch refreshes. Refer [Stream Processing with SnappyData](howto/use_stream_processing_with_snappydata.md) </br>
4242

4343
* **Approximate Query Processing(AQP)** </br>
44-
When dealing with huge data sets, for example, IoT sensor streaming time-series data, it may not be possible to provision the data in-memory, and if left at the source (say Hadoop or S3) your analytic query processing can take too long. In SnappyData, you can create one or more stratified data samples on the full data set. The query engine automatically uses these samples for aggregation queries, and a nearly accurate answer returned to clients. This can be immensely valuable when visualizing a trend, plotting a graph or bar chart. Refer [AQP](aqp.md)
44+
When dealing with huge data sets, for example, IoT sensor streaming time-series data, it may not be possible to provision the data in-memory, and if left at the source (say Hadoop or S3) your analytic query processing can take too long. In SnappyData, you can create one or more stratified data samples on the full data set. The query engine automatically uses these samples for aggregation queries, and a nearly accurate answer returned to clients. This can be immensely valuable when visualizing a trend, plotting a graph or bar chart. Refer [AQP](sde/index.md).
4545

4646
* **Access from anywhere** </br>
47-
You can use JDBC, ODBC, REST, or any of the Apache Spark APIs. The product is fully compatible with Apache Spark 2.1.1. SnappyData natively supports modern visualization tools such as [TIBCO Spotfire](howto/connecttibcospotfire.md), [Tableau](howto/tableauconnect.md), and [Qlikview](setting_up_jdbc_driver_qlikview.md). Refer
47+
You can use JDBC, ODBC, REST, or any of the Apache Spark APIs. The product is fully compatible with Apache Spark 2.1.1 to 2.1.3. SnappyData natively supports modern visualization tools such as [TIBCO Spotfire](howto/connecttibcospotfire.md), [Tableau](howto/tableauconnect.md), and [Qlikview](setting_up_jdbc_driver_qlikview.md).
4848

4949

5050
## Downloading and Installing SnappyData
5151
You can download and install the latest version of SnappyData from [github](https://github.com/TIBCOSoftware/snappydata/releases) or you can download the enterprise version that is TIBCO ComputeDB from [here](https://edelivery.tibco.com/storefront/index.ep).
52-
Refer to the [documentation](/install.md) for installation steps.
52+
Refer to the [documentation](install/index.md) for installation steps.
5353

5454
## Getting Started
5555
Multiple options are provided to get started with SnappyData. Easiest way to get going with SnappyData is on your laptop. You can also use any of the following options:
@@ -61,7 +61,7 @@ Multiple options are provided to get started with SnappyData. Easiest way to get
6161
* Docker
6262
* Kubernetes
6363

64-
You can find more information on options for running SnappyData [here](/quickstart.md).
64+
You can find more information on options for running SnappyData [here](quickstart/index.md).
6565

6666
## Quick Test to Measure Performance of SnappyData vs Apache Spark
6767

@@ -132,7 +132,7 @@ For more details, refer [https://github.com/sbt/sbt/issues/3618](https://github.
132132

133133

134134
## Building from Source
135-
If you would like to build SnappyData from source, refer to the [documentation on building from source](/install/building_from_source.md).
135+
If you would like to build SnappyData from source, refer to the [documentation on building from source](install/building_from_source.md).
136136

137137

138138
## How is SnappyData Different than Apache Spark?

docs/Images/logo.png

9.27 KB
Loading

docs/Images/vsd/vsd-connection-stats.png

100755100644
File mode changed.

docs/Images/vsd/vsd_applications.png

100755100644
File mode changed.

docs/Images/vsd/vsd_applications_2.png

100755100644
File mode changed.

docs/Images/vsd/vsd_cpu.png

100755100644
File mode changed.

docs/Images/vsd/vsd_memory.png

100755100644
File mode changed.

docs/Images/vsd/vsd_memory_2.png

100755100644
File mode changed.

docs/Images/vsd/vsd_statements.png

100755100644
File mode changed.

docs/Images/vsd/vsd_tables.png

100755100644
File mode changed.

docs/Images/vsd/vsd_tables_2.png

100755100644
File mode changed.

docs/Images/vsd/vsd_tables_3.png

100755100644
File mode changed.

docs/Images/vsd/vsd_transactions.png

100755100644
File mode changed.

docs/Images/vsd/vsd_transactions_2.png

100755100644
File mode changed.

docs/LICENSE.md

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
## LICENSE
1+
# License
2+
3+
The source code is distributed with Apache License 2.0. Users can download and deploy it in production.
4+
Full text of the license is below.
5+
26

37
Apache License
48
Version 2.0, January 2004
@@ -188,7 +192,7 @@
188192
same "printed page" as the copyright notice for easier
189193
identification within third-party archives.
190194

191-
Copyright 2018 SnappyData Inc.
195+
Copyright © 2017-2021 TIBCO Software Inc. All rights reserved.
192196

193197
Licensed under the Apache License, Version 2.0 (the "License");
194198
you may not use this file except in compliance with the License.

docs/additional_files/open_source_components.md

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,9 +44,4 @@ The high level capabilities of the **Community Edition** are listed in the follo
4444
|Use encrypted password instead of clear text password | X |
4545
|Restrict Table, View, Function creation even in user’s own schema | X |
4646
|LDAP security interface | X |
47-
|Visual Statistics Display (VSD) tool for system statistics (gfs) files(*) | |
4847
|GemFire connector | |
49-
50-
(*) NOTE: The graphical Visual Statistics Display (VSD) tool to see the system statistics (gfs) files is not OSS
51-
and was never shipped with SnappyData. It is available from [GemTalk Systems](https://gemtalksystems.com/products/vsd/)
52-
or [Pivotal GemFire](https://network.pivotal.io/products/pivotal-gemfire) under their own respective licenses.

docs/affinity_modes/connector_mode.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Specifically, to run on a cluster, the SparkContext can connect to several types
1111

1212
**Key Points:**
1313

14-
* Can work with SnappyData store from a compatible Spark distribution (2.1.1)
14+
* Can work with SnappyData store from a compatible Spark distribution (2.1.1 to 2.1.3)
1515

1616
* Spark application executes in its own independent JVM processes
1717

docs/affinity_modes/embedded_mode.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ In this mode, one can write Spark programs using jobs. For more details, refer t
2323

2424
Also, you can use [SnappySQL](../howto/use_snappy_shell.md) to create and query tables.
2525

26-
You can either [start SnappyData members](../howto/start_snappy_cluster/) using the `snappy-start-all.sh` script or you can start them individually.
26+
You can either [start SnappyData members](../howto/start_snappy_cluster.md) using the `snappy-start-all.sh` script or you can start them individually.
2727

2828
Having the Spark computation embedded in the same JVM allows us to do several optimizations at the query planning level. For example:
2929

docs/affinity_modes/index.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# Affinity Modes
2+
In this section, the various modes available for colocation of related data and computation is discussed.
3+
4+
You can run the SnappyData store in the following modes:
5+
6+
* [Local Mode](local_mode.md): Used mainly for development, where the client application, the executors, and data store are all running in the same JVM
7+
8+
* [Embedded SnappyData Store Mode](embedded_mode.md): The Spark computations and in-memory data store run colocated in the same JVM
9+
10+
* [SnappyData Smart Connector Mode](connector_mode.md): Allows you to work with the SnappyData store cluster from any compatible Spark distribution

docs/affinity_modes/local_mode.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,7 +68,7 @@ To start SnappyData store you need to create a SnappySession in your program:
6868

6969
**Example**: **Launch Apache Spark shell and provide SnappyData dependency as a Spark package**:
7070

71-
If you already have Spark 2.1.1 installed in your local machine you can directly use `--packages` option to download the SnappyData binaries.
71+
If you already have Spark 2.1.1 to 2.1.3 installed in your local machine you can directly use `--packages` option to download the SnappyData binaries.
7272

7373
```pre
7474
./bin/spark-shell --packages "TIBCOSoftware:snappydata:1.3.0-s_2.11"

docs/apidocsintro.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
## API Documentation
22

3-
* Details about **SnappyData Spark Extension APIs** can be found [here](/reference/API_Reference/apireference_guide.md).
3+
* Details about **SnappyData Spark Extension APIs** can be found [here](reference/API_Reference/apireference_guide.md).
44

55
* Details of all the **other API reference for SnappyData **can be found [here](http://tibcosoftware.github.io/snappydata/apidocs).

docs/aqp_aws.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
# Using <!--iSight-Cloud-->SnappyData CloudBuilder
2-
<!--iSight-Cloud-->CloudBuilder is a cloud-based service that allows for instant visualization of analytic query results on large datasets. Powered by the SnappyData Synopsis Data Engine ([SDE](aqp.md)), users interact with <!--iSight-Cloud-->CloudBuilder to populate the synopsis engine with the right data sets and accelerate SQL queries by using the engine to provide latency bounded responses to large complex aggregate queries.
2+
<!--iSight-Cloud-->CloudBuilder is a cloud-based service that allows for instant visualization of analytic query results on large datasets. Powered by the SnappyData Synopsis Data Engine ([SDE](sde/index.md)), users interact with <!--iSight-Cloud-->CloudBuilder to populate the synopsis engine with the right data sets and accelerate SQL queries by using the engine to provide latency bounded responses to large complex aggregate queries.
33

4-
<!--iSight-Cloud-->CloudBuilder uses Apache Zeppelin as the front end notebook to display results and allows users to build powerful notebooks representing key elements of their business in a matter of minutes.
4+
<!--iSight-Cloud-->CloudBuilder uses Apache Zeppelin as the front end notebook to display results and allows users to build powerful notebooks representing key elements of their business in a matter of minutes.
55

6-
The service provides a web URL that spins up a cluster instance on AWS or users can download the <!--iSight-Cloud-->CloudBuilder EC2 script to configure a custom sized cluster, to create and render powerful visualizations of their big data sets with the click of a button.
6+
The service provides a web URL that spins up a cluster instance on AWS or users can download the <!--iSight-Cloud-->CloudBuilder EC2 script to configure a custom sized cluster, to create and render powerful visualizations of their big data sets with the click of a button.
77
With <!--iSight-Cloud-->CloudBuilder, you can speed up the process of understanding what your data is telling you, and move on to the task of organizing your business around those insights rapidly.
88

99
In this document, the features provided by SnappyData for analyzing your data is described. It also provides details for deploying a SnappyData Cloud cluster on AWS using either the CloudFormation service or by using the EC2 scripts.

docs/architecture/cluster_architecture.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ A SnappyData cluster is a peer-to-peer (P2P) network comprised of three distinct
99

1010
![ClusterArchitecture](../GettingStarted_Architecture.png)
1111

12-
SnappyData also has multiple deployment options. For more information refer to, [Deployment Options](../deployment.md).
12+
SnappyData also has multiple deployment options. For more information refer to, [Deployment Options](../affinity_modes/index.md).
1313

1414
## Interacting with SnappyData
1515

docs/architecture/index.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# SnappyData Concepts
2+
3+
4+
The topic explains the following fundamental concepts of SnappyData:
5+
6+
* [Core Components](core_components.md)
7+
* [SnappyData Cluster Architecture](cluster_architecture.md)
8+
* [Hybrid Cluster Manager](hybrid_cluster_manager.md)
9+
* [Distributed Transactions](../consistency/index.md)
10+
* [Affinity Modes](../affinity_modes/index.md)
11+

docs/best_practices.md

Lines changed: 0 additions & 20 deletions
This file was deleted.

0 commit comments

Comments
 (0)