Skip to content

Commit 6a4ece2

Browse files
committed
initial work
0 parents  commit 6a4ece2

68 files changed

Lines changed: 2781 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
secret/
2+
bin/
3+
*.so
4+
*.h

.travis.yml

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
sudo: false
2+
language: go
3+
4+
go:
5+
- "1.9"
6+
- "1.10"
7+
- "1.11"
8+
- tip
9+
10+
os:
11+
- linux
12+
#- osx
13+
14+
matrix:
15+
allow_failures:
16+
- go: tip
17+
fast_finish: true
18+
19+
install:
20+
- go get ./...
21+
- go get -u github.com/alecthomas/gometalinter
22+
- gometalinter --install
23+
24+
script:
25+
- scripts/format.sh
26+
- scripts/test.sh

AUTHORS

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
go-reader is maintained by Spatial Current, Inc.
2+
3+
Authors:
4+
5+
* Patrick Dufour (pjdufour)

CONTRIBUTING.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Contributing to go-reader
2+
3+
## Contributor License Agreement
4+
5+
Thank you for your interest in contributing. You will first need to agree to the license. Simply post the following paragraph, with "your name" and "your github account" substituted as needed, to the first issue [#1](https://github.com/spatialcurrent/go-reader/issues/1). Otherwise, you can email us [here](mailto:opensource@spatialcurrent.io?subject=CLA).
6+
7+
I, **< YOUR NAME > (< YOUR GITHUB ACCOUNT >)**, agree to the license terms. My contributions to this repo are granted to **Spatial Current, Inc.** under a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable license and/or copyright is transferred.
8+
9+
## Versioning
10+
11+
This library is still in alpha (0.x.x), so versioning is not semantic yet.
12+
13+
## Authors
14+
15+
See [AUTHORS](https://github.com/spatialcurrent/go-reader/blob/master/AUTHORS) for a list of contributors.

LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2018 Spatial Current, Inc.
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

README.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
[![Build Status](https://travis-ci.org/spatialcurrent/go-reader-writer.svg)](https://travis-ci.org/spatialcurrent/go-reader-writer) [![Go Report Card](https://goreportcard.com/badge/spatialcurrent/go-reader-writer)](https://goreportcard.com/report/spatialcurrent/go-reader-writer) [![GoDoc](https://godoc.org/github.com/spatialcurrent/go-reader-writer?status.svg)](https://godoc.org/github.com/spatialcurrent/go-reader-writer) [![license](http://img.shields.io/badge/license-MIT-red.svg?style=flat)](https://github.com/spatialcurrent/go-reader-writer/blob/master/LICENSE)
2+
3+
# go-reader-writer
4+
5+
# Description
6+
7+
**go-reader-writer** (aka GRW) is a simple library for managing reading/writing of resources. GRW can read from `bzip2`, `gzip`, `snappy`, and `zip` resources and write to `gzip` and `snappy` resources.
8+
9+
Using cross compilers, this library can also be called by other languages. This library is cross compiled into a Shared Object file (`*.so`). The Shared Object file can be called by `C`, `C++`, and `Python` on Linux machines. See the examples folder for patterns that you can use. This library is also compiled to pure `JavaScript` using [GopherJS](https://github.com/gopherjs/gopherjs).
10+
11+
# Usage
12+
13+
**CLI**
14+
15+
You can use the command line tool to convert between formats.
16+
17+
```
18+
Usage: grw [-input_uri INPUT_URI] [-input_compression [bzip2|gzip|snappy|zip|none]] [-output_uri OUTPUT_URI] [-output_compression [bzip2|gzip|snappy|zip|none]] [-verbose] [-version]
19+
Options:
20+
-aws_access_key_id string
21+
Defaults to value of environment variable AWS_ACCESS_KEY_ID
22+
-aws_default_region string
23+
Defaults to value of environment variable AWS_DEFAULT_REGION.
24+
-aws_secret_access_key string
25+
Defaults to value of environment variable AWS_SECRET_ACCESS_KEY.
26+
-aws_session_token string
27+
Defaults to value of environment variable AWS_SESSION_TOKEN.
28+
-help
29+
Print help
30+
-input_buffer_size int
31+
the input reader buffer size (default 4096)
32+
-input_compression string
33+
Stream input compression algorithm for nodes, using: bzip2, gzip, snappy, zip, or none.
34+
-input_uri string
35+
"stdin" or uri to input file (default "stdin")
36+
-output_append
37+
append output to resource
38+
-output_buffer_size int
39+
the output writer buffer size (default 4096)
40+
-output_compression string
41+
Stream input compression algorithm for nodes, using: bzip2, gzip, snappy, zip, or none.
42+
-output_uri string
43+
"stdout" or uri to output resource (default "stdout")
44+
-version
45+
Prints version to stdout
46+
```
47+
48+
**Go**
49+
50+
You can import **go-reader-writer** as a library with:
51+
52+
```go
53+
import (
54+
"github.com/spatialcurrent/go-reader-writer/grw"
55+
)
56+
...
57+
```
58+
59+
See [grw](https://godoc.org/github.com/spatialcurrent/go-reader-writer/grw) in GoDoc for information on how to use Go API.
60+
61+
**JavaScript**
62+
63+
```html
64+
<html>
65+
<head>
66+
<script src="https://...grw.js"></script>
67+
</head>
68+
<body>
69+
<script>
70+
grw.open(uri, "none", function(text){ ... })
71+
...
72+
</script>
73+
</body>
74+
</html>
75+
```
76+
77+
# Examples:
78+
79+
TBD
80+
81+
# Building
82+
83+
**CLI**
84+
85+
The command line go-reader-wrtier program can be built with the `scripts/build_cli.sh` script.
86+
87+
**JavaScript**
88+
89+
You can compile go-reader-writer to pure JavaScript with the `scripts/build_javascript.sh` script.
90+
91+
**Shared Object**
92+
93+
The `scripts/build_so.sh` script is used to build a Shared Object (`*.go`), which can be called by `C`, `C++`, and `Python` on Linux machines.
94+
95+
**Changing Destination**
96+
97+
The default destination for build artifacts is `go-reader/bin`, but you can change the destination with a CLI argument. For building on a Chromebook consider saving the artifacts in `/usr/local/go/bin`, e.g., `bash scripts/build_cli.sh /usr/local/go/bin`
98+
99+
# Contributing
100+
101+
[Spatial Current, Inc.](https://spatialcurrent.io) is currently accepting pull requests for this repository. We'd love to have your contributions! Please see [Contributing.md](https://github.com/spatialcurrent/go-reader-writer/blob/master/CONTRIBUTING.md) for how to get started.
102+
103+
# License
104+
105+
This work is distributed under the **MIT License**. See **LICENSE** file.

cmd/grw.js/main.go

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
// =================================================================
2+
//
3+
// Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
// Released as open source under the MIT License. See LICENSE file.
5+
//
6+
// =================================================================
7+
8+
// grw.js is the Javascript version of go-reader-writer (GRW).
9+
//
10+
package main
11+
12+
import (
13+
"github.com/gopherjs/gopherjs/js"
14+
"github.com/pkg/errors"
15+
"github.com/spatialcurrent/go-reader-writer/grw"
16+
)
17+
18+
func main() {
19+
js.Global.Set("grw", map[string]interface{}{
20+
"version": grw.Version,
21+
"fetch": FetchResource,
22+
})
23+
}
24+
25+
func FetchResource(uri string, alg string, callback func(...interface{}) *js.Object) {
26+
27+
go func() {
28+
29+
r, _, err := grw.ReadHTTPFile(uri, alg, false)
30+
if err != nil {
31+
callback("", errors.Wrap(err, "error opening resource from uri "+uri).Error())
32+
return
33+
}
34+
35+
b, err := r.ReadAll()
36+
if err != nil {
37+
callback("", errors.Wrap(err, "error reading from resource at uri "+uri).Error())
38+
return
39+
}
40+
41+
err = r.Close()
42+
if err != nil {
43+
callback("", errors.Wrap(err, "error closing resource at "+uri).Error())
44+
return
45+
}
46+
47+
callback(string(b), nil)
48+
}()
49+
50+
}

0 commit comments

Comments
 (0)