Skip to content

Commit a31f465

Browse files
pjdufourpjdufour
committed
0.0.2 release
1 parent 3c50463 commit a31f465

19 files changed

+651
-83
lines changed

‎.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -1 +1,3 @@
11
bin
2+
*.so
3+
*.h

‎README.md

+60-8
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,11 @@
44

55
# Description
66

7-
**go-simple-serializer** (aka GSS) is a simple library for serializing/deserializing objects. GSS supports `csv`, `hcl`, `hcl2`, `json`, `jsonl`, `toml`, `yaml`. `hcl` and `hcl2` implementation is fragile and very much in `alpha`.
7+
**go-simple-serializer** (aka GSS) is a simple library for serializing/deserializing objects.
8+
9+
GSS supports `bson`, `csv`, `tsv`, `hcl`, `hcl2`, `json`, `jsonl`, `properties`, `toml`, `yaml`. `hcl` and `hcl2` implementation is fragile and very much in `alpha`.
10+
11+
Using cross compilers, this library can also be called by other languages. This library is cross compiled into a Shared Object file (`*.so`). The Shared Object file can be called by `C`, `C++`, and `Python` on Linux machines. See the examples folder for patterns that you can use. This library is also compiled to pure `JavaScript` using [GopherJS](https://github.com/gopherjs/gopherjs).
812

913
# Usage
1014

@@ -13,14 +17,18 @@
1317
You can use the command line tool to convert between formats.
1418

1519
```
16-
Usage: gss -i INPUT_FORMAT -o OUTPUT_FORMAT
20+
Usage: gss -i INPUT_FORMAT -o OUTPUT_FORMAT [-h HEADER] [-c COMMENT]
1721
Options:
22+
-c string
23+
The input comment character, e.g., #. Commented lines are not sent to output.
24+
-h string
25+
The input header if the stdin input has no header.
1826
-help
1927
Print help.
2028
-i string
21-
The input format: csv, hcl, hcl2, json, jsonl, toml, yaml
29+
The input format: bson, csv, tsv, hcl, hcl2, json, jsonl, properties, toml, yaml
2230
-o string
23-
The output format: csv, hcl, hcl2, json, jsonl, toml, yaml
31+
The output format: bson, csv, tsv, hcl, hcl2, json, jsonl, properties, toml, yaml
2432
-version
2533
Prints version to stdout.
2634
```
@@ -33,8 +41,18 @@ You can import **go-simple-serializer** as a library with:
3341
import (
3442
"github.com/spatialcurrent/go-simple-serializer/gss"
3543
)
44+
```
45+
46+
The `Convert`, `Deserialize`, and `Serialize` functions are the core functions to use.
47+
48+
```go
49+
...
50+
output_string, err := gss.Convert(input_string, input_format, input_header, input_comment, output_format)
3651
...
37-
output_string, err := gss.Convert(input_string, input_format, output_format)
52+
output = map[string]interface{}{}
53+
err := gss.Deserialize(input, format, input_header, input_comment, &output)
54+
...
55+
output_string, err := gss.Serialize(input, format)
3856
...
3957
```
4058

@@ -48,28 +66,62 @@ import (
4866
<body>
4967
<script>
5068
var input = "{\"a\":1}";
51-
var output = gss.convert(input, "json", "yaml")
69+
var output = gss.convert(input, "json", "yaml", )
5270
...
71+
// You can also pass the input header for a csv/tsv that has none
72+
var output = gss.convert(input, "csv", "json", {"header": ["a","b"]})
5373
</script>
5474
</body>
5575
</html>
5676
```
5777

5878
**Android**
5979

60-
The `go-simple-serializer` code is available under `com.spatialcurrent.gss`. For example,
80+
The `go-simple-serializer` code is available for use in Android applications under `com.spatialcurrent.gss`. For example,
6181

6282
```java
6383
import com.spatialcurrent.gss.Gss;
6484
...
65-
String output_format = Gss.convert(input_string, input_format, output_format);
85+
String output_format = Gss.convert(input_string, input_format, input_header, input_comment, output_format);
6686
...
6787
```
6888

89+
**C**
90+
91+
A variant of the `Convert` function is exported in a Shared Object file (`*.so`), which can be called by `C`, `C++`, and `Python` programs on Linux machines. For example:
92+
93+
```
94+
char *input_string = "<YOUR INPUT>";
95+
char *output_string;
96+
err = Convert(input_string, input_format, input_header_csv, input_comment, output_format, &output_string);
97+
```
98+
99+
The Go function definition defined in `plugins/gss/main.go` uses `*C.char` for all input except `output_string` which uses a double pointer (`**C.char`) to write to the output.
100+
101+
```
102+
func Convert(input_string *C.char, input_format *C.char, input_header *C.char, input_comment *C.char, output_format *C.char, output_string **C.char) *C.char
103+
```
104+
105+
For complete patterns for `C`, `C++`, and `Python`, see the `examples`.
106+
69107
# Releases
70108

71109
**go-simple-serializer** is currently in **alpha**. See releases at https://github.com/spatialcurrent/go-simple-serializer/releases.
72110

111+
# Examples
112+
113+
`.gitignore` file to jsonl
114+
115+
```
116+
cat .gitignore | ./gss -i csv -h pattern -o jsonl
117+
```
118+
119+
Get language from `.travis.yml` and set to variable
120+
121+
```
122+
language=$(cat .travis.yml | ./gss_linux_amd64 -i yaml -o json -c '#' | jq .language -r)
123+
```
124+
73125
# Building
74126

75127
**CLI**

‎cmd/gss.js/main.go

+32-3
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
package main
1414

1515
import (
16+
"fmt"
1617
"github.com/spatialcurrent/go-simple-serializer/gss"
1718
)
1819

@@ -21,7 +22,7 @@ import (
2122
"honnef.co/go/js/console"
2223
)
2324

24-
var GO_GSS_VERSION = "0.0.1"
25+
var GO_GSS_VERSION = "0.0.2"
2526

2627
func main() {
2728
js.Global.Set("gss", map[string]interface{}{
@@ -30,8 +31,36 @@ func main() {
3031
})
3132
}
3233

33-
func Convert(input_string string, input_format string, output_format string) string {
34-
output_string, err := gss.Convert(input_string, input_format, output_format)
34+
func Convert(input_string string, input_format string, output_format string, options *js.Object) string {
35+
36+
m := map[string]interface{}{}
37+
for _, key := range js.Keys(options) {
38+
m[key] = options.Get(key).Interface()
39+
}
40+
41+
input_header := []string{}
42+
input_comment := ""
43+
44+
if v, ok := m["header"]; ok {
45+
switch v.(type) {
46+
case []string:
47+
input_header = v.([]string)
48+
case []interface{}:
49+
input_header = make([]string, 0, len(v.([]interface{})))
50+
for _, h := range v.([]interface{}) {
51+
input_header = append(input_header, fmt.Sprint(h))
52+
}
53+
}
54+
}
55+
56+
if v, ok := m["input_comment"]; ok {
57+
switch v := v.(type) {
58+
case string:
59+
input_comment = v
60+
}
61+
}
62+
63+
output_string, err := gss.Convert(input_string, input_format, input_header, input_comment, output_format)
3564
if err != nil {
3665
console.Log(err.Error())
3766
return ""

‎cmd/gss/main.go

+17-5
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import (
1212
"fmt"
1313
"io/ioutil"
1414
"os"
15+
"strings"
1516
)
1617

1718
import (
@@ -22,22 +23,28 @@ import (
2223
"github.com/spatialcurrent/go-simple-serializer/gss"
2324
)
2425

25-
var GO_GSS_VERSION = "0.0.1"
26+
var GO_GSS_VERSION = "0.0.2"
27+
var GO_GSS_FORMATS = []string{"csv", "tsv", "hcl", "hcl2", "json", "jsonl", "properties", "toml", "yaml"}
2628

2729
func printUsage() {
28-
fmt.Println("Usage: gss -i INPUT_FORMAT -o OUTPUT_FORMAT")
30+
fmt.Println("Usage: gss -i INPUT_FORMAT -o OUTPUT_FORMAT [-h HEADER] [-c COMMENT]")
2931
}
3032

3133
func main() {
3234

3335
var input_format string
36+
var input_header_text string
37+
var input_comment string
38+
3439
var output_format string
3540

3641
var version bool
3742
var help bool
3843

39-
flag.StringVar(&input_format, "i", "", "The input format: csv, hcl, hcl2, json, jsonl, toml, yaml")
40-
flag.StringVar(&output_format, "o", "", "The output format: csv, hcl, hcl2, json, jsonl, toml, yaml")
44+
flag.StringVar(&input_format, "i", "", "The input format: "+strings.Join(GO_GSS_FORMATS, ", "))
45+
flag.StringVar(&input_header_text, "h", "", "The input header if the stdin input has no header.")
46+
flag.StringVar(&input_comment, "c", "", "The input comment character, e.g., #. Commented lines are not sent to output.")
47+
flag.StringVar(&output_format, "o", "", "The output format: "+strings.Join(GO_GSS_FORMATS, ", "))
4148
flag.BoolVar(&version, "version", false, "Prints version to stdout.")
4249
flag.BoolVar(&help, "help", false, "Print help.")
4350

@@ -82,7 +89,12 @@ func main() {
8289
os.Exit(1)
8390
}
8491

85-
output_string, err := gss.Convert(string(input_bytes), input_format, output_format)
92+
input_header := make([]string, 0)
93+
if len(input_header_text) > 0 {
94+
input_header = strings.Split(input_header_text, ",")
95+
}
96+
97+
output_string, err := gss.Convert(string(input_bytes), input_format, input_header, input_comment, output_format)
8698
if err != nil {
8799
fmt.Println(errors.Wrap(err, "Error converting"))
88100
os.Exit(1)

‎examples/c/Makefile

+17
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# =================================================================
2+
#
3+
# Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
# Released as open source under the MIT License. See LICENSE file.
5+
#
6+
# =================================================================
7+
8+
build_so:
9+
go build -o gss.so -buildmode=c-shared github.com/spatialcurrent/go-simple-serializer/plugins/gss
10+
build_c:
11+
mkdir bin && gcc -o bin/gss_test_c test.c -L. -l:gss.so
12+
build: build_so build_c
13+
run:
14+
LD_LIBRARY_PATH=. bin/gss_test_c
15+
run_python:
16+
clean:
17+
rm -f gss.h gss.so bin/gss_test_c

‎examples/c/gss_test_c

8.3 KB
Binary file not shown.

‎examples/c/test.c

+34
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
// =================================================================
2+
//
3+
// Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
// Released as open source under the MIT License. See LICENSE file.
5+
//
6+
// =================================================================
7+
8+
#include <stdio.h>
9+
#include <string.h>
10+
#include <stdlib.h>
11+
12+
#include "gss.h"
13+
14+
int
15+
main(int argc, char **argv) {
16+
char *err;
17+
18+
char *input_string = "{\"a\":\"b\",\"c\":[\"d\"]}";
19+
char *output_string;
20+
21+
printf("%s\n", input_string);
22+
23+
err = Convert(input_string, "json", "", "", "yaml", &output_string);
24+
25+
if (err != NULL) {
26+
fprintf(stderr, "error: %s\n", err);
27+
free(err);
28+
return 1;
29+
}
30+
31+
printf("%s\n", output_string);
32+
33+
return 0;
34+
}

‎examples/cpp/Makefile

+16
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# =================================================================
2+
#
3+
# Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
# Released as open source under the MIT License. See LICENSE file.
5+
#
6+
# =================================================================
7+
8+
build_so:
9+
go build -o gss.so -buildmode=c-shared github.com/spatialcurrent/go-simple-serializer/plugins/gss
10+
build_cpp:
11+
mkdir -p bin && g++ -o bin/gss_test_cpp test.cpp -L. -l:gss.so
12+
build: build_so build_cpp
13+
run:
14+
LD_LIBRARY_PATH=. bin/gss_test_cpp
15+
clean:
16+
rm -f gss.h gss.so bin/gss_test_cpp

‎examples/cpp/test.cpp

+68
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
// =================================================================
2+
//
3+
// Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
// Released as open source under the MIT License. See LICENSE file.
5+
//
6+
// =================================================================
7+
8+
#include <iostream>
9+
#include <string>
10+
#include <cstring>
11+
#include "gss.h"
12+
13+
// Conv is a example of a C++ function that can convert between formats using some std::string variables.
14+
// In production, you would want to write the function definition to match the use case.
15+
char* conv(std::string input_string, std::string input_format, std::string input_header, std::string input_comment, std::string output_format, char** output_string_c) {
16+
17+
char *input_string_c = new char[input_string.length() + 1];
18+
std::strcpy(input_string_c, input_string.c_str());
19+
char *input_format_c = new char[input_format.length() + 1];
20+
std::strcpy(input_format_c, input_format.c_str());
21+
char *input_header_c = new char[input_header.length() + 1];
22+
std::strcpy(input_header_c, input_header.c_str());
23+
char *input_comment_c = new char[input_comment.length() + 1];
24+
std::strcpy(input_comment_c, input_comment.c_str());
25+
char *output_format_c = new char[output_format.length() + 1];
26+
std::strcpy(output_format_c, output_format.c_str());
27+
28+
char *err = Convert(input_string_c, input_format_c, input_header_c, input_comment_c, output_format_c, output_string_c);
29+
30+
free(input_string_c);
31+
free(input_format_c);
32+
free(input_header_c);
33+
free(input_comment_c);
34+
free(output_format_c);
35+
36+
return err;
37+
38+
}
39+
40+
int main(int argc, char **argv) {
41+
42+
// Since Go requires non-const values, we must define our parameters as variables
43+
// https://stackoverflow.com/questions/4044255/passing-a-string-literal-to-a-function-that-takes-a-stdstring
44+
std::string input_string("{\"a\":\"b\",\"c\":[\"d\"]}");
45+
std::string input_format("json");
46+
std::string input_header("");
47+
std::string input_comment("");
48+
std::string output_format("yaml");
49+
char *output_char_ptr;
50+
51+
// Write input to stderr
52+
std::cout << input_string << std::endl;
53+
54+
char *err = conv(input_string, input_format, input_header, input_comment, output_format, &output_char_ptr);
55+
if (err != NULL) {
56+
// Write output to stderr
57+
std::cerr << std::string(err) << std::endl;
58+
// Return exit code indicating error
59+
return 1;
60+
}
61+
std::string output_string = std::string(output_char_ptr);
62+
63+
// Write output to stdout
64+
std::cout << output_string << std::endl;
65+
66+
// Return exit code indicating success
67+
return 0;
68+
}

‎examples/python/Makefile

+13
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# =================================================================
2+
#
3+
# Copyright (C) 2018 Spatial Current, Inc. - All Rights Reserved
4+
# Released as open source under the MIT License. See LICENSE file.
5+
#
6+
# =================================================================
7+
8+
build:
9+
go build -o gss.so -buildmode=c-shared github.com/spatialcurrent/go-simple-serializer/plugins/gss
10+
run:
11+
LD_LIBRARY_PATH=. python test.py
12+
clean:
13+
rm -f gss.h gss.so

0 commit comments

Comments
 (0)