Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(gno.land): add read-only metadata support to packages #3740

Draft
wants to merge 19 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions docs/gno-tooling/cli/gnokey/querying-a-network.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ Below is a list of queries a user can make with `gnokey`:
- `vm/qfile` - returns package contents for a given pkgpath
- `vm/qeval` - evaluates an expression in read-only mode on and returns the results
- `vm/qrender` - shorthand for evaluating `vm/qeval Render("")` for a given pkgpath
- `vm/qmeta` - returns the value for a package metadata field

Let's see how we can use them.

Expand Down Expand Up @@ -225,6 +226,14 @@ gnokey query vm/qrender --data "gno.land/r/demo/wugnot:balance/g125em6arxsnj49vx
To see how this was achieved, check out `wugnot`'s `Render()` function.
:::

## `vm/qmeta`

`vm/qmeta` allows us to read the metadata of a realm or package. For example:

```bash
gnokey query vm/qmeta --data "gno.land/r/demo/boards:field_name"` |
```

## Conclusion

That's it! 🎉
Expand Down
39 changes: 36 additions & 3 deletions gno.land/pkg/keyscli/addpkg.go
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
"context"
"flag"
"fmt"
"strings"

"github.com/gnolang/gno/gno.land/pkg/sdk/vm"
gno "github.com/gnolang/gno/gnovm/pkg/gnolang"
Expand All @@ -21,6 +22,7 @@
PkgPath string
PkgDir string
Deposit string
Meta commands.StringArr
}

func NewMakeAddPkgCmd(rootCfg *client.MakeTxCfg, io commands.IO) *commands.Command {
Expand Down Expand Up @@ -62,6 +64,12 @@
"",
"deposit coins",
)

fs.Var(
&c.Meta,
"meta",
"metadata fields",
)
}

func execMakeAddPkg(cfg *MakeAddPkgCfg, args []string, io commands.IO) error {
Expand Down Expand Up @@ -107,6 +115,30 @@
panic(fmt.Sprintf("found an empty package %q", cfg.PkgPath))
}

// parse metadata fields
var metadata []*vm.MetaField

Check failure on line 119 in gno.land/pkg/keyscli/addpkg.go

View workflow job for this annotation

GitHub Actions / Run gno.land suite / Go Lint / lint

Consider pre-allocating `metadata` (prealloc)
for _, s := range cfg.Meta {
parts := strings.SplitN(s, "=", 2)
if len(parts) != 2 {
return errors.New("invalid metadata field format, expected field=value")
}

Check warning on line 124 in gno.land/pkg/keyscli/addpkg.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/keyscli/addpkg.go#L121-L124

Added lines #L121 - L124 were not covered by tests

name := strings.TrimSpace(parts[0])
if name == "" {
return errors.New("empty metadata field name")
}

Check warning on line 129 in gno.land/pkg/keyscli/addpkg.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/keyscli/addpkg.go#L126-L129

Added lines #L126 - L129 were not covered by tests

var value []byte
if v := parts[1]; strings.TrimSpace(v) != "" {
value = []byte(v)
}

Check warning on line 134 in gno.land/pkg/keyscli/addpkg.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/keyscli/addpkg.go#L131-L134

Added lines #L131 - L134 were not covered by tests

metadata = append(metadata, &vm.MetaField{
Name: name,
Value: value,
})

Check warning on line 139 in gno.land/pkg/keyscli/addpkg.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/keyscli/addpkg.go#L136-L139

Added lines #L136 - L139 were not covered by tests
}

// parse gas wanted & fee.
gaswanted := cfg.RootCfg.GasWanted
gasfee, err := std.ParseCoin(cfg.RootCfg.GasFee)
Expand All @@ -115,9 +147,10 @@
}
// construct msg & tx and marshal.
msg := vm.MsgAddPackage{
Creator: creator,
Package: memPkg,
Deposit: deposit,
Creator: creator,
Package: memPkg,
Deposit: deposit,
Metadata: metadata,
}
tx := std.Tx{
Msgs: []std.Msg{msg},
Expand Down
6 changes: 6 additions & 0 deletions gno.land/pkg/sdk/vm/errors.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
// NOTE: these are meant to be used in conjunction with pkgs/errors.
type (
InvalidPkgPathError struct{ abciError }
InvalidPkgMetaError struct{ abciError }
NoRenderDeclError struct{ abciError }
PkgExistError struct{ abciError }
InvalidStmtError struct{ abciError }
Expand All @@ -28,6 +29,7 @@
)

func (e InvalidPkgPathError) Error() string { return "invalid package path" }
func (e InvalidPkgMetaError) Error() string { return "invalid package metadata" }

Check warning on line 32 in gno.land/pkg/sdk/vm/errors.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/errors.go#L32

Added line #L32 was not covered by tests
func (e NoRenderDeclError) Error() string { return "render function not declared" }
func (e PkgExistError) Error() string { return "package already exists" }
func (e InvalidStmtError) Error() string { return "invalid statement" }
Expand All @@ -52,6 +54,10 @@
return errors.Wrap(InvalidPkgPathError{}, msg)
}

func ErrInvalidPkgMeta(msg string) error {
return errors.Wrap(InvalidPkgMetaError{}, msg)

Check warning on line 58 in gno.land/pkg/sdk/vm/errors.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/errors.go#L57-L58

Added lines #L57 - L58 were not covered by tests
}

func ErrInvalidStmt(msg string) error {
return errors.Wrap(InvalidStmtError{}, msg)
}
Expand Down
20 changes: 20 additions & 0 deletions gno.land/pkg/sdk/vm/handler.go
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@
QueryFuncs = "qfuncs"
QueryEval = "qeval"
QueryFile = "qfile"
QueryMeta = "qmeta"
)

func (vh vmHandler) Query(ctx sdk.Context, req abci.RequestQuery) abci.ResponseQuery {
Expand All @@ -95,6 +96,8 @@
res = vh.queryEval(ctx, req)
case QueryFile:
res = vh.queryFile(ctx, req)
case QueryMeta:
res = vh.queryMeta(ctx, req)

Check warning on line 100 in gno.land/pkg/sdk/vm/handler.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/handler.go#L99-L100

Added lines #L99 - L100 were not covered by tests
default:
return sdk.ABCIResponseQueryFromError(
std.ErrUnknownRequest(fmt.Sprintf(
Expand Down Expand Up @@ -201,6 +204,23 @@
return
}

// queryMeta returns the value for a package metadata field.
func (vh vmHandler) queryMeta(ctx sdk.Context, req abci.RequestQuery) (res abci.ResponseQuery) {
parts := strings.SplitN(string(req.Data), ":", 2)
if len(parts) != 2 {
panic("expected <pkgpath>:<field_name> syntax in query input data")

Check warning on line 211 in gno.land/pkg/sdk/vm/handler.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/handler.go#L208-L211

Added lines #L208 - L211 were not covered by tests
}

pkgPath, name := parts[0], parts[1]
result, err := vh.vm.QueryMeta(ctx, pkgPath, name)
if err != nil {
res = sdk.ABCIResponseQueryFromError(err)
return
}
res.Data = result
return

Check warning on line 221 in gno.land/pkg/sdk/vm/handler.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/handler.go#L214-L221

Added lines #L214 - L221 were not covered by tests
}

// ----------------------------------------
// misc

Expand Down
83 changes: 81 additions & 2 deletions gno.land/pkg/sdk/vm/keeper.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
goerrors "errors"
"fmt"
"io"
Expand Down Expand Up @@ -37,10 +39,15 @@
)

const (
maxAllocTx = 500_000_000
maxAllocQuery = 1_500_000_000 // higher limit for queries
maxAllocQuery = 1_500_000_000 // higher limit for queries
maxAllocTx = 500_000_000
maxMetaFieldValueSize = 1_000_000 // maximum size for package metadata field values in bytes
maxMetaFields = 10 // maximum number of package metadata fields
maxToolDescriptionLenght = 100
)

var reToolName = regexp.MustCompile(`^[a-z]+[_a-z0-9]{5,16}$`)

// vm.VMKeeperI defines a module interface that supports Gno
// smart contracts programming (scripting).
type VMKeeperI interface {
Expand Down Expand Up @@ -353,6 +360,18 @@
return ErrTypeCheck(err)
}

// Set package metadata
for _, f := range msg.Metadata {
if f.Name == "tools" {
// The "tools" metadata field must have a valid pre-defined structure
if err := validateToolsMetaField(f.Value); err != nil {
return ErrInvalidPkgMeta(err.Error())
}

Check warning on line 369 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L365-L369

Added lines #L365 - L369 were not covered by tests
}

gnostore.SetPackageMetaField(msg.Package.Path, f.Name, f.Value)

Check warning on line 372 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L372

Added line #L372 was not covered by tests
}

// Pay deposit from creator.
pkgAddr := gno.DerivePkgAddr(pkgPath)

Expand Down Expand Up @@ -843,6 +862,27 @@
}
}

func (vm *VMKeeper) QueryMeta(ctx sdk.Context, pkgPath, name string) ([]byte, error) {
var (
res []byte
store = vm.newGnoTransactionStore(ctx) // throwaway (never committed)
)

found := store.IteratePackageMeta(pkgPath, func(field string, value []byte) bool {
if field == name {
res = make([]byte, base64.StdEncoding.EncodedLen(len(value)))
base64.StdEncoding.Encode(res, value)
return true
}
return false

Check warning on line 877 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L865-L877

Added lines #L865 - L877 were not covered by tests
})

if !found {
return nil, fmt.Errorf("metadata field for package %s not found: %s", pkgPath, name) // TODO: XSS protection
}
return res, nil

Check warning on line 883 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L880-L883

Added lines #L880 - L883 were not covered by tests
}

// logTelemetry logs the VM processing telemetry
func logTelemetry(
gasUsed int64,
Expand Down Expand Up @@ -874,3 +914,42 @@
metric.WithAttributes(attributes...),
)
}

func validateToolsMetaField(value []byte) error {
var v struct {
Tools []struct {
Name string `json:"name"`
Weight float64 `json:"weight"`
Description string `json:"description,omitempty"`
} `json:"tools,omitempty"`
}

if err := json.Unmarshal(value, &v); err != nil {
return fmt.Errorf("invalid tools field format: %s", err)

Check failure on line 928 in gno.land/pkg/sdk/vm/keeper.go

View workflow job for this annotation

GitHub Actions / Run gno.land suite / Go Lint / lint

non-wrapping format verb for fmt.Errorf. Use `%w` to format errors (errorlint)
}

Check warning on line 929 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L918-L929

Added lines #L918 - L929 were not covered by tests

if len(v.Tools) == 0 {
return errors.New("tools list is empty")
}

Check warning on line 933 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L931-L933

Added lines #L931 - L933 were not covered by tests

var totalWeight float64
for _, t := range v.Tools {
if len(t.Description) > maxToolDescriptionLenght {
return fmt.Errorf("maximum length for tool description is %d", maxToolDescriptionLenght)
}

Check warning on line 939 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L935-L939

Added lines #L935 - L939 were not covered by tests

if !reToolName.MatchString(t.Name) {
return fmt.Errorf(
"invalid tool name: %s (minimum 6 chars, lowercase alphanumeric with underscore)",
t.Name,
)
}

Check warning on line 946 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L941-L946

Added lines #L941 - L946 were not covered by tests

totalWeight += t.Weight

Check warning on line 948 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L948

Added line #L948 was not covered by tests
}

if totalWeight != 1 {
return errors.New("the sum of all tool weights must be 1")
}
return nil

Check warning on line 954 in gno.land/pkg/sdk/vm/keeper.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/keeper.go#L951-L954

Added lines #L951 - L954 were not covered by tests
}
2 changes: 1 addition & 1 deletion gno.land/pkg/sdk/vm/msg_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func TestMsgAddPackage_ValidateBasic(t *testing.T) {
{
name: "valid message",
msg: NewMsgAddPackage(creator, pkgPath, files),
expectSignBytes: `{"creator":"g14ch5q26mhx3jk5cxl88t278nper264ces4m8nt","deposit":"",` +
expectSignBytes: `{"creator":"g14ch5q26mhx3jk5cxl88t278nper264ces4m8nt","deposit":"","metadata":null,` +
`"package":{"files":[{"body":"package test\n\t\tfunc Echo() string {return \"hello world\"}",` +
`"name":"test.gno"}],"name":"test","path":"gno.land/r/namespace/test"}}`,
expectErr: nil,
Expand Down
46 changes: 40 additions & 6 deletions gno.land/pkg/sdk/vm/msgs.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import (
"fmt"
"regexp"
"strings"

"github.com/gnolang/gno/gnovm"
Expand All @@ -12,15 +13,26 @@
"github.com/gnolang/gno/tm2/pkg/std"
)

var reMetaFieldName = regexp.MustCompile(`^[a-zA-Z0-9][a-zA-Z0-9_-]*$`)

//----------------------------------------
// MsgAddPackage

// MsgAddPackage - create and initialize new package
type MsgAddPackage struct {
Creator crypto.Address `json:"creator" yaml:"creator"`
Package *gnovm.MemPackage `json:"package" yaml:"package"`
Deposit std.Coins `json:"deposit" yaml:"deposit"`
}
type (
// MsgAddPackage - create and initialize new package
MsgAddPackage struct {
Creator crypto.Address `json:"creator" yaml:"creator"`
Package *gnovm.MemPackage `json:"package" yaml:"package"`
Deposit std.Coins `json:"deposit" yaml:"deposit"`
Metadata []*MetaField `json:"metadata" yaml:"metadata"`
}

// MetaField - package metadata field
MetaField struct {
Name string `json:"name" yaml:"name"`
Value []byte `json:"value" yaml:"value"`
}
)

var _ std.Msg = MsgAddPackage{}

Expand Down Expand Up @@ -60,6 +72,28 @@
if !msg.Deposit.IsValid() {
return std.ErrInvalidCoins(msg.Deposit.String())
}

if len(msg.Metadata) > maxMetaFields {
return ErrInvalidPkgMeta("maximum number of metadata fields exceeded")
}

Check warning on line 78 in gno.land/pkg/sdk/vm/msgs.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/msgs.go#L77-L78

Added lines #L77 - L78 were not covered by tests

seenFields := make(map[string]struct{}, len(msg.Metadata))
for _, f := range msg.Metadata {
if !reMetaFieldName.Match([]byte(f.Name)) {
return ErrInvalidPkgMeta("invalid metadata field name")
}

Check warning on line 84 in gno.land/pkg/sdk/vm/msgs.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/msgs.go#L82-L84

Added lines #L82 - L84 were not covered by tests

if _, exists := seenFields[f.Name]; exists {
return ErrInvalidPkgMeta("metadata contains duplicated fields")
}

Check warning on line 88 in gno.land/pkg/sdk/vm/msgs.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/msgs.go#L86-L88

Added lines #L86 - L88 were not covered by tests

if len(f.Value) > maxMetaFieldValueSize {
return ErrInvalidPkgMeta("metadata field value is too large")
}

Check warning on line 92 in gno.land/pkg/sdk/vm/msgs.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/msgs.go#L90-L92

Added lines #L90 - L92 were not covered by tests

seenFields[f.Name] = struct{}{}

Check warning on line 94 in gno.land/pkg/sdk/vm/msgs.go

View check run for this annotation

Codecov / codecov/patch

gno.land/pkg/sdk/vm/msgs.go#L94

Added line #L94 was not covered by tests
}

// XXX validate files.
return nil
}
Expand Down
4 changes: 4 additions & 0 deletions gno.land/pkg/sdk/vm/package.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,12 @@ var Package = amino.RegisterPackage(amino.NewPackage(
MsgRun{}, "m_run",
MsgAddPackage{}, "m_addpkg", // TODO rename both to MsgAddPkg?

// other
MetaField{}, "MetaField",

// errors
InvalidPkgPathError{}, "InvalidPkgPathError",
InvalidPkgMetaError{}, "InvalidPkgMetaError",
NoRenderDeclError{}, "NoRenderDeclError",
PkgExistError{}, "PkgExistError",
InvalidStmtError{}, "InvalidStmtError",
Expand Down
Loading
Loading