Skip to content
This repository was archived by the owner on Oct 31, 2023. It is now read-only.

Commit 0ea6bd5

Browse files
authored
Release 0.2.4 (#31)
* feat(go): implement list all functions (#29) * doc: add contributing help and doc comments (#30) * Add CHANGELOG entry * chore: Release am_list version 0.2.4
1 parent 04b580b commit 0ea6bd5

16 files changed

+430
-123
lines changed

CHANGELOG.md

+13-5
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
## [Unreleased] - ReleaseDate
1111

12+
## [Version 0.2.4] - 2023-07-06
13+
14+
### Fixed
15+
16+
- [Go] The `list` subcommand now can also list all functions in a
17+
project.
18+
1219
## [Version 0.2.3] - 2023-07-04
1320

1421
### Added
@@ -68,9 +75,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
6875
### Security
6976

7077
<!-- next-url -->
71-
[Unreleased]: https://github.com/autometrics-dev/am_list/compare/v0.2.3...HEAD
78+
[Unreleased]: https://github.com/autometrics-dev/am_list/compare/v0.2.4...HEAD
79+
[Version 0.2.4]: https://github.com/autometrics-dev/am_list/compare/v0.2.3...v0.2.4
7280
[Version 0.2.3]: https://github.com/autometrics-dev/am_list/compare/v0.2.2...v0.2.3
73-
[Version 0.2.2]: https://github.com/gagbo/am_list/compare/v0.2.1...v0.2.2
74-
[Version 0.2.1]: https://github.com/gagbo/am_list/compare/v0.2.0...v0.2.1
75-
[Version 0.2.0]: https://github.com/gagbo/am_list/compare/v0.1.0...v0.2.0
76-
[Version 0.1.0]: https://github.com/gagbo/am_list/releases/tag/v0.1.0
81+
[Version 0.2.2]: https://github.com/autometrics-dev/am_list/compare/v0.2.1...v0.2.2
82+
[Version 0.2.1]: https://github.com/autometrics-dev/am_list/compare/v0.2.0...v0.2.1
83+
[Version 0.2.0]: https://github.com/autometrics-dev/am_list/compare/v0.1.0...v0.2.0
84+
[Version 0.1.0]: https://github.com/autometrics-dev/am_list/releases/tag/v0.1.0

CONTRIBUTING.md

+63
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,68 @@
11
# Contributing
22

3+
## Implementing language support
4+
5+
To add `am_list` support for a new language, you mostly need to know the usage
6+
patterns of autometrics in this language, and understand how to make and use
7+
tree-sitter queries.
8+
9+
### Function detection architecture
10+
11+
The existing implementations can also be used as a stencil to add a new
12+
language. They all follow the same pattern:
13+
- a `src/{lang}.rs` which implements walking down a directory and collecting the
14+
`ExpectedAmLabel` labels on the files it traverses. This is the location where
15+
the `crate::ListAmFunctions` trait is implemented.
16+
- a `src/{lang}/queries.rs` which is a wrapper around a tree-sitter query, using
17+
the rust bindings.
18+
- a `runtime/queries/{lang}` folder which contains the actual queries being
19+
passed to tree-sitter:
20+
+ the `.scm` files are direct queries, ready to be used "as-is",
21+
+ the `.scm.tpl` files are format strings, which are meant to be templated at
22+
runtime according to other detection happening in the file. For example,
23+
when autometrics is a wrapper function that can be renamed in a file
24+
(`import { autometrics as am } from '@autometrics/autometrics'` in
25+
typescript), we need to change the exact query to detect wrapper usage in
26+
the file.
27+
28+
### Building queries
29+
30+
The hardest part about supporting a language is to build the queries for it. The
31+
best resources to create those are on the main website of tree-sitter library:
32+
33+
- the help section on [Query
34+
syntax](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax)
35+
helps understand how to create queries, and
36+
- the [playground](https://tree-sitter.github.io/tree-sitter/playground) allows
37+
to paste some text and see both the syntax tree and the current highlight from
38+
a query. This enables creating queries incrementally.
39+
40+
![Screenshot of the Tree-sitter playground on their website](./assets/contributing/ts-playground.png)
41+
42+
If you're using Neovim, it also has a very good [playground
43+
plugin](https://github.com/nvim-treesitter/playground) you can use.
44+
45+
The goal is to test the query on the common usage patterns of autometrics, and
46+
make sure that you can match the function names. There are non-trivial
47+
limitations about what a tree-sitter query can and cannot match, so if you're
48+
blocked, don't hesitate to create a discussion or an issue to ask a question,
49+
and we will try to solve the problem together.
50+
51+
### Using queries
52+
53+
Once you have you query, it's easier to wrap it in a Rust structure to keep
54+
track of the indices of the "captures" created within the query. It is best to
55+
look at how the queries are used in the existing implementations and try to
56+
replicate this, seeing how the bindings are used. In order of complexity, you
57+
should look at:
58+
- the [Go](./src/go/queries.rs) implementation, which has a very simple query.
59+
- the [Typescript](./src/typescript/queries.rs) implementation, which uses the
60+
result of a first query to dynamically create a second one, and collect the
61+
results of everything.
62+
- the [Rust](./src/rust/queries.rs) implementation, which uses dynamically
63+
created queries, but also uses a recursion pattern to keep track of modules
64+
declared in-file.
65+
366
## Release management
467

568
The complete release cycle is handled and automated by the combination of `cargo-dist` and `cargo-release`.

Cargo.lock

+5-5
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

+1-1
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
[package]
22
name = "am_list"
3-
version = "0.2.3"
3+
version = "0.2.4"
44
edition = "2021"
55
repository = "https://github.com/autometrics-dev/am_list"
66
authors = ["Fiberplane <[email protected]>", "Gerry Agbobada <[email protected]>"]

assets/contributing/ts-playground.png

276 KB
Loading

runtime/queries/go/all_functions.scm

+5
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
((package_clause
2+
(package_identifier) @pack.name)
3+
4+
(function_declaration
5+
name: (identifier) @func.name))

runtime/queries/go/autometrics.scm

+3-4
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
(
2-
(package_clause
1+
((package_clause
32
(package_identifier) @pack.name)
43

54
(comment) @dir.comment
@@ -8,6 +7,6 @@
87
.
98
(function_declaration
109
name: (identifier) @func.name)
11-
(#match? @dir.comment "^//autometrics:(inst|doc)")
12-
)
10+
(#match? @dir.comment "^//autometrics:(inst|doc)"))
11+
1312

src/go.rs

+34-106
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,12 @@
1-
use crate::{AmlError, ExpectedAmLabel, ListAmFunctions, Result, FUNC_NAME_CAPTURE};
2-
use itertools::Itertools;
1+
mod queries;
2+
3+
use crate::{ExpectedAmLabel, ListAmFunctions, Result};
4+
use queries::{AllFunctionsQuery, AmQuery};
35
use rayon::prelude::*;
46
use std::{collections::HashSet, fs::read_to_string, path::Path};
5-
use tree_sitter::{Parser, Query};
6-
use tree_sitter_go::language;
77
use walkdir::{DirEntry, WalkDir};
88

9-
const PACK_NAME_CAPTURE: &str = "pack.name";
10-
9+
/// Implementation of the Go support for listing autometricized functions.
1110
#[derive(Clone, Copy, Debug, Default)]
1211
pub struct Impl {}
1312

@@ -51,116 +50,45 @@ impl ListAmFunctions for Impl {
5150

5251
list.par_extend(source_mod_pairs.par_iter().filter_map(move |path| {
5352
let source = read_to_string(path).ok()?;
54-
let names = list_function_names(&source).unwrap_or_default();
55-
Some(
56-
names
57-
.into_iter()
58-
.map(move |(module, function)| ExpectedAmLabel { module, function })
59-
.collect::<Vec<_>>(),
60-
)
53+
let query = AmQuery::try_new().ok()?;
54+
let names = query.list_function_names(&source).unwrap_or_default();
55+
Some(names)
6156
}));
6257

6358
let mut result = Vec::with_capacity(PREALLOCATED_ELEMS);
6459
result.extend(list.into_iter().flatten());
6560
Ok(result)
6661
}
6762

68-
fn list_all_functions(&mut self, _project_root: &Path) -> Result<Vec<ExpectedAmLabel>> {
69-
unimplemented!("listing all functions in Golang")
70-
}
71-
}
72-
73-
fn new_parser() -> Result<Parser> {
74-
let mut parser = Parser::new();
75-
parser.set_language(language())?;
76-
Ok(parser)
77-
}
63+
fn list_all_functions(&mut self, project_root: &Path) -> Result<Vec<ExpectedAmLabel>> {
64+
const PREALLOCATED_ELEMS: usize = 100;
65+
let mut list = HashSet::with_capacity(PREALLOCATED_ELEMS);
7866

79-
fn query_builder() -> Result<(Query, u32, u32)> {
80-
let query = Query::new(
81-
language(),
82-
include_str!("../runtime/queries/go/autometrics.scm"),
83-
)?;
84-
let idx = query
85-
.capture_index_for_name(FUNC_NAME_CAPTURE)
86-
.ok_or_else(|| AmlError::MissingNamedCapture(FUNC_NAME_CAPTURE.into()))?;
87-
let mod_idx = query
88-
.capture_index_for_name(PACK_NAME_CAPTURE)
89-
.ok_or_else(|| AmlError::MissingNamedCapture(PACK_NAME_CAPTURE.into()))?;
90-
Ok((query, idx, mod_idx))
91-
}
67+
let walker = WalkDir::new(project_root).into_iter();
68+
let mut source_mod_pairs = Vec::with_capacity(PREALLOCATED_ELEMS);
69+
source_mod_pairs.extend(walker.filter_entry(Self::is_valid).filter_map(|entry| {
70+
let entry = entry.ok()?;
71+
Some(
72+
entry
73+
.path()
74+
.to_str()
75+
.map(ToString::to_string)
76+
.unwrap_or_default(),
77+
)
78+
}));
9279

93-
fn list_function_names(source: &str) -> Result<Vec<(String, String)>> {
94-
let mut parser = new_parser()?;
95-
let (query, idx, mod_idx) = query_builder()?;
96-
let parsed_source = parser.parse(source, None).ok_or(AmlError::Parsing)?;
80+
list.par_extend(source_mod_pairs.par_iter().filter_map(move |path| {
81+
let source = read_to_string(path).ok()?;
82+
let query = AllFunctionsQuery::try_new().ok()?;
83+
let names = query.list_function_names(&source).unwrap_or_default();
84+
Some(names)
85+
}));
9786

98-
let mut cursor = tree_sitter::QueryCursor::new();
99-
// TODO(maint): the complexity/type tetris needs to go down.
100-
cursor
101-
.matches(&query, parsed_source.root_node(), source.as_bytes())
102-
.map(|capture| {
103-
let module = capture
104-
.nodes_for_capture_index(mod_idx)
105-
.next()
106-
.map(|node| node.utf8_text(source.as_bytes()).map(ToString::to_string))
107-
.transpose()?;
108-
let fn_name = capture
109-
.nodes_for_capture_index(idx)
110-
.next()
111-
.map(|node| node.utf8_text(source.as_bytes()).map(ToString::to_string))
112-
.transpose()?;
113-
Ok((module, fn_name))
114-
})
115-
.filter_map_ok(|(module, fn_name)| Some((module?, fn_name?)))
116-
.collect::<std::result::Result<Vec<_>, _>>()
117-
.map_err(|_: anyhow::Error| AmlError::InvalidText)
87+
let mut result = Vec::with_capacity(PREALLOCATED_ELEMS);
88+
result.extend(list.into_iter().flatten());
89+
Ok(result)
90+
}
11891
}
11992

12093
#[cfg(test)]
121-
mod tests {
122-
use super::*;
123-
124-
#[test]
125-
fn detect_simple() {
126-
let source = r#"
127-
package lambda
128-
129-
//autometrics:inst
130-
func the_one() {
131-
return nil
132-
}
133-
"#;
134-
135-
let list = list_function_names(source).unwrap();
136-
137-
assert_eq!(list.len(), 1);
138-
assert_eq!(list[0], ("lambda".to_string(), "the_one".to_string()));
139-
}
140-
141-
#[test]
142-
fn detect_legacy() {
143-
let source = r#"
144-
package lambda
145-
146-
func not_the_one() {
147-
}
148-
149-
//autometrics:doc
150-
func sandwiched_function() {
151-
return nil
152-
}
153-
154-
func not_that_one_either() {
155-
}
156-
"#;
157-
158-
let list = list_function_names(source).unwrap();
159-
160-
assert_eq!(list.len(), 1);
161-
assert_eq!(
162-
list[0],
163-
("lambda".to_string(), "sandwiched_function".to_string())
164-
);
165-
}
166-
}
94+
mod tests;

0 commit comments

Comments
 (0)