Skip to content

Commit cec4f7d

Browse files
AidChengtopshihunjerry609ztygodjuanlou1217
authored
[buck2]REFECTOR: Refactorising and Integrating a Generic API model and WebSocket (#1909)
* refactor: some modules (#4) * feat: add refactor orion module for migration Signed-off-by: Wangyan <topshihun@foxmail.com> * fix: add CL for build Signed-off-by: Wangyan <topshihun@foxmail.com> * refactor: some modules Signed-off-by: Wangyan <topshihun@foxmail.com> --------- Signed-off-by: Wangyan <topshihun@foxmail.com> Signed-off-by: Aid_C <57825561+AidCheng@users.noreply.github.com> Co-authored-by: Aid_C <57825561+AidCheng@users.noreply.github.com> * [orion]REFACTOR: Integrate generic jupiter model with refactored orion structure Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * fix: enforce mount_path semantics and stabilize builds (#1836) * fix: enforce mount_path semantics and stabilize builds Consolidate mount_path handling with buck2 prechecks and fix BuildDTO repo handling to avoid moves. Signed-off-by: jerry609 <1772030600@qq.com> style: format code with cargo fmt Signed-off-by: jerry609 <1772030600@qq.com> chore(orion): remove unused mount helpers Signed-off-by: jerry609 <1772030600@qq.com> fix(orion): retry buck2 targets once on failure Signed-off-by: jerry609 <1772030600@qq.com> fix(orion): remount and retry target discovery on failure Signed-off-by: jerry609 <1772030600@qq.com> style(orion): format buck_controller after retry changes Signed-off-by: jerry609 <1772030600@qq.com> fix(orion): annotate changes type for retry path Signed-off-by: jerry609 <1772030600@qq.com> fix(orion): appease clippy for mount guards Signed-off-by: jerry609 <1772030600@qq.com> test(common): avoid clippy field reassignment lint Signed-off-by: jerry609 <1772030600@qq.com> * test(ceres): fix clippy lints in tests Signed-off-by: jerry609 <1772030600@qq.com> --------- Signed-off-by: jerry609 <1772030600@qq.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * Feature/comment reanchor (#1874) * refactor(code_review): add anchor and position to locat code thread * feat: add reanchor thread function * fix: seaORM repeated * fix: rebase error * fix: lcs_len error * fix: fix CR problems * fix: error Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * feat(theme): Complete theme color migration for multiple modules (#1875) * feat(theme): Complete theme color system migration for CL, Issue, Queue, OrionClient, and Settings modules * feat(theme): Complete theme color system migration for CL, Issue, Queue, OrionClient, and Settings modules * feat(theme): Complete theme color system migration for CL, Issue, Queue, OrionClient, and Settings modules * feat(theme): Complete theme color system migration for CL, Issue, Queue, OrionClient, and Settings modules Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * refactor(mono): convert third-party and buckal-bundles init to script (#1876) Signed-off-by: MYUU <1405758738@qq.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * chore: move log models under buck2 types (#1878) Signed-off-by: lvy010 <17338770572@163.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * test(mono): add multi-user support and files-list API (#1881) * test(mono): Add CL merge and update-branch integration test Signed-off-by: miliu2cc <miliu2cc@gmail.com> * test(mono): Add CL merge and update-branch integration test Signed-off-by: miliu2cc <miliu2cc@gmail.com> * test(mono): add multi-user support and files-list API Signed-off-by: miliu2cc <miliu2cc@gmail.com> * test(mono): add multi-user support and files-list API Signed-off-by: miliu2cc <miliu2cc@gmail.com> --------- Signed-off-by: miliu2cc <miliu2cc@gmail.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * fix(script):cleanup per-repo .git after processing (#1884) Signed-off-by: MYUU <1405758738@qq.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * feat: implement build trigger system (#1880) * feat: implement build trigger system Signed-off-by: allure <1550220889@qq.com> * feat: refactor build trigger system and fix error Signed-off-by: allure <1550220889@qq.com> --------- Signed-off-by: allure <1550220889@qq.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * fix(orion): skip incompatible targets for buck2 builds (#1885) Signed-off-by: jerry609 <1772030600@qq.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * fixed(cl): update UI for checks log section (#1887) Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * refactor: some modules (#4) * feat: add refactor orion module for migration Signed-off-by: Wangyan <topshihun@foxmail.com> * fix: add CL for build Signed-off-by: Wangyan <topshihun@foxmail.com> * refactor: some modules Signed-off-by: Wangyan <topshihun@foxmail.com> --------- Signed-off-by: Wangyan <topshihun@foxmail.com> Signed-off-by: Aid_C <57825561+AidCheng@users.noreply.github.com> Co-authored-by: Aid_C <57825561+AidCheng@users.noreply.github.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * [buck2]REFACTOR: Integrate orion and orion server with generic api and model interface Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * FIX: use correct ProjectRelativePath crate Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * UPDATE: cargo fmt Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * [orion] Replace worker-server commun with WS msg Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * [orion]FIX: update branch and fix conflicts * [Orion]FIX: workflow error Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * [orion]UPDATE: fix based on copilot suggestions Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * cargo fmt Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * UPDATE: temporary allow unused started at in build info Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * FIX: remove unused import Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * fmt Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> * [jupiter]REMOVE: duplicate migration Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Aid_C <57825561+AidCheng@users.noreply.github.com> * [jupiter]FIX: temporary remove task.rs Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> --------- Signed-off-by: Wangyan <topshihun@foxmail.com> Signed-off-by: Aid_C <57825561+AidCheng@users.noreply.github.com> Signed-off-by: AidCheng <cn.aiden.cheng@gmail.com> Signed-off-by: jerry609 <1772030600@qq.com> Signed-off-by: MYUU <1405758738@qq.com> Signed-off-by: lvy010 <17338770572@163.com> Signed-off-by: miliu2cc <miliu2cc@gmail.com> Signed-off-by: allure <1550220889@qq.com> Co-authored-by: TOP <89700178+topshihun@users.noreply.github.com> Co-authored-by: Jerry Zhang <1772030600@qq.com> Co-authored-by: TianYi <2739022972@qq.com> Co-authored-by: zhaokang <zhaokang.r2cn@isrc.iscas.ac.cn> Co-authored-by: MYUU <52773565+zhoujiaqi30@users.noreply.github.com> Co-authored-by: lvy010 <17338770572@163.com> Co-authored-by: miliu2cc <98928567+miliu2cc@users.noreply.github.com> Co-authored-by: Yao <1550220889@qq.com> Co-authored-by: zyd123-cmd <yingdong.r2cn@isrc.iscas.ac.cn> Co-authored-by: Tianxing Ye <benjamin.747@outlook.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
1 parent b27cc90 commit cec4f7d

File tree

30 files changed

+915
-753
lines changed

30 files changed

+915
-753
lines changed

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -141,6 +141,7 @@ redis-test = "1.0.1"
141141
bincode = "2.0.1"
142142
rustls = "0.23.36"
143143
object_store = "0.13.1"
144+
parse-display = "0.8.2"
144145
qlean = "0.2.2"
145146
toml = "0.9.8"
146147

api-model/Cargo.toml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,3 +12,6 @@ path = "src/lib.rs"
1212
[dependencies]
1313
serde = { workspace = true, features = ["derive"] }
1414
utoipa = { workspace = true }
15+
anyhow = {workspace = true}
16+
thiserror = {workspace = true}
17+
parse-display = {workspace = true}

api-model/src/buck2/api.rs

Lines changed: 37 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -9,32 +9,58 @@
99
//! - Used by both server and mono crates only
1010
1111
use serde::{Deserialize, Serialize};
12+
use utoipa::ToSchema;
1213

13-
use crate::buck2::types::{ProjectRelativePath, Status};
14+
use crate::buck2::{status::Status, types::ProjectRelativePath};
1415

1516
/// Parameters required to build a task.
1617
#[allow(dead_code)]
17-
#[derive(Debug)]
18+
#[derive(Debug, Deserialize, Serialize, ToSchema)]
1819
pub struct TaskBuildRequest {
1920
/// The repository base path
2021
pub repo: String,
2122
/// The change list link (URL)
22-
pub cl: String,
23+
pub cl_link: String,
24+
//TODO: for old database only, delete after updated
25+
pub cl_id: i64,
2326
/// The list of file diff changes
2427
pub changes: Vec<Status<ProjectRelativePath>>,
28+
/// Buck2 target path (e.g. //app:server). Optional for backward compatibility.
29+
#[serde(default, alias = "targets_path")]
30+
pub targets: Option<Vec<String>>,
31+
}
32+
33+
impl TaskBuildRequest {
34+
pub fn targets(&self) -> Vec<String> {
35+
self.targets.clone().unwrap_or_default()
36+
}
37+
}
38+
39+
/// Request structure for Retry a build
40+
#[derive(Debug, Clone, Deserialize, ToSchema)]
41+
pub struct RetryBuildRequest {
42+
pub build_id: String,
43+
pub cl_link: String,
44+
pub cl_id: i64,
45+
pub changes: Vec<Status<ProjectRelativePath>>,
46+
pub targets: Option<Vec<String>>,
2547
}
2648

2749
/// Result of a task build operation containing status and metadata. Used by Orion-Server
2850
#[allow(dead_code)]
29-
#[derive(Debug, Deserialize, Serialize)]
30-
pub struct TaskBuildResult {
31-
/// Whether the build operation was successful
32-
pub success: bool,
51+
#[derive(Debug, Deserialize, Serialize, ToSchema)]
52+
pub struct OrionBuildResult {
3353
/// Unique identifier for the build task
34-
pub id: String,
35-
/// Process exit code (None if not yet completed)
36-
#[serde(skip_serializing_if = "Option::is_none")]
37-
pub exit_code: Option<i32>,
54+
pub build_id: String,
55+
/// Current status of the build (e.g., "queued", "running", "success", "failure")
56+
pub status: String,
3857
/// Human-readable status or error message
3958
pub message: String,
4059
}
60+
61+
/// Response structure for build-related API endpoints in Orion-Server
62+
#[derive(Debug, Deserialize, Serialize, ToSchema)]
63+
pub struct OrionServerResponse {
64+
pub task_id: String,
65+
pub results: Vec<OrionBuildResult>,
66+
}

api-model/src/buck2/mod.rs

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
pub mod api;
2+
pub mod status;
23
pub mod types;
34
pub mod ws;

api-model/src/buck2/status.rs

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,171 @@
1+
/*
2+
* Copyright (c) Meta Platforms, Inc. and affiliates.
3+
*
4+
* This source code is dual-licensed under either the MIT license found in the
5+
* LICENSE-MIT file in the root directory of this source tree or the Apache
6+
* License, Version 2.0 found in the LICENSE-APACHE file in the root directory
7+
* of this source tree. You may select, at your option, one of the
8+
* above-listed licenses.
9+
*/
10+
11+
use std::{fs, path::Path};
12+
13+
use anyhow::Context as _;
14+
use thiserror::Error;
15+
use utoipa::ToSchema;
16+
17+
pub use crate::buck2::types::ProjectRelativePath;
18+
19+
#[derive(Debug, PartialEq, Eq, Hash, Clone, serde::Serialize, serde::Deserialize, ToSchema)]
20+
pub enum Status<Path> {
21+
Modified(Path),
22+
Added(Path),
23+
Removed(Path),
24+
}
25+
26+
#[derive(Error, Debug)]
27+
enum StatusParseError {
28+
#[error("Unexpected line format: {0}")]
29+
UnexpectedFormat(String),
30+
#[error("Unknown line prefix: {0}")]
31+
UnknownPrefix(String),
32+
}
33+
34+
impl Status<ProjectRelativePath> {
35+
/// Creates a new Modified status from a file path string
36+
pub fn modified(path: &str) -> Self {
37+
Self::Modified(ProjectRelativePath::new(path))
38+
}
39+
40+
/// Creates a new Added status from a file path string
41+
pub fn added(path: &str) -> Self {
42+
Self::Added(ProjectRelativePath::new(path))
43+
}
44+
45+
/// Creates a new Removed status from a file path string
46+
pub fn removed(path: &str) -> Self {
47+
Self::Removed(ProjectRelativePath::new(path))
48+
}
49+
50+
fn from_str(value: &str) -> anyhow::Result<Self> {
51+
let mut it = value.chars();
52+
let typ = it.next();
53+
if it.next() != Some(' ') {
54+
return Err(StatusParseError::UnexpectedFormat(value.to_owned()).into());
55+
}
56+
let path = ProjectRelativePath::new(it.as_str());
57+
match typ {
58+
Some('A') => Ok(Self::Added(path)),
59+
Some('M') => Ok(Self::Modified(path)),
60+
Some('R') => Ok(Self::Removed(path)),
61+
Some('D') => Ok(Self::Removed(path)), // used by jujutsu
62+
_ => Err(StatusParseError::UnknownPrefix(value.to_owned()).into()),
63+
}
64+
}
65+
}
66+
67+
impl<Path> Status<Path> {
68+
pub fn get(&self) -> &Path {
69+
match self {
70+
Status::Modified(x) => x,
71+
Status::Added(x) => x,
72+
Status::Removed(x) => x,
73+
}
74+
}
75+
76+
pub fn map<'a, T: 'a>(&'a self, f: impl FnOnce(&'a Path) -> T) -> Status<T> {
77+
match self {
78+
Status::Modified(x) => Status::Modified(f(x)),
79+
Status::Added(x) => Status::Added(f(x)),
80+
Status::Removed(x) => Status::Removed(f(x)),
81+
}
82+
}
83+
84+
pub fn try_map<T, E>(&self, f: impl FnOnce(&Path) -> Result<T, E>) -> Result<Status<T>, E> {
85+
Ok(match self {
86+
Status::Modified(x) => Status::Modified(f(x)?),
87+
Status::Added(x) => Status::Added(f(x)?),
88+
Status::Removed(x) => Status::Removed(f(x)?),
89+
})
90+
}
91+
92+
pub fn into_map<T>(self, f: impl FnOnce(Path) -> T) -> Status<T> {
93+
match self {
94+
Status::Modified(x) => Status::Modified(f(x)),
95+
Status::Added(x) => Status::Added(f(x)),
96+
Status::Removed(x) => Status::Removed(f(x)),
97+
}
98+
}
99+
100+
pub fn into_try_map<T, E>(self, f: impl FnOnce(Path) -> Result<T, E>) -> Result<Status<T>, E> {
101+
Ok(match self {
102+
Status::Modified(x) => Status::Modified(f(x)?),
103+
Status::Added(x) => Status::Added(f(x)?),
104+
Status::Removed(x) => Status::Removed(f(x)?),
105+
})
106+
}
107+
}
108+
109+
pub fn read_status(path: &Path) -> anyhow::Result<Vec<Status<ProjectRelativePath>>> {
110+
parse_status(
111+
&fs::read_to_string(path).with_context(|| format!("When reading `{}`", path.display()))?,
112+
)
113+
}
114+
115+
fn parse_status(data: &str) -> anyhow::Result<Vec<Status<ProjectRelativePath>>> {
116+
data.lines()
117+
.map(Status::from_str)
118+
.collect::<anyhow::Result<Vec<_>>>()
119+
}
120+
121+
#[cfg(test)]
122+
mod tests {
123+
use super::*;
124+
125+
#[test]
126+
fn test_status() {
127+
let src = r#"
128+
M proj/foo.rs
129+
M bar.rs
130+
A baz/file.txt
131+
R quux.js
132+
"#;
133+
assert_eq!(
134+
parse_status(&src[1..]).unwrap(),
135+
vec![
136+
Status::Modified(ProjectRelativePath::new("proj/foo.rs")),
137+
Status::Modified(ProjectRelativePath::new("bar.rs")),
138+
Status::Added(ProjectRelativePath::new("baz/file.txt")),
139+
Status::Removed(ProjectRelativePath::new("quux.js"))
140+
]
141+
);
142+
}
143+
144+
#[test]
145+
fn test_status_error() {
146+
assert!(parse_status("X quux.js").is_err());
147+
assert!(parse_status("notaline").is_err());
148+
assert!(parse_status("not a line").is_err());
149+
}
150+
151+
#[test]
152+
fn test_status_constructors() {
153+
let modified = Status::modified("foo/modified.rs");
154+
let modified_parsed = Status::from_str("M foo/modified.rs").unwrap();
155+
assert!(matches!(modified, Status::Modified(_)));
156+
assert_eq!(modified, modified_parsed);
157+
assert_eq!(modified.get().as_str(), "foo/modified.rs");
158+
159+
let added = Status::added("foo/added.rs");
160+
let added_parsed = Status::from_str("A foo/added.rs").unwrap();
161+
assert!(matches!(added, Status::Added(_)));
162+
assert_eq!(added, added_parsed);
163+
assert_eq!(added.get().as_str(), "foo/added.rs");
164+
165+
let removed = Status::removed("foo/removed.rs");
166+
let removed_parsed = Status::from_str("R foo/removed.rs").unwrap();
167+
assert!(matches!(removed, Status::Removed(_)));
168+
assert_eq!(removed, removed_parsed);
169+
assert_eq!(removed.get().as_str(), "foo/removed.rs");
170+
}
171+
}

api-model/src/buck2/types.rs

Lines changed: 40 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,9 @@
11
//! Types related to Buck2 build system.
22
3+
use parse_display::Display;
34
use serde::{Deserialize, Serialize};
45
use utoipa::ToSchema;
56

6-
/// Request from server to worker to build a target.
7-
#[allow(dead_code)]
8-
#[derive(Debug, Clone, PartialEq, Eq, Hash, Serialize, Deserialize, ToSchema)]
9-
pub enum Status<Path> {
10-
Modified(Path),
11-
Added(Path),
12-
Removed(Path),
13-
}
14-
157
/// Task phase when in buck2 build
168
#[allow(dead_code)]
179
#[derive(Debug, Clone, Serialize, Deserialize, ToSchema, PartialEq, Eq)]
@@ -20,23 +12,52 @@ pub enum TaskPhase {
2012
RunningBuild,
2113
}
2214

23-
/// Represents a file path relative to the project root.
24-
#[allow(dead_code)]
25-
#[derive(Clone, Debug, Hash, PartialEq, Eq, Deserialize, Serialize)]
15+
#[derive(Clone, Debug, Hash, PartialEq, Eq, Display, Deserialize, Serialize, ToSchema)]
2616
pub struct ProjectRelativePath(String);
2717

2818
impl ProjectRelativePath {
29-
/// Creates a new ProjectRelativePath from a string slice.
3019
pub fn new(path: &str) -> Self {
3120
Self(path.to_owned())
3221
}
3322

34-
/// Attempts to create a ProjectRelativePath from an absolute path and a base path.
35-
pub fn from_abs(abs_path: &str, base: &str) -> Option<Self> {
36-
let opt = abs_path
37-
.strip_prefix(base)
38-
.map(|s| s.trim_start_matches("/"));
39-
opt.map(|s| Self(s.to_owned()))
23+
pub fn join(&self, suffix: &str) -> Self {
24+
if self.0.is_empty() {
25+
Self(suffix.to_owned())
26+
} else {
27+
Self(format!("{}/{}", self.0, suffix))
28+
}
29+
}
30+
31+
/// ```
32+
/// use api-models::types::ProjectRelativePath;
33+
/// assert_eq!(
34+
/// ProjectRelativePath::new("foo/bar.bzl").extension(),
35+
/// Some("bzl")
36+
/// );
37+
/// assert_eq!(
38+
/// ProjectRelativePath::new("foo/bar.bzl/baz").extension(),
39+
/// None
40+
/// );
41+
/// assert_eq!(ProjectRelativePath::new("foo/bar/baz").extension(), None);
42+
/// ```
43+
pub fn extension(&self) -> Option<&str> {
44+
self.0
45+
.as_str()
46+
.rsplit_once('/')
47+
.unwrap_or_default()
48+
.1
49+
.rsplit_once('.')
50+
.map(|x| x.1)
51+
}
52+
53+
pub fn as_str(&self) -> &str {
54+
&self.0
55+
}
56+
}
57+
58+
impl AsRef<str> for ProjectRelativePath {
59+
fn as_ref(&self) -> &str {
60+
self.as_str()
4061
}
4162
}
4263

0 commit comments

Comments
 (0)