Skip to content

Commit 3ee3c59

Browse files
authored
chore: add feluda semantic versioning blog (#245)
1 parent 3996eca commit 3ee3c59

File tree

1 file changed

+111
-0
lines changed

1 file changed

+111
-0
lines changed
Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
---
2+
name: How to make sure too many cooks don’t spoil the broth
3+
excerpt: A custom approach to do multi-package semantic versioning in a Python monorepo
4+
author: Aatman Vaidya, Priyash Shah
5+
project: Feluda
6+
date: 2025-04-01
7+
tags: devlog
8+
cover:
9+
---
10+
11+
In Feluda, functionalities are abstracted into something we call **operators**, which are built keeping in mind the need to process data in various modalities (text, audio, video, images, hybrid) and various languages. You can think of operators as plugins that you can mix and match to perform different analysis on your data. This abstraction also allows others to write operators for their unique use case and use them within Feluda.
12+
Feluda acts like a conductor in an orchestra where it brings together all the operators to let them do their function. For example, if you wish to cluster large amounts of videos, then operators will help you convert a video to an embedding, and then use embeddings to cluster them. While we encourage the community to build and use their own operators, we also provide some core operators out of the box with Feluda, which are maintained within the same Feluda repository.
13+
14+
### **The Challenge**
15+
16+
In a monorepo with multiple Python packages we wanted a dynamic system that could automatically maintain and update package versions. As manually tracking changes and version numbers is tedious and not manageable as number of packages increase, we needed a solution that could:
17+
18+
* Automatically detect which packages have changed
19+
* Determine the appropriate version bump based on commit messages (follow [semantic versioning](https://semver.org/) rules)
20+
* Update package versions in pyproject.toml files
21+
* Create corresponding Git tags
22+
* Handle all of this while maintaining semantic versioning principles
23+
24+
Here is a demo folder structure we wanted to cater too
25+
```
26+
.(root)
27+
├── package1/
28+
│ ├── package1.py
29+
│ ├── test_package1.py
30+
│ ├── pyproject.toml
31+
│ └── README.md
32+
├── package2/
33+
│ ├── package2.py
34+
│ ├── test_package2.py
35+
│ ├── pyproject.toml
36+
│ └── README.md
37+
└── package3/
38+
├── package3.py
39+
├── test_package3.py
40+
├── pyproject.toml
41+
└── README.md
42+
```
43+
44+
### **The Solution**
45+
46+
We wrote a [custom script](https://github.com/tattle-made/feluda/blob/main/scripts/semantic_release_workflow.py) ([github action](https://github.com/tattle-made/feluda/blob/main/.github/workflows/merge-main.yml)) that analyzes commit messages between two **git commits** and automatically handles version management. Here's how it works:
47+
48+
Package Discovery
49+
The script starts by discovering all Python packages located in our monorepo. It looks for packages in two locations:
50+
51+
- The root package (named "feluda" in our case)
52+
- Any package inside the **operators** directory (as each operator is a folder inside it)
53+
54+
Each package must have a valid pyproject.toml file containing the current version number.
55+
56+
Analyzing Changes
57+
For each package, the script:
58+
59+
* Retrieves all commits affecting that package between two specified Git commits range.
60+
* Analyzes each commit message using [conventional commit format](https://www.conventionalcommits.org/en/v1.0.0/).
61+
* Determines the highest priority version bump needed
62+
63+
The commit analysis follows these rules:
64+
65+
- Commits containing **"breaking change"** trigger a **major** version bump
66+
- Commits starting with **"feat:"** trigger a **minor** version bump
67+
- Other conventional commits **(fix:, chore:, etc.)** trigger a **patch** bump
68+
69+
Version Management
70+
Once the script determines the necessary version bump, it:
71+
72+
* Checks if a Git tag already exists for the new version
73+
* Updates the version in pyproject.toml if needed
74+
* Creates a new Git tag using the format specified in pyproject.toml
75+
76+
The script also includes comprehensive error handling to ensure reliability, it validates repository paths and package structures, handles missing or malformed pyproject.toml files, manages git command failures gracefully and provides clear error messages for troubleshooting.
77+
78+
#### Here is a detailed flowchart of this approach \- [https://github.com/tattle-made/feluda/wiki/Release-Management\#semantic-versioning](https://github.com/tattle-made/feluda/wiki/Release-Management#semantic-versioning)
79+
80+
### **Best Practices and Benefits of this Approach**
81+
82+
To make the most of this automation:
83+
84+
1. Always use conventional commit messages:
85+
86+
```
87+
feat: add new feature
88+
fix: resolve bug
89+
chore: update dependencies
90+
```
91+
92+
2. Include proper tag format in **pyproject.toml**:
93+
94+
```
95+
[tool.semantic_release.branches.main]
96+
tag_format = "{name}-{version}"
97+
```
98+
99+
3. Run the script as part of your CI/CD pipeline between a range of commits:
100+
```
101+
python semantic_release_workflow.py <previous_commit> <current_commit>
102+
```
103+
104+
This automated approach offers several advantages:
105+
106+
1. **Consistency**: Version numbers always reflect the nature of changes made
107+
2. **Efficiency**: Eliminates manual version management reducing human error in version numbering.
108+
3. **Traceability**: Version changes are directly linked to commit history
109+
110+
### **Link to the [Custom Python Script](https://github.com/tattle-made/feluda/blob/main/scripts/semantic_release_workflow.py)**
111+
### **Link to the [Github Action](https://github.com/tattle-made/feluda/blob/main/.github/workflows/merge-main.yml)**

0 commit comments

Comments
 (0)