Replace go-swagger with huma

[TheFox0x7]

Feature Description

Disclaimer: I'm working on it currently to see if it's doable and so far I think it is. If this gets accepted I'd like to author it.

---

I'd like to suggest dropping go-swagger based generation in favor of automatically generated docs from code with huma

This would bring following benefits:

• No need to update swagger comments
• Always up to date spec with current api
• Autogenerated openapi 3.1 and 3.0 (in both json and yaml formats) plus json schema for it
• Works with chi and few other frameworks/routers so it's more portable

Downsides:

• A rather larger work to get this up
• Drops openapi 2.0 support unless we also provide a converter for it
• Some endpoints might be harder to convert (but could possibly be registered anyway despite not being managed by huma)
• Contexts would need to be examined as function signature is func (context.Context, *struct) (*struct, error), context being the std lib one.¹

Screenshots

No response

Footnotes

1. Semi related: I find current router system with multiple different functions and contexts being accepted really confusing so maybe it's an area that can be simplified a bit? ↩

[lunny]

It seems to be more user-friendly for Chi routers.

[wxiaoguang]

It seems to be more user-friendly for Chi routers.

Gitea isn't really using Chi, it is using a heavily customized router system. Actually we could completely drop Chi by adding some more code since Gitea's "router framework" has been complex enough.

[TheFox0x7]

All integrations are just adapters for it so it isn't a big problem... I hope.

Gitea isn't really using Chi, it is using a heavily customized router system. Actually we could completely drop Chi by adding some more code since Gitea's "router framework" has been complex enough.

Oh. Oh that explains why it's almost like chi but different enough to be more confusing. And why I have trouble wrapping my head around it at times. And why it takes any as a handler... it explains a lot yeah.

[yp05327]

since Gitea's "router framework" has been complex enough.

I really hope it can be used independently by other projects.
I'm already using it in other PJs, but Gitea includes many other unrelated libs.

[TheFox0x7]

off topic

Out of curiosity - why? I find it very confusing compared to basic chi and besides port of the Combo system from macaron (which is a nice feature), I don't see a feature I'd prefer over other frameworks.
If I'd have to compare it based on some browsing of golang web frameworks gitea's router would be some crossover between labstack/echo and gofiber/fiber with c.JSON and c.HTML based responses, but without obviously typed handlers (I don't mean they aren't validated but it's more of a question what any in m.Get(pattern string, h ...any) can be? Without descending into the code I have no way of knowing what is and isn't valid there).

Maybe there's a killer feature which I don't see - hence the question. I'm not even suggesting to swap it, as I know it won't be a good use of anyone's time and won't be accepted. I'm mostly curious what's the big benefit I'm overlooking.

[wxiaoguang]

That's really a long history, it can't simply say it's right or wrong. Some key points are like this:

• any was introduced by (or maybe it is inherited from Macaron)
  • Move macaron to chi (Move macaron to chi #14293)
• Then a lot of customizations and wrappers like:
  • Defer closing the gitrepo until the end of the wrapped context functions (Defer closing the gitrepo until the end of the wrapped context functions #15653)
  • Clean-up HookPreReceive and restore functionality for pushing non-standard refs (Clean-up HookPreReceive and restore functionality for pushing non-standard refs #16705)
  • Propagate context and ensure git commands run in request context (Propagate context and ensure git commands run in request context #17868)
• Then try to remove unnecessary wrappers:
  • Refactor web route (Refactor web route #24080)
• And try to support patterns which are not able to be handled by Chi and simplify code:
  • Refactor arch route handlers (Refactor arch route handlers #32972)
  • Refactor package (routes and error handling, npm peer dependency) (Refactor package (routes and error handling, npm peer dependency) #33111)

---

ps: in my mind, c.JSON and c.HTML are not a good design. All handlers should return response (I have made many web frameworks in different languages many times, including Java, PHP, Go, I found that return response is the most maintainable approach)

[yp05327]

off topic

yes. it is off topic, just ignore my comment. 😉

[wxiaoguang]

Maybe there's a killer feature which I don't see - hence the question. I'm not even suggesting to swap it, as I know it won't be a good use of anyone's time and won't be accepted. I'm mostly curious what's the big benefit I'm overlooking.

At the moment, there is a blocker reason: many middlewares and handlers have different "ctx" arguments: for example: APIContext, PrivateContext, ArtifactContext, (Web)Context. (ps: some of these contexts were already abused due to various reasons, I left some comments in code when I found them)

Golang's generic type system is quite weak, so the router method functions have to accept "any" at the moment (I added some runtime validation during "init" stage in some refactoring PRs to make sure there won't be low-level mistakes)

[TheFox0x7]

I meant my comment was off topic for the proposal but your comment got me interested in the subject as I get really confused around that area anyway :)
There are 4 different contexts all which have a context.Context somewhere - for me it's a bit too confusing to understand it fairly quickly, compared to for example regular handlers.

ps: in my mind, c.JSON and c.HTML are not a good design. All handlers should return response (I have made many web frameworks in different languages many times, including Java, PHP, Go, I found that return response is the most maintainable approach)

I'm not a fan either especially since they don't often have a reason to be part of the struct except for less func arguments. I think that (response,error) style returns are nice for the handler implementation, especially in api area - except the parts where it makes more sense to write the response directly - case and point markup rendering endpoints. Though the more I think about it the more "It depends on the usecase".
Then again I can't say I authored any web frameworks so it's mostly a new person perspective. On paper I'd put all metadata (logged in user and so on) in context but it's probably not the best idea for reasons I can't think of at the moment...

More on topic - writing an adapter for gitea router with APIContext is possible

func New(r *web.Router, config huma.Config) huma.API {
	return huma.NewAPI(config, &teaAdapter{router: r})
}

type teaAdapter struct {
	router *web.Router
}
type subApiContext struct {
	*context.APIContext
	op *huma.Operation
}

// AppendHeader implements huma.Context.
func (s *subApiContext) AppendHeader(name string, value string) {
	s.RespHeader().Add(name, value)
}

// BodyReader implements huma.Context.
func (s *subApiContext) BodyReader() io.Reader {
	return s.Req.Body
}

// BodyWriter implements huma.Context.
func (s *subApiContext) BodyWriter() io.Writer {
	return s.Resp
}

// Context implements huma.Context.
func (s *subApiContext) Context() goctx.Context {
	return s
}

// EachHeader implements huma.Context.
func (s *subApiContext) EachHeader(cb func(name string, value string)) {
	for name, values := range s.Req.Header {
		for _, value := range values {
			cb(name, value)
		}
	}
}

// GetMultipartForm implements huma.Context.
func (s *subApiContext) GetMultipartForm() (*multipart.Form, error) {
	err := s.Req.ParseMultipartForm(8 * 1024) //Temp because I know it's there somewhere but it's a draft
	return s.Req.MultipartForm, err
}

// Header implements huma.Context.
func (s *subApiContext) Header(name string) string {
	return s.Req.Header.Get(name)
}

// Host implements huma.Context.
func (s *subApiContext) Host() string {
	return s.Req.Host
}

// Method implements huma.Context.
func (s *subApiContext) Method() string {
	return s.Req.Method
}

// Operation implements huma.Context.
func (s *subApiContext) Operation() *huma.Operation {
	return s.op
}

// Param implements huma.Context.
func (s *subApiContext) Param(name string) string {
	return s.Req.PathValue(name)
}

// Query implements huma.Context.
func (s *subApiContext) Query(name string) string {
	return queryparam.Get(s.Req.URL.RawQuery, name)
}

// SetHeader implements huma.Context.
func (s *subApiContext) SetHeader(name string, value string) {
	s.Resp.Header().Set(name, value)
}

// SetReadDeadline implements huma.Context.
func (s *subApiContext) SetReadDeadline(deadline time.Time) error {
	return huma.SetReadDeadline(s.Resp, deadline)
}

// SetStatus implements huma.Context.
func (s *subApiContext) SetStatus(code int) {
	s.Resp.WriteHeader(code)
}

// Status implements huma.Context.
// Subtle: this method shadows the method (APIContext).Status of subApiContext.APIContext.
func (s *subApiContext) Status() int {
	return s.WrittenStatus()
}

// TLS implements huma.Context.
func (s *subApiContext) TLS() *tls.ConnectionState {
	return s.Req.TLS
}

// URL implements huma.Context.
func (s *subApiContext) URL() url.URL {
	return *s.Req.URL
}

// Version implements huma.Context.
func (s *subApiContext) Version() huma.ProtoVersion {
	return huma.ProtoVersion{
		Proto:      s.Req.Proto,
		ProtoMajor: s.Req.ProtoMajor,
		ProtoMinor: s.Req.ProtoMinor,
	}
}

// Handle implements huma.Adapter.
func (t *teaAdapter) Handle(op *huma.Operation, handler func(ctx huma.Context)) {
	t.router.Methods(op.Method, op.Path, func(apiCtx *context.APIContext) {

		handler(&subApiContext{op: op, APIContext: apiCtx})
	})
}

// ServeHTTP implements huma.Adapter.
func (t *teaAdapter) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	t.ServeHTTP(w, r)
}

func RoutesV2() *web.Router {
	router := web.NewRouter()

	router.Use(securityHeaders())
	if setting.CORSConfig.Enabled {
		router.Use(cors.Handler(cors.Options{
			AllowedOrigins:   setting.CORSConfig.AllowDomain,
			AllowedMethods:   setting.CORSConfig.Methods,
			AllowCredentials: setting.CORSConfig.AllowCredentials,
			AllowedHeaders:   append([]string{"Authorization", "X-Gitea-OTP"}, setting.CORSConfig.Headers...),
			MaxAge:           int(setting.CORSConfig.MaxAge.Seconds()),
		}))
	}
	router.Use(context.APIContexter())
	router.Use(apiAuth(buildAuthGroup())) 

	api := New(router, huma.DefaultConfig("Gitea API", setting.AppVer))
	api.OpenAPI().Servers = []*huma.Server{
		&huma.Server{URL: setting.AppURL + "api/v1/"},
	}

	huma.Get(api, "/version", misc.Version)
	huma.Get(api, "/signing-key.gpg", misc.SigningKey)
	huma.Get(api, "/gitignore/templates", misc.ListGitignoresTemplates)
	huma.Get(api, "/gitignore/templates/{name}", misc.GetGitignoreTemplateInfo)
	huma.Get(api, "/licenses", misc.ListLicenseTemplates)
	huma.Get(api, "/licenses/{name}", misc.GetLicenseTemplateInfo)
	huma.Get(api, "/label/templates", misc.ListLabelTemplates)
	huma.Get(api, "/label/templates/{name}", misc.GetLabelTemplate)

	settingsAPI := huma.NewGroup(api, "/settings")
	huma.Get(settingsAPI, "/ui", settings.GetGeneralUISettings)
	huma.Get(settingsAPI, "/api", settings.GetGeneralAPISettings)
	huma.Get(settingsAPI, "/attachment", settings.GetGeneralAttachmentSettings)
	huma.Get(settingsAPI, "/repository", settings.GetGeneralRepoSettings)

	return router
}

[wxiaoguang]

I think that (response,error) style returns are nice for the handler implementation, especially in api area

I think we shouldn't allow returning an "error" directly. It would be abused a lot (using "err" as user error has been abused a lot in current code base). The reason is that some error contains internal sensitive information, for example: database IP, git filesystem path, internal URL, etc.

Golang also has a weak error system, we are not able to use any error directly for end users. For safety, we need to only process the errors we could handle.

And return response could cover the "error" case: response is an interface, just wrap the error and return.

---

writing an adapter for gitea router with APIContext is possible

The approach looks pretty good. I am not familiar with huma so I can't comment too much for that part. For the API part, for the Req/Context part:

• Chi's PathValue is (might) not the same as Golang s.Req.PathValue(name). And there is our own RouterPathGroup to use regexp to match multiple path segments (Update: fortunately at the moment RouterPathGroup is only used for "packages" api)
• How to make middlewares work?
• Maybe we need to completely drop Chi before introducing huma? Do they three (Chi, our customization, huma) work together?

[TheFox0x7]

Don't look too much into the implementation - it's mainly a test if it could be done fairly simply and it's based on humachi wrapper which had a comment that it can be replaced with that function in go 1.22+. I wasn't looking into it that much and I'm sure there are more issues with it but it works as a demo.

How to make middlewares work?

Good question, middlewares in huma have 2 layers, global router level ones (so web.Router middleware) and huma layer ones (with PR I'm running which adds groups and group layer for huma). So some middlewares might need adoption (for example the auth one)
Registering middlewares per route is... slightly annoying but a wrapper would probably solve it.

func Demo() {
    api := New(router, huma.DefaultConfig("Gitea API", setting.AppVer))
    huma.Get(api, "/version", misc.Version) // No middleware adding by default, unless with func (o *huma.Operation){} which does have access to middlewares - but I don't think it's a correct approach.

    // Alternatively:
   operation := Operation{
		OperationID: GenerateOperationID(http.MethodGet, "/version" , VersionOutput), //Version output is a wrapper for struct with version - see later example
		Summary:     GenerateSummary(http.MethodGet, "/version" , VersionOutput),
		Method:      http.MethodGet,
		Path:        "/version" ,
                Middlewares: Middlewares{func(ctx huma.Context, next func(huma.Context)) {
			// Some middleware, which has access to [huma.Context](https://pkg.go.dev/github.com/danielgtaylor/huma/v2#Context)
			next(ctx)
		}},
    }
    huma.Register(api, operation, misc.Version)
}

Discussion about route specific middlewares

Maybe we need to completely drop Chi before introducing huma? Do they three (Chi, our customization, huma) work together?

I don't think that's needed. It did start and work as expected so I think it does properly, though rewrite of actual handlers will be required, which is the main pain point.

---

I probably should give a small example of such rewrites so we're on the same page and have something to look at:

With a really simple version endpoint

import (
	"context" //huma uses stdlib context
	"code.gitea.io/gitea/modules/setting"
)

type VersionOutput struct { 
	Body struct {
		Version string `json:"version" example:"1.23" doc:"Returns the version of the Gitea application"`
	} // Response body contains a json, with a string "version". Example for that string is provided with docs
}

// Version shows the version of the Gitea server
func Version(ctx context.Context, _ *struct{}) (*VersionOutput, error) {
        // empty struct because endpoint does not accept input
	resp := &VersionOutput{}
	resp.Body.Version = setting.AppVer
	return resp, nil 
}

With a bit more complex LicenseTemplateInfo

type LicenseTemplateInfoOutput struct {
	Body api.LicenseTemplateInfo  // struct is already defined so we can reuse it. I haven't added tags to it 
}

// Returns information about a gitignore template
func GetLicenseTemplateInfo(ctx context.Context, input *struct {
	Name string `path:"name", example:"MIT"`
}) (*LicenseTemplateInfoOutput, error) {
        // note that input now has data about where the parameter is (in path) and an example payload (MIT)
	name := input.Name // not really needed

	text, err := options.License(name)
	if err != nil {
		locale := ctx.Value("locale").(translation.Locale) // HACK. 
		return nil, huma.Error404NotFound(locale.TrString("error.not_found"), err) // given what you mentioned about errors it might not be the best idea all the time. I haven't looked into error response that much
	}

	response := api.LicenseTemplateInfo{
		Key:  name,

[openapi.json](https://github.com/user-attachments/files/18929640/openapi.json)

		Name: name,
		URL:  fmt.Sprintf("%sapi/v1/licenses/%s", setting.AppURL, url.PathEscape(name)),
		Body: string(text),
		// This is for combatibilty with the GitHub API. This Text is for some reason added to each License response.
		Implementation: "Create a text file (typically named LICENSE or LICENSE.txt) in the root of your source code and copy the text of the license into the file",
	}

	return &LicenseTemplateInfoOutput{Body: response}, nil
}

generated json api (with a few more routes): openapi.json

And version endpoint (with a different UI but they are swappable):

[wxiaoguang]

I probably should give a small example of such rewrites so we're on the same page and have something to look at:

Thank you for the demo. IMO we'd better to start from the most complex (hardest) part, including middlewares. If the most complex (hardest) part could be resolved, then the simple APIs could also be resolved. Vice versa is not the case, we might get stuck when handling the complex APIs.

And by the way & FYI, it seems that some APIs were not well-designed, for example this newly found one: #33683 (comment) , GET /repos/{owner}/{repo}/git/refs/{ref} it sometimes returns an object, sometimes returns a list. Some special cases need to considered together.

[TheFox0x7]

Yeah, some will be more difficult than the rest - I got stuck on middlewares as I don't really understand how they are handled right now.
I think storing every required value in context would be best (locale, user and so on), unless there's a reason not to do it or a better solution... or just get the APIContext from the context. That could work too but I'm not entirely sure how well Not that well, things might not be initialized until all middlewares are ported which would take a while - for example I got stuck on apiCtx.Repo.Repository not existing when trying to port refs endpoint.
I'd be in favor of loading things like that more explicitly instead of via middlewares but that's another topic entirely and I'm not too sure if it even makes sense or is it just because I find it confusing.

As for that particular endpoint, I guess it's a choice between a custom schema with oneOf in response, or splitting the endpoint into a filtered list and and ref query, second one being breaking but more obvious for end user what will they get in response.
It's not based on github's api as it has git/matching-refs for the list endpoint, so it might be better to split it up - but that's a later discussion.