feature: enable passing custom default response class #440
Conversation
requirements.txt
Outdated
| @@ -1,3 +1,4 @@ | |||
| fastapi >=0.100 | |||
| uvicorn[standard] >=0.29.0 | |||
| pyzmq >=22.0.0 | |||
| orjson==3.10.15 | |||
There was a problem hiding this comment.
let's keep this dependency optional
There was a problem hiding this comment.
Sure, right now I'm adding it to the test and perf requirements in the _requirements directory. Is there a better place for it?
src/litserve/server.py
Outdated
| @@ -414,7 +414,7 @@ async def health(request: Request) -> Response: | |||
|
|
|||
| @self.app.get(self.info_path, dependencies=[Depends(self.setup_auth())]) | |||
| async def info(request: Request) -> Response: | |||
| return JSONResponse( | |||
| return ORJSONResponse( | |||
There was a problem hiding this comment.
this is not a performance critical section. I would suggest applying this to the predict method.
There was a problem hiding this comment.
Got it! In the latest commit I'm hard-coding the ORJSONResponse class while registering the /predict endpoint, but it's only temporary for the sake of the benchmarks. In the target scenario this would be user's choice
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #440 +/- ##
===================================
- Coverage 89% 89% -0%
===================================
Files 31 31
Lines 2041 2028 -13
===================================
- Hits 1824 1805 -19
- Misses 217 223 +6 🚀 New features to boost your workflow:
|
|
@aniketmaurya do I interpret correctly that the failing job means that it's not worth it to include the |
| @@ -1,3 +1,4 @@ | |||
| uvloop | |||
| tenacity | |||
| jsonargparse | |||
| orjson==3.10.15 | |||
There was a problem hiding this comment.
requirements.txt contains the main dependency. These are optional.
There was a problem hiding this comment.
You asked to make this dependency optional in one of the comments
| @@ -15,3 +15,4 @@ openai>=1.12.0 | |||
| pillow | |||
| numpy <2.0 | |||
| pytest-retry>=1.6.3 | |||
| orjson==3.10.15 | |||
What does this PR do?
Fixes #436 .
Before submitting
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in GitHub issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃