Merge pull request #54 from tjmlabs/feat/first-release

Feat/first release

Author: Jonathan-Adly

.github/pull-request-template.md

@@ -4,23 +4,40 @@
 
 ## Type of change
 
-<!-- Please delete options that are not relevant. -->
+<!-- Please select ONE option that best describes your changes -->
+<!-- Your selection will determine the semantic version bump -->
 
-- [ ] Bug fix (non-breaking change which fixes an issue)
-- [ ] New feature (non-breaking change which adds functionality)
-- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
-- [ ] Internal (changes that do not affect the application's functionality, changes in admin site, documentation improvements, refactor, settings) 
+- [ ] 🚨 Breaking change - causes existing functionality to break (triggers MAJOR version bump)
+- [ ] ✨ New feature - adds functionality in a non-breaking way (triggers MINOR version bump)
+- [ ] 🐛 Bug fix - fixes an issue without changing functionality (triggers PATCH version bump)
+- [ ] 🏠 Internal - changes that don't affect external functionality (no version bump)
+      (e.g., refactoring, docs, tests, admin site changes)
 
-## Checklist:
+## Release Impact
+
+<!-- The following will be automated based on your type of change selection above -->
+- Current version: <!-- e.g., v1.2.3 -->
+- Next version will be: <!-- will be automatically determined -->
+
+## Checklist
 
 - [ ] My code follows the style guidelines 
 - [ ] My code is tested with 99%+ coverage
 - [ ] I have commented my code, particularly in hard-to-understand areas
 - [ ] My changes generate no new warnings
+- [ ] I have updated the documentation to reflect my changes
+- [ ] I have added/updated tests to cover my changes
+
+## Breaking Changes
+
+<!-- If you checked "Breaking change" above, please detail the breaks here -->
+<!-- This will be included in the release notes -->
+
+## Migration Guide
 
+<!-- If you checked "Breaking change" above, please provide migration instructions -->
+<!-- This will be included in the release notes -->
 
+## Additional Notes
 
-## To Do before merging:
-- [ ] If applicable, changes have been merged to the cloud branch
-- [ ] The readme and documentation have been updated
-- [ ] If my changes break or alters current functionality, I have upgraded the SDKs to reflect these changes 
+<!-- Any other information that would be helpful to reviewers -->

readme.md

@@ -114,15 +114,11 @@ Please check swagger documentation endpoint (v1/docs) for endpoints. More compre
 
 You can import an openAPI spec (for example for Postman) from the swagger documentation endpoint at `v1/docs/openapi.json`
 
-## Roadmap for 1.0 Release
-
-1.  Complete Documentation for the API
-
-## Roadmap for 2.0 Release
+## Roadmap
 
 1. Independent Evals for quality + latency
 2. Full Demo with Generative Models
-3. Typescript SDK
+3. Automated SDKs for popular languages other than Python
 
 ## Getting Started (Local Setup)
 

web/api/models.py

@@ -178,7 +178,7 @@ async def embed_document(self) -> None:
         """
         # Constants
         EMBEDDINGS_URL = settings.EMBEDDINGS_URL
-        EMBEDDINGS_BATCH_SIZE = 6
+        EMBEDDINGS_BATCH_SIZE = 3
         DELAY_BETWEEN_BATCHES = 1  # seconds
 
         # Helper function to send a batch of images to the embeddings service
@@ -227,6 +227,10 @@ async def send_batch(
                         "Failed to get embeddings from the embeddings service."
                     )
                 out = await response.json()
+                if "output" not in out or "data" not in out["output"]:
+                    raise ValidationError(
+                        f"Failed to get embeddings from the embeddings service. Repsonse: {out}"
+                    )
                 logger.info(
                     f"Got embeddings for batch of {len(images)} images. delayTime: {out['delayTime']}, executionTime: {out['executionTime']}"
                 )
@@ -239,6 +243,9 @@ async def send_batch(
             base64_images[i : i + EMBEDDINGS_BATCH_SIZE]
             for i in range(0, len(base64_images), EMBEDDINGS_BATCH_SIZE)
         ]
+        logger.info(
+            f"Split document {self.name} into {len(batches)} batches for embedding"
+        )
 
         try:
             # we save the document first, then save the pages