-
Notifications
You must be signed in to change notification settings - Fork 67
feat(cloud): Add log reading capabilities to SyncAttempt #786
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(cloud): Add log reading capabilities to SyncAttempt #786
Conversation
- Add get_log_text_tail() method for retrieving last N lines of logs - Add get_full_log_text() method for complete log retrieval - Use Config API /v1/attempts/get_for_job endpoint for log access - Handle both structured (LogEvents) and unstructured (LogRead) formats - Fix field names to match Config API schema (events vs logEvents) - Follow existing PyAirbyte patterns for lazy loading and caching Co-Authored-By: AJ Steers <[email protected]>
Original prompt from AJ Steers
|
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
👋 Greetings, Airbyte Team Member!Here are some helpful tips and reminders for your convenience. Testing This PyAirbyte VersionYou can test this version of PyAirbyte using the following: # Run PyAirbyte CLI from this branch:
uvx --from 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757984248-sync-attempt-log-operations' pyairbyte --help
# Install PyAirbyte from this branch for development:
pip install 'git+https://github.com/airbytehq/PyAirbyte.git@devin/1757984248-sync-attempt-log-operations' Helpful ResourcesPR Slash CommandsAirbyte Maintainers can execute the following slash commands on your PR:
Community SupportQuestions? Join the #pyairbyte channel in our Slack workspace. |
airbyte/cloud/sync_results.py
Outdated
def get_log_text_tail(self, num_lines: int = 1000) -> str: | ||
"""Return the last N lines of log text for this attempt. | ||
Args: | ||
num_lines: Maximum number of lines to return from the end of the logs. | ||
Defaults to 1000 lines. | ||
Returns: | ||
String containing the last N lines of log text, with lines separated by newlines. | ||
""" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't add anything meaningful. Let's remove this method.
- Address GitHub comment from @aaronsteers - Keep only get_full_log_text() method for complete log retrieval - Simplify SyncAttempt log interface to single method Co-Authored-By: AJ Steers <[email protected]>
…757984248-sync-attempt-log-operations
…757984248-sync-attempt-log-operations
9cf622f
into
devin/1757543890-sync-attempt-abstraction
This PR targets the following PR:
SyncAttempt
abstraction for fetching job attempts #781feat(cloud): Add log reading capabilities to SyncAttempt
Summary
Adds two new methods to the
SyncAttempt
class to enable log retrieval from Cloud sync attempts:get_log_text_tail(num_lines: int = 1000) -> str
: Returns the last N lines of log textget_full_log_text() -> str
: Returns complete log text for the attemptBoth methods use the existing Config API integration (
_fetch_attempt_info()
) and handle both structured (LogEvents
) and unstructured (LogRead
) log formats returned by the/v1/attempts/get_for_job
endpoint.Review & Testing Checklist for Human
get_full_log_text()
doesn't cause memory issuesNotes
_fetch_attempt_info()
events
vslogLines
) were verified against the Config API OpenAPI schema_make_config_api_request()
pattern with proper noqa commentsLink to Devin run: https://app.devin.ai/sessions/946366cdd71140c88397678ef6fb16fd
Requested by: @aaronsteers