Skip to content

Commit 1aee5b7

Browse files
committed
handle multiline tweets
1 parent 15f5c72 commit 1aee5b7

File tree

2 files changed

+6
-3
lines changed

2 files changed

+6
-3
lines changed

Diff for: scraper.py

+6-2
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,12 @@ def extract_tweets(containers):
2121
By.XPATH, './/div[@data-testid="User-Name"]//span').text
2222

2323
# Extract the tweet text
24-
tweet = container.find_element(
25-
By.XPATH, './/div[@data-testid="tweetText"]').text
24+
tweet_texts = container.find_element(
25+
By.XPATH, './/div[@data-testid="tweetText"]')
26+
27+
tweet = ""
28+
for text in tweet_texts.find_elements(By.XPATH, './/span'):
29+
tweet += text.text+" "
2630

2731
# Add to the lists
2832
print(f"User: {user}")

Diff for: tweets.csv

-1
This file was deleted.

0 commit comments

Comments
 (0)