v0.8 - subpage support added among other things!

Author: mpentler

README.md

@@ -20,9 +20,9 @@ Before you get this up and running there are two other things you need to do fir
 * VBIT2 (Peter Kwan): https://github.com/peterkvt80/vbit2
 
 ## Setup
-After getting these up and running there are some setup tasks to do. Rename config.py-default to config.py and open the file in a text editor. You need to add:
+After getting these up and running there are some setup tasks to do. Rename config.py-default to config.py and open the file in a text editor. You need to:
 
-1) Your Twitter access tokens: You can find a good guide for doing this here - https://iag.me/socialmedia/how-to-create-a-twitter-app-in-8-easy-steps/ You will need to pick a unique name for the app. Which is annoying. Pick anything you want that isn't taken. Maybe add your name at the end.
+1) Add your Twitter access tokens: You can find a good guide for doing this here - https://iag.me/socialmedia/how-to-create-a-twitter-app-in-8-easy-steps/ You will need to pick a unique name for the app. Which is annoying. Pick anything you want that isn't taken. Maybe add your name at the end.
 2) Check where your pages will be saved to: Change the tti_path line too if you've changed your Teefax location. You can also customise the page number.
 3) Change the theme if you want: Theme support is detailed below
 
@@ -34,7 +34,7 @@ When you've setup your config.py you can run the script like this example that g
 
 The script will constantly update your chosen page number (default is 153 - chosen because it used to be used for this purpose on Teefax in the past) in the main teletext folder (which defaults in VBIT2 to /home/pi/teletext/).
 
-All of the files in that folder are sent across to the TV every so often, therefore the script constantly overwrites it with new tweets (up to 5 - space permitting!) so that it will update on your screen.
+All of the files in that folder are sent across to the TV every so often, therefore the script constantly overwrites it with new tweets (up to 5 - space permitting!) so that it will update on your screen. Up to 99 subpages of tweets will be displayed.
 
 ## Theme support
 In the config.py file you can see a theme section. The teletext level I am using supports 7 colours:
@@ -53,7 +53,6 @@ You can also change the colours of a tweet poster's username, timestamp, and the
 Finally, the title text at the top can be changed, although there is a 30 character limit or things look wonky.
 
 ## Notes
-* At this moment in time the script reads 5 tweets. Further versions will improve on this by writing multiple tweets, possibly in subpages :-O
 * New tweets are grabbed every 60 seconds by default. This is configurable with the -d option, but you do have be aware of the Twitter API limits.
 * Emoji stripping is now included. Mostly.
 * Due to the limited teletext character set substitutions are implemented. Things like replacing underscores with hyphens and also making sure # works correctly. You also have to replace curly apostrophes with straight ones, as that's all the specification allows

changelog.md

@@ -1,7 +1,14 @@
 ## Changelog
 
+### v0.8
+- major feature addition: subpage support to handle a larger number of tweets
+- added cycle_time parameter to config, this is cycle time between subpages
+- added count argument to specify number of tweets to download
+- a few more bug fixes: line 24 now writable and no more dropped tweets (they go to subpages instead)
+- moved a few bits of code around for logic and readability
+
 ### v0.7
-- compressed three different tweet output functions into one! i guess this is slower to run but i like smaller, cleaner files...
+- compressed three different tweet output functions into one!
 - fixed still display bug in write_home_timeline function
 
 ### v0.6

config.py-default

@@ -7,7 +7,7 @@ access_key = "XXXXXXXX-xxXXxXXxxXxxxXxXXxXxXxXxxxXxxxxXxXXxXxxXX"
 access_secret = "XxXXXXXXXXxxxXXXxXXxXxXxxXXXXXxXxxXXXXx"
 
 # theme section
-# page title - 30 chars maximum
+# page title - 27 chars maximum
 page_title = "TELETEXT TWITTER"
 # display colours - must be in quotes! allowed colours are:
 # red, green, yellow, blue, magenta, cyan, white
@@ -20,3 +20,4 @@ timestamp_colour = "yellow"
 # miscellaneous
 tti_path = "/home/pi/teletext/"
 page_number = 153
+cycle_time = 20

teletext-twitter/__main__.py

@@ -23,17 +23,18 @@ def parse_args():
 
     parser.add_argument("-m", "--mode", type=str, help="choose between different modes - home, user or search")
     parser.add_argument("-q", "--query", type=str, help="a search query, either a search term or a username. hashtags supported if you put quotes around the string")
-    parser.add_argument("-d", "--delay", type=int, default=60, help="seconds between timeline scrapes (minimum is 60 seconds - lower values have no effect)")
-    parser.add_argument("-v", "--version", action="version", version="0.7")
+    parser.add_argument("-c", "--count", type=int, default=5, help="number of tweets to download (default is 5, maximum is 800)")
+    parser.add_argument("-d", "--delay", type=int, default=60, help="seconds between timeline scrapes (default is 60 seconds - lower values have no effect)")
+    parser.add_argument("-v", "--version", action="version", version="0.8")
     parser.add_argument("-Q", "--quiet", action="store_true", default=False, help="suppresses all output to the terminal except warnings and errors")
 
     args = parser.parse_args()
+    args.count = min(args.count, 800)
     args.delay = max(60, args.delay)
 
     if args.mode == "search" and not args.query:
         print("[!] Search mode selected but no query specfied with -q. Exiting...", file=sys.stderr)
         sys.exit(1)
-
     if args.mode == "user" and not args.query:
         print("[!] User timeline mode selected but no username specified. Exiting...", file=sys.stderr)
         sys.exit(1)
@@ -48,23 +49,23 @@ def main():
 
     while True:
         try:
-            write_header(config)
             if args.mode == "home":
                 if not args.quiet:
                     print("[*] Beginning home timeline scraping", file=sys.stdout)
-                write_tweets(twitter_object, args.mode, config)
+                write_tweets(twitter_object, args.mode, args.count, config)
             elif args.mode == "search":
                 if not args.quiet:
                     print("[*] Getting recent tweets containing: " + args.query, file=sys.stdout)
-                write_tweets(twitter_object, args.mode, config, args.query)
+                write_tweets(twitter_object, args.mode, args.count, config, args.query)
             elif args.mode == "user":
                 if not args.quiet:
                     print("[*] Getting recent tweets from user: @{}".format(args.query), file=sys.stdout)
-                write_tweets(twitter_object, args.mode, config, args.query)
+                write_tweets(twitter_object, args.mode, args.count, config, args.query)
             if not args.quiet:
                 print("[*] Page updated. Waiting {} seconds until next scrape".format(args.delay), file=sys.stdout)
         except OSError as e:
-            print("[!] Error accessing teletext data file, exiting: {}".format(e.strerror), file=sys.stderr)
+            print("[!] Error accessing teletext data file, {}".format(e.strerror), file=sys.stderr)
+            print("[!] Exiting...", file=sys.stderr)
             sys.exit(1)
         except twitter.error.TwitterError as e:
             for error in e.message:

teletext-twitter/output.py

@@ -29,27 +29,32 @@ def write_tweet_line(file, line_num, line, config):
     string += "\r\n"
     file.write(string)
 
-def write_header(config): # write a header for the page and pop a nice banner at the top
-    page_title = config["page_title"]
+def write_header(subpage, config, mode): # write a header for the page and pop a nice banner at the top
+    page_title = config["page_title"] + " " + "{:02d}".format(subpage)
     logo_spacer = " " * (39 - (4 + len(page_title) + 5))
-    with open(config["tti_path"] + "P" + str(config["page_number"]) + ".tti", "w+") as file:
-        file.write("DE,Autogenerated by Teletext-Twitter\r\n")
-        file.write("PN," + str(config["page_number"]) + "00\r\n")
-        file.write("SC,0000\r\n")
-        file.write("PS,8000\r\n")
+    with open(config["tti_path"] + "P" + str(config["page_number"]) + ".tti", mode) as file:
+        if mode == "w+":
+            file.write("DE,Autogenerated by Teletext-Twitter\r\n")
+            file.write("SC,0000\r\n")
+            file.write("PS,8000\r\n")
+            file.write("CT," + str(config["cycle_time"]) + ",T\r\n")
+        file.write("PN," + str(config["page_number"]) + "{:02}\r\n".format(subpage))
         file.write("OL,1," + ESCAPE + chr(text_colours[config["header_colour"]]) + SET_BACKGROUND +
                    DOUBLE_HEIGHT + ESCAPE + chr(text_colours["white"]) +
                    page_title + logo_spacer + ESCAPE + chr(mosaic_colours["cyan"]) + TWITTER_BIRD + "\r\n")
         file.write("OL,3," + ESCAPE + chr(mosaic_colours[config["header_separator"]]) + (chr(35) * 39) + "\r\n")
 
-def write_tweets(twitter_object, mode, config, query=""): # grab the latest timeline - only 5 tweets for now
+def write_tweets(twitter_object, mode, count, config, query=None): # grab the latest timeline
     if mode == "home":
-        statuses = twitter_object.GetHomeTimeline(count = 5)
+        statuses = twitter_object.GetHomeTimeline(count=count)
     elif mode == "search":
-        statuses = twitter_object.GetSearch(term=query, result_type="recent", count=5)
+        statuses = twitter_object.GetSearch(term=query, result_type="recent", count=count)
     elif mode == "user":
-        statuses = twitter_object.GetUserTimeline(screen_name=query, count=5)
+        statuses = twitter_object.GetUserTimeline(screen_name=query, count=count)
+
     line_position = 4
+    subpage = 1
+    write_header(subpage, config, "w+")
 
     for status in statuses: # iterate through our responses
         tweet_text = clean_tweet(status.text)
@@ -58,10 +63,12 @@ def write_tweets(twitter_object, mode, config, query=""): # grab the latest time
         tweet_username = status.user.screen_name
         tweet_text = textwrap.wrap(tweet_text, 38) # make sure our lines fit on the screen
 
-        if (line_position + len(tweet_text) + 1) > 24: # are we going to fill the page?
-            break # yep! dump the last tweet!
-
         with open(config["tti_path"] + "P" + str(config["page_number"]) + ".tti", "a") as file:
+            post_length = len(tweet_text) + 1 # how long is our next tweet? (including info line)
+            if (line_position + post_length) > 25: # are we about to go over the page?
+                subpage += 1 # yes! new page, please
+                write_header(subpage, config, "a") # append mode to do subpages
+                line_position = 4 # and reset our cursor
             write_tweet_info(file, line_position, tweet_username, tweet_human_time, config)
             line_position += 1
             for line in tweet_text: