Building a YouTube-Twitter Bot, Part 3: Checking for New Video and Tweeting Link

At the end of the last post I mentioned that I might put these posts on hold for until 2021. While posts were on hold, the development was not; at least not until just before Christmas. The bot is actually live and tweeting!!! Please go follow it, it’s called @YTBotbyPS. YouTube Bot by princelySid is where the name comes from.

The problem with coming back to writing is that you’ve forgotten almost everything you did to get to the point you’re now at, development-wise, and you have your editor leave messages like this on the post you started but didn’t finish

lol, another screenshot

Luckily for us we have git-diff so at the very least I can see which files changed from the last post to this one. But first, let’s recap: we’re building a bot that given a list of YouTube channels, it’ll tweet out snippets of a newly posted video. Below is the list of planned activities:

  1. Get list of my subscriptions from YouTube.
  2. Mark channels as music or blog and whether they’re Kenyan.
  3. For each channel get a list of recent videos and store that in a database.
  4. Take the last video link and download the video.
  5. Clip the video to 40s.
  6. Tweet that video (title, link to YouTube)
  7. Save link to tweet, video in database
  8. Check every 30 minutes if there’s a new video, if yes start from 4.

In my last post we did step 3, for every channel we got a channel data and video data and stored that in a database. In this post we’ll be doing step 6 and 8. If you’d like to follow along with the code, you can find it in the repo: https://github.com/princelySid/yt-bot. This whole post will be talking about code on the tweeting branch so switch to that.

Once you’re on the right branch, you’ll need to run the command pipenv install --dev to install all the dependencies and then run the tests: pipenv run pytest. The output should look like this:

all tests should pass

In this branch we only added one new dependency:

  1. twython - a pure python wrapper for the Twitter API

Now for the first time we’re going create some resources outside our app. We need two things for this to work, a Twitter account and a Twitter app. I’m hoping the how-to of the first is obvious.

As for the second you’ll need to go to https://developer.twitter.com and apply for a developer account(I don’t know how long this will take. I’ve been playing with Twitter’s API for years so I’ve always had access). Once you have access you want to navigate to Projects & Apps and create a new stand-alone app. You’ll want to set the App Permissions you’ll want to set it read-write. Finally, you’ll want to Consumer Keys from the Keys & Token tabs, these will be saved as environment variables.

We also add a few new environment variables, all connected to how we’ll be able to tweet:

  • TW_API_KEY - consumer key taken from the Twitter app that you registered
  • TW_API_SECRET - consumer secret taken from the Twitter app you registered
  • TW_OAUTH_TOKEN - Add once your app gets access to your Twitter profile, explained in the following section
  • TW_OAUTH_SECRET - Add once your app gets access to your Twitter profile

To get the TW_OAUTH_TOKEN, TW_OAUTH_SECRET variables for you’ll need to run python data/tw_auth.py.

It’ll ask you to open a link, which if you give authorisation will give you a code that you need to input exactly into your terminal. Finally the program will give you the variables that you’ll need to add your .env file.

Authorise the app

code that you’ll get

What the output of the whole program looks like. Do what it tells you

At this point you can write scripts that interact with Twitter via your account.

At this point we need a script that goes through each channel in out database and checks if there’s new videos and if there is tweets it. Luckily for us we wrote a script that does most of that last week, only it didn’t send a tweet out. So for next part I’ll only be focusing on parts that interact with Twitter.

In line 14 in the file tweet_videos.py. We have a method that gives us the twython object, and makes sure that it’s authenticated properly so that we can interact with Twitter properly. This is what we call in line 32.

In line 56 we take the data from the feed and channel and formats it for Twitter. This ensures that text carries the details of video in this format: [category] New Video from channel name: video name _video link_ but if the text length is too long it drops the video name.

In line 59 with have the send_tweet method, which does one thing: send out the tweet.

Again, we’ve not added to much compared to what we did last week. The rest of the code is almost identical to the file we created last week.

There is one last thing to do we want to be able to run this script every so often. For this I chose the easiest option for the and used cron and wrote a bash script:

#!/bin/bash

VENV_PYTHON="/path/to/virtual/environment/bin/activate"
PROJECT="/path/to/yt-bot"
PY_FILE="data/tweet_videos.py"
LOG_FILE="path/to/log/tweet.log"

source "$VENV_PYTHON" && cd "$PROJECT" && python "$PY_FILE" >> "$LOG_FILE" 2>&1

This file activates our virtual environment, changes to our project folder then runs out python file and pipes the output from that into a log file(so that you can track errors)

You can get the value for VENV_PYTHON by running pipenv --venv in your project folder and adding /bin/activate to the output

don’t forget to add /bin/activate when adding it to your bash script

  • PROJECT - the path to the project folder, you’ll need to change this
  • PY_FILE - the path to python file that tweets, no need to change this
  • LOG_FILE - the path to the log file, you’ll need to change this I would suggest creating a folder called log in your project and using log/tweet.log.

Finally you want to add a command to cron by running crontab -e in the terminal and adding this to the end of the file and saving it:*/15 3-20 * * * /bin/bash /path/to/yt-bot/data/cron.sh

I won’t go into the complexities of cron but what this does is run the script every 15mins between 3-20(my server is on UCT time). If you’d like to read more on cron you can do so here.

If you’ve made it to this point without any errors then the log file should begin to look like this:

Well done!! You have now have bot that tweets out YouTube links. If you’ve managed to deploy this, or you have any questions, please let me know in the comments or on twitter. Would love to see what you’re doing.

Next post we’ll be probably be working on the steps we skips starting with downloading videos. See you then. Peace!!!