This project has a visual representation available on my Portfolio.
https://portfolio.owenthe.dev/portal-bot
Portal Bot is a mini-project that I made to help check college portals automatically.
One of the great joys of applying to college is checking the student portal on a relatively frequent basis. As a high schooler, you’re basically taught that the bane of your existence is checking these portals all the time, to make sure that everything gets in to all the colleges.
But, come on, checking 8 portals every x days is a tedious, time-consuming process that nobody wants to do. And so, I created portal bot (also known as big college lazy), so I could have some fun with Selenium and the Slack API.
The architecture of Portal Bot is pretty simple – log into a portal using Selenium, take a screenshot, post it to Slack. Simple, right? Well, yes, mostly.
The first part of this entire thing is dealing with the nightmare that is educational software. Some schools use Slate by Technolutions, some use a Salesforce thing, and Northeastern and CMU decide to just go their own way, so that’s pretty fun.
Dealing with schools that use Slate was pretty easy. Fill in the username/password, click Login, wait about 5 seconds, and we’re home free. That’s the easy part.
Dealing with the Salesforce schools was a bit more challenging. The actual underlying code has some pretty random variable names, but no big deal. I can still put in the username and password, but the waiting periods are longer because the software is a lot heavier. Additionally, some schools have separate areas for the actual application and financial aid, and I never got around to programming that into portal bot.
The last two schools that I applied to – Northeastern and CMU – oh boy are those fun to work with. Northeastern first.
The Northeastern portal is a lovely mix of ASP software from many generations of computers, updated over the years to make sense, in some way? Not sure how to describe it. Regardless, finding all the fields to fill out and buttons to click isn’t a huge deal. The slight issue is that sometimes the portal will not log in, and you have to log in again. Not a huge issue, just requires a try/catch block so that if this does occur, everything is repeated again. It takes about 10 or so seconds to log in fully, but nothing crazy.
CMU was the school that I could not crack for Portal Bot. Instead of using a static username/password, their portal exclusively uses external authorizations, through Google, Facebook, and LinkedIn. Being the boneheaded self I was, I decided to choose Google for the authorization. Of course, Google has a pretty complex login system that, to my best guess, does a whole shit load of heuristic analysis to see if you’re a robot or not. Every time I tried visiting the site on a Selenium instance, it kept asking for phone number confirmation. I could technically get around this, by pointing the phone number on the account to one that I can control (such as Twilio, for instance), but this requires a huge amount of effort and is prone to a lot of errors.
At the end of all of this, the next step is to upload the screenshots that portal bot takes to a Slack channel exclusively for Portal Bot. Figuring out how to deal with Slack’s API wasn’t that difficult, thankfully. The only stumbling blocks I encountered was using proper app authorizations, and knowing how to properly make an API request to upload an image. Nothing crazy, though.
Since I started Portal Bot, there’s been a few revisions to the code. One of the more major revisions was automatically picking up the length of the browser window. In revision 1, I was manually setting the length of the window – so if any text or information was added, part of the capture was cut off. The fix for this involved executing JavaScript to determine the page height, and setting the browser window height accordingly.
Another minor revision was that I changed when the script executed from 10:30 AM to 4:30 PM. Sometimes schools take until the end of the day to process any new documents.
Lastly, when I moved the server that Portal Bot was running on from a PC to a Raspberry Pi, I did have to rebase the code on using a Chrome driver, and setting up the Chrome driver on a Raspberry Pi requires you to add this flag: –disable-features=VizDisplayCompositor before it can work properly. Otherwise, it will crash on every run. Running Portal Bot on a RPi4 4 GB RAM model works really well, so no complaints there.
And that’s Portal Bot! A quick and dirty script I made to help with the college portal checking process. I’ll be posting the source code online at some point in May-June 2020 if you want to take a look at how it works.
