Archive for Python/Shell Scripts

textme.py: Have important emails come in as text messages (for free!)

It’s sometimes more relaxing to forward texts from certain people to your cell phone as text messages, rather than compulsively checking your email or being notified by the stupid iPhone new mail sound of every spam message you get.

textmeI’ve uploaded a Python script here that does just that. It’s in zip format because I’ve also bundled the necessary Python dependencies with it since it can be tricky sometimes installing your own in Python (twilio-pythonsix, and httplib2). If you already have these or prefer to install them yourself, you can just download the Python script by itself here.

This script logs you in to Gmail and checks your email every 10 seconds. If the from field of a new email matches a string that you specify, it sends you a text message including the sender, subject, send time, and first 100 characters of the email.

How to make it work ASAP (note: requires making a free account on a web texting service, but it’s quick, free, and better yet hip):

  1. Download the script and dependencies I referenced above.
  2. Sign up for a free trial account at https://www.twilio.com. I think you may have to provide a credit card number, but I’ve had an account there for a while, and they seem like a respectable company (they’ve never charged me). Of course, you may decide you really like this and want to pay them a few cents per text message to remove the note on each message that it was sent from a Twilio trial account (it doesn’t bother me).
  3. Fill in the following lines at the top of the script. The first three you’ll get from making your Twilio account, the next two are your Gmail login info to check your inbox every 10 seconds, then put in the cell number you’d like the free text messages to go to and the regular expression pattern to search the from field for in order for it to send the message. [tip: to forward messages from Bob B and Jane J, put pattern = “Bob B|Jane J”; the middle character is a pipe, found above the return key, and means “or”].

ACCOUNT_SID = “your twilio id”
AUTH_TOKEN = “your twilio token”
twilionum = “your twilio phone nmber”
username = “your gmail username”
password = “your gmail password”
cellnumber = “the number you want to receive text messages at, eg 2038675309”
pattern = “regular expression (aka pattern, eg a name) to search the from field”

  1. Launch the script by changing to the directory where textme.py is, and type python textme.py. It should continue running and sending you text messages until the process is terminated and/or the computer goes to sleep or restarts. So, it’s probably easiest to have this running somewhere on a server that you know won’t restart for a while.
  2. I hope it’s helpful! As always, email with questions or comments.

A Python Tutorial for Economists

It’s been a few months since I’ve posted here; blogging was a bit taboo this summer at work (though it turns out I found plenty of other ways to raise red flags for the Cyber Security team using just Python + the Interwebs).

Working in an office with several other research assistants who were proficient with some statistical scripting languages (Stata, SAS), I began to think there’s probably a niche for a more general-purpose language in academic social science research (as well as in automating some of the tasks involved with casework around the office). I was already using Python in much of my work. What started out as a few trips to coworkers’ desks to help them write this or that script quickly turned into a few pages of notes, and that turned into some thirty pages of charts, explanations, and instructional tasks. (I must note, the final formatting was inspired by the style of my linear algebra lecture notes from last semester.)

I presented a version of Python for Economists to some coworkers at the FTC Bureau of Economics in July. I’ve been a student of three different college classes that taught Python from scratch, but I’ve never seen a way of teaching Python that I thought was appropriate for students already familiar with scripting languages such as Stata. I focus on two broad applications of Python I’ve found very useful in social science research: web scraping and textual processing (including regular expressions).

Downloads:

  • PDF of the booklet (34 pages, colored Python syntax highlighting)
  • Zipped supporting materials used in the exercises

Have your computer email you when an action completes

I can’t imagine doing distributed or large batch work without an easy-to-use script that emails me once a command terminates. Before that, I was just going insane with checking on them all the time to see if my process was done or if an error has occurred, leaving the computer idle. I’ve called it notify (linked here, or pasted at end), and you can use it as a shell script to quickly call with other commands.

I issue it in the terminal like this:
$ python ~/reallylongscript.py; notify
The semicolon separates lines, and as soon as the first statement terminates (whether it terminates with success or with error), the notification script will be called.

You’ll get an email with a body like:

New notification from brunonia.

Where “brunonia” is the name of my computer (this script will determine the computer’s name automatically, which is very helpful when you need to know which of several computers needs attention!).

It’s been really useful for:

  • Running a long action on a remote machine, even if it’s as simple as deleting all the files in a large directory to free up space, so I don’t have to keep checking to see when it’s done.
  • Running a command on several machines over ssh (usually with screen in zombie mode) to let me know if a process fails and needs to be investigated and/or restarted. (The only downside for me so far is that if the filesystem croaks or some similar problem affects all nodes at once, my inbox needs a bit of clean-up work!).

This is where I got the base code from, and they have a few other ideas on how to use it. This strategy can be easily integrated into already-existing scripts to keep you in the loop as they run. I suppose one could also write a shell script using Python’s os.system() method (or similar newer ones) and sys.argv list that takes as parameters commands to run, and sends the email after all commands in sys.argv have finished.

#!/usr/bin/python

import smtplib, platform

me = platform.node()

fromaddr = 'myusername@gmail.com'
toaddrs  = 'myusername@gmail.com'
msg = 'New notification from '+me+'.'  

# Credentials (if needed)
username = 'myusername'
password = 'mypassword'  

# The actual mail send
server = smtplib.SMTP('smtp.gmail.com:587')
server.starttls()
server.login(username,password)
server.sendmail(fromaddr, toaddrs, msg)
server.quit()

PawPrints from a Macbook

Update: You can now also use PawPrints from any computer by following these instructions from Brown. They’re probably much easier. Also, the print driver is already installed on the latest Mac operating systems anyway.

Audience: The intended audience of this post is Brown students who have CS department logins and use Macs, and from time to time need to print at the libraries. I suspect this strategy can be adapted for Windows users or for similar systems at other schools. Let me know if you have anything to share.

If you have a CS department account and SSH set up on your laptop, you don’t need to bother searching for a free computer in the SciLi next time you need to print. If you run Mac OS, copy the script below into an empty text file (or you can download it here.) You should name the file “pawprint” (with no extension) and change my CS department username (ambell) to your own.

The file should be put in /usr/local/bin. If your account doesn’t have have admin capabilities for your laptop, you may need to use a command like “sudo mv pawprint /usr/local/bin.” Then navigate to that directory, and add executable permissions to the file: type into the terminal “sudo chmod +x pawprint.”

To print a file from your Mac to PawPrints, just navigate to the directory where your files are and type pawprint filenames. You can type as many file names as you want at a time from your laptop, then walk over to a print station. If you entered multiple filenames, I believe PawPrints will bundle them all into one print item, named after the first file.

Thanks to Chris Sulawko for suggesting a few days ago that I try printing to PawPrints over ssh!

#!/bin/bash

echo “starting print of:” $@
scp $@ ambell@ssh.cs.brown.edu:~
ssh ambell@ssh.cs.brown.edu lpr -P pawprints $@
ssh ambell@ssh.cs.brown.edu rm -f $@
echo “files have been sent to pawprints”
exit

Translation:

  1. Tell the user the parameters passed to the function
  2. Copy the files to your home directory on your departmental account.
  3. Order the print of the files.
  4. Remove the files.
  5. Tell the user that the files have been sent.