Hack of the day: enhancing Eastern Bank deposit notification emails

By | December 25, 2021

I have my Eastern Bank account configured to email me notifications about checking deposits. There are three problems with these notifications:

  1. They include only the amount of the deposit, not whom it’s from.
  2. They only put the deposit amount in the body of the email, not in the subject.
  3. They email me about every deposit, including the ones I already know about since I’m the one who made them. I want to be notified about direct deposits, but I really don’t want to be notified about check deposits. Eastern Bank’s online banking platform doesn’t support this distinction.

(Century Bank’s online banking platform had the same problem. I was hoping things would approve in this area after the acquisition by Eastern Bank, but alas no.)

This finally annoyed me enough today that I decided to do something about it. After implementing the hack described below, each deposit notification email is handled in one of three ways:

  1. If the deposit is already in my bookkeeping system (GnuCash), the email is simply discarded; I don’t need to see it!
  2. Otherwise, an attempt is made to log into Eastern Bank online banking automatically to fetch a description of the deposit from the list of recent transactions in my checking account. If successful, then the deposit amount and description are appended to the Subject of the email before it is put in my inbox.
  3. If step 2 fails for whatever reason, the email is put in my inbox as is.

All that sounds pretty straightforward, but there are actually a lot of moving parts under the surface.

Logging into Eastern Bank online banking automatically requires storing my username and password somewhere I feel comfortable with and enabling automatic capturing of the multi-factor authentication (MFA) code sent by Eastern Bank when it decides that MFA is required to log in.

I am certainly not going to do all this on my mail server in the cloud, because wow holy cow that is not a good idea! So instead I do the actual processing on my home server in my basement, which is protected by enough layers of paranoid security that I’m OK with it. (The risk calculus I’ve done to conclude this is interesting, but that’s a topic for another blog posting.)

So let’s talk about the nuts and bolts of how all this works. This rule in my procmailrc file on my mail server forwards the incoming deposit notification email to my home server and waits for it to process the email and return either the modified email or nothing:

:0
* ^From: .*Eastern Bank
* ^Subject: Eastern Bank Deposit Alert
{
:0 fw : $MAILDIR/deposit-notification$LOCKEXT
|tf=/tmp/notif.$$; curl –silent –data-binary '@-' http://fillmein/deposit_notification.cgi > $tf; if [ -s $tf ]; then cat $tf; e=0; else e=1; fi; exit $e
:0
* ^X-In-GnuCash-File: Yes
/dev/null
}
view raw procmailrc hosted with ❤ by GitHub

(Notice that in this gist and others I have replaced sensitive or site-specific strings with “fillmein”. You will need to do the needful if you want t use any of this stuff!)

If the script on my home server returns a non-empty response, the procmail rule assumes that it is a filtered version of the message and uses it; otherwise the filter fails and the message is preserved as-is.

After filtering the message the procmail rule above checks if the script has added the “X-In-GnuCash-File” header to it. If so, it discards that message because that means I already know about the deposit and don’t need to see the notification about it.

On my home server, the CGI script called above looks like this:

#!/bin/bash -e
TF=/fillmein/deposit_notification.$$
trap "rm -f $TF" EXIT
echo "Content-type: message/rfc822"
echo
if /fillmein/eastern-deposit-notification.py >| $TF; then
cat $TF
fi

The directory this script lives in is protected by a .htaccess file which restricts access to it so only my mail server and a few other local machines and networks can talk to it:

<FilesMatch "^deposit_notification\.cgi$">
Order deny,allow
Deny from all
Allow from fillmein
Allow from fillmein
Allow from fillmein
Allow from fillmein
</FilesMatch>
view raw htaccess hosted with ❤ by GitHub

This brings us to the “meat” of this whole endeavor, the script that parses the notification email, checks my GnuCash file, and logs into Eastern Bank to fetch the deposit description if necessary:

#!/usr/bin/env python3
# Objective: decide whether to notify about a deposit notification email
# from Eastern Bank, and if so, try to fetch a description of the deposit
# from online banking.
#
# 1. Read notification email from Eastern Bank on stdin.
# 2. Extract deposit amount from email.
# 3. Check if deposit is in GnuCash file. If so, then:
# a. Add a header to the message indicating that for procmail to read.
# b. Spit the modified message to stdout.
# c. Exit.
# 4. Try to log into online banking, requesting MFA by email.
# 5. Wait for MFA code to be saved to the filesystem by a different script.
# 6. Finish logging in using saved MFA code and then delete it.
# 7. Go to checking account page.
# 8. Find recent transaction with matching amount.
# 9. Add description of transaction to email.
# 10. Spit the modified message to stdout.
# 11. Exit.
import datetime
import os
from pandas import Timestamp
import pprint
import re
from selenium import webdriver
from selenium.common.exceptions import (
NoSuchElementException,
StaleElementReferenceException,
)
from selenium.webdriver.common.action_chains import ActionChains # noqa
from selenium.webdriver.common.keys import Keys
import sys
import time
import xml.etree.ElementTree as ET
gnucash_file = os.path.expanduser('/fillmein')
mfacode_file = os.path.expanduser('/fillmein/mfacode_eastern.txt')
password_file = os.path.expanduser('/fillmein')
ns = {
'gnc': 'http://www.gnucash.org/XML/gnc',
'act': 'http://www.gnucash.org/XML/act',
'trn': 'http://www.gnucash.org/XML/trn',
'ts': 'http://www.gnucash.org/XML/ts',
'split': 'http://www.gnucash.org/XML/split',
}
crlf = False
wait_time = 10
username_xpath = '//*[@id="AuthenticationFG.USER_PRINCIPAL"]'
password_xpath = '//*[@id="AuthenticationFG.ACCESS_CODE_1"]'
login_button_xpath = '//*[@id="VALIDATE_CREDENTIALS1"]'
account_xpath = '//span[contains(text(), "PREMIER CHECKING")]'
mfa_button_xpath = '//p[contains(text(), "Text Message")]'
mfa_code_xpath = '//*[@id="AuthenticationFG.RSA_GENERATED_OTP"]'
username = 'fillmein'
password = open(password_file).read().strip()
def get_email():
global crlf
if len(sys.argv) == 2:
email = open(sys.argv[1]).read()
else:
email = sys.stdin.read()
if '\r\n' in email:
email = re.sub(r'\r\n', r'\n', email)
crlf = True
return email
def print_email(email):
if crlf:
email = re.sub(r'\n', r'\r\n', email)
print(email)
def get_deposit_amount(email):
r = r'A deposit in the amount of \$\s*([\d.]+)'
match = re.search(r, email)
if not match:
raise Exception(f'No match for {r} in email')
return float(match.group(1))
def is_in_gnucash_file(amount):
tree = ET.parse(gnucash_file)
root = tree.getroot()
accounts = root.findall('*/gnc:account', ns)
# Assumes accounts are sorted by depth
assets_guid = None
for account in accounts:
name = account.find('act:name', ns).text
guid = account.find('act:id', ns).text
if not assets_guid and name == 'Assets':
assets_guid = guid
continue
parent = account.find('act:parent', ns)
if parent is not None:
parent = parent.text
if assets_guid and name == 'Checking' and parent == assets_guid:
checking_guid = guid
break
else:
raise Exception('Failed to find checking account in GnuCash file')
all_transactions = (t for t in root.findall('*/gnc:transaction', ns))
before = (datetime.date.today() datetime.timedelta(days=7)).\
strftime('%F')
recent_transactions = (
t for t in all_transactions
if t.find('trn:date-posted/ts:date', ns).text > before
)
checking_splits = (
s
for t in recent_transactions
for s in t.findall('trn:splits/trn:split', ns)
if s.find('split:account', ns).text == checking_guid
)
deposit_amounts = (
int(match.group(1)) / 100
for s in checking_splits
for value in s.findall('split:value', ns)
for match in (re.match(r'([0-9]+)/100', value.text),)
if match is not None
)
return(amount in deposit_amounts)
def reset_mfa_code():
try:
os.remove(os.path.realpath(mfacode_file))
except FileNotFoundError:
pass
def get_mfa_code(wait=30):
when = time.time() + wait
while time.time() < when:
try:
code = open(mfacode_file).read().strip()
break
except FileNotFoundError:
time.sleep(1)
else:
raise Exception('Failed to read MFA code')
os.remove(os.path.realpath(mfacode_file))
return(code)
def wait_for_stale(elt):
end = time.time() + wait_time
while time.time() < end:
try:
elt.find_elements_by_id('nosuchid')
time.sleep(1)
except StaleElementReferenceException:
break
else:
raise Exception("New page didn't load")
def wait_for_xpath(driver, xpath):
end = time.time() + wait_time
while time.time() < end:
try:
return(driver.find_element_by_xpath(xpath))
except NoSuchElementException:
time.sleep(1)
print(driver.page_source, file=sys.stderr)
raise Exception(f'No match for xpath {xpath}')
def find_one_of_several(driver, xpaths, timeout=wait_time):
end_at = time.time() + timeout
while time.time() < end_at:
for xpath in xpaths:
try:
driver.find_element_by_xpath(xpath)
return xpath
except Exception:
continue
time.sleep(1)
else:
print(driver.page_source, file=sys.stderr)
raise TimeoutError(f'Could not find any of {xpaths}')
def add_description_from_web(amount, email):
options = webdriver.ChromeOptions()
options.add_argument('–disable-gpu')
options.add_argument('headless')
driver = webdriver.Chrome(options=options)
try:
driver.get('https://online.easternbank.com')
driver.find_element_by_xpath(username_xpath).send_keys(username)
password_field = driver.find_element_by_xpath(password_xpath)
password_field.send_keys(password)
password_field.send_keys(Keys.ENTER)
wait_for_stale(password_field)
xpath = find_one_of_several(driver, (account_xpath, mfa_button_xpath))
if xpath == mfa_button_xpath:
button = driver.find_element_by_xpath(xpath)
reset_mfa_code()
# This should work but doesn't. I have no idea why. It clicks in
# the wrong place. :shrug:
# ActionChains(driver).move_to_element(button).click().perform()
# So instead I fetch the anchor link and get it directly rather
# than clicking.
anchor = button.find_element_by_xpath('..')
driver.get(anchor.get_attribute('href'))
code_input = wait_for_xpath(driver, mfa_code_xpath)
mfa_code = get_mfa_code()
code_input.send_keys(mfa_code)
code_input.send_keys(Keys.ENTER)
wait_for_stale(code_input)
account_tile = wait_for_xpath(driver, account_xpath)
account_tile.click()
wait_for_stale(account_tile)
wait_for_xpath(driver, '//td[contains(@ng-if,"row.colorCode")]')
then = datetime.date.today() datetime.timedelta(days=7)
rows = []
for row in driver.find_elements_by_tag_name('tr'):
columns = [c.get_attribute('innerHTML')
for c in row.find_elements_by_tag_name('td')]
rows.append(columns)
if not any(c for c in columns
if c.startswith('+') and c.endswith(f'{amount:,.2f}')):
continue
if Timestamp(columns[0]) < then:
continue
# So gross
description = re.search(r'</span>(.*?)\n', columns[1]).group(1)
break
else:
pprint.pprint(rows, stream=sys.stderr)
print(driver.page_source, file=sys.stderr)
print(f'Did not find {amount} deposit in online banking',
file=sys.stderr)
return
description = re.sub(r'^(?:Preauthorized|Transfer) Credit ', '',
description)
email = re.sub(r'^(Subject: .*)', f'\\1 (${amount}, {description})',
email, 1, re.M)
print(email)
sys.exit()
finally:
driver.quit()
def main():
email = get_email()
deposit_amount = get_deposit_amount(email)
print(f'Deposit amount is {deposit_amount}', file=sys.stderr)
if is_in_gnucash_file(deposit_amount):
email = re.sub(r'\n\n', '\nX-In-GnuCash-File: Yes\n\n', email, 1)
print_email(email)
sys.exit()
add_description_from_web(deposit_amount, email)
sys.exit(1)
if __name__ == '__main__':
main()

It’s worth noting that this script assumes that the GnuCash data file is uncompressed, since that’s how I have GnuCash configured. You’d have to modify the script to uncompress the file when reading it if you keep yours compressed.

If you don’t use GnuCash you could modify script so that instead of parsing a GnuCash file, it fetches the deposit description from online banking and add “X-In-GnuCash-File” to the header if it’s “Mobile Check Deposit” or a similar description that you don’t care about.

The function add_description_from_web is where the logging-into-online-banking magic happens, using Selenium. The tricky part there is handling MFA. That requires more infrastructure! If the site asks for MFA, the script asks it to send a text message, which is not terribly secure (I wish Eastern Bank MFA supported TOTP!) but more secure than email. This causes a text message to be sent to my Android phone. There, Tasker + AutoNotification intercepts the message, extracts the MFA code from it, and sends it to my home server. Here’s the Tasker description (the XML is here; it’s too ungainly and ugly to be worth embedding, so you can just download it):

Profile: Forward Eastern MFA Codes
Settings: Restore: no
Event: AutoNotification Intercept [ Configuration:Event Behaviour: true
Notification Type: Only Created Notifications
Persistency Type: Non-Persistent Only
Notification Apps: Messages
Notification Text: Eastern Bank will not call you for this code. ]
Enter Task: Send Eastern MFA Code
Task: Send Eastern MFA Code
A1: Variable Search Replace [
Variable: %antext
Search: Your security code is \d+
Multi-Line: On
One Match Only: On
Store Matches In Array: %mfamatch ]
A2: Variable Search Replace [
Variable: %mfamatch1
Search: \d+
One Match Only: On
Store Matches In Array: %mfacode ]
A3: HTTP Request [
Method: GET
URL: http://fillmein/mfacode.cgi?source=eastern&c=%mfacode1
Timeout (Seconds): 30 ]
A4: AutoNotification Cancel [
Configuration: Id: %anid
Timeout (Seconds): 20 ]

This is the mfacode.cgi script that Tasker is posting the MFA code to on my home server:

#!/bin/bash -e
declare -A params
while read kv; do
key="${kv%%=*}"
if [ ! "$key" ]; then
continue
fi
if [[ "$kv" =~ = ]]; then
value="${kv#*=}"
else
value=""
fi
params[$key]="$value"
done < <(echo "${QUERY_STRING}" | sed -E "s/&/\n/g")
# For debugging
#for key in "${!params[@]}"; do
# echo "params[$key]=${params[$key]}"
#done
#exit
safe() {
echo "$1" | tr -c -d — -_A-Za-z0-9
}
source=$(safe "${params[source]}")
code=$(safe "${params[c]}")
D=/fillmein
if [ -n "$source" ]; then
F="$D/mfacode_$source.txt"
else
F="$D/mfacode.txt"
fi
if [ -n "$code" ]; then
echo "$code" >| $F.tmp
mv -f $F.tmp $F
fi
echo "Content-Type: text/plain"
echo
echo ok
view raw mfacode.cgi hosted with ❤ by GitHub
Print Friendly, PDF & Email
Share

Leave a Reply

Your email address will not be published.