Deploying big Spacy NLP models on AWS Lambda + S3

Xoel López Barata
4 min readAug 20, 2020
Source: Unsplash

Hey y’all!

A few weeks ago, I started using Spacy to detect locations in job descriptions. Spacy is an NLP library that lets you do pretty powerful stuff out-of-the-box and get things done fast.

Everything was working fine locally. But NoiceJobs (my project) is hosted on Heroku and uses the cheapest dynos possible, with only 0,5GB of RAM. For running simple apps that’s enough, but ML code is normally more memory and CPU-intensive, so when I deployed the new version of the app on Heroku I’d get memory quota exceeded errors all the time.

Some AWS engineers jumped into the conversation and after some back-and-forth, we came to the conclusion that AWS could be a good solution for my problem

I had used Flask on AWS Lambda on the past with Zappa and liked the easiness of the deployment process and fast is to get a small app running without too much hassle (most of the time)

Zappa lets you deploy Django or Flask apps on AWS Lambda, but I’d rather use Flask for something simple like this to keep memory usage and the bundle size as low as possible.

I made a smaller but functioning version of the code I run for my app, and you can find it in this GitHub repo:

The README has all the info on how to run it locally and deploy it so I won’t get you bored repeating it here. If you have any doubts write to me on Twitter and I’ll try to help you.

What I’ll do instead is telling you what didn’t work and what worked for me, so that you don’t waste your time the next time.

Loading models — the wrong way (for AWS Lambda)

Xoel López Barata

Freelance data scientist and software developer. On Twitter: @xoelipedes.