Skip to content

1stdibs/lambdium

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

lambdium

headless chrome + selenium webdriver in AWS Lambda

This uses the binaries from the serverless-chrome project to prototype running headless chromium with selenium-webdriver in AWS Lambda. I've also bundled the chromedriver binary so the browser can be interacted with using the Webdriver Protocol.

Background

The function interacts with headless Chromium process using chromedriver and a popular webdriver node.js client library.

This is highly experimental and not all chromedriver functions will work. Check issues for known issues.

Since this Lambda function is written using node.js, you can run almost any script written for selenium-webdriver. Example scripts can be found in the examples directory.

Installation

If you don't want to build the archive, you can also download a pre-built AWS Lambda *.zip that you can upload directly to a new node6.10 runtime function using the AWS CLI or Console with at least 256mb of memory. Visit lambdium releases.

Requirements

  • An AWS Account
  • Terraform (optional but highly recommended for function creation and deploy)
  • node.js + npm
  • make

Fetching dependencies

The headless chromium binary is too large for Github, you need to fetch it using a script bundled in this repository. Marco Lüthy has an excellent post on Medium about how he built chromium for for AWS Lambda here.

    $ ./scripts/fetch-dependencies.sh

Building Lambda .zip archive

    $ make

Creating and Deploying Using Terraform

This will create the function using Terraform with all required permissions.

    $ make deploy

The optional DEBUG_ENV environment variable will log additional information to Cloudwatch. The PATH environment variable points to the bin directory in this project—this is required in order to launch the chromedriver process.

Usage

If dependencies are installed and make deploy succeeds you can have Lambda run a script. There's an example of a selenium-webdriver simple script in the examples/ directory that the Lambda function can now run.

Expected JSON input for the function: {"Base64Script": "<Base64 Encoding of Selenium Script>"} (this can also be provided as an environment variable named BASE64_SCRIPT).

To run the example Selenium script, you can use the example with the AWS CLI in the scripts directory. It takes care of base64 encoding the file:

    $ scripts/invoke.sh

To use your own selenium-webdriver script:

    $ scripts/invoke.sh ~/Desktop/my-script.js

Related projects

About

prototype of headless chrome + selenium webdriver in AWS Lambda w/ terraform

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 52.0%
  • HCL 27.3%
  • Shell 13.2%
  • Makefile 7.5%