New Job
Advanced options
Authentication
For password-protected sites: upload a profile.tar.gz (a browser session that is already logged in), then pick it here. See the README for how to create one.
Crawl options
Also crawl linked files hosted on these domains (e.g. a download/file server).