logged-in crawls

[pooneh-nb]

Hi there,
I'm wondering if "logged-in crawls" can be done using TrackerRadar. The crawler needs to preserve logged-in status to Facebook/Google.

[kdzwinel]

Hey @pooneh-nb ! Sorry for a late response. crawlerConductor can't currently do it, but a single crawler can. You have to create one non-incognito context and reuse it via browserContext param to crawl multiple sites. Sth like this:

const browser = /*create browser*/
const loggedInContext = browser.defaultBrowserContext();

// data between crawls will not be cleared
const data1 = await crawler(new URL('https://example.com'), {browserContext: loggedInContext});
const data2 = await crawler(new URL('https://example.com'), {browserContext: loggedInContext});
…

LMK if you have any follow up questions!