Cookies not working – again (regression?)

[p3k]

If I read #83 correctly, it should have been merged into master already.

However, after I installed this package via git clone / npm install today, I get an error message in the console output when I try to render a page with a cookie:

# curl -o test.pdf -XPOST -d '{"url": "http://github.com", "cookies": [{ "name": "foo", "value": "bar" }]}' -H "Content-Type: application/json" http://localhost:9000/api/render
2019-05-06T10:54:08.359Z - info: [render-core.js] Rendering with opts: {
  "cookies": [
    {
      "name": "foo",
      "value": "bar"
    }
  ],
  "scrollPage": false,
  "emulateScreenMedia": true,
  "ignoreHttpsErrors": false,
  "html": null,
  "viewport": {
    "width": 1600,
    "height": 1200
  },
  "goto": {
    "waitUntil": "networkidle2"
  },
  "output": "pdf",
  "pdf": {
    "format": "A4",
    "printBackground": true
  },
  "screenshot": {
    "type": "png",
    "fullPage": true
  },
  "failEarly": false,
  "url": "http://github.com"
}
2019-05-06T10:54:08.542Z - info: [render-core.js] Set browser viewport..
2019-05-06T10:54:08.543Z - info: [render-core.js] Emulate @media screen..
2019-05-06T10:54:08.544Z - info: [render-core.js] Setting cookies..
2019-05-06T10:54:08.623Z - error: [render-core.js] Error when rendering page: Error: Protocol error (Network.setCookies): Target closed.
2019-05-06T10:54:08.623Z - error: [render-core.js] Error: Protocol error (Network.setCookies): Target closed.
    at Promise (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:202:56)
    at new Promise (<anonymous>)
    at CDPSession.send (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:201:12)
    at Object.render (/home/tobi/Projects/url-to-pdf-api/src/core/render-core.js:90:20)
    at process._tickCallback (internal/process/next_tick.js:68:7)
2019-05-06T10:54:08.623Z - info: [render-core.js] Closing browser..
2019-05-06T10:54:08.626Z - error: [error-logger.js] Request headers: host=localhost:9000, user-agent=curl/7.64.1, accept=*/*, content-type=application/json, content-length=76
2019-05-06T10:54:08.626Z - error: [error-logger.js] Request parameters:
2019-05-06T10:54:08.626Z - error: [error-logger.js] Request body: url=http://github.com, cookies=[name=foo, value=bar]
2019-05-06T10:54:08.626Z - error: [error-logger.js] { Error: Protocol error (Network.setCookies): Target closed.
    at Promise (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:202:56)
    at new Promise (<anonymous>)
    at CDPSession.send (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:201:12)
    at Object.render (/home/tobi/Projects/url-to-pdf-api/src/core/render-core.js:90:20)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  message: 'Protocol error (Network.setCookies): Target closed.' } 'Error: Protocol error (Network.setCookies): Target closed.\n    at Promise (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:202:56)\n    at new Promise (<anonymous>)\n    at CDPSession.send (/home/tobi/Projects/url-to-pdf-api/node_modules/puppeteer/lib/Connection.js:201:12)\n    at Object.render (/home/tobi/Projects/url-to-pdf-api/src/core/render-core.js:90:20)\n    at process._tickCallback (internal/process/next_tick.js:68:7)'
POST /api/render 500 268.897 ms - 88

Furthermore, the cookies do not come through at all when using GET:

# curl -o Downloads/test.pdf "http://localhost:9000/api/render?url=https://github.com&cookies[0][name]=foo&cookies[0][value]=bar"
2019-05-06T10:58:56.969Z - info: [render-core.js] Rendering with opts: {
  "cookies": [],
  "scrollPage": false,
  "emulateScreenMedia": true,
  "ignoreHttpsErrors": false,
  "html": null,
  "viewport": {
    "width": 1600,
    "height": 1200
  },
  "goto": {
    "waitUntil": "networkidle2"
  },
  "output": "pdf",
  "pdf": {
    "format": "A4",
    "printBackground": true,
    "margin": {}
  },
  "screenshot": {
    "type": "png",
    "fullPage": true,
    "clip": {}
  },
  "failEarly": false,
  "url": "https://github.com"
}

Would be grateful for assistance or clarification.

[p3k]

Update: Testing the same requests with the Docker container provided via microbox/node-url-to-pdf-api displayed a hint to a potential fix:

2019-05-06T11:06:10.493951843Z (node:1) UnhandledPromiseRejectionWarning: Error: Protocol error (Network.deleteCookies): At least one of the url and domain needs to be specified undefined

And in fact, with a domain property the cookie works:

# curl -o test.pdf -XPOST -d '{"url": "http://github.com", "cookies": [{ "name": "foo", "value": "bar", "domain": "localhost" }]}' -H "Content-Type: application/json" http://localhost:9000/api/render
2019-05-06T11:11:26.137Z - info: [render-core.js] Rendering with opts: {
  "cookies": [
    {
      "name": "foo",
      "value": "bar",
      "domain": "github.com"
    }
  ],
  "scrollPage": false,
  "emulateScreenMedia": true,
  "ignoreHttpsErrors": false,
  "html": null,
  "viewport": {
    "width": 1600,
    "height": 1200
  },
  "goto": {
    "waitUntil": "networkidle2"
  },
  "output": "pdf",
  "pdf": {
    "format": "A4",
    "printBackground": true
  },
  "screenshot": {
    "type": "png",
    "fullPage": true
  },
  "failEarly": false,
  "url": "http://github.com"
}
2019-05-06T11:11:26.322Z - info: [render-core.js] Set browser viewport..
2019-05-06T11:11:26.323Z - info: [render-core.js] Emulate @media screen..
2019-05-06T11:11:26.324Z - info: [render-core.js] Setting cookies..
2019-05-06T11:11:26.328Z - info: [render-core.js] Goto url http://github.com ..
2019-05-06T11:11:29.746Z - info: [render-core.js] Rendering ..
2019-05-06T11:11:32.614Z - info: [render-core.js] Closing browser..
POST /api/render 200 6544.997 ms - 10299087

So, the fix for one half of this issue (the POST part) is obviously to add to the README that a cookie’s domain or url property is required.

[kimmobrunfeldt]

I haven't personally used the cookies option so the workings are unknown for me. I'm happy to merge a PR which improves readme.