Skip to content

prohetamine/proxy-checker

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

41 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

logo

README is available in the following languages: Russian | English

Proxy Checker

proxy-checker - The most versatile proxy parser and proxy checker.

Why ?

I, and therefore many people, need to get a proxy to bypass the ban on IP with a large number of requests. This module allows you to perform almost all tasks related to proxy preparation. I, and therefore many people, need to build proxies individually for some sites, I also took care of this. I hope my contribution will make it easier for someone life.

Get started

Install the npm module @prohetamine/proxy-checker

$ npm install @prohetamine/proxy-checker

or

$ yarn add @prohetamine/proxy-checker

Examples and description

Connecting the module

const proxy = require('@prohetamine/proxy-checker')

The load function collects proxies and easily returns them via all and random.

object
key value default value required information
port number false none used for the port filter
debug boolean false none used for debugging
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  const count = await proxy.load({
    port: 8888,
    debug: true
  })

  console.log(count) // Number of collected proxies with port 8888
})()

The loadInterval function collects proxies with an interval of N milliseconds and easily returns them via all and random.

options default info
function () => { /* ... */} callback function returns in a single argument the number of IP loaded
int 5000 interval in milliseconds
object { started, port, debug } additional options
object
key value default value mandatory information
started boolean false no use for instant download
port boolean false no used to filter ports
debug boolean false none used for debugging
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  const kill = await proxy.loadInterval(count => {
    console.log(`count parse: ${count}`) // Number of collected proxies with port 8080
  }, 60000, { started: true, debug: false, port: 8080 })

  console.log(proxy.all({ port: 8888 })) // All downloaded IP addresses with port 8888
  console.log(proxy.random({ port: 80 })) // One random IP address with port 80

  setInterval(() => {
    if (proxy.all().length > 1488) {
      console.log(proxy.all())
      kill() // kills the current loadInterval
    }
  }, 5000)
})()

The all function returns an array of IP addresses when it is loaded load or loadInterval and false when the load has not yet occurred.

object
key value default value required information
port number false none used for the port filter
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  const count = await proxy.load({
    debug: true
  })

  console.log(proxy.all({ port: 1488 })) // All downloaded IP addresses with port 1488
})()

The random function returns a single random IP address when it is loaded load or loadInterval and false when the load has not yet occurred.

object
key value default value required information
port number false none used for the port filter
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  const count = await proxy.load({
    debug: true
  })

  console.log(proxy.random({ port: 8888 })) // One random IP address with port 8888
})()

The checkerInterval function checks proxies with an interval of N milliseconds for a specific site and easily returns them via get.all and get.random.

parameters default value information
string null unique identifier
object { url, isBrowser, browserConfig, timeout, stream, debug, indicators, session, onData } additional options
object
key value default value mandatory information
url string null no website link
isBrowser boolean false none used for checking through the browser
trashIgnore boolean false none used for optimization, removes loading of media, xhr, fetch, websocket, manifest, image, stylesheet, font, script
browserConfig object { ... } none used to configure the launcher
onData function () => { ... } none used to get the body of a successful request (works only for browser)
timeout int 10000 none interval in milliseconds
stream int 2 none number of concurrent threads
debug boolean false none used for debugging
indicators array [] none indicators of correctly loaded data
session string false none path to save valid IP addresses after completion or saving
quarantineMode boolean false none used to quarantine non-working proxies
array[object]
key value default value required information
keyword string no yes used to search the page by keyword
selector string no yes used for searching on a page by dom node
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  await proxy.loadInterval(count => {
    console.log(`count parse: ${count}`) // Number of collected proxies
  }, 60000, { started: true, debug: false })

  const { key, kill, save, clean } = await proxy.checkerInterval('chaturbate', {
    url: 'https://chaturbate.com/diana_smiley/',
    isBrowser: true,
    trashIgnore: true,
    browserConfig: proxy => ({
      headless: true,
      ignoreHTTPSErrors: true,
      args: [`--proxy-server=${proxy}`],
    }),
    timeout: 60000,
    stream: 10,
    session: __dirname + '/chaturbate.json',
    debug: true,
    indicators: [{
      //keyword: 'close_entrance_terms',
      selector: '#close_entrance_terms'
    }] // List other metrics in the objects if they are required
  })

  setInterval(() => {
    const proxys = proxy.get(key).all()
    console.clear()
    console.log(proxys, proxys.length)

    if (proxys.length > 20) {
      kill() // Kills the current checkerInterval and saves the session
      save() // Only saves the session
    }
  }, 3000)
})()
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  await proxy.loadInterval(count => {
    console.log(`count parse: ${count}`) // The number of proxies collected with the port
  }, 60000, { started: true, debug: false })

  const keyName = null // A unique identifier for IP accumulation, if set to null, will be generated automatically.

  const { key, kill, save, clean } = await proxy.checkerInterval(keyName, {
    url: 'https://github.com/prohetamine',
    timeout: 30000,
    stream: 20,
    session: __dirname + '/github.json',
    debug: true,
    indicators: [{
      keyword: 'Prohetamine',
      //selector: '#close_entrance_terms'
    }, {
      keyword: 'Stas'
    }] // List other metrics in the objects if they are required
  })

  setInterval(() => {
    const proxys = proxy.get(key).all()
    console.clear()
    console.log(proxys, proxys.length)

    if (proxys.length > 20) {
      kill() // Kills the current checkerInterval and saves the session
      save() // Only saves the session
    }
  }, 3000)
})()

The get.all function returns an array of checked checkerInterval IP addresses by a unique identifier and false when the download has not yet occurred.

get(key)
parameters default value required information
string null yes unique identifier
object
key value default value required information
port number false none used for the port filter
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  await proxy.loadInterval(count => {
    console.log(`count parse: ${count}`) // Number of collected proxies
  }, 60000, { started: true, debug: false })

  const { key, kill, save, clean } = await proxy.checkerInterval('chaturbate', {
    url: 'https://chaturbate.com/diana_smiley/',
    isBrowser: true,
    debugBrowser: false,
    timeout: 60000,
    stream: 10,
    session: __dirname + '/chaturbate.json',
    debug: true,
    indicators: [{
      //keyword: 'close_entrance_terms',
      selector: '#close_entrance_terms'
    }] // List other metrics in the objects if they are required
  })

  setInterval(() => {
    const proxys = proxy.get(key).all()
    console.clear()
    console.log(proxys, proxys.length)

    if (proxys.length > 20) {
      kill() // Kills the current checkerInterval and saves the session
      save() // Only saves the session
    }
  }, 3000)
})()

The function get.random returns one random of the checked checkerInterval IP addresses by a unique identifier and false when the download has not yet occurred.

get(key)
parameters default value required information
string null yes unique identifier
object
key value default value required information
port number false none used for the port filter
const proxy = require('@prohetamine/proxy-checker')

;(async () => {
  await proxy.loadInterval(count => {
    console.log(`count parse: ${count}`) // Number of collected proxies
  }, 60000, { started: true, debug: false })

  const { key, kill, save, clean } = await proxy.checkerInterval('chaturbate', {
    url: 'https://chaturbate.com/diana_smiley/',
    isBrowser: true,
    debugBrowser: false,
    timeout: 60000,
    stream: 10,
    session: __dirname + '/chaturbate.json',
    debug: true,
    indicators: [{
      //keyword: 'close_entrance_terms',
      selector: '#close_entrance_terms'
    }] // List other metrics in the objects if they are required
  })

  setInterval(() => {
    console.log(proxy.get(key).random())
  }, 3000)
})()

Other examples from this project can be found here: examples

Contacts

My Telegram: @prohetamine, channel

Email: [email protected]

Donat money: patreon

If you have any questions and/or suggestions, please email me in telegram, if you find any bugs also let me know, I will be very grateful.

Releases

No releases published

Packages

No packages published