This package has been deprecated

Author message:

this package has been deprecated

cex-crawler

1.0.8 • Public • Published

CEX Crawler Build Status Coverage Status

Crawls the CEX store you want (*.webuy.com), getting the available stores and the products based on search terms and filters.

Installation (NPM)

$ npm install cex-crawler

Installation (Github)

$ git clone https://github.com/Dionakra/cex-crawler.git
$ cd cex-crawler
$ npm install

Available methods

setStore(country)

Set the country you want to get the info. Available countries are:

  • AUSTRALIA
  • CANARY_ISLANDS
  • INDIA
  • IRELAND
  • ITALY
  • MEXICO
  • NETHERLANDS
  • POLAND
  • PORTUGAL
  • SPAIN
  • UNITED_KINGDOM
  • UNITED_STATES

Examples

const { setCountry, STORE } = require('cex-crawler');

// Setting the country with the JSON
setCountry(STORE.SPAIN);

// Setting the country with the code
setCountry('es');

getStores()

Retrieve the active stores at the selected country (United Kingdom as default)

Example

var { getStores, setCountry, STORE } = require('cex-crawler');

// Set your country
setCountry(STORE.SPAIN);
getStores().then( data => console.log(data));

This returns a JSON with the store id (used later for searching) and its name:

[ { id: '1075', name: 'Algeciras San Antonio' },
  ...
]

getProducts(opts)

Get the products using the options to search at *.webuy.com, adding some custom fields like the URL to the images and the full URL to the product (it is relative inside CEX webpage). You can set this options to perform custom searches. The available fields are:

const FIELD = {
  QUERY: 'stext',
  CATEGORY: 'catid',
  STORE: 'refinebystore',
  PRICE: 'price', // &price=MIN-MAX
  PAGE: 'page',
  COUNTER: 'counter',
  SORTING: 'sortOn'
};

The sorting option is also a enumerated field. You can get the sorting options with the SORT variable:

const SORT = {
  PRICE_ASC: 'sellprice_asc',
  PRICE_DESC: 'sellprice_desc',
  NAME_ASC: 'box_name_asc',
  NAME_DESC: 'box_name_desc',
  RATING: 'box_rating_desc'
}

Example

const { getProducts, setCountry, FIELD, SORT } = require('cex-crawler');

setCountry(FIELD.SPAIN);
let opts = {
  [FIELD.QUERY]: 'Bayonetta',
  [FIELD.SORTING]: SORT.PRICE_ASC,
  [FIELD.PRICE]: '0-20'
}
getProducts(opts).then(data => console.log(data));

This returns:

[ { id: '8420266977458',
    url: 'https://es.webuy.com/product.php?sku=8420266977458',
    name: 'Bayonetta: Bloody Fate',
    category: 'DVD',
    subcategory: 'DVD Anime',
    currency: 'EUR',
    unit_price: 3,
    cash_price: 0.7,
    exchange_price: 1.1,
    stock: 1,
    images:
     [ large: 'https://es.webuy.com/product_images/DVD/DVD%20Anime/8420266977458_l.jpg',
       medium: 'https://es.webuy.com/product_images/DVD/DVD%20Anime/8420266977458_m.jpg',
       small: 'https://es.webuy.com/product_images/DVD/DVD%20Anime/8420266977458_s.jpg' ] }
  ...
]

getStock(sku)

Returns the available stock of the product in your country, with its stock number.

Example

const { getProductStock, setCountry, STORE} = require('cex-crawler');

setCountry(STORE.UNITED_KINGDOM);
const stockUK = await getProductStock(5055277000746);
console.log("Stock of Bayonetta (Xbox 360) in United Kingdom:");
console.log(stockUK);

This returns:

Stock of Bayonetta (Xbox 360) in United Kingdom:
[ { id: '286', name: 'Acocks Green', stock: '2' },
  { id: '270', name: 'Aldershot', stock: '3' },
  { id: '109', name: 'Andover', stock: '1' },
  { id: '3052', name: 'Arnold', stock: '2' },
  ...
]

Built With

  • Axios - HTTP/S requests tool
  • Cheerio - Scraping tool
  • QS - QueryString tool

Readme

Keywords

Package Sidebar

Install

npm i cex-crawler

Weekly Downloads

9

Version

1.0.8

License

GPL-3.0

Last publish

Collaborators

  • dionakra