Use Cheerio in an Angular project

Question

I've got an Angular web-scraping project that I'm trying to use Cheerio in. I have included Cheerio into my angular.json file. When I include Cheerio into my App component I get the following error message:

index.js:43 Uncaught ReferenceError: global is not defined
    at Object../node_modules/buffer/index.js (index.js:43)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/readable-stream/node_modules/safe-buffer/index.js (index.js:2)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/readable-stream/lib/_stream_readable.js (_stream_readable.js:55)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/readable-stream/readable-browser.js (readable-browser.js:1)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/stream-browserify/index.js (index.js:28)
    at __webpack_require__ (bootstrap:79)

I fixed this error by typing this into index.html as @Leandro Matilla suggested:

if (global === undefined) {
    var global = window;
}

But now I get a different error:

util.js:103 Uncaught ReferenceError: Buffer is not defined
    at Object../node_modules/core-util-is/lib/util.js (util.js:103)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/readable-stream/lib/_stream_readable.js (_stream_readable.js:67)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/readable-stream/readable-browser.js (readable-browser.js:1)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/stream-browserify/index.js (index.js:28)
    at __webpack_require__ (bootstrap:79)
    at Object../node_modules/cheerio/node_modules/parse5/lib/parser/parser_stream.js (parser_stream.js:3)
    at __webpack_require__ (bootstrap:79)

How do I fix this error message? I've tried other solutions on StackOverflow, but none of them have worked...

"...Cheerio, an implementation of core jQuery designed specifically for the server." Why do you want to use a server lib in an Angular project? — Phix, Feb 12 '20 at 18:07
@Phix Because I'm making a web scraper, and as far as my knowledge goes jQuery works on the current DOM, not provided data. — Johannes, Feb 12 '20 at 19:59
jQuery is a library to help with DOM manipulation, there's no DOM in node. Scraping is done on the server. — Phix, Feb 12 '20 at 21:34
You're getting the `Buffer` error because `Buffer` doesn't exist in the browser. **This package is designed to run in a NodeJS environment**. — Phix, Feb 12 '20 at 23:25

score 0 · Answer 1 · answered Feb 12 '20 at 18:05

0

Add this to your index.html

<script>
    if (global === undefined) {
        var global = window;
    }
</script>

answered Feb 12 '20 at 18:05

Leandro Matilla

629
3
12

This solution fixed the first error, but now I get a different error (See my edited question). – Johannes Feb 12 '20 at 20:07
install the buffer package using npm or yarn. And add the following into polyfills.ts () – Leandro Matilla Feb 12 '20 at 20:09
global.Buffer = global.Buffer || require('buffer').Buffer; – Leandro Matilla Feb 12 '20 at 20:09
If the answer resolve the first problem, close the ask. And make one new – Leandro Matilla Feb 12 '20 at 20:10
https://stackoverflow.com/questions/50371593/angular-6-uncaught-referenceerror-buffer-is-not-defined – Leandro Matilla Feb 12 '20 at 20:10

Kyle Burkett · Answer 2 · 2020-02-14T17:02:24.737

0

You shouldn't have to add it to angular.json. Here's how you should do it. Note: as @phix correctly points out, cheerio doesn't appear to be intended for front-end use. To that point, any manipulation you're attempting to do with cheerio can likely be done more clearly using Angular alone.

First, install it either with npm or yarn.

npm install cheerio

Then, add it to your component:

import * as cheerio from 'cheerio';

You should then be able to use it, like in the documentation:

const $ = cheerio.load('<ul id="fruits">...</ul>');

More examples of usage: https://www.npmjs.com/package/cheerio

Edit: As mentioned in the comments, it seems like cheerio relies on server side libraries. You need to switch to the Node framework to use this package. A guide on setting up Cheerio and Node: https://buttercms.com/blog/web-scraping-with-nodejs-and-cheerio

edited Feb 14 '20 at 17:02

answered Feb 12 '20 at 18:06

Kyle Burkett

1,161
11
26

This solution doesn't work. I still get `global is not defined`. – Johannes Feb 12 '20 at 20:06
@Xydez in the comments they correctly suggest using Nodejs. I would switch to Node for the framework. Selenium also works well as a scraper and would work with node or python. It would be unique to run a web scraper on the front end using something like an iframe, I've never seen that done. Best approach is server side, and, you will find lots of guides to help. Here's one for Node and cheerio: https://buttercms.com/blog/web-scraping-with-nodejs-and-cheerio – Kyle Burkett Feb 14 '20 at 17:00

Use Cheerio in an Angular project

2 Answers2