Bugfix: The result page of crawlPage API reported an error. Expose the browser and let the user decide to close the browser

Author: coder-hxl

README.md

@@ -36,6 +36,9 @@ We can do the following:
       + [Choose crawling mode](#Choose-crawling-mode)
       + [Multiple crawler application instances](#Multiple-crawler-application-instances)
     * [Crawl page](#Crawl-page)
+      + [jsdom](#jsdom)
+      + [browser](#browser)
+      + [page](#page)
     * [Crawl interface](#Crawl-interface)
     * [Crawl files](#Crawl-files)
     * [Start polling](#Start-polling)
@@ -49,7 +52,6 @@ We can do the following:
     * [crawlPage](#crawlPage)
        + [Type](#Type-2)
        + [Example](#Example-2)
-       + [About page](#About-page)
     * [crawlData](#crawlData)
        + [Type](#Type-3)
        + [Example](#Example-3)
@@ -107,7 +109,7 @@ const myXCrawl = xCrawl({
 myXCrawl.startPolling({ d: 1 }, () => {
   // Call crawlPage API to crawl Page
   myXCrawl.crawlPage('https://www.youtube.com/').then((res) => {
-    const { jsdom } = res.data // By default, the JSDOM library is used to parse Page
+    const { browser, jsdom } = res // By default, the JSDOM library is used to parse Page
 
     // Get the cover image element of the Promoted Video
     const imgEls = jsdom.window.document.querySelectorAll(
@@ -127,6 +129,9 @@ myXCrawl.startPolling({ d: 1 }, () => {
       requestConfig,
       fileConfig: { storeDir: path.resolve(__dirname, './upload') }
     })
+      
+    // Close the browser
+    browser.close()
   })
 })
 ```
@@ -206,10 +211,27 @@ const myXCrawl = xCrawl({
 })
 
 myXCrawl.crawlPage('https://xxx.com').then(res => {
-  const { jsdom, page } = res.data
+  const { jsdom, browser, page } = res
+  
+  // Close the browser
+  browser.close()
 })
 ```
 
+#### jsdom
+
+Refer to [jsdom](https://github.com/jsdom/jsdom) for specific usage.
+
+#### browser
+
+**Purpose of calling close: **browser will keep running, so the file will not be terminated. Do not call [crawlPage](#crawlPage) or [page](#page) if you need to use it later. When you modify the properties of the browser object, it will affect the browser inside the crawlPage of the crawler instance, the returned page, and the browser, because the browser is shared within the crawlPage API of the crawler instance.
+
+Refer to [browser](https://pptr.dev/api/puppeteer.browser) for specific usage.
+
+#### page
+
+The page attribute can be used for interactive operations such as events. For details, refer to [page](https://pptr.dev/api/puppeteer.page).
+
 ### Crawl interface
 
 Crawl interface data through [crawlData()](#crawlData)
@@ -276,7 +298,10 @@ myXCrawl. startPolling({ h: 2, m: 30 }, (count, stopPolling) => {
   // will be executed every two and a half hours
   // crawlPage/crawlData/crawlFile
   myXCrawl.crawlPage('https://xxx.com').then(res => {
-    const { jsdom, page } = res.data
+    const { jsdom, browser, page } = res
+    
+    // Close the browser
+    browser.close()
   })
 })
 ```
@@ -480,15 +505,14 @@ const myXCrawl = xCrawl({ timeout: 10000 })
 
 // crawlPage API
 myXCrawl.crawlPage('https://xxx.com/xxxx').then((res) => {
-  const { jsdom, page } = res.data
+  const { jsdom, browser, page } = res
   console.log(jsdom.window.document.querySelector('title')?.textContent)
+    
+  // Close the browser
+  browser.close()
 })
 ```
 
-#### About page
-
-The page attribute can be used for interactive operations such as events. For details, refer to [page](https://pptr.dev/api/puppeteer.page).
-
 ### crawlData
 
 crawlData is the method of the crawler instance, which is usually used to crawl APIs to obtain JSON data and so on.
@@ -771,10 +795,9 @@ interface FileInfo {
 ```ts
 interface CrawlPage {
   httpResponse: HTTPResponse | null // The type of HTTPResponse in the puppeteer library
-  data: {
-    page: Page // The type of Page in the puppeteer library
-    jsdom: JSDOM // The type of JSDOM in the jsdom library
-  }
+  browser // The type of Browser in the puppeteer library
+  page: Page // The type of Page in the puppeteer library
+  jsdom: JSDOM // The type of JSDOM in the jsdom library
 }
 ```
 

docs/cn.md

@@ -29,19 +29,27 @@ crawlPage API 内部使用 [puppeteer](https://github.com/puppeteer/puppeteer) 
 # 目录
 
 - [安装](#安装)
+
 - [示例](#示例)
+
 - [核心概念](#核心概念)
     * [创建应用](#创建应用)
       + [一个爬虫应用实例](#一个爬虫应用实例)
       + [选择爬取模式](#选择爬取模式)
       + [多个爬虫应用实例](#多个爬虫应用实例)
     * [爬取页面](#爬取页面)
+    
+      + [jsdom](#jsdom)
+      + [browser](#browser)
+      + [page](#page)
+    
     * [爬取接口](#爬取接口)
     * [爬取文件](#爬取文件)
     * [启动轮询](#启动轮询)
     * [请求间隔时间](#请求间隔时间)
     * [requestConfig 选项的多种写法](#requestConfig-选项的多种写法)
     * [获取结果的多种方式](#获取结果的多种方式)
+    
 - [API](#API)
     * [xCrawl](#xCrawl)
        + [类型](#类型-1)
@@ -52,13 +60,13 @@ crawlPage API 内部使用 [puppeteer](https://github.com/puppeteer/puppeteer) 
     * [crawlData](#crawlData)
        + [类型](#类型-3)
        + [示例](#示例-3)
-       + [关于 page](#关于-page)
     * [crawlFile](#crawlFile)
        + [类型](#类型-4)
        + [示例](#示例-4)
     * [startPolling](#startPolling)
        + [类型](#类型-5)
        + [示例](#示例-5)
+    
 - [类型](#类型-6)
     * [AnyObject](#AnyObject)
     * [Method](#Method)
@@ -76,6 +84,7 @@ crawlPage API 内部使用 [puppeteer](https://github.com/puppeteer/puppeteer) 
     * [CrawlResCommonArrV1](#CrawlResCommonArrV1)
     * [FileInfo](#FileInfo)
     * [CrawlPage](#CrawlPage)
+    
 - [更多](#更多)
 
 ## 安装
@@ -106,7 +115,7 @@ const myXCrawl = xCrawl({
 myXCrawl.startPolling({ d: 1 }, () => {
   // 调用 crawlPage API 爬取 Page
   myXCrawl.crawlPage('https://www.bilibili.com/guochuang/').then((res) => {
-    const { jsdom } = res.data // 默认使用了 JSDOM 库解析 Page
+    const { browser, jsdom } = res // 默认使用了 JSDOM 库解析 Page
 
     // 获取轮播图片元素
     const imgEls = jsdom.window.document.querySelectorAll('.chief-recom-item img')
@@ -120,6 +129,9 @@ myXCrawl.startPolling({ d: 1 }, () => {
       requestConfig, 
       fileConfig: { storeDir: path.resolve(__dirname, './upload') } 
     })
+      
+    // 关闭浏览器
+    browser.close()
   })
 })
 ```
@@ -198,10 +210,27 @@ import xCrawl from 'x-crawl'
 const myXCrawl = xCrawl({ timeout: 10000 })
 
 myXCrawl.crawlPage('https://xxx.com').then(res => {
-  const { jsdom, page } = res.data
+  const { jsdom, browser, page } = res
+  
+  // 关闭浏览器
+  browser.close()
 })
 ```
 
+#### jsdom
+
+具体使用参考 [jsdom](https://github.com/jsdom/jsdom) 。
+
+#### browser
+
+**调用 close 的目的：**browser 会一直保持运行，造成文件不会终止。如果后面还需要用到 [crawlPage](#crawlPage) 或者 [page](#page) 请勿调用。当您修改 browser 对象的属性时，会对该爬虫实例的 crawlPage 内部的 browser 和返回的 page 以及 browser 造成影响，因为 browser 在爬虫实例的 crawlPage API 内是共享的。
+
+具体使用参考 [browser](https://pptr.dev/api/puppeteer.browser) 。
+
+#### page
+
+page 属性可以做事件之类的交互操作，具体使用参考 [page](https://pptr.dev/api/puppeteer.page) 。
+
 ### 爬取接口
 
 通过 [crawlData()](#crawlData) 爬取接口数据
@@ -268,7 +297,9 @@ myXCrawl.startPolling({ h: 2, m: 30 }, (count, stopPolling) => {
   // 每隔两个半小时会执行一次
   // crawlPage/crawlData/crawlFile
   myXCrawl.crawlPage('https://xxx.com').then(res => {
-    const { jsdom, page } = res.data
+    const { jsdom, browser, page } = res
+    
+    browser.close()
   })
 })
 ```
@@ -471,15 +502,14 @@ const myXCrawl = xCrawl({ timeout: 10000 })
 
 // crawlPage API
 myXCrawl.crawlPage('https://xxx.com/xxx').then((res) => {
-  const { jsdom, page } = res.data
+  const { jsdom, browser, page } = res
   console.log(jsdom.window.document.querySelector('title')?.textContent)
+  
+  // 关闭浏览器
+  browser.close()
 })
 ```
 
-#### 关于 page 
-
-page 属性可以做事件之类的交互操作，具体使用参考 [page](https://pptr.dev/api/puppeteer.page) 。
-
 ### crawlData
 
 crawl 是爬虫实例的方法，通常用于爬取 API ，可获取 JSON 数据等等。
@@ -764,10 +794,9 @@ interface FileInfo {
 ```ts
 interface CrawlPage {
   httpResponse: HTTPResponse | null // puppeteer 库的 HTTPResponse 类型
-  data: {
-    page: Page // puppeteer 库的 Page 类型
-    jsdom: JSDOM // jsdom 库的 JSDOM 类型
-  }
+  browser: Browser // puppeteer 库的 Browser 类型
+  page: Page // puppeteer 库的 Page 类型
+  jsdom: JSDOM // jsdom 库的 JSDOM 类型
 }
 ```
 

package.json

@@ -1,7 +1,7 @@
 {
   "private": true,
   "name": "x-crawl",
-  "version": "3.1.2",
+  "version": "3.2.0",
   "author": "coderHXL",
   "description": "x-crawl is a flexible nodejs crawler library. ",
   "license": "MIT",
@@ -11,8 +11,8 @@
     "build-dts": "tsc && prettier --write ./publish/src",
     "build-strict": "pnpm test-dev && pnpm build && pnpm test-pro",
     "start": "rollup --config script/start.mjs",
-    "test-dev": "jest test/modal/test.ts dev",
-    "test-pro": "jest test/modal/test.ts pro",
+    "test-dev": "jest test/modal/test.ts dev --detectOpenHandles",
+    "test-pro": "jest test/modal/test.ts pro --detectOpenHandles",
     "prettier": "prettier --write ."
   },
   "dependencies": {