|
1 | 1 | # [v4.0.0](https://github.com/coder-hxl/x-crawl/compare/v3.3.0...v4.0.0) (2023-03-27)
|
2 | 2 |
|
3 |
| -### Features |
| 3 | +### 🚨 Breaking Changes |
4 | 4 |
|
5 | 5 | - The crawlPage API can add batch requests.
|
6 | 6 | - The crawlPage API remove JSDOM.
|
| 7 | + |
| 8 | +### 🚀 Features |
| 9 | + |
7 | 10 | - Document updates.
|
8 | 11 |
|
9 | 12 | # [v3.3.0](https://github.com/coder-hxl/x-crawl/compare/v3.2.12...v3.3.0) (2023-03-24)
|
10 | 13 |
|
11 |
| -### Features |
| 14 | +### 🚀 Features |
12 | 15 |
|
13 | 16 | - The crawlerPage API crawling page allows to carry Cookies (for login and other operations).
|
14 | 17 |
|
15 | 18 | # [v3.2.12](https://github.com/coder-hxl/x-crawl/compare/v3.2.11...v3.2.12) (2023-03-23)
|
16 | 19 |
|
17 |
| -### Bug Fixes |
| 20 | +### 🐞 Bug Fixes |
18 | 21 |
|
19 | 22 | - Document jump fix
|
20 | 23 |
|
21 | 24 | # [v3.2.11](https://github.com/coder-hxl/x-crawl/compare/v3.2.10...v3.2.11) (2023-03-22)
|
22 | 25 |
|
23 |
| -### Bug Fixes |
| 26 | +### 🚀 Features |
24 | 27 |
|
25 |
| -- Fix crawlPage API internal error |
| 28 | +- Test updates, unit test all APIs |
26 | 29 |
|
27 |
| -### Features |
| 30 | +### 🐞 Bug Fixes |
28 | 31 |
|
29 |
| -- Test updates, unit test all APIs |
| 32 | +- Fix crawlPage API internal error |
30 | 33 |
|
31 | 34 | # [v3.2.10](https://github.com/coder-hxl/x-crawl/compare/v3.2.9...v3.2.10) (2023-03-21)
|
32 | 35 |
|
33 |
| -### Features |
| 36 | +### 🚀 Features |
34 | 37 |
|
35 | 38 | - Update documents
|
36 | 39 |
|
37 | 40 | # [v3.2.9](https://github.com/coder-hxl/x-crawl/compare/v3.2.8...v3.2.9) (2023-03-20)
|
38 | 41 |
|
39 |
| -### Features |
| 42 | +### 🚀 Features |
40 | 43 |
|
41 | 44 | - Update dependency
|
42 | 45 |
|
43 | 46 | # [v3.2.8](https://github.com/coder-hxl/x-crawl/compare/v3.2.7...v3.2.8) (2023-02-19)
|
44 | 47 |
|
45 |
| -### Bug Fixes |
| 48 | +### 🐞 Bug Fixes |
46 | 49 |
|
47 | 50 | - Internal type adjustment.
|
48 | 51 | - Catch crawlPage API errors.
|
49 | 52 |
|
50 | 53 | # [v3.2.7](https://github.com/coder-hxl/x-crawl/compare/v3.2.6...v3.2.7) (2023-03-14)
|
51 | 54 |
|
52 |
| -### Features |
| 55 | +### 🚀 Features |
53 | 56 |
|
54 | 57 | - Update documents
|
55 | 58 |
|
56 | 59 | # [v3.2.6](https://github.com/coder-hxl/x-crawl/compare/v3.2.5...v3.2.6) (2023-03-14)
|
57 | 60 |
|
58 |
| -### Features |
| 61 | +### 🚀 Features |
59 | 62 |
|
60 | 63 | - Update documents
|
61 | 64 |
|
62 | 65 | # [v3.2.5](https://github.com/coder-hxl/x-crawl/compare/v3.2.4...v3.2.5) (2023-03-12)
|
63 | 66 |
|
64 |
| -### Features |
| 67 | +### 🚀 Features |
65 | 68 |
|
66 | 69 | - Update documents
|
67 | 70 |
|
68 | 71 | # [v3.2.4](https://github.com/coder-hxl/x-crawl/compare/v3.2.3...v3.2.4) (2023-03-09)
|
69 | 72 |
|
70 |
| -### Features |
| 73 | +### 🚀 Features |
71 | 74 |
|
72 | 75 | - Update documents
|
73 | 76 |
|
74 | 77 | # [v3.2.3](https://github.com/coder-hxl/x-crawl/compare/v3.2.2...v3.2.3) (2023-03-08)
|
75 | 78 |
|
76 |
| -### Features |
| 79 | +### 🚀 Features |
77 | 80 |
|
78 | 81 | - Update documents
|
79 | 82 |
|
80 | 83 | # [v3.2.2](https://github.com/coder-hxl/x-crawl/compare/v3.2.1...v3.2.2) (2023-03-07)
|
81 | 84 |
|
82 |
| -### Features |
| 85 | +### 🚀 Features |
83 | 86 |
|
84 | 87 | - Update documents
|
85 | 88 |
|
86 | 89 | # [v3.2.1](https://github.com/coder-hxl/x-crawl/compare/v3.2.0...v3.2.1) (2023-03-07)
|
87 | 90 |
|
88 |
| -### Features |
| 91 | +### 🚀 Features |
89 | 92 |
|
90 | 93 | - Update documents
|
91 | 94 |
|
92 | 95 | # [v3.2.0](https://github.com/coder-hxl/x-crawl/compare/v3.1.1...v3.2.0) (2023-03-06)
|
93 | 96 |
|
94 |
| -### Bug Fixes |
| 97 | +### 🚀 Features |
95 | 98 |
|
96 |
| -- The result page of crawlPage API reported an error. Expose the browser and let the user decide to close the browser |
| 99 | +- crawlPage API return value update |
97 | 100 |
|
98 |
| -### Features |
| 101 | +### 🐞 Bug Fixes |
99 | 102 |
|
100 |
| -- crawlPage API return value update |
| 103 | +- The result page of crawlPage API reported an error. Expose the browser and let the user decide to close the browser |
101 | 104 |
|
102 | 105 | # [v3.1.1](https://github.com/coder-hxl/x-crawl/compare/v3.1.0...v3.1.1) (2023-02-05)
|
103 | 106 |
|
104 |
| -### Bug Fixes |
| 107 | +### 🐞 Bug Fixes |
105 | 108 |
|
106 | 109 | - publish type is missing
|
107 | 110 |
|
108 | 111 | # [v3.1.0](https://github.com/coder-hxl/x-crawl/compare/v3.0.0...v3.1.0) (2023-03-05)
|
109 | 112 |
|
110 |
| -### Features |
| 113 | +### 🚀 Features |
111 | 114 |
|
112 | 115 | - The callback function adds a stopPolling parameter, which can be called to stop subsequent polling operations. The stop polling of the startPolling API is determined by the user
|
113 | 116 |
|
114 | 117 | - Update documents
|
115 | 118 |
|
116 | 119 | # [v3.0.0](https://github.com/coder-hxl/x-crawl/compare/v2.4.2...v3.0.0) (2023-03-04)
|
117 | 120 |
|
118 |
| -### Features |
| 121 | +### 🚨 Breaking Changes |
119 | 122 |
|
120 | 123 | - The three APIs fetchPage, fetchData, and fetchFile are renamed to crawlPage, crawlData, and crawlFile respectively
|
| 124 | + |
| 125 | +### 🚀 Features |
| 126 | + |
121 | 127 | - Multiple ways of writing requestConfig options
|
| 128 | + |
122 | 129 | - All types are exposed
|
123 | 130 |
|
124 | 131 | # [v2.4.2](https://github.com/coder-hxl/x-crawl/compare/v2.4.1...v2.4.2) (2023-03-04)
|
125 | 132 |
|
126 |
| -### Features |
| 133 | +### 🚀 Features |
127 | 134 |
|
128 | 135 | - Update documents
|
129 | 136 |
|
130 | 137 | # [v2.4.1](https://github.com/coder-hxl/x-crawl/compare/v2.4.0...v2.4.1) (2023-03-04)
|
131 | 138 |
|
132 |
| -### Bug Fixes |
| 139 | +### 🐞 Bug Fixes |
133 | 140 |
|
134 | 141 | - Interval time doesn't work
|
135 | 142 |
|
136 | 143 | # [v2.4.0](https://github.com/coder-hxl/x-crawl/compare/v2.3.0...v2.4.0) (2023-03-03)
|
137 | 144 |
|
138 |
| -### Features |
| 145 | +### 🚀 Features |
139 | 146 |
|
140 | 147 | - Update documents
|
141 | 148 |
|
142 | 149 | # [v2.3.0](https://github.com/coder-hxl/x-crawl/compare/v2.2.1...v2.3.0) (2023-03-02)
|
143 | 150 |
|
144 |
| -### Features |
| 151 | +### 🚨 Breaking Changes |
145 | 152 |
|
146 | 153 | - fetchHTML API rename fetchPage
|
147 | 154 |
|
148 | 155 | # [v2.2.1](https://github.com/coder-hxl/x-crawl/compare/v2.2.0...v2.2.1) (2023-03-01)
|
149 | 156 |
|
150 |
| -### Features |
| 157 | +### 🚀 Features |
151 | 158 |
|
152 | 159 | - Update documents
|
153 | 160 |
|
154 | 161 | # [v2.2.0](https://github.com/coder-hxl/x-crawl/compare/v2.1.0...v2.2.0) (2023-02-28)
|
155 | 162 |
|
156 |
| -### Features |
| 163 | +### 🚀 Features |
157 | 164 |
|
158 | 165 | - Parameter config name correction
|
159 | 166 | - Update documents
|
160 | 167 |
|
161 | 168 | # [v2.1.0](https://github.com/coder-hxl/x-crawl/compare/v2.0.0...v2.1.0) (2023-02-28)
|
162 | 169 |
|
163 |
| -### Features |
| 170 | +### 🚀 Features |
164 | 171 |
|
165 | 172 | - The fetchHTML API results remove content options
|
166 | 173 | - The fetchHTML API cancels the incoming header attribute
|
167 | 174 | - The running mode of crawling is highlighted
|
168 | 175 |
|
169 | 176 | # [v2.0.0](https://github.com/coder-hxl/x-crawl/compare/v1.1.1...v2.0.0) (2023-02-27)
|
170 | 177 |
|
171 |
| -### Features |
| 178 | +### 🚨 Breaking Changes |
172 | 179 |
|
173 |
| -- Significantly changed |
174 | 180 | - Create a crawler method, created by calling
|
175 | 181 | - The fetchHTML API uses puppeteer to crawl HTML
|
176 | 182 | - The fetchPolling API renamed to startPolling, removed year and month
|
177 | 183 |
|
178 | 184 | # [v1.1.1](https://github.com/coder-hxl/x-crawl/compare/v1.1.0...v1.1.1) (2023-02-22)
|
179 | 185 |
|
180 |
| -### Features |
| 186 | +### 🚀 Features |
181 | 187 |
|
182 | 188 | - Add jump to a detailed type
|
183 | 189 |
|
184 | 190 | # [v1.1.0](https://github.com/coder-hxl/x-crawl/compare/v1.0.1...v1.1.0) (2023-02-21)
|
185 | 191 |
|
186 |
| -### Features |
| 192 | +### 🚀 Features |
187 | 193 |
|
188 | 194 | - The fetchFile API uses async for save file operations and sorts errors
|
189 | 195 | - The fetchFile API fileConfig can pass in non-existing path
|
190 | 196 |
|
191 | 197 | # [v1.0.1](https://github.com/coder-hxl/x-crawl/compare/v1.0.0...v1.0.1) (2023-02-20)
|
192 | 198 |
|
193 |
| -### Features |
| 199 | +### 🚀 Features |
194 | 200 |
|
195 | 201 | - Sorting of error messages and fetchData/fetchFile API results
|
196 | 202 |
|
197 | 203 | # [v1.0.0](https://github.com/coder-hxl/x-crawl/compare/v0.4.0...v1.0.0) (2023-02-17)
|
198 | 204 |
|
199 |
| -### Features |
| 205 | +### 🚨 Breaking Changes |
200 | 206 |
|
201 | 207 | - Added the method of getting the request result through the callback function, The result of each request can be obtained through the callback function, and the total result can be obtained through PromIse
|
202 | 208 | - API internal refactoring
|
203 | 209 |
|
204 | 210 | # [v0.4.0](https://github.com/coder-hxl/x-crawl/compare/v0.3.1...v0.4.0) (2023-02-16)
|
205 | 211 |
|
206 |
| -### Features |
| 212 | +### 🚀 Features |
207 | 213 |
|
208 | 214 | - The fetchFile API config can provide the extension of the downloaded file
|
209 | 215 |
|
210 | 216 | # [v0.3.1](https://github.com/coder-hxl/x-crawl/compare/v0.3.0...v0.3.1) (2023-02-10)
|
211 | 217 |
|
212 |
| -### Features |
| 218 | +### 🚀 Features |
213 | 219 |
|
214 | 220 | - Update documents
|
215 | 221 |
|
216 | 222 | # [v0.3.0](https://github.com/coder-hxl/x-crawl/compare/v0.2.0...v0.3.0) (2023-02-10)
|
217 | 223 |
|
218 |
| -### Features |
| 224 | +### 🚨 Breaking Changes |
219 | 225 |
|
220 |
| -- Add proxy option |
221 | 226 | - Rename the data object raw of fetchHTML return value to html
|
222 | 227 |
|
| 228 | +### 🚀 Features |
| 229 | + |
| 230 | +- Add proxy option |
| 231 | + |
223 | 232 | # [v0.2.0](https://github.com/coder-hxl/x-crawl/compare/v0.1.5...v0.2.0) (2023-02-09)
|
224 | 233 |
|
225 |
| -### Features |
| 234 | +### 🚀 Features |
226 | 235 |
|
227 | 236 | - Add polling function
|
228 | 237 |
|
229 | 238 | # [v0.1.5](https://github.com/coder-hxl/x-crawl/compare/v0.1.4...v0.1.5) (2023-02-05)
|
230 | 239 |
|
231 |
| -### Bug Fixes |
| 240 | +### 🐞 Bug Fixes |
232 | 241 |
|
233 | 242 | - Fix fetchFile API file save is lost
|
234 | 243 |
|
235 | 244 | # [v0.1.4](https://github.com/coder-hxl/x-crawl/compare/v0.1.3...v0.1.4) (2023-02-05)
|
236 | 245 |
|
237 |
| -### Features |
| 246 | +### 🚀 Features |
238 | 247 |
|
239 | 248 | - Add chalk library
|
240 | 249 |
|
241 | 250 | # [v0.1.3](https://github.com/coder-hxl/x-crawl/compare/v0.1.2...v0.1.3) (2023-02-05)
|
242 | 251 |
|
243 |
| -### Features |
| 252 | +### 🚀 Features |
244 | 253 |
|
245 | 254 | - fetchHTML API exposes more content
|
246 | 255 |
|
247 | 256 | # [v0.1.2](https://github.com/coder-hxl/x-crawl/compare/v0.1.1...v0.1.2) (2023-02-02)
|
248 | 257 |
|
249 |
| -### Features |
| 258 | +### 🚀 Features |
250 | 259 |
|
251 | 260 | - Add requst mode option: async/sync
|
252 | 261 | - Document adjustment
|
253 | 262 |
|
254 | 263 | # [v0.1.1](https://github.com/coder-hxl/x-crawl/compare/v0.1.0...v0.1.1) (2023-01-31)
|
255 | 264 |
|
256 |
| -### Features |
| 265 | +### 🚀 Features |
257 | 266 |
|
258 | 267 | - fetchHTML API parameter can be Object type
|
259 | 268 |
|
260 | 269 | # [v0.1.0](https://github.com/coder-hxl/x-crawl/compare/v0.0.3...v0.1.0) (2023-01-30)
|
261 | 270 |
|
262 |
| -### Features |
| 271 | +### 🚀 Features |
263 | 272 |
|
264 | 273 | - fetch API renamed to fetchData API
|
265 | 274 | - fetchData and fetchFile request handling
|
266 | 275 |
|
267 | 276 | # [v0.0.3](https://github.com/coder-hxl/x-crawl/compare/v0.0.2...v0.0.3) (2023-01-29)
|
268 | 277 |
|
269 |
| -### Features |
| 278 | +### 🚀 Features |
270 | 279 |
|
271 | 280 | - Request Protocol
|
272 | 281 | - Use jest test
|
273 | 282 |
|
274 | 283 | # [v0.0.2](https://github.com/coder-hxl/x-crawl/compare/v0.0.1...v0.0.2) (2023-01-28)
|
275 | 284 |
|
276 |
| -### Features |
| 285 | +### 🐞 Bug Fixes |
277 | 286 |
|
278 | 287 | - Add jsdom type
|
279 | 288 |
|
280 | 289 | - Combined configuration when the basic configuration is not undefined will be processed
|
281 | 290 |
|
282 | 291 | # v0.0.1 (2023-01-28)
|
283 | 292 |
|
284 |
| -- Feat:Crawl is a Nodejs multifunctional crawler library. Provide configuration to batch fetch HTML, JSON, images, etc. |
| 293 | +### 🚀 Features |
| 294 | + |
| 295 | +- Feat:First release of x-crawl |
0 commit comments