diff --git a/README.md b/README.md index d2b9c62c90..83757b22f3 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,7 @@ ## Quick Start This short tutorial will set you up to start using Apify SDK in a minute or two. -If you want to learn more, proceed to the [Apify Platform](https://docs.apify.com/sdk/js/docs/guides/apify-platform) +If you want to learn more, proceed to the [Apify Platform](https://docs.apify.com/sdk/js/docs/concepts/actor-lifecycle) guide that will take you step by step through running your Actor on Apify's platform. Apify SDK requires [Node.js](https://nodejs.org/en/) 16 or later. Add Apify SDK to any Node.js project by running: @@ -33,7 +33,7 @@ await Actor.setValue('OUTPUT', { await Actor.exit(); ``` -> You can also install the [`crawlee`](https://npmjs.org/crawlee) module, as it now provides the crawlers that were previously exported by Apify SDK. If you don't plan to use crawlers in your Actors, then you don't need to install it. Keep in mind that neither `playwright` nor `puppeteer` are bundled with `crawlee` in order to reduce install size and allow greater flexibility. That's why we manually install it with NPM. You can choose one, both, or neither. For more information and example please check [`documentation.`](https://docs.apify.com/sdk/js/docs/guides/apify-platform#running-crawlee-code-as-an-actor) +> You can also install the [`crawlee`](https://npmjs.org/crawlee) module, as it now provides the crawlers that were previously exported by Apify SDK. If you don't plan to use crawlers in your Actors, then you don't need to install it. Keep in mind that neither `playwright` nor `puppeteer` are bundled with `crawlee` in order to reduce install size and allow greater flexibility. That's why we manually install it with NPM. You can choose one, both, or neither. For more information and example please check [`documentation.`](https://docs.apify.com/sdk/js/docs/concepts/actor-lifecycle#running-crawlee-code-as-an-actor) ## Support @@ -42,7 +42,7 @@ For questions, you can ask on [Stack Overflow](https://stackoverflow.com/questio ## Upgrading -Visit the [Upgrading Guide](https://docs.apify.com/sdk/js/docs/upgrading) to find out what changes you might want to make, and, if you encounter any issues, join our [Discord server](https://discord.gg/jyEM2PRvMU) for help! +Visit the [Upgrading Guide](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3) to find out what changes you might want to make, and, if you encounter any issues, join our [Discord server](https://discord.gg/jyEM2PRvMU) for help! ## Contributing diff --git a/docs/02_concepts/02_request_storage.mdx b/docs/02_concepts/02_request_storage.mdx index ea4f5a08a9..51bd1b90e2 100644 --- a/docs/02_concepts/02_request_storage.mdx +++ b/docs/02_concepts/02_request_storage.mdx @@ -53,7 +53,7 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/docs/03_guides/call_actor.mdx b/docs/03_guides/call_actor.mdx index 45530e8bee..d718a6d5e3 100644 --- a/docs/03_guides/call_actor.mdx +++ b/docs/03_guides/call_actor.mdx @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify platform, you can do things like set up a scheduler to run your actor early in the morning. diff --git a/docs/03_guides/puppeteer_with_proxy.mdx b/docs/03_guides/puppeteer_with_proxy.mdx index 0174ad7fbe..7fa10c4df0 100644 --- a/docs/03_guides/puppeteer_with_proxy.mdx +++ b/docs/03_guides/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!roa-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/src/actor.ts b/src/actor.ts index c0ebd1b074..1c83f94c28 100644 --- a/src/actor.ts +++ b/src/actor.ts @@ -201,7 +201,7 @@ export interface ApifyEnv { * Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue * store their data. Typically, it is set to ./storage. If omitted, you should define the * APIFY_TOKEN environment variable instead. See more info on combination of this and - * APIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/guides/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR) + * APIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/concepts/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR) */ localStorageDir: string | null; diff --git a/website/docusaurus.config.js b/website/docusaurus.config.js index a61a62d97d..9d1bdcd7e0 100644 --- a/website/docusaurus.config.js +++ b/website/docusaurus.config.js @@ -47,17 +47,10 @@ module.exports = { items: [ { type: 'doc', - docId: 'overview', + docId: 'introduction/introduction', label: 'Docs', position: 'left', - activeBaseRegex: 'guides|overview', - }, - { - type: 'doc', - docId: '/examples', - label: 'Examples', - position: 'left', - activeBaseRegex: 'examples', + activeBaseRegex: 'guides|overview|introduction', }, { to: 'reference', diff --git a/website/versioned_docs/version-1.3/examples/call_actor.md b/website/versioned_docs/version-1.3/examples/call_actor.md index 94c3cc9df5..de9a6b2890 100644 --- a/website/versioned_docs/version-1.3/examples/call_actor.md +++ b/website/versioned_docs/version-1.3/examples/call_actor.md @@ -10,7 +10,7 @@ sends it to your email using the [`apify/send-mail`](https://apify.com/apify/sen To make the example work, you'll need an [Apify account](https://my.apify.com/). Go to the [Account - Integrations](https://my.apify.com/account#/integrations) page to obtain your API token and set it to the -[`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, or run the script using the Apify CLI. If you deploy this actor +[`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set up a scheduler to run your actor early in the morning. To see what other actors are available, visit the [Apify Store](https://apify.com/store). diff --git a/website/versioned_docs/version-1.3/examples/puppeteer_with_proxy.md b/website/versioned_docs/version-1.3/examples/puppeteer_with_proxy.md index 8ab9912f06..baf500e9c9 100644 --- a/website/versioned_docs/version-1.3/examples/puppeteer_with_proxy.md +++ b/website/versioned_docs/version-1.3/examples/puppeteer_with_proxy.md @@ -5,7 +5,7 @@ id: puppeteer-with-proxy --- This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). To make it work, you'll -need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your +need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. > To run this example on the Apify Platform, select the `apify/actor-node-puppeteer-chrome` image for your Dockerfile. diff --git a/website/versioned_docs/version-1.3/guides/request_storage.md b/website/versioned_docs/version-1.3/guides/request_storage.md index 0c63c1d27e..655108f137 100644 --- a/website/versioned_docs/version-1.3/guides/request_storage.md +++ b/website/versioned_docs/version-1.3/guides/request_storage.md @@ -5,7 +5,7 @@ id: request-storage --- The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./apify_storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./apify_storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -51,7 +51,7 @@ const crawler = new Apify.CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -84,7 +84,7 @@ const crawler = new Apify.PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-2.3/examples/call_actor.md b/website/versioned_docs/version-2.3/examples/call_actor.md index 52a616d7b6..dd4651ad4d 100644 --- a/website/versioned_docs/version-2.3/examples/call_actor.md +++ b/website/versioned_docs/version-2.3/examples/call_actor.md @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://my.apify.com/). Go to the [Account - Integrations](https://my.apify.com/account#/integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set up a scheduler to run your actor early in the morning. diff --git a/website/versioned_docs/version-2.3/examples/puppeteer_with_proxy.md b/website/versioned_docs/version-2.3/examples/puppeteer_with_proxy.md index 1c1d255224..4dbd219192 100644 --- a/website/versioned_docs/version-2.3/examples/puppeteer_with_proxy.md +++ b/website/versioned_docs/version-2.3/examples/puppeteer_with_proxy.md @@ -6,7 +6,7 @@ title: Puppeteer with proxy This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). To make it work, you'll need an Apify account with access to the proxy. -Visit the [Apify platform introduction](/docs/guides/apify-platform) to find +Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. > To run this example on the Apify Platform, select the `apify/actor-node-puppeteer-chrome` image for your Dockerfile. diff --git a/website/versioned_docs/version-2.3/guides/request_storage.md b/website/versioned_docs/version-2.3/guides/request_storage.md index 2a2a9ba86c..dbc30a6695 100644 --- a/website/versioned_docs/version-2.3/guides/request_storage.md +++ b/website/versioned_docs/version-2.3/guides/request_storage.md @@ -4,7 +4,7 @@ title: Request Storage --- The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./apify_storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./apify_storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -50,7 +50,7 @@ const crawler = new Apify.CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -83,7 +83,7 @@ const crawler = new Apify.PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.0/examples/call_actor.mdx b/website/versioned_docs/version-3.0/examples/call_actor.mdx index aa83ada658..4e295adaca 100644 --- a/website/versioned_docs/version-3.0/examples/call_actor.mdx +++ b/website/versioned_docs/version-3.0/examples/call_actor.mdx @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set up a scheduler to run your actor early in the morning. diff --git a/website/versioned_docs/version-3.0/examples/puppeteer_with_proxy.mdx b/website/versioned_docs/version-3.0/examples/puppeteer_with_proxy.mdx index 73126e30af..4592f63959 100644 --- a/website/versioned_docs/version-3.0/examples/puppeteer_with_proxy.mdx +++ b/website/versioned_docs/version-3.0/examples/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/website/versioned_docs/version-3.0/guides/request_storage.mdx b/website/versioned_docs/version-3.0/guides/request_storage.mdx index b7b1301db4..f1d1e9ca33 100644 --- a/website/versioned_docs/version-3.0/guides/request_storage.mdx +++ b/website/versioned_docs/version-3.0/guides/request_storage.mdx @@ -7,7 +7,7 @@ import ApiLink from '@site/src/components/ApiLink'; import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -53,7 +53,7 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.1/examples/call_actor.mdx b/website/versioned_docs/version-3.1/examples/call_actor.mdx index ea5488ed99..f452690f70 100644 --- a/website/versioned_docs/version-3.1/examples/call_actor.mdx +++ b/website/versioned_docs/version-3.1/examples/call_actor.mdx @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this Actor to the Apify Cloud, you can do things like set up a scheduler to run your Actor early in the morning. diff --git a/website/versioned_docs/version-3.1/examples/puppeteer_with_proxy.mdx b/website/versioned_docs/version-3.1/examples/puppeteer_with_proxy.mdx index 0174ad7fbe..7fa10c4df0 100644 --- a/website/versioned_docs/version-3.1/examples/puppeteer_with_proxy.mdx +++ b/website/versioned_docs/version-3.1/examples/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!roa-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/website/versioned_docs/version-3.1/guides/request_storage.mdx b/website/versioned_docs/version-3.1/guides/request_storage.mdx index 494148998f..377554a403 100644 --- a/website/versioned_docs/version-3.1/guides/request_storage.mdx +++ b/website/versioned_docs/version-3.1/guides/request_storage.mdx @@ -7,7 +7,7 @@ import ApiLink from '@site/src/components/ApiLink'; import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -53,7 +53,7 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.2/api-typedoc.json b/website/versioned_docs/version-3.2/api-typedoc.json index 2241ae0392..636ae98d49 100644 --- a/website/versioned_docs/version-3.2/api-typedoc.json +++ b/website/versioned_docs/version-3.2/api-typedoc.json @@ -31753,7 +31753,7 @@ "summary": [ { "kind": "text", - "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/guides/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" + "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/concepts/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" } ] }, diff --git a/website/versioned_docs/version-3.2/examples/call_actor.mdx b/website/versioned_docs/version-3.2/examples/call_actor.mdx index aa83ada658..4e295adaca 100644 --- a/website/versioned_docs/version-3.2/examples/call_actor.mdx +++ b/website/versioned_docs/version-3.2/examples/call_actor.mdx @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set up a scheduler to run your actor early in the morning. diff --git a/website/versioned_docs/version-3.2/examples/puppeteer_with_proxy.mdx b/website/versioned_docs/version-3.2/examples/puppeteer_with_proxy.mdx index 0174ad7fbe..7fa10c4df0 100644 --- a/website/versioned_docs/version-3.2/examples/puppeteer_with_proxy.mdx +++ b/website/versioned_docs/version-3.2/examples/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!roa-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/website/versioned_docs/version-3.2/guides/request_storage.mdx b/website/versioned_docs/version-3.2/guides/request_storage.mdx index b7b1301db4..f1d1e9ca33 100644 --- a/website/versioned_docs/version-3.2/guides/request_storage.mdx +++ b/website/versioned_docs/version-3.2/guides/request_storage.mdx @@ -7,7 +7,7 @@ import ApiLink from '@site/src/components/ApiLink'; import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -53,7 +53,7 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.3/api-typedoc.json b/website/versioned_docs/version-3.3/api-typedoc.json index d4508b6eac..6fc6da0470 100644 --- a/website/versioned_docs/version-3.3/api-typedoc.json +++ b/website/versioned_docs/version-3.3/api-typedoc.json @@ -33846,7 +33846,7 @@ "summary": [ { "kind": "text", - "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/guides/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" + "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/concepts/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" } ] }, diff --git a/website/versioned_docs/version-3.3/examples/call_actor.mdx b/website/versioned_docs/version-3.3/examples/call_actor.mdx index aa83ada658..4e295adaca 100644 --- a/website/versioned_docs/version-3.3/examples/call_actor.mdx +++ b/website/versioned_docs/version-3.3/examples/call_actor.mdx @@ -11,7 +11,7 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set up a scheduler to run your actor early in the morning. diff --git a/website/versioned_docs/version-3.3/examples/puppeteer_with_proxy.mdx b/website/versioned_docs/version-3.3/examples/puppeteer_with_proxy.mdx index 0174ad7fbe..7fa10c4df0 100644 --- a/website/versioned_docs/version-3.3/examples/puppeteer_with_proxy.mdx +++ b/website/versioned_docs/version-3.3/examples/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!roa-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/website/versioned_docs/version-3.3/guides/request_storage.mdx b/website/versioned_docs/version-3.3/guides/request_storage.mdx index b7b1301db4..f1d1e9ca33 100644 --- a/website/versioned_docs/version-3.3/guides/request_storage.mdx +++ b/website/versioned_docs/version-3.3/guides/request_storage.mdx @@ -7,7 +7,7 @@ import ApiLink from '@site/src/components/ApiLink'; import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/concepts/actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -53,7 +53,7 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.4/overview.md b/website/versioned_docs/version-3.4/01_introduction/index.md similarity index 83% rename from website/versioned_docs/version-3.4/overview.md rename to website/versioned_docs/version-3.4/01_introduction/index.md index 6804fd2a35..f49ab9c1e2 100644 --- a/website/versioned_docs/version-3.4/overview.md +++ b/website/versioned_docs/version-3.4/01_introduction/index.md @@ -1,7 +1,9 @@ --- -id: overview +id: introduction title: Apify SDK for JavaScript sidebar_label: Overview +slug: /overview +description: 'The official library for creating Apify Actors in JavaScript, providing tools for web crawling, scraping, and automation at scale.' --- The Apify SDK for JavaScript is the official library for creating [Apify Actors](https://docs.apify.com/platform/actors) using JavaScript or TypeScript. It provides useful features like Actor lifecycle management, local storage emulation, and Actor event handling. @@ -25,9 +27,9 @@ Actors can be executed locally or on the [Apify platform](https://docs.apify.com ## Quick start -To create and run Actors using Apify Console, check out [Apify Console documentation](https://docs.apify.com/platform/console). For creating and running JavaScript Actors locally, refer to the [Apify platform guide](./guides/apify-platform). +To create and run Actors using Apify Console, check out [Apify Console documentation](https://docs.apify.com/platform/console). For creating and running JavaScript Actors locally, refer to the [Actor lifecycle guide](concepts/actor-lifecycle). -Explore the [Examples](./examples) section to see the SDK in action, and refer to the [Guides](./guides) section for a deeper understanding of the SDK's features and best practices. +Explore the Guides section in the sidebar for a deeper understanding of the SDK's features and best practices. ## Installation diff --git a/website/versioned_docs/version-3.4/01_introduction/quick-start.md b/website/versioned_docs/version-3.4/01_introduction/quick-start.md new file mode 100644 index 0000000000..3b2efb7341 --- /dev/null +++ b/website/versioned_docs/version-3.4/01_introduction/quick-start.md @@ -0,0 +1,91 @@ +--- +id: quick-start +title: Quick start +sidebar_label: Quick start +description: 'Get started with the Apify SDK for JavaScript by creating your first Actor and learning the basics.' +--- + +Learn how to create and run Actors using the Apify SDK for JavaScript. + +--- + +## Step 1: Create Actors + +To create and run Actors in Apify Console, refer to the [Console documentation](https://docs.apify.com/platform/actors/development/quick-start/web-ide). + +To create an Actor on your computer, use the [Apify CLI](/cli): + +```bash +apify create my-first-actor +``` + +The CLI will prompt you to select a template. After you choose a template, the CLI creates a new folder called `my-first-actor`, downloads and extracts the selected template, and installs dependencies using npm. + +## Step 2: Run Actors + +To run the Actor, you can use the [`apify run` command](/cli/docs/reference#apify-run): + +```bash +cd my-first-actor +apify run +``` + +This command: + +- Starts the Actor with the appropriate environment variables for local running +- Configures it to use local storages from the `storage` folder + +The Actor input, for example, will be in `storage/key_value_stores/default/INPUT.json`. + +## Step 3: Understand Actor structure + +All JavaScript Actor templates follow the same structure. + +The `.actor` directory contains the [Actor configuration](https://docs.apify.com/platform/actors/development/actor-config), such as the Actor's definition and input schema, and the Dockerfile necessary to run the Actor on the Apify platform. + +The Actor's runtime dependencies are specified in the `package.json` file. + +The Actor's source code is typically in the `src` folder (or root for simple Actors). The main entry point is usually `src/main.js` or `src/main.ts`: + +```js +import { Actor, log } from 'apify'; + +// Initialize the Actor +await Actor.init(); + +// Get input from the Actor +const input = await Actor.getInput(); +log.info('Actor input:', input); + +// Your Actor logic goes here +await Actor.pushData({ message: 'Hello, world!' }); + +// Gracefully exit the Actor +await Actor.exit(); +``` + +### Key methods + +- **`Actor.init()`** - Initializes the Actor runtime, sets up storage, and prepares the environment +- **`Actor.getInput()`** - Retrieves input data passed to the Actor +- **`Actor.pushData()`** - Stores data in the default dataset +- **`Actor.exit()`** - Gracefully shuts down the Actor and saves its state + +## Next steps + +### Guides + +To see how you can integrate the Apify SDK with some of the most popular web scraping libraries, check out our guides for working with: + +- [CheerioCrawler](../guides/cheerio-crawler) +- [PuppeteerCrawler](../guides/puppeteer-crawler) +- [PlaywrightCrawler](../guides/playwright-crawler) + +### Concepts + +To learn more about the features of the Apify SDK and how to use them, check out the Concepts section in the sidebar, especially: + +- [Actor lifecycle](../concepts/actor-lifecycle) +- [Request storage](../concepts/request-storage) +- [Result storage](../concepts/result-storage) +- [Proxy management](../concepts/proxy-management) diff --git a/website/versioned_docs/version-3.4/guides/apify_platform.mdx b/website/versioned_docs/version-3.4/02_concepts/01_actor_lifecycle.mdx similarity index 98% rename from website/versioned_docs/version-3.4/guides/apify_platform.mdx rename to website/versioned_docs/version-3.4/02_concepts/01_actor_lifecycle.mdx index 0ce1386ae9..b39083e3fb 100644 --- a/website/versioned_docs/version-3.4/guides/apify_platform.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/01_actor_lifecycle.mdx @@ -1,7 +1,7 @@ --- -id: apify-platform -title: Apify Platform -description: Apify platform - large-scale and high-performance web scraping +id: actor-lifecycle +title: Actor lifecycle +description: Understanding Actor lifecycle, initialization, and the Apify platform --- import ApiLink from '@site/src/components/ApiLink'; @@ -150,7 +150,7 @@ Your script will be uploaded to and built on the Apify platform so that it can b ## Usage on Apify platform -You can also develop your actor in an online code editor directly on the platform (you'll need an Apify Account). Let's go to the [Actors](https://console.apify.com/actors) page in the app, click _Create new_ and then go to the _Source_ tab and start writing the code or paste one of the examples from the [Examples](../examples) section. +You can also develop your actor in an online code editor directly on the platform (you'll need an Apify Account). Let's go to the [Actors](https://console.apify.com/actors) page in the app, click _Create new_ and then go to the _Source_ tab and start writing the code or paste one of the examples from the Guides section in the sidebar. ## Storages diff --git a/website/versioned_docs/version-3.4/guides/request_storage.mdx b/website/versioned_docs/version-3.4/02_concepts/02_request_storage.mdx similarity index 89% rename from website/versioned_docs/version-3.4/guides/request_storage.mdx rename to website/versioned_docs/version-3.4/02_concepts/02_request_storage.mdx index b7b1301db4..51bd1b90e2 100644 --- a/website/versioned_docs/version-3.4/guides/request_storage.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/02_request_storage.mdx @@ -7,7 +7,7 @@ import ApiLink from '@site/src/components/ApiLink'; import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The Apify SDK has several request storage types that are useful for specific tasks. The requests are stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](/docs/guides/apify-platform) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](./actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. Typically, you will be developing the code on your local computer and thus set the `APIFY_LOCAL_STORAGE_DIR` environment variable. Once the code is ready, you will deploy it to the Apify platform, where it will automatically set the `APIFY_TOKEN` environment variable and thus use cloud storage. No code changes are needed. @@ -53,11 +53,11 @@ const crawler = new CheerioCrawler({ }); ``` -To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/examples/puppeteer-crawler) example. +To see more detailed example of how to use the request queue with a crawler, see the [Puppeteer Crawler](/docs/guides/puppeteer-crawler) example. ## Request list -The request list is not a storage per se - it represents the list of URLs to crawl that is stored in a run memory (or optionally in default [Key-Value Store](../guides/result-storage#key-value-store) associated with the run, if specified). The list is used for the crawling of a large number of URLs, when you know all the URLs which should be visited by the crawler and no URLs would be added during the run. The URLs can be provided either in code or parsed from a text file hosted on the web. +The request list is not a storage per se - it represents the list of URLs to crawl that is stored in a run memory (or optionally in default [Key-value Store](./result-storage#key-value-store) associated with the run, if specified). The list is used for the crawling of a large number of URLs, when you know all the URLs which should be visited by the crawler and no URLs would be added during the run. The URLs can be provided either in code or parsed from a text file hosted on the web. Request list is created exclusively for the actor run and only if its usage is explicitly specified in the code. Its usage is optional. @@ -86,7 +86,7 @@ const crawler = new PuppeteerCrawler({ }); ``` -To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/examples/puppeteer-with-proxy) example. +To see more detailed example of how to use the request list with a crawler, see the [Puppeteer with proxy](/docs/guides/puppeteer-with-proxy) example. ## Which one to choose? diff --git a/website/versioned_docs/version-3.4/guides/result_storage.mdx b/website/versioned_docs/version-3.4/02_concepts/03_result_storage.mdx similarity index 94% rename from website/versioned_docs/version-3.4/guides/result_storage.mdx rename to website/versioned_docs/version-3.4/02_concepts/03_result_storage.mdx index f587d07282..5a1ef2293c 100644 --- a/website/versioned_docs/version-3.4/guides/result_storage.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/03_result_storage.mdx @@ -6,7 +6,7 @@ title: Result Storage import ApiLink from '@site/src/components/ApiLink'; The Apify SDK has several result storage types that are useful for specific tasks. The data is stored either on local disk to a directory defined by the -`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](../guides/apify-platform) under the user account +`APIFY_LOCAL_STORAGE_DIR` environment variable, or on the [Apify platform](./actor-lifecycle) under the user account identified by the API token defined by the `APIFY_TOKEN` environment variable. If neither of these variables is defined, by default Apify SDK sets `APIFY_LOCAL_STORAGE_DIR` to `./storage` in the current working directory and prints a warning. @@ -67,7 +67,7 @@ const value = await store.getValue('some-key'); await store.setValue('some-key', null); ``` -To see a real-world example of how to get the input from the key-value store, see the [Screenshots](../examples/capture-screenshot) example. +To see a real-world example of how to get the input from the key-value store, see the [Screenshots](../guides/capture-screenshot) example. ## Dataset @@ -75,7 +75,7 @@ Datasets are used to store structured data where each object stored has the same can imagine a dataset as a table, where each object is a row and its attributes are columns. Dataset is an append-only storage - you can only add new records to it but you cannot modify or remove existing records. -When the dataset is stored on the [Apify platform](../guides/apify-platform), you can export its data to the following formats: HTML, +When the dataset is stored on the [Apify platform](./actor-lifecycle), you can export its data to the following formats: HTML, JSON, CSV, Excel, XML and RSS. The datasets are displayed on the actor run details page and in the [Storage](https://console.apify.com/storage) section in the Apify Console. The actual data is exported using the [Get dataset items](https://apify.com/docs/api/v2#/reference/datasets/item-collection/get-items) Apify API endpoint. This @@ -113,4 +113,4 @@ await dataset.pushData({ foo: 'bar' }); await dataset.pushData([{ foo: 'bar2', col2: 'val2' }, { col3: 123 }]); ``` -To see how to use the dataset to store crawler results, see the [Cheerio Crawler](../examples/cheerio-crawler) example. +To see how to use the dataset to store crawler results, see the [Cheerio Crawler](../guides/cheerio-crawler) example. diff --git a/website/versioned_docs/version-3.4/guides/proxy_management.mdx b/website/versioned_docs/version-3.4/02_concepts/04_proxy_management.mdx similarity index 98% rename from website/versioned_docs/version-3.4/guides/proxy_management.mdx rename to website/versioned_docs/version-3.4/02_concepts/04_proxy_management.mdx index a271a063fb..2a4b05456b 100644 --- a/website/versioned_docs/version-3.4/guides/proxy_management.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/04_proxy_management.mdx @@ -21,7 +21,7 @@ for your scraping needs. If you already subscribed to Apify Proxy or have proxy URLs of your own, you can start using them immediately in only a few lines of code. -> If you want to use Apify Proxy, make sure that your [scraper is logged in](../guides/apify-platform). +> If you want to use Apify Proxy, make sure that your [scraper is logged in](./actor-lifecycle). @@ -90,7 +90,7 @@ allows you to pass a `sessionId` parameter. It will then be used to create a `sessionId`-`proxyUrl` pair, and subsequent `newUrl()` calls with the same `sessionId` will always return the same `proxyUrl`. This is extremely useful in scraping, because you want to create the impression of a real user. See the -[session management guide](../guides/session-management) and +[session management guide](./session-management) and `SessionPool` class for more information on how keeping a real session helps you avoid blocking. diff --git a/website/versioned_docs/version-3.4/guides/session_management.mdx b/website/versioned_docs/version-3.4/02_concepts/05_session_management.mdx similarity index 100% rename from website/versioned_docs/version-3.4/guides/session_management.mdx rename to website/versioned_docs/version-3.4/02_concepts/05_session_management.mdx diff --git a/website/versioned_docs/version-3.4/guides/environment_variables.mdx b/website/versioned_docs/version-3.4/02_concepts/06_environment_variables.mdx similarity index 97% rename from website/versioned_docs/version-3.4/guides/environment_variables.mdx rename to website/versioned_docs/version-3.4/02_concepts/06_environment_variables.mdx index 86a6cd9fe3..6496b60d9c 100644 --- a/website/versioned_docs/version-3.4/guides/environment_variables.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/06_environment_variables.mdx @@ -9,7 +9,7 @@ import { CrawleeApiLink } from '@site/src/components/CrawleeLinks'; The following is a list of the environment variables used by Apify SDK that are available to the user. The SDK is capable of running without any env vars present, but certain features will only become available after env vars are properly set. You can use [Apify CLI](https://github.com/apify/apify-cli) -to set the env vars for you. [Apify platform](../guides/apify-platform) also sets the variables automatically. +to set the env vars for you. [Apify platform](./actor-lifecycle) also sets the variables automatically. ## Important env vars diff --git a/website/versioned_docs/version-3.4/guides/docker_images.mdx b/website/versioned_docs/version-3.4/02_concepts/07_docker_images.mdx similarity index 99% rename from website/versioned_docs/version-3.4/guides/docker_images.mdx rename to website/versioned_docs/version-3.4/02_concepts/07_docker_images.mdx index f0cfc479be..e2fe6ab6e5 100644 --- a/website/versioned_docs/version-3.4/guides/docker_images.mdx +++ b/website/versioned_docs/version-3.4/02_concepts/07_docker_images.mdx @@ -16,7 +16,7 @@ import tsNodeDocker from '!!raw-loader!./docker_node_ts.txt'; import jsBrowserDocker from '!!raw-loader!./docker_browser_js.txt'; import tsBrowserDocker from '!!raw-loader!./docker_browser_ts.txt'; -Running headless browsers in Docker requires a lot of setup to do it right. But there's no need to worry about that, because we already created base images that you can freely use. We use them every day on the [Apify Platform](./apify-platform). +Running headless browsers in Docker requires a lot of setup to do it right. But there's no need to worry about that, because we already created base images that you can freely use. We use them every day on the [Apify Platform](./actor-lifecycle). All images can be found in their [GitHub repo](https://github.com/apify/apify-actor-docker) and in our [DockerHub](https://hub.docker.com/orgs/apify). diff --git a/website/versioned_docs/version-3.4/guides/pay_per_event.mdx b/website/versioned_docs/version-3.4/02_concepts/08_pay_per_event.mdx similarity index 100% rename from website/versioned_docs/version-3.4/guides/pay_per_event.mdx rename to website/versioned_docs/version-3.4/02_concepts/08_pay_per_event.mdx diff --git a/website/versioned_docs/version-3.4/guides/apify_platform_init_exit.ts b/website/versioned_docs/version-3.4/02_concepts/apify_platform_init_exit.ts similarity index 100% rename from website/versioned_docs/version-3.4/guides/apify_platform_init_exit.ts rename to website/versioned_docs/version-3.4/02_concepts/apify_platform_init_exit.ts diff --git a/website/versioned_docs/version-3.4/guides/apify_platform_main.ts b/website/versioned_docs/version-3.4/02_concepts/apify_platform_main.ts similarity index 100% rename from website/versioned_docs/version-3.4/guides/apify_platform_main.ts rename to website/versioned_docs/version-3.4/02_concepts/apify_platform_main.ts diff --git a/website/versioned_docs/version-3.4/guides/code/actor_charge.ts b/website/versioned_docs/version-3.4/02_concepts/code/actor_charge.ts similarity index 100% rename from website/versioned_docs/version-3.4/guides/code/actor_charge.ts rename to website/versioned_docs/version-3.4/02_concepts/code/actor_charge.ts diff --git a/website/versioned_docs/version-3.4/guides/code/conditional_actor_charge.ts b/website/versioned_docs/version-3.4/02_concepts/code/conditional_actor_charge.ts similarity index 100% rename from website/versioned_docs/version-3.4/guides/code/conditional_actor_charge.ts rename to website/versioned_docs/version-3.4/02_concepts/code/conditional_actor_charge.ts diff --git a/website/versioned_docs/version-3.4/guides/docker_browser_js.txt b/website/versioned_docs/version-3.4/02_concepts/docker_browser_js.txt similarity index 100% rename from website/versioned_docs/version-3.4/guides/docker_browser_js.txt rename to website/versioned_docs/version-3.4/02_concepts/docker_browser_js.txt diff --git a/website/versioned_docs/version-3.4/guides/docker_browser_ts.txt b/website/versioned_docs/version-3.4/02_concepts/docker_browser_ts.txt similarity index 100% rename from website/versioned_docs/version-3.4/guides/docker_browser_ts.txt rename to website/versioned_docs/version-3.4/02_concepts/docker_browser_ts.txt diff --git a/website/versioned_docs/version-3.4/guides/docker_node_js.txt b/website/versioned_docs/version-3.4/02_concepts/docker_node_js.txt similarity index 100% rename from website/versioned_docs/version-3.4/guides/docker_node_js.txt rename to website/versioned_docs/version-3.4/02_concepts/docker_node_js.txt diff --git a/website/versioned_docs/version-3.4/guides/docker_node_ts.txt b/website/versioned_docs/version-3.4/02_concepts/docker_node_ts.txt similarity index 100% rename from website/versioned_docs/version-3.4/guides/docker_node_ts.txt rename to website/versioned_docs/version-3.4/02_concepts/docker_node_ts.txt diff --git a/website/versioned_docs/version-3.4/examples/accept_user_input.mdx b/website/versioned_docs/version-3.4/03_guides/accept_user_input.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/accept_user_input.mdx rename to website/versioned_docs/version-3.4/03_guides/accept_user_input.mdx diff --git a/website/versioned_docs/version-3.4/examples/accept_user_input.ts b/website/versioned_docs/version-3.4/03_guides/accept_user_input.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/accept_user_input.ts rename to website/versioned_docs/version-3.4/03_guides/accept_user_input.ts diff --git a/website/versioned_docs/version-3.4/examples/add_data_to_dataset.mdx b/website/versioned_docs/version-3.4/03_guides/add_data_to_dataset.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/add_data_to_dataset.mdx rename to website/versioned_docs/version-3.4/03_guides/add_data_to_dataset.mdx diff --git a/website/versioned_docs/version-3.4/examples/add_data_to_dataset.ts b/website/versioned_docs/version-3.4/03_guides/add_data_to_dataset.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/add_data_to_dataset.ts rename to website/versioned_docs/version-3.4/03_guides/add_data_to_dataset.ts diff --git a/website/versioned_docs/version-3.4/examples/basic_crawler.mdx b/website/versioned_docs/version-3.4/03_guides/basic_crawler.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/basic_crawler.mdx rename to website/versioned_docs/version-3.4/03_guides/basic_crawler.mdx diff --git a/website/versioned_docs/version-3.4/examples/basic_crawler.ts b/website/versioned_docs/version-3.4/03_guides/basic_crawler.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/basic_crawler.ts rename to website/versioned_docs/version-3.4/03_guides/basic_crawler.ts diff --git a/website/versioned_docs/version-3.4/examples/call_actor.mdx b/website/versioned_docs/version-3.4/03_guides/call_actor.mdx similarity index 93% rename from website/versioned_docs/version-3.4/examples/call_actor.mdx rename to website/versioned_docs/version-3.4/03_guides/call_actor.mdx index aa83ada658..d718a6d5e3 100644 --- a/website/versioned_docs/version-3.4/examples/call_actor.mdx +++ b/website/versioned_docs/version-3.4/03_guides/call_actor.mdx @@ -11,8 +11,8 @@ and sends it to your email using the [`apify/send-mail`](https://apify.com/apify To make the example work, you'll need an [Apify account](https://console.apify.com/). Go to the [Settings - Integrations](https://console.apify.com/account?tab=integrations) page to obtain your API token -and set it to the [`APIFY_TOKEN`](/docs/guides/environment-variables#APIFY_TOKEN) environment variable, -or run the script using the Apify CLI. If you deploy this actor to the Apify Cloud, you can do things like set +and set it to the [`APIFY_TOKEN`](/docs/concepts/environment-variables#apify_token) environment variable, +or run the script using the Apify CLI. If you deploy this actor to the Apify platform, you can do things like set up a scheduler to run your actor early in the morning. To see what other actors are available, visit the [Apify Store](https://apify.com/store). diff --git a/website/versioned_docs/version-3.4/examples/cheerio_crawler.mdx b/website/versioned_docs/version-3.4/03_guides/cheerio_crawler.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/cheerio_crawler.mdx rename to website/versioned_docs/version-3.4/03_guides/cheerio_crawler.mdx diff --git a/website/versioned_docs/version-3.4/examples/cheerio_crawler.ts b/website/versioned_docs/version-3.4/03_guides/cheerio_crawler.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/cheerio_crawler.ts rename to website/versioned_docs/version-3.4/03_guides/cheerio_crawler.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_all_links.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_all_links.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_all_links.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_all_links.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_all_links_cheerio.ts b/website/versioned_docs/version-3.4/03_guides/crawl_all_links_cheerio.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_all_links_cheerio.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_all_links_cheerio.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_all_links_playwright.ts b/website/versioned_docs/version-3.4/03_guides/crawl_all_links_playwright.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_all_links_playwright.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_all_links_playwright.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_all_links_puppeteer.ts b/website/versioned_docs/version-3.4/03_guides/crawl_all_links_puppeteer.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_all_links_puppeteer.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_all_links_puppeteer.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_multiple_urls.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_multiple_urls.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_multiple_urls_cheerio.ts b/website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_cheerio.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_multiple_urls_cheerio.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_cheerio.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_multiple_urls_playwright.ts b/website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_playwright.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_multiple_urls_playwright.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_playwright.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_multiple_urls_puppeteer.ts b/website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_puppeteer.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_multiple_urls_puppeteer.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_multiple_urls_puppeteer.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_relative_links.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_relative_links.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_relative_links.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_relative_links.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_relative_links_all.ts b/website/versioned_docs/version-3.4/03_guides/crawl_relative_links_all.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_relative_links_all.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_relative_links_all.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_relative_links_same_hostname.ts b/website/versioned_docs/version-3.4/03_guides/crawl_relative_links_same_hostname.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_relative_links_same_hostname.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_relative_links_same_hostname.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_relative_links_same_subdomain.ts b/website/versioned_docs/version-3.4/03_guides/crawl_relative_links_same_subdomain.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_relative_links_same_subdomain.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_relative_links_same_subdomain.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_single_url.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_single_url.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_single_url.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_single_url.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_single_url.ts b/website/versioned_docs/version-3.4/03_guides/crawl_single_url.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_single_url.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_single_url.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_sitemap.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_sitemap.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_sitemap.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_sitemap.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_sitemap_cheerio.ts b/website/versioned_docs/version-3.4/03_guides/crawl_sitemap_cheerio.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_sitemap_cheerio.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_sitemap_cheerio.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_sitemap_playwright.ts b/website/versioned_docs/version-3.4/03_guides/crawl_sitemap_playwright.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_sitemap_playwright.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_sitemap_playwright.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_sitemap_puppeteer.ts b/website/versioned_docs/version-3.4/03_guides/crawl_sitemap_puppeteer.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_sitemap_puppeteer.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_sitemap_puppeteer.ts diff --git a/website/versioned_docs/version-3.4/examples/crawl_some_links.mdx b/website/versioned_docs/version-3.4/03_guides/crawl_some_links.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_some_links.mdx rename to website/versioned_docs/version-3.4/03_guides/crawl_some_links.mdx diff --git a/website/versioned_docs/version-3.4/examples/crawl_some_links.ts b/website/versioned_docs/version-3.4/03_guides/crawl_some_links.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/crawl_some_links.ts rename to website/versioned_docs/version-3.4/03_guides/crawl_some_links.ts diff --git a/website/versioned_docs/version-3.4/examples/forms.mdx b/website/versioned_docs/version-3.4/03_guides/forms.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/forms.mdx rename to website/versioned_docs/version-3.4/03_guides/forms.mdx diff --git a/website/versioned_docs/version-3.4/examples/forms.ts b/website/versioned_docs/version-3.4/03_guides/forms.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/forms.ts rename to website/versioned_docs/version-3.4/03_guides/forms.ts diff --git a/website/versioned_docs/version-3.4/examples/map.ts b/website/versioned_docs/version-3.4/03_guides/map.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/map.ts rename to website/versioned_docs/version-3.4/03_guides/map.ts diff --git a/website/versioned_docs/version-3.4/examples/map_and_reduce.mdx b/website/versioned_docs/version-3.4/03_guides/map_and_reduce.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/map_and_reduce.mdx rename to website/versioned_docs/version-3.4/03_guides/map_and_reduce.mdx diff --git a/website/versioned_docs/version-3.4/examples/playwright_crawler.mdx b/website/versioned_docs/version-3.4/03_guides/playwright_crawler.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/playwright_crawler.mdx rename to website/versioned_docs/version-3.4/03_guides/playwright_crawler.mdx diff --git a/website/versioned_docs/version-3.4/examples/playwright_crawler.ts b/website/versioned_docs/version-3.4/03_guides/playwright_crawler.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/playwright_crawler.ts rename to website/versioned_docs/version-3.4/03_guides/playwright_crawler.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_capture_screenshot.mdx b/website/versioned_docs/version-3.4/03_guides/puppeteer_capture_screenshot.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_capture_screenshot.mdx rename to website/versioned_docs/version-3.4/03_guides/puppeteer_capture_screenshot.mdx diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_crawler.mdx b/website/versioned_docs/version-3.4/03_guides/puppeteer_crawler.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_crawler.mdx rename to website/versioned_docs/version-3.4/03_guides/puppeteer_crawler.mdx diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_crawler.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_crawler.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_crawler.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_crawler.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_crawler_crawler_utils_snapshot.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_crawler_utils_snapshot.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_crawler_crawler_utils_snapshot.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_crawler_utils_snapshot.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_crawler_page_screenshot.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_page_screenshot.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_crawler_page_screenshot.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_page_screenshot.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_crawler_utils_snapshot.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_utils_snapshot.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_crawler_utils_snapshot.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_crawler_utils_snapshot.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_page_screenshot.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_page_screenshot.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_page_screenshot.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_page_screenshot.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_recursive_crawl.mdx b/website/versioned_docs/version-3.4/03_guides/puppeteer_recursive_crawl.mdx similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_recursive_crawl.mdx rename to website/versioned_docs/version-3.4/03_guides/puppeteer_recursive_crawl.mdx diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_recursive_crawl.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_recursive_crawl.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_recursive_crawl.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_recursive_crawl.ts diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_with_proxy.mdx b/website/versioned_docs/version-3.4/03_guides/puppeteer_with_proxy.mdx similarity index 82% rename from website/versioned_docs/version-3.4/examples/puppeteer_with_proxy.mdx rename to website/versioned_docs/version-3.4/03_guides/puppeteer_with_proxy.mdx index 0174ad7fbe..7fa10c4df0 100644 --- a/website/versioned_docs/version-3.4/examples/puppeteer_with_proxy.mdx +++ b/website/versioned_docs/version-3.4/03_guides/puppeteer_with_proxy.mdx @@ -9,7 +9,7 @@ import CrawlSource from '!!raw-loader!roa-loader!./puppeteer_with_proxy.ts'; This example demonstrates how to load pages in headless Chrome / Puppeteer over [Apify Proxy](https://docs.apify.com/proxy). -To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/guides/apify-platform) to find how to log into your account from the SDK. +To make it work, you'll need an Apify account with access to the proxy. Visit the [Apify platform introduction](/docs/concepts/actor-lifecycle) to find how to log into your account from the SDK. :::tip diff --git a/website/versioned_docs/version-3.4/examples/puppeteer_with_proxy.ts b/website/versioned_docs/version-3.4/03_guides/puppeteer_with_proxy.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/puppeteer_with_proxy.ts rename to website/versioned_docs/version-3.4/03_guides/puppeteer_with_proxy.ts diff --git a/website/versioned_docs/version-3.4/examples/reduce.ts b/website/versioned_docs/version-3.4/03_guides/reduce.ts similarity index 100% rename from website/versioned_docs/version-3.4/examples/reduce.ts rename to website/versioned_docs/version-3.4/03_guides/reduce.ts diff --git a/website/versioned_docs/version-3.4/examples/tsconfig.json b/website/versioned_docs/version-3.4/03_guides/tsconfig.json similarity index 100% rename from website/versioned_docs/version-3.4/examples/tsconfig.json rename to website/versioned_docs/version-3.4/03_guides/tsconfig.json diff --git a/website/versioned_docs/version-3.4/guides/typescript_actor.mdx b/website/versioned_docs/version-3.4/03_guides/typescript_setup.mdx similarity index 100% rename from website/versioned_docs/version-3.4/guides/typescript_actor.mdx rename to website/versioned_docs/version-3.4/03_guides/typescript_setup.mdx diff --git a/website/versioned_docs/version-3.4/upgrading/upgrading_v1.md b/website/versioned_docs/version-3.4/04_upgrading/upgrading_v1.md similarity index 100% rename from website/versioned_docs/version-3.4/upgrading/upgrading_v1.md rename to website/versioned_docs/version-3.4/04_upgrading/upgrading_v1.md diff --git a/website/versioned_docs/version-3.4/upgrading/upgrading_v2.md b/website/versioned_docs/version-3.4/04_upgrading/upgrading_v2.md similarity index 100% rename from website/versioned_docs/version-3.4/upgrading/upgrading_v2.md rename to website/versioned_docs/version-3.4/04_upgrading/upgrading_v2.md diff --git a/website/versioned_docs/version-3.4/upgrading/upgrading_v3.md b/website/versioned_docs/version-3.4/04_upgrading/upgrading_v3.md similarity index 100% rename from website/versioned_docs/version-3.4/upgrading/upgrading_v3.md rename to website/versioned_docs/version-3.4/04_upgrading/upgrading_v3.md diff --git a/website/versioned_docs/version-3.4/api-typedoc.json b/website/versioned_docs/version-3.4/api-typedoc.json index 93b74af34f..6f161df301 100644 --- a/website/versioned_docs/version-3.4/api-typedoc.json +++ b/website/versioned_docs/version-3.4/api-typedoc.json @@ -37657,7 +37657,7 @@ "summary": [ { "kind": "text", - "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/guides/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" + "text": "Defines the path to a local directory where KeyValueStore, Dataset, and RequestQueue\nstore their data. Typically, it is set to ./storage. If omitted, you should define the\nAPIFY_TOKEN environment variable instead. See more info on combination of this and\nAPIFY_TOKEN [here](https://docs.apify.com/sdk/js/docs/concepts/environment-variables#combinations-of-apify_local_storage_dir-and-apify_token)(CRAWLEE_STORAGE_DIR)" } ] }, @@ -45881,7 +45881,7 @@ }, { "kind": "text", - "text": "](https://npmjs.org/crawlee) module. We've written a guide to help you easily migrate from v2 to v3. Visit the [Upgrading Guide](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3) to find out what changes you need to make (especially the section related to this very [Apify SDK](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3#apify-sdk)), and, if you encounter any issues, join our [Discord server](https://discord.gg/jyEM2PRvMU) for help!\n\n## Quick Start\n\nThis short tutorial will set you up to start using Apify SDK in a minute or two.\nIf you want to learn more, proceed to the [Apify Platform](https://docs.apify.com/sdk/js/docs/guides/apify-platform)\nguide that will take you step by step through running your Actor on Apify's platform.\n\nApify SDK requires [Node.js](https://nodejs.org/en/) 16 or later. Add Apify SDK to any Node.js project by running:\n\n" + "text": "](https://npmjs.org/crawlee) module. We've written a guide to help you easily migrate from v2 to v3. Visit the [Upgrading Guide](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3/upgrading-to-v3) to find out what changes you need to make (especially the section related to this very [Apify SDK](https://docs.apify.com/sdk/js/docs/upgrading/upgrading-to-v3/upgrading-to-v3#apify-sdk)), and, if you encounter any issues, join our [Discord server](https://discord.gg/jyEM2PRvMU) for help!\n\n## Quick Start\n\nThis short tutorial will set you up to start using Apify SDK in a minute or two.\nIf you want to learn more, proceed to the [Apify Platform](https://docs.apify.com/sdk/js/docs/concepts/actor-lifecycle)\nguide that will take you step by step through running your Actor on Apify's platform.\n\nApify SDK requires [Node.js](https://nodejs.org/en/) 16 or later. Add Apify SDK to any Node.js project by running:\n\n" }, { "kind": "code", diff --git a/website/versioned_docs/version-3.4/readme/introduction.md b/website/versioned_docs/version-3.4/readme/introduction.md deleted file mode 100644 index ee83e2f9a5..0000000000 --- a/website/versioned_docs/version-3.4/readme/introduction.md +++ /dev/null @@ -1,19 +0,0 @@ ---- ---- - -# Apify SDK: The scalable web crawling and scraping library for JavaScript - - - -[![npm version](https://badge.fury.io/js/apify.svg)](https://www.npmjs.com/package/apify) - -Apify SDK simplifies the development of web crawlers, scrapers, data extractors and web automation jobs. -It provides tools to manage and automatically scale a pool of headless browsers, -to maintain queues of URLs to crawl, store crawling results to a local filesystem or into the cloud, -rotate proxies and much more. -The SDK is available as the [`apify`](https://www.npmjs.com/package/apify) NPM package. -It can be used either stand-alone in your own applications -or in [actors](https://docs.apify.com/actor) -running on the [Apify Cloud](https://apify.com/). - -**View full documentation, guides and examples on the [Apify SDK project website](https://sdk.apify.com)** diff --git a/website/versioned_docs/version-3.4/readme/overview.md b/website/versioned_docs/version-3.4/readme/overview.md deleted file mode 100644 index aad3d4d637..0000000000 --- a/website/versioned_docs/version-3.4/readme/overview.md +++ /dev/null @@ -1,34 +0,0 @@ ---- ---- - -## Overview - -The Apify SDK is available as the [`apify`](https://www.npmjs.com/package/apify) NPM package and it provides the following tools: - -- [`Actor`](https://sdk.apify.com/api/apify/class/Actor) - Serves as an alternative approach to the static helpers exported from the package. - This class can be used to control the current actor run and to interact with the actor's environment. - -- [`ApifyClient`](https://sdk.apify.com/api/apify/class/ApifyClient) - Allows user to interact with the Apify platform from code, control and schedule actors on the platform and access the result data stores. - -- [`Configuration`](https://sdk.apify.com/api/apify/class/Configuration) - Helper class encapsulating the configuration of the current actor run. - -- [`PlatformEventManager`](https://sdk.apify.com/api/apify/class/PlatformEventManager) - Event emitter for the platform and SDK events. Can be used to track actor run performance or serverless container migration. - -- [`ProxyConfiguration`](https://sdk.apify.com/api/apify/class/ProxyConfiguration) - Configures connection to a proxy server with the provided options. Setting proxy configuration in your crawlers automatically configures them to use the selected proxies for all connections. The proxy servers are managed by Apify Proxy. - -- [`RequestQueue`](https://sdk.apify.com/api/apify/class/RequestQueue) - Represents a queue of URLs to crawl, - which is stored either on a local filesystem or in the [Apify Cloud](https://apify.com). The queue is used - for deep crawling of websites, where you start with several URLs and then recursively follow links to other pages. - The data structure supports both breadth-first and depth-first crawling orders. - -- [`Dataset`](https://sdk.apify.com/api/apify/class/Dataset) - Provides a store for structured data and enables their export - to formats like JSON, JSONL, CSV, XML, Excel or HTML. The data is stored on a local filesystem or in the Apify Cloud. - Datasets are useful for storing and sharing large tabular crawling results, such as a list of products or real estate offers. - -- [`KeyValueStore`](https://sdk.apify.com/api/apify/class/KeyValueStore) - A simple key-value store for arbitrary data - records or files, along with their MIME content type. It is ideal for saving screenshots of web pages, PDFs - or to persist the state of your crawlers. The data is stored on a local filesystem or in the Apify Cloud. - -Additionally, the package provides various helper functions to simplify running your code on the Apify Cloud and thus -take advantage of its pool of proxies, job scheduler, data storage, etc. -For more information, see the [Apify SDK Programmer's Reference](https://sdk.apify.com). diff --git a/website/versioned_docs/version-3.4/readme/support.md b/website/versioned_docs/version-3.4/readme/support.md deleted file mode 100644 index e3375dab48..0000000000 --- a/website/versioned_docs/version-3.4/readme/support.md +++ /dev/null @@ -1,24 +0,0 @@ ---- ---- - -## Support - -If you find any bug or issue with the Apify SDK, please [submit an issue on GitHub](https://github.com/apify/apify-js/issues). -For questions, you can ask on [Stack Overflow](https://stackoverflow.com/questions/tagged/apify) or contact support@apify.com - -## Contributing - -Your code contributions are welcome and you'll be praised to eternity! -If you have any ideas for improvements, either submit an issue or create a pull request. -For contribution guidelines and the code of conduct, -see [CONTRIBUTING.md](https://github.com/apify/apify-js/blob/master/CONTRIBUTING.md). - -## License - -This project is licensed under the Apache License 2.0 - -see the [LICENSE.md](https://github.com/apify/apify-js/blob/master/LICENSE.md) file for details. - -## Acknowledgments - -Many thanks to [Chema Balsas](https://www.npmjs.com/~jbalsas) for giving up the `apify` package name -on NPM and renaming his project to [jsdocify](https://www.npmjs.com/package/jsdocify). diff --git a/website/versioned_sidebars/version-3.4-sidebars.json b/website/versioned_sidebars/version-3.4-sidebars.json index 506aad00fc..194b608277 100644 --- a/website/versioned_sidebars/version-3.4-sidebars.json +++ b/website/versioned_sidebars/version-3.4-sidebars.json @@ -1,66 +1,48 @@ { - "docs": [ + "sidebar": [ { "type": "doc", - "label": "Overview", - "id": "overview" + "id": "introduction/introduction" + }, + { + "type": "doc", + "id": "introduction/quick-start" }, { "type": "category", - "label": "Guides", - "link": { - "type": "generated-index", - "title": "Guides", - "slug": "/guides", - "keywords": ["guides"] - }, + "label": "Concepts", + "collapsed": true, "items": [ - "guides/apify-platform", - "guides/request-storage", - "guides/result-storage", - "guides/environment-variables", - "guides/proxy-management", - "guides/session-management", - "guides/pay-per-event", - "guides/type-script-actor", - "guides/docker-images" + { + "type": "autogenerated", + "dirName": "02_concepts" + } ] }, { "type": "category", - "label": "Examples", - "link": { - "type": "generated-index", - "title": "Examples", - "slug": "/examples", - "keywords": ["examples"] - }, + "label": "Guides", + "collapsed": true, "items": [ { "type": "autogenerated", - "dirName": "examples" + "dirName": "03_guides" } ] }, { "type": "category", "label": "Upgrading", - "link": { - "type": "generated-index", - "title": "Upgrading", - "slug": "/upgrading", - "keywords": ["upgrading"] - }, + "collapsed": true, "items": [ { "type": "autogenerated", - "dirName": "upgrading" + "dirName": "04_upgrading" } ] }, { "type": "doc", - "label": "Changelog", "id": "changelog" } ]