Skip to content

Commit 2d61588

Browse files
quanruquanru
authored
feat: add playwright demo (#57)
1 parent d6736c2 commit 2d61588

21 files changed

Lines changed: 581 additions & 42 deletions

‎playwright-demo/.gitignore‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,8 @@
1+
package-lock.json
2+
.env
3+
14
# Midscene.js dump files
25
midscene_run/midscene-report
36
midscene_run/dump-logger
47

5-
package-lock.json
8+
midscene_run/cache

‎playwright-demo/README.md‎

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
1-
# playwright-demo
1+
# Playwright Demo
2+
3+
This is a demo to show how to use Playwright to do some automation tasks.
4+
5+
If you want to use Playwright with Vitest, please refer to [puppeteer-with-vitest-demo](../puppeteer-with-vitest-demo) for the usage.
26

37
## Steps
48

@@ -15,26 +19,19 @@ Refer to this document if your want to use other models like Qwen: https://midsc
1519

1620
### Run demo
1721

18-
run e2e test
19-
2022
```bash
21-
pnpm install
23+
npm install
2224

23-
# run e2e test
24-
pnpm run e2e
25+
# run demo.ts
26+
npx tsx demo.ts
2527

26-
# prefer using cache
27-
pnpm run e2e:cache
28+
# run extract-data.ts
29+
npx tsx extract-data.ts
2830

29-
# run e2e with playwright ui, remember to click the little "Play" button on the upper-left corner
30-
pnpm run e2e:ui
31-
32-
# run e2e with playwright ui + cache
33-
pnpm run e2e:ui:cache
31+
# run demo with a `.runYaml` call
32+
npx tsx demo-run-yaml.ts
3433
```
3534

36-
After the above command executes successfully, the console will output: `Midscene - report file updated: ./current_cwd/midscene_run/report/some_id.html.` You can open this file in a browser to view the report.
37-
3835
# Reference
3936

4037
https://midscenejs.com/integrate-with-playwright.html

‎playwright-demo/demo-run-yaml.ts‎

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
import { chromium } from "playwright";
2+
import { PlaywrightAgent } from "@midscene/web/playwright";
3+
import "dotenv/config";
4+
5+
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
6+
Promise.resolve(
7+
(async () => {
8+
const browser = await chromium.launch({
9+
headless: true, // 'true' means we can't see the browser window
10+
args: ["--no-sandbox", "--disable-setuid-sandbox"],
11+
});
12+
13+
const page = await browser.newPage();
14+
await page.setViewportSize({
15+
width: 1280,
16+
height: 800,
17+
});
18+
19+
await page.goto("https://www.ebay.com");
20+
await sleep(5000);
21+
22+
const agent = new PlaywrightAgent(page);
23+
24+
// 👀 run YAML with agent
25+
const { result } = await agent.runYaml(`
26+
tasks:
27+
- name: search
28+
flow:
29+
- ai: input 'Headphones' in search box, click search button
30+
- sleep: 3000
31+
32+
- name: query
33+
flow:
34+
- aiQuery: "{itemTitle: string, price: Number}[], find item in list and corresponding price"
35+
name: headphones
36+
- aiNumber: "What is the price of the first headphone?"
37+
- aiBoolean: "Is the price of the headphones more than 1000?"
38+
- aiString: "What is the name of the first headphone?"
39+
- aiLocate: "What is the location of the first headphone?"
40+
`);
41+
42+
console.log(result);
43+
44+
await browser.close();
45+
})()
46+
);

‎playwright-demo/demo.ts‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
import { chromium } from "playwright";
2+
import { PlaywrightAgent } from "@midscene/web/playwright";
3+
import "dotenv/config"; // read environment variables from .env file
4+
5+
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
6+
7+
8+
Promise.resolve(
9+
(async () => {
10+
const browser = await chromium.launch({
11+
headless: true, // 'true' means we can't see the browser window
12+
args: ["--no-sandbox", "--disable-setuid-sandbox"],
13+
});
14+
15+
const page = await browser.newPage();
16+
await page.setViewportSize({
17+
width: 1280,
18+
height: 768,
19+
});
20+
await page.goto("https://www.ebay.com");
21+
await sleep(5000); // 👀 init Midscene agent
22+
const agent = new PlaywrightAgent(page);
23+
24+
// 👀 type keywords, perform a search
25+
await agent.aiAction('type "Headphones" in search box, hit Enter');
26+
27+
// 👀 wait for the loading
28+
await agent.aiWaitFor("there is at least one headphone item on page");
29+
// or you may use a plain sleep:
30+
// await sleep(5000);
31+
32+
// 👀 understand the page content, find the items
33+
const items = await agent.aiQuery(
34+
"{itemTitle: string, price: Number}[], find item in list and corresponding price"
35+
);
36+
console.log("headphones in stock", items);
37+
38+
const isMoreThan1000 = await agent.aiBoolean("Is the price of the headphones more than 1000?");
39+
console.log("isMoreThan1000", isMoreThan1000);
40+
41+
const price = await agent.aiNumber("What is the price of the first headphone?");
42+
console.log("price", price);
43+
44+
const name = await agent.aiString("What is the name of the first headphone?");
45+
console.log("name", name);
46+
47+
const location = await agent.aiLocate("What is the location of the first headphone?");
48+
console.log("location", location);
49+
50+
// 👀 assert by AI
51+
await agent.aiAssert("There is a category filter on the left");
52+
53+
// 👀 click on the first item
54+
await agent.aiTap("the first item in the list");
55+
56+
await browser.close();
57+
})()
58+
);

‎playwright-demo/extract-data.ts‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
import { chromium } from "playwright";
2+
import { PlaywrightAgent } from "@midscene/web/playwright";
3+
import "dotenv/config"; // read environment variables from .env file
4+
5+
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
6+
7+
Promise.resolve(
8+
(async () => {
9+
const browser = await chromium.launch({
10+
headless: false, // set to 'false' to see the browser window for demo
11+
args: ["--no-sandbox", "--disable-setuid-sandbox"],
12+
});
13+
14+
const page = await browser.newPage();
15+
await page.setViewportSize({
16+
width: 1280,
17+
height: 768,
18+
});
19+
20+
// Load the contacts demo page (replace with your actual file path or URL)
21+
await page.goto("https://lf3-static.bytednsdoc.com/obj/eden-cn/nupipfups/Midscene/contacts3.html");
22+
23+
// await sleep(2000);
24+
25+
// 🤖 Initialize Midscene agent
26+
const agent = new PlaywrightAgent(page);
27+
28+
console.log("🚀 Starting Smart Contacts Demo with Midscene AI");
29+
console.log("================================================");
30+
31+
// ✨ FEATURE DEMO 1: aiRightClick - Right-click on a contact
32+
console.log("\n1. 🖱️ Testing aiRightClick feature...");
33+
await agent.aiRightClick("Alice Johnson", { deepThink: true });
34+
await sleep(1000);
35+
console.log("✅ Successfully right-clicked on Alice Johnson's contact card");
36+
37+
// Click on "Copy Info" option in context menu
38+
await agent.aiTap("Copy Info");
39+
await sleep(1000);
40+
console.log("✅ Successfully triggered 'Copy Info' action from context menu");
41+
42+
// ✨ FEATURE DEMO 2: aiQuery with domIncluded - Extract contact data including hidden attributes
43+
console.log("\n2. 📊 Testing aiQuery with domIncluded feature...");
44+
const contactsData = await agent.aiQuery(
45+
"{name: string, id: number, company: string, department: string, avatarUrl: string}[], extract all contact information including hidden avatarUrl attributes",
46+
{ domIncluded: true }
47+
);
48+
console.log("✅ Successfully extracted contact data with hidden attributes:");
49+
console.log(JSON.stringify(contactsData, null, 2));
50+
51+
// ✨ FEATURE DEMO 3: aiBoolean with domIncluded - Check for ID fields
52+
console.log("\n3. ❓ Testing aiBoolean with domIncluded feature...");
53+
const isId1 = await agent.aiBoolean(
54+
"is First contact's id is 1?",
55+
{ domIncluded: true }
56+
);
57+
console.log("✅ Is First contact's id is 1?", isId1);
58+
59+
// ✨ FEATURE DEMO 4: aiNumber - with domIncluded - Count contacts
60+
console.log("\n4. 🔢 Testing aiNumber with domIncluded feature...");
61+
const contactCount = await agent.aiNumber("First contact's id?", { domIncluded: true });
62+
console.log("✅ First contact's id:", contactCount);
63+
64+
// ✨ FEATURE DEMO 5: aiString with domIncluded - Get first contact's ID
65+
console.log("\n5. 🆔 Testing aiString with domIncluded feature...");
66+
const firstContactId = await agent.aiString(
67+
"What is the Avatar URL of the first contact?",
68+
{ domIncluded: true }
69+
);
70+
console.log("✅ First contact's Avatar URL:", firstContactId);
71+
72+
console.log("\n🎉 Smart Contacts Demo completed!");
73+
console.log("================================================");
74+
console.log("✨ Midscene features demonstrated:");
75+
console.log(" • aiRightClick() with deepThink - Custom context menus");
76+
console.log(" • aiQuery() with domIncluded - Extract hidden ID attributes");
77+
console.log(" • aiBoolean() with domIncluded - DOM-based boolean checks");
78+
console.log(" • aiNumber() - with domIncluded - Hidden ID attributes");
79+
console.log(" • aiString() with domIncluded - Extract hidden Avatar URL values");
80+
81+
// Keep browser open for a few seconds to see the results
82+
await sleep(3000);
83+
await browser.close();
84+
})()
85+
);

‎playwright-demo/package.json‎

Lines changed: 20 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,21 @@
11
{
2-
"name": "playwright-demo",
3-
"private": true,
4-
"version": "0.0.1",
5-
"type": "module",
6-
"scripts": {
7-
"e2e": "playwright test --config=playwright.config.ts",
8-
"e2e:cache": "cross-env MIDSCENE_CACHE=true playwright test --config=playwright.config.ts",
9-
"e2e:ui": "playwright test --config=playwright.config.ts --ui",
10-
"e2e:ui:cache": "cross-env MIDSCENE_CACHE=true playwright test --config=playwright.config.ts --ui",
11-
"postinstall": "pnpm exec playwright install"
12-
},
13-
"devDependencies": {
14-
"@midscene/web": "latest",
15-
"@playwright/test": "1.52.0",
16-
"@types/jest": "~29.5.14",
17-
"@types/node": "~22.7.9",
18-
"cross-env": "7.0.3",
19-
"dotenv": "16.4.5",
20-
"eslint-plugin-prettier": "~5.2.1",
21-
"rimraf": "~6.0.1",
22-
"typescript": "~5.6.3"
23-
},
24-
"publishConfig": {
25-
"access": "public"
26-
}
27-
}
2+
"name": "playwright-demo",
3+
"private": true,
4+
"version": "1.0.0",
5+
"description": "> quick start",
6+
"main": "index.js",
7+
"type": "module",
8+
"scripts": {
9+
"test": "tsx demo.ts",
10+
"test-yaml": "tsx demo-run-yaml.ts"
11+
},
12+
"author": "",
13+
"license": "MIT",
14+
"devDependencies": {
15+
"@midscene/web": "latest",
16+
"@playwright/test": "^1.54.1",
17+
"dotenv": "^16.4.5",
18+
"playwright": "1.54.1",
19+
"tsx": "4.20.1"
20+
}
21+
}

‎playwright-testing-demo/.gitignore‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
# Midscene.js dump files
2+
midscene_run/midscene-report
3+
midscene_run/dump-logger
4+
5+
package-lock.json

playwright-demo/.npmrc renamed to playwright-testing-demo/.npmrc

File renamed without changes.

‎playwright-testing-demo/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# playwright-testing-demo
2+
3+
## Steps
4+
5+
### Preparation
6+
7+
create `.env` file
8+
9+
```shell
10+
# replace by your gpt-4o api key
11+
OPENAI_API_KEY="YOUR_TOKEN"
12+
```
13+
14+
Refer to this document if your want to use other models like Qwen: https://midscenejs.com/choose-a-model
15+
16+
### Run demo
17+
18+
run e2e test
19+
20+
```bash
21+
pnpm install
22+
23+
# run e2e test
24+
pnpm run e2e
25+
26+
# prefer using cache
27+
pnpm run e2e:cache
28+
29+
# run e2e with playwright ui, remember to click the little "Play" button on the upper-left corner
30+
pnpm run e2e:ui
31+
32+
# run e2e with playwright ui + cache
33+
pnpm run e2e:ui:cache
34+
```
35+
36+
After the above command executes successfully, the console will output: `Midscene - report file updated: ./current_cwd/midscene_run/report/some_id.html.` You can open this file in a browser to view the report.
37+
38+
# Reference
39+
40+
https://midscenejs.com/integrate-with-playwright.html
41+
https://midscenejs.com/api.html

playwright-demo/e2e/ebay-search.spec.ts renamed to playwright-testing-demo/e2e/ebay-search.spec.ts

File renamed without changes.

0 commit comments

Comments
 (0)