Skip to content

Commit 61d59fc

Browse files
rickstaamikezupperAuthorityNullChuckChaineliteprox
authored
docs(ai): add ai-subnet documentation (#549)
* docs(ai): add on-chain discovery documentation This commit adds documentation on how Orchestrators can broadcast their AI subnet service URIs to AI subnet Gateways, enabling their discovery on the subnet for job execution. * docs(ai): improve subsection titles and discovery text This commit improves the subsection titles and Discovery text. * docs(ai): improve introduction text * ci(ai): add AI pipelines document structure * docs(ai): add artibtrum explorer discovery steps This commit adds a guide on how AI nodes can advertise their AI URI using the Arbitrum explorer. * docs(ai): add AI API reference This commit adds an initial draft for the AI Subnet API reference. It is not yet linked to an upstream API endpoint so the playground is not yet working. * chore(ai): cleanup redundant OpenAPI entries * ci(ai): add AI docs codeowners * docs(ai): add pipelines documentation This commit adds the initial draft for the documentation that explains the pipelines available on the network. * docs(ai): fix broken images This commit ensures that images are working when deployed on the remote. * feat(ai): temporarily use external paths for pipeline images This commit replaces the relative image paths used in the pipelines documentation with external paths. We can change this back if we found a way to use relative paths inside mermaid graphs. * feat(ai): add contribution pages scaffold This commit adds a general scaffold for the contribution pages. It is a quick draft and should not yet be published to the upstream docs. * feat(ai): add contribution pages scaffold (#553) This commit adds a general scaffold for the contribution pages. It is a quick draft and should not yet be published to the upstream docs. * Livepeer.Cloud SPE Milestone #1 Deliverables: Add Gateway Docs * docs(ai): improve AI subnet introduction This commit improves the AI subnet introduction page. * fix(ai): fix kickstart collab card link This commit ensures that the collaboration card is linked correctly. * refactor(ai): improve pipeline mermaid diagrams This commit improves the styling used in the pipeline mermaid diagrams. * refactor(ai): improve pipeline documentation This commit updates the pipelines documentation to increase readability. * fix(ai): implement temporary solution for mermaid image issue This commit implements a temporary solution to address the issue of images not displaying correctly when embedded in a mermaid graph on the deployed site. Can be replaced with relative path if mintlify fixes the issue (see https://mintlify-users.slack.com/archives/C05D00NS55F/p1714485030542069). * feat(ai): add builders page This commit adds the builders page and a showcase to show dApps already build on the AI subnet. * refactor(ai): add cloud API disclaimer This commit adds a small disclaimer to the API documentation that states that the livepeer.cloud API is not production ready. * feat(ai): add orchestrator setup guide This commit adds the orchestrator setup guide. * refactor(ai): improve showcase descriptions This commit improves the descriptions of the projects in the showcase. * refactor(ai): highlight showcase keywords This commit highlights some keywords in the showcase descriptions. * feat(ai): add AI gateway docs This commit adds the documentation for setting up a AI gateway on the AI subnet. It also improves the project structure. * refactor(ai): adopt impersonal writing style (#556) This commit updates the text to reflect an impersonal writing style, consistent with the ecosystem's voice. It eliminates personal pronouns such as "our" and "we". * fix(ai): fix broken AI gateway links This commit fixes several broken internal links in the AI gateway docs. * refactor(ai): improve AI subnet introduction section This commit improves the readability of the AI subnet introduction section. * refactor(ai): hide contribution/SDK pages and cleanup pipelines This commit hides the contribution and SDK pages and cleansup the pipeline pages. * refactor(ai): improve text formatting This commit applies some last text formatting improvements. * addressed review comments from Rick * refactor(ai): add model parameter callout to API docs This commit adds a model parameter callout to the API docs to explain that not all parameters are used by all models. * chore(ai): fix small typo This commit fixes some small typos. * refactor(ai): fix spelling This commit fixes some small spelling errors. * made Prerequisites - more concise * chore(ai): add Analytics token This commit adds the Google Analytics GA4 token to the documentation config. * docs(gateway): apply prettier mdx formatting This commit applies the prettier formatting suggestions on the Gateway docs (see https://prettier.io/blog/2021/11/25/2.5.0.html). * refactor(ai): gateway former naming comment This commit makes it more clear that Gateways were formaly named Broadcasters. * refactor(ai): simplify binary install This commit simplifies the binary install of the AI subnet by giving users one simple download url they can use to downloade the latest `ai-video` branch binary. This was made possible in livepeer/go-livepeer@fb9764b. * docs(ai): apply community feedback This commit applies the feedback I received from the community about things that could be improved. Co-authored-by: Authority_Null <[email protected]> Co-authored-by: Chuck Chain <[email protected]> * docs(ai): add RealViz warm comment This commit adds a comment letting users know that several orchestrators have the https://huggingface.co/SG161222/RealVisXL_V4.0 model warm on their GPU. * docs(ai): fix incorrect RealViLightning model link This commit ensures that the RealVizLightning model link is pointing to the right url. * chore(ai): update AI subnet OpenAPI spec This commit updates the AI subnet OpenApi spec so that the reference documentation correctly represents the new `num_inference_steps` parameter introduced in livepeer/ai-runner#80. * docs: replace -broadcaster flag with -gateway in AI subnet documentation (#564) This commit replaces the `broadcaster` flag with the new `gateway` flag. See livepeer/go-livepeer#3048 for more information. * docs(ai): remove pricePerUnit requirement for AI docs (#565) This commit removes the dependency on the `pricePerUnit` flag in the AI orchestrator documentation now that livepeer/go-livepeer#3047 has been merged. * docs(ai): add RealVisXL to I2I pipeline This commit adds the [SG161222/RealVisXL_V4.0_Lightning](https://huggingface.co/SG161222/RealVisXL_V4.0_Lightning) model to the supported models on the I2I pipeline documentation page. * docs: add Gateway network page This commit adds a Gateway page under the core-concepts category, explaining the role and functions of a Gateway. * docs: improve Gateway configuration wording * chore: apply prettier to Gateway node page * Update gateways.mdx Clarified gateways as the type of node, and talked about hosted gateways as something that individual entities may run. * docs: fix Gateway node page links This commit fixes some broken links in the Gateway node page. * docs(ai): remove double subnet requirements text This commit removes duplicate text in the AI subnet requirements section. * docs(ai): address community feedback This commit clarifies points that the community identified as unclear. * docs(ai): enhance introduction This commit strengthens the AI Subnet introduction, providing clearer and more detailed information about its capabilities and benefits. * docs(ai): add DeepCache optimization method This commit adds the DeepCache optimization method to the AI models configuration page. This method can be used to speedup the pipelines to up to 50%. * docs(ai): replace dApp with application This commit replaces the parts that mention dApps with applications since the AI subnet can serve both web2 and web3 applications. * docs(ai): ensure GPU binaries are listed This commit ensures the GPU binaries are listed in the Orchestrator binary setup guide. * docs(ai): fix incorrect binary format This commit fixes a small syntax error in the AI subnet GPU binary download url. * docs(ai): add HuggingFace token read permission comment This commit ensures that people are aware that they only need to create a HuggingFace token with read permissions. * docs(ai): improve introduction This commit improves the introduction to make clear that orchestrators can have multiple workers. It also improves the diagram. * docs(ai): change dApp to app This commit ensures that all references to dApp are replaced by app or application. * docs(ai): apply some small text improvements This commit applies some small text improvements to the AI documentation. * docs(ai): update AI API reference This commit updates the AI API reference to include the new `nsfw` feature (see livepeer/ai-runner@272ac74). * refactor(ai) improve introduction wording (#570) * Update verbiage in introduction * docs(ai): mention AI Video subnet --------- Co-authored-by: Elite Encoder <[email protected]> --------- Co-authored-by: Mike Zupper <[email protected]> Co-authored-by: Authority_Null <[email protected]> Co-authored-by: Chuck Chain <[email protected]> Co-authored-by: Elite Encoder <[email protected]> Co-authored-by: Mike Zupper <[email protected]> Co-authored-by: Doug Petkanics <[email protected]>
1 parent 36b3902 commit 61d59fc

38 files changed

+2356
-21
lines changed

‎.github/CODEOWNERS

+2
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
# Default reviewers for the AI documentation.
2+
ai/ @rickstaa

‎.gitignore

+2-1
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
node_modules
22
.DS_Store
33

4-
# IDEs
4+
# IDE
55
.vscode
6+
*.code-workspace

‎ai/api-reference/ai-openapi-schema.yml

+345
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,345 @@
1+
openapi: 3.1.0
2+
info:
3+
title: Livepeer AI Runner
4+
description: An application to run AI pipelines
5+
version: 0.1.0
6+
servers:
7+
- url: https://dream-gateway.livepeer.cloud
8+
description: Livepeer Cloud Community Gateway
9+
paths:
10+
/text-to-image:
11+
post:
12+
summary: Text To Image
13+
operationId: text_to_image
14+
requestBody:
15+
content:
16+
application/json:
17+
schema:
18+
$ref: '#/components/schemas/TextToImageParams'
19+
required: true
20+
responses:
21+
'200':
22+
description: Successful Response
23+
content:
24+
application/json:
25+
schema:
26+
$ref: '#/components/schemas/ImageResponse'
27+
'400':
28+
description: Bad Request
29+
content:
30+
application/json:
31+
schema:
32+
$ref: '#/components/schemas/HTTPError'
33+
'500':
34+
description: Internal Server Error
35+
content:
36+
application/json:
37+
schema:
38+
$ref: '#/components/schemas/HTTPError'
39+
'422':
40+
description: Validation Error
41+
content:
42+
application/json:
43+
schema:
44+
$ref: '#/components/schemas/HTTPValidationError'
45+
security:
46+
- HTTPBearer: []
47+
/image-to-image:
48+
post:
49+
summary: Image To Image
50+
operationId: image_to_image
51+
requestBody:
52+
content:
53+
multipart/form-data:
54+
schema:
55+
$ref: '#/components/schemas/Body_image_to_image_image_to_image_post'
56+
required: true
57+
responses:
58+
'200':
59+
description: Successful Response
60+
content:
61+
application/json:
62+
schema:
63+
$ref: '#/components/schemas/ImageResponse'
64+
'400':
65+
description: Bad Request
66+
content:
67+
application/json:
68+
schema:
69+
$ref: '#/components/schemas/HTTPError'
70+
'500':
71+
description: Internal Server Error
72+
content:
73+
application/json:
74+
schema:
75+
$ref: '#/components/schemas/HTTPError'
76+
'422':
77+
description: Validation Error
78+
content:
79+
application/json:
80+
schema:
81+
$ref: '#/components/schemas/HTTPValidationError'
82+
security:
83+
- HTTPBearer: []
84+
/image-to-video:
85+
post:
86+
summary: Image To Video
87+
operationId: image_to_video
88+
requestBody:
89+
content:
90+
multipart/form-data:
91+
schema:
92+
$ref: '#/components/schemas/Body_image_to_video_image_to_video_post'
93+
required: true
94+
responses:
95+
'200':
96+
description: Successful Response
97+
content:
98+
application/json:
99+
schema:
100+
$ref: '#/components/schemas/VideoResponse'
101+
'400':
102+
description: Bad Request
103+
content:
104+
application/json:
105+
schema:
106+
$ref: '#/components/schemas/HTTPError'
107+
'500':
108+
description: Internal Server Error
109+
content:
110+
application/json:
111+
schema:
112+
$ref: '#/components/schemas/HTTPError'
113+
'422':
114+
description: Validation Error
115+
content:
116+
application/json:
117+
schema:
118+
$ref: '#/components/schemas/HTTPValidationError'
119+
security:
120+
- HTTPBearer: []
121+
components:
122+
schemas:
123+
APIError:
124+
properties:
125+
msg:
126+
type: string
127+
title: Msg
128+
type: object
129+
required:
130+
- msg
131+
title: APIError
132+
Body_image_to_image_image_to_image_post:
133+
properties:
134+
prompt:
135+
type: string
136+
title: Prompt
137+
image:
138+
type: string
139+
format: binary
140+
title: Image
141+
model_id:
142+
type: string
143+
title: Model Id
144+
default: ''
145+
strength:
146+
type: number
147+
title: Strength
148+
default: 0.8
149+
guidance_scale:
150+
type: number
151+
title: Guidance Scale
152+
default: 7.5
153+
negative_prompt:
154+
type: string
155+
title: Negative Prompt
156+
default: ''
157+
safety_check:
158+
type: boolean
159+
title: Safety Check
160+
default: true
161+
seed:
162+
type: integer
163+
title: Seed
164+
num_images_per_prompt:
165+
type: integer
166+
title: Num Images Per Prompt
167+
default: 1
168+
type: object
169+
required:
170+
- prompt
171+
- image
172+
- model_id
173+
title: Body_image_to_image_image_to_image_post
174+
Body_image_to_video_image_to_video_post:
175+
properties:
176+
image:
177+
type: string
178+
format: binary
179+
title: Image
180+
model_id:
181+
type: string
182+
title: Model Id
183+
default: ''
184+
height:
185+
type: integer
186+
title: Height
187+
default: 576
188+
width:
189+
type: integer
190+
title: Width
191+
default: 1024
192+
fps:
193+
type: integer
194+
title: Fps
195+
default: 6
196+
motion_bucket_id:
197+
type: integer
198+
title: Motion Bucket Id
199+
default: 127
200+
noise_aug_strength:
201+
type: number
202+
title: Noise Aug Strength
203+
default: 0.02
204+
seed:
205+
type: integer
206+
title: Seed
207+
type: object
208+
required:
209+
- image
210+
- model_id
211+
title: Body_image_to_video_image_to_video_post
212+
HTTPError:
213+
properties:
214+
detail:
215+
$ref: '#/components/schemas/APIError'
216+
type: object
217+
required:
218+
- detail
219+
title: HTTPError
220+
HTTPValidationError:
221+
properties:
222+
detail:
223+
items:
224+
$ref: '#/components/schemas/ValidationError'
225+
type: array
226+
title: Detail
227+
type: object
228+
title: HTTPValidationError
229+
HealthCheck:
230+
properties:
231+
status:
232+
type: string
233+
title: Status
234+
default: OK
235+
type: object
236+
title: HealthCheck
237+
ImageResponse:
238+
properties:
239+
images:
240+
items:
241+
$ref: '#/components/schemas/Media'
242+
type: array
243+
title: Images
244+
type: object
245+
required:
246+
- images
247+
title: ImageResponse
248+
Media:
249+
properties:
250+
url:
251+
type: string
252+
title: Url
253+
seed:
254+
type: integer
255+
title: Seed
256+
nsfw:
257+
type: boolean
258+
title: Nsfw
259+
type: object
260+
required:
261+
- url
262+
- seed
263+
- nsfw
264+
title: Media
265+
TextToImageParams:
266+
properties:
267+
model_id:
268+
type: string
269+
title: Model Id
270+
default: ''
271+
prompt:
272+
type: string
273+
title: Prompt
274+
height:
275+
type: integer
276+
title: Height
277+
width:
278+
type: integer
279+
title: Width
280+
guidance_scale:
281+
type: number
282+
title: Guidance Scale
283+
default: 7.5
284+
negative_prompt:
285+
type: string
286+
title: Negative Prompt
287+
default: ''
288+
safety_check:
289+
type: boolean
290+
title: Safety Check
291+
default: true
292+
seed:
293+
type: integer
294+
title: Seed
295+
num_inference_steps:
296+
type: integer
297+
title: Num Inference Steps
298+
default: 50
299+
num_images_per_prompt:
300+
type: integer
301+
title: Num Images Per Prompt
302+
default: 1
303+
type: object
304+
required:
305+
- prompt
306+
- model_id
307+
title: TextToImageParams
308+
ValidationError:
309+
properties:
310+
loc:
311+
items:
312+
anyOf:
313+
- type: string
314+
- type: integer
315+
type: array
316+
title: Location
317+
msg:
318+
type: string
319+
title: Message
320+
type:
321+
type: string
322+
title: Error Type
323+
type: object
324+
required:
325+
- loc
326+
- msg
327+
- type
328+
title: ValidationError
329+
VideoResponse:
330+
properties:
331+
frames:
332+
items:
333+
items:
334+
$ref: '#/components/schemas/Media'
335+
type: array
336+
type: array
337+
title: Frames
338+
type: object
339+
required:
340+
- frames
341+
title: VideoResponse
342+
securitySchemes:
343+
HTTPBearer:
344+
type: http
345+
scheme: bearer

‎ai/api-reference/health.mdx

+3
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
---
2+
openapi: get /health
3+
---

‎ai/api-reference/image-to-image.mdx

+21
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
---
2+
openapi: post /image-to-image
3+
---
4+
5+
<Info>
6+
The public [Livepeer.cloud](https://www.livepeer.cloud/) Gateway used in this
7+
guide is intended for experimentation and is not guaranteed for production
8+
use. It is a free, non-token-gated, but rate-limited service designed for
9+
testing purposes. For production-ready applications, consider setting up your
10+
own Gateway node or partnering with one via the `ai-video` channel on
11+
[Discord](https://discord.gg/livepeer).
12+
</Info>
13+
14+
<Note>
15+
Please note that the **optimal** parameters for a given model may vary
16+
depending on the specific model and use case. The parameters provided in this
17+
guide are not model-specific and should be used as a starting point.
18+
Additionally, some models may have parameters such as `guiding_scale` and
19+
`num_inference_steps` disabled by default. For more information on
20+
model-specific parameters, please refer to the respective model documentation.
21+
</Note>

0 commit comments

Comments
 (0)