Skip to content

Commit a71cbca

Browse files
3y33y3
committed
wip
1 parent e3f01b9 commit a71cbca

File tree

15 files changed

+439
-33
lines changed

15 files changed

+439
-33
lines changed

‎.yfm

+3-5
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,19 @@ allowHTML: true
22
langs: ['en','ru']
33

44
mtimes: true
5+
authors: true
6+
contributors: true
57
vcs:
68
enabled: true
7-
type: github
89
owner: diplodoc-platform
910
repo: docs
10-
token: aaa
11-
initialCommit: 8b7762007064bb25537bab4d3d1705fc50b1ebda
11+
branch: master
1212

1313
search:
1414
enabled: false
1515
index: true
1616
provider: algolia
1717
appId: L6T02M2L5A
18-
apiKey: 6b76bcff90a5f89ad2cc996e7f8c2f6e
19-
searchKey: 544ccb72d154b30b63a51aa4a1bde3bb
2018
indexPrefix: diplodoc
2119

2220
resources:

‎en/releases/v6.0.md

+116
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,116 @@
1+
# Diplodoc v6.0: New Level of Performance
2+
3+
## Major Improvements
4+
5+
### Performance Optimization
6+
- Significant CLI tools performance improvement (2x base speedup)
7+
- Parallel document processing with up to 10x speedup on multi-core systems
8+
- Improved TOC (Table of Contents) processing
9+
- Resource loading optimization and caching
10+
- Optimized handling of large projects
11+
12+
We have been making gradual performance improvements since version 4.50.0 and are now ready to announce the results. We have achieved a base speedup of 2x thanks to optimized document processing algorithms, improved caching, and more efficient resource utilization. Additionally, we have added parallel document processing, which further accelerates the build depending on the number of cores in the system. In some cases, especially when working with large projects on multi-core servers, we observed a 10x speedup. This significant improvement allows developers to get build results faster and work more efficiently with documentation.
13+
14+
Our system now efficiently processes documentation with over 50,000 articles, including individual articles containing more than 10 MB of text and markup. This enables the use of Diplodoc for creating and maintaining large-scale documentation projects without performance degradation.
15+
16+
### Extended Features
17+
- Page Constructor support for modern landing pages
18+
- Built-in search system with Algolia support
19+
- Enhanced RTL (Right-to-Left) support for Arabic and Hebrew
20+
- Version Control Systems integration (GitHub)
21+
22+
### Enhanced Table of Contents (TOC) System
23+
- Extended variable templating support for `href` and `label` fields
24+
- Automatic extraction of headings from referenced files for the `name` field
25+
- Improved handling of includes in TOC with more careful file system operations
26+
- More flexible documentation structure configuration through frontmatter
27+
28+
We have significantly improved the Table of Contents (TOC) system, making it more flexible and user-friendly. Now more fields support variable templating, allowing for dynamic link and label generation. The `name` field can now automatically extract headings from the file it references, simplifying documentation maintenance and keeping it up-to-date. We have also improved the handling of includes in TOC, making it more careful when working with the file system, which is especially important for large projects with many nested directories.
29+
30+
### Redirect System
31+
- Built-in support for simple file-to-file redirects based on browser redirects
32+
- Set of hooks for generating configurations for nginx and other proxy servers
33+
- Flexible configuration of redirect rules
34+
- Support for complex redirect scenarios for large projects
35+
36+
While developing the Extension API, we recognized the need to introduce a new functionality in the system - handling redirects. By default, we implemented a simple file-to-file redirect system that works based on browser redirects. For more complex scenarios, when generating configurations for nginx or other proxy servers is required, the system includes a set of hooks for convenient work with redirects. This allows flexible configuration of redirection rules depending on project requirements and infrastructure.
37+
38+
### Enhanced Configuration System
39+
- New configuration format with environment variables support
40+
- Extended settings validation capabilities
41+
- Improved presets and variables handling
42+
- More flexible settings override system
43+
- Systematized command-line parameters with unified naming style
44+
- Complete synchronization of settings between configuration file and command-line parameters
45+
46+
We have systematized command-line parameters for more intuitive use. Previously, we had several "disabling" parameters that were difficult to remember: `--disableLiquid`, `--lintDisabled`, `--buildDisabled`, `--needToSanitizeHtml`. We removed unnecessary parameters and standardized the remaining ones to follow a unified naming convention:
47+
48+
- `--disableLiquid` → `--no-template`
49+
- `--lintDisabled` → `--no-lint`
50+
- `--needToSanitizeHtml false` → `--no-sanitize-html`
51+
52+
This systematization makes the command-line interface more understandable and predictable for users.
53+
54+
We have also synchronized project build settings between command-line parameters and the configuration file. Now, for all main parameters in the configuration file, there is a corresponding command-line parameter. This has allowed us to add new parameters:
55+
56+
- `--langs` - manage documentation languages
57+
- `--search` - configure search parameters
58+
- `--changelogs` - manage changelog list
59+
- `--mtimes` - configure timestamps
60+
61+
### Extension API and Hook System
62+
- New extension system for developers with full TypeScript support
63+
- Built-in hook system for extending functionality at various stages
64+
- Modular design for creating independent, reusable extensions
65+
- Documented API for creating plugins and integrations
66+
- Integration examples with popular tools
67+
- Flexible extension lifecycle management
68+
69+
We have already developed several extensions using the new system, which can be used as examples for creating your own extensions:
70+
71+
- **GenericIncluder** - extension for automatic Table of Contents (TOC) generation. Analyzes the file system, finds Markdown files, and automatically creates a TOC structure based on their location and content. When generating the TOC, it takes into account the file title and its weight specified in the frontmatter, allowing for flexible customization of the documentation structure. This significantly simplifies the organization of large documentation projects.
72+
73+
- **GithubVcsConnector** - extension for collecting information about contributors and article authors from GitHub. Gathers metadata about commits, authors, and contributors, which can then be displayed in the documentation interface. This allows users to see who and when made changes to the documentation, increasing transparency in the development process and acknowledging community contributions.
74+
75+
- **AlgoliaSearch** - integration with Algolia for full-text search capabilities. Automatically indexes documentation content in Algolia, configures search suggestions and filters, and provides fast and relevant search across the entire documentation with multi-language support.
76+
77+
- **LocalSearch** - extension for local documentation search using the lunr.js library. Creates a search index on the client side, enabling fast search without connecting to external services. Supports fuzzy search, phrase search, and relevance-based result ranking. Particularly useful for projects requiring offline operation or with data confidentiality restrictions.
78+
79+
These extensions demonstrate various capabilities of the new extension system and can serve as a starting point for developing your own plugins. Each of them implements its own set of hooks and utilizes different aspects of the extension API.
80+
81+
## Technical Details
82+
83+
### New Configuration Options
84+
```yaml
85+
sanitizeHtml: true
86+
allowHtml: true
87+
addMapFile: false
88+
removeHiddenTocItems: false
89+
mergeIncludes: false
90+
staticContent: false
91+
```
92+
93+
### Enhanced Extension System
94+
- Base Program class for all CLI commands
95+
- Hook system for functionality extension
96+
- Configuration and logging management
97+
- Extension registration interface
98+
99+
## Getting Started
100+
101+
1. Update dependencies to the latest version
102+
2. Check compatibility of existing configurations
103+
3. Review new documentation
104+
4. Configure parameters according to your needs
105+
106+
### Minor Technical Improvements
107+
108+
- Landing pages now fully support frontmatter, allowing customization of their metadata and behavior
109+
- Using Latex and Mermaid syntax no longer requires the `--allow-custom-resources` flag, these features are now available by default
110+
- More secure file system operations - fixed a number of vulnerabilities that allowed access to files outside the project
111+
112+
## Additional Information
113+
114+
- [Full Documentation](https://diplodoc.com/docs)
115+
- [Migration Guide](https://diplodoc.com/docs/migration)
116+
- [Usage Examples](https://diplodoc.com/docs/examples)

‎en/toc.yaml

+5
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,11 @@ navigation:
2929
items:
3030
- name: Overview
3131
href: about.md
32+
- name: Releases
33+
labeled: true
34+
items:
35+
- name: Diplodoc v6.0
36+
href: releases/v6.0.md
3237
- name: Quick start
3338
href: how-it-work.md
3439
- name: Yandex Flavored Markdown

‎extensions/README.md renamed to ‎ru/extensions/README.md

File renamed without changes.

‎ru/extensions/commands/build/services/search-service.md

+155
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,155 @@
1+
# Search Service
2+
3+
Search Service предоставляет функциональность для организации поиска по документации. Сервис поддерживает различные типы поисковых систем, включая локальный поиск и интеграцию с внешними поисковыми сервисами.
4+
5+
## Основные возможности
6+
7+
- Создание и управление поисковым индексом
8+
- Поддержка различных поисковых систем (lunr.js, Algolia и др.)
9+
- Гибкая настройка параметров поиска
10+
- Интеграция с внешними поисковыми сервисами
11+
- Кэширование результатов поиска
12+
13+
## Hooks
14+
15+
Search Service предоставляет следующие hooks:
16+
17+
### `Index`
18+
```typescript
19+
searchHooks.Index.tap('MyExtension', (index) => {
20+
// Модификация поискового индекса
21+
return index;
22+
});
23+
```
24+
Позволяет модифицировать поисковый индекс перед его использованием.
25+
26+
### `Search`
27+
```typescript
28+
searchHooks.Search.tap('MyExtension', (query, options) => {
29+
// Обработка поискового запроса
30+
return results;
31+
});
32+
```
33+
Позволяет модифицировать поисковый запрос или его результаты.
34+
35+
## Примеры использования
36+
37+
### 1. Добавление пользовательских полей в индекс
38+
```typescript
39+
export class Extension {
40+
apply(program: Build) {
41+
getBaseHooks(program).BeforeAnyRun.tap('MySearchExtension', (run) => {
42+
const searchHooks = getSearchHooks(run.search);
43+
44+
searchHooks.Index.tap('MySearchExtension', (index) => {
45+
// Добавление пользовательских полей в индекс
46+
return {
47+
...index,
48+
customField: 'value'
49+
};
50+
});
51+
});
52+
}
53+
}
54+
```
55+
56+
### 2. Модификация поискового запроса
57+
```typescript
58+
export class Extension {
59+
apply(program: Build) {
60+
getBaseHooks(program).BeforeAnyRun.tap('MySearchExtension', (run) => {
61+
const searchHooks = getSearchHooks(run.search);
62+
63+
searchHooks.Search.tap('MySearchExtension', (query, options) => {
64+
// Добавление пользовательской логики поиска
65+
const modifiedQuery = this.processQuery(query);
66+
return modifiedQuery;
67+
});
68+
});
69+
}
70+
71+
private processQuery(query: string): string {
72+
// Логика обработки запроса
73+
return query;
74+
}
75+
}
76+
```
77+
78+
### 3. Интеграция с внешним поисковым сервисом
79+
```typescript
80+
export class Extension {
81+
constructor(private apiKey: string) {}
82+
83+
apply(program: Build) {
84+
getBaseHooks(program).BeforeAnyRun.tap('MySearchExtension', (run) => {
85+
const searchHooks = getSearchHooks(run.search);
86+
87+
searchHooks.Search.tapPromise('MySearchExtension', async (query, options) => {
88+
// Использование внешнего поискового сервиса
89+
const results = await this.externalSearch(query, options);
90+
return results;
91+
});
92+
});
93+
}
94+
95+
private async externalSearch(query: string, options: SearchOptions) {
96+
// Логика интеграции с внешним сервисом
97+
return results;
98+
}
99+
}
100+
```
101+
102+
## Конфигурация
103+
104+
Search Service поддерживает следующие параметры конфигурации:
105+
106+
```typescript
107+
interface SearchConfig {
108+
// Тип поисковой системы
109+
type: 'local' | 'algolia' | 'custom';
110+
111+
// Параметры для локального поиска
112+
local?: {
113+
// Настройки lunr.js
114+
lunr?: {
115+
fields: string[];
116+
boost: Record<string, number>;
117+
};
118+
};
119+
120+
// Параметры для Algolia
121+
algolia?: {
122+
appId: string;
123+
apiKey: string;
124+
indexName: string;
125+
};
126+
127+
// Параметры для пользовательской поисковой системы
128+
custom?: {
129+
// Пользовательские настройки
130+
};
131+
}
132+
```
133+
134+
## Best Practices
135+
136+
1. **Оптимизация индекса**
137+
- Используйте только необходимые поля для индексации
138+
- Настраивайте веса полей в соответствии с их важностью
139+
- Регулярно обновляйте индекс при изменении контента
140+
141+
2. **Обработка запросов**
142+
- Реализуйте кэширование результатов поиска
143+
- Обрабатывайте ошибки поиска
144+
- Предоставляйте fallback механизмы
145+
146+
3. **Интеграция с внешними сервисами**
147+
- Используйте асинхронные операции для внешних запросов
148+
- Реализуйте механизм повторных попыток
149+
- Обрабатывайте ограничения API
150+
151+
## См. также
152+
153+
- [TOC Service](./toc-service.md) - для работы со структурой документации
154+
- [Markdown Service](./markdown-service.md) - для обработки контента
155+
- [Leading Service](./leading-service.md) - для работы со страницами оглавления

0 commit comments

Comments
 (0)