Skip to content

Commit

Permalink
Lets add docusaurus
Browse files Browse the repository at this point in the history
  • Loading branch information
rubenvdlinde committed Feb 8, 2025
1 parent 12069a5 commit 8a7ea7d
Show file tree
Hide file tree
Showing 6 changed files with 455 additions and 11 deletions.
59 changes: 59 additions & 0 deletions .github/workflows/documentation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Documentation

on:
push:
branches:
- documentation
pull_request:
branches:
- documentation

jobs:
deploy:
name: Deploy Documentation
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: 18
cache: npm

- name: Install dependencies
run: |
cd website
npm ci
- name: Build website
run: |
cd website
npm run build
# Deploy to GitHub Pages
- name: Deploy to GitHub Pages
uses: peaceiris/actions-gh-pages@v3
if: github.ref == 'refs/heads/documentation'
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: ./website/build
user_name: github-actions[bot]
user_email: 41898282+github-actions[bot]@users.noreply.github.com

# Notify on failure
- name: Notify on failure
if: failure()
uses: actions/github-script@v6
with:
script: |
github.rest.issues.create({
owner: context.repo.owner,
repo: context.repo.repo,
title: 'Documentation build failed',
body: 'The documentation build failed. Please check the workflow logs.'
})
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Open Registers

Open Registers provides the ability to work with objects based on [`schema.json`](https://json-schema.org/).
Open Registers provides a way to quicly build and deploy standardized registers based on [NLGov REST API Design Rules](https://logius-standaarden.github.io/API-Design-Rules/) and [Common Ground Principles](https://common-ground.nl/common-ground-principes/). It is based on based on the concepts of defining object types in [`schema.json`](https://json-schema.org/) and storing objects in configurable source.

## What is Open Registers?

Expand All @@ -12,7 +12,7 @@ Registers provide APIs for consumption.

Registers can also apply additional logic to objects, such as validation that is not applicable through the [`schema.json`](https://json-schema.org/) format.

## Features
## Key Features

- 📦 **Object Management**: Work with objects based on [`schema.json`](https://json-schema.org/).
- 🗂️ **Register System**: Manage collections of object types.
Expand Down Expand Up @@ -60,7 +60,7 @@ Open Register makes these principles accessible to any organization by providing

| Feature | Description | Benefits |
|---------|-------------|-----------|
| 💾 [Object Storage](docs/object-storage.md) | Flexible storage backend selection per register | Storage flexibility, system integration, scalability |
| 💾 [Storing Objects](docs/storing-objects.md) | Configure how and where register data is stored | Storage flexibility, system integration, scalability |
| 📝 [Audit Trails](docs/audit-trails.md) | Complete history of all object changes | Compliance, accountability, change tracking |
|[Time Travel](docs/time-travel.md) | View and restore previous object states | Data recovery, historical analysis, version control |
| 🔒 [Object Locking](docs/object-locking.md) | Prevent concurrent modifications | Data integrity, process management, conflict prevention |
Expand All @@ -75,7 +75,7 @@ Open Register makes these principles accessible to any organization by providing
|[Elasticsearch](docs/elasticsearch.md) | Advanced search and analytics capabilities | Performance, insights, complex queries |
| 📋 [Schema Import & Sharing](docs/schema-import.md) | Import schemas from Schema.org, OAS, GGM, and share via Open Catalogi | Standards compliance, reuse, collaboration |
| 🔔 [Events & Webhooks](docs/events.md) | React to object changes with events and webhooks | Integration, automation, real-time updates |
| 🔄 [Data Extension](docs/data-extension.md) | Automatically include related entities in responses | Efficient data retrieval, reduced API calls, complete context |
| 🔄 [Data Extension](docs/data-extension.md) | Include related entities in responses using _extend | Reduced API calls, complete context, efficient data retrieval |
| ✂️ [Data Filtering](docs/data-filtering.md) | Select specific properties to return | Data minimalization, GDPR compliance, efficient responses |
| 🔍 [Advanced Search](docs/advanced-search.md) | Filter objects using flexible property-based queries | Precise filtering, complex conditions, efficient results |
| 🗑️ [Object Deletion](docs/object-deletion.md) | Soft deletion with retention and recovery | Data safety, compliance, lifecycle management |
Expand Down
103 changes: 96 additions & 7 deletions docs/data-extension.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,104 @@
# Data Extension (_extend)

Data Extension allows you to automatically include related entities in API responses, reducing the need for multiple API calls and providing complete context in a single request.
Data Extension allows you to automatically include related entities in API responses, reducing the need for multiple API calls and providing complete context in a single request. This is useful when you need to retrieve related data for a specific object or collection an lowers the number of API calls needed therby reducing the load on the server and improving performence client side.

The extend patern is based was orginally developed for the [Open Catalogi](https://opencatalogi.org) project and is now available in the ObjectStore API. Its baed on the extend functionality of [Zaak gericht werken] but brought in line with p[NLGov REST API Design Rules](https://logius-standaarden.github.io/API-Design-Rules/) by adding a _ prefix to the parameter


## Overview

The _extend parameter provides:
- Automatic inclusion of related entities
- Support for nested relations
- Resolution of both ID and URL references
- Configurable depth limits
- Circular reference handling
The extension system provides:
- Automatic inclusion of related objects
- Nested object resolution
- Circular reference protection
- Depth limiting
- Performance optimization

## Extension Patterns

### Basic Extension
Extend a single property:
- `?_extend=author` - Include full author object
- `?_extend=category` - Include full category object
- `?_extend=files` - Include file metadata

### Nested Extension
Extend nested properties:
- `?_extend=author.organization` - Include author with their organization
- `?_extend=department.employees` - Include department with all employees
- `?_extend=project.tasks.assignee` - Include project with tasks and their assignees

### Multiple Extensions
Combine multiple extensions:
- `?_extend=author,category,comments` - Include multiple related objects
- `?_extend=files,metadata,relations` - Include all related data
- `?_extend=all` - Include all possible relations on the root object

## Extension Logic

1. **Depth Control**
- Maximum depth of 3 levels
- Prevents infinite recursion
- Optimizes response size
- Manages performance

2. **Circular Detection**
- Identifies circular references
- Prevents infinite loops
- Maintains data integrity
- Optimizes response

3. **Type Handling**
- Single object relations
- Array of objects
- File references
- External references

## Performance Considerations

1. **Query Optimization**
- Efficient database queries
- Batch loading
- Cache utilization
- Resource management

2. **Response Size**
- Selective extension
- Depth limiting
- Data filtering
- Compression

3. **Caching**
- Response caching
- Relation caching
- Cache invalidation
- Cache optimization

## Best Practices

1. **Extension Selection**
- Request only needed relations
- Consider response size
- Use appropriate depth
- Plan for performance

2. **Query Design**
- Use specific extensions
- Combine related requests
- Optimize query patterns
- Consider caching

3. **Error Handling**
- Handle missing relations
- Validate extension paths
- Manage timeouts
- Provide fallbacks

## Related Features

- [Object Relations](object-relations.md) - Base for extensions
- [Data Selection](data-selection.md) - Combine with property selection
- [Content Search](content-search.md) - Search within extended data

## Extension Types

Expand Down
148 changes: 148 additions & 0 deletions docs/storing-objects.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Storing Objects

Open Register provides flexible storage options for objects, allowing organizations to store their data in various backends while maintaining a consistent interface.

## Overview

The storage system supports:
- Multiple storage backends
- Transparent data access
- Unified querying interface
- Consistent audit logging
- Schema validation across stores

## Storage Options

### Nextcloud Internal Store (Default)
Default storage using Nextcloud's database:
- Objects stored as JSON in MariaDB/MySQL
- Full CRUD operations
- Built-in versioning
- Automatic audit trails
- Integrated with Nextcloud permissions

### Relational Databases
Traditional SQL databases:
- MariaDB/MySQL
- PostgreSQL
- JSON column types
- SQL query optimization
- Transaction support

### Document Stores
NoSQL databases optimized for JSON documents:
- MongoDB support
- Native JSON storage
- Schema-less flexibility
- High performance queries
- Scalable architecture

### EAV (Entity-Attribute-Value)
Integration with systems like "Uit Betrouwbare Bron":
- Dynamic attribute storage
- Flexible schema support
- Legacy system compatibility
- Custom attribute mapping
- Historical data support

## Storage Configuration

Each register defines:
- Storage backend type
- Connection details
- Schema mapping
- Query optimization
- Cache settings

## Implementation Details

### Internal Store
```sql
CREATE TABLE objects (
id INT PRIMARY KEY,
uuid VARCHAR(255),
object JSON,
metadata JSON
)
```

### Document Store
```json
{
"_id": "object_id",
"uuid": "unique_id",
"object": {
"property": "value"
},
"metadata": {
"created": "timestamp"
}
}
```

### Relational Store
```sql
CREATE TABLE objects (
id INT PRIMARY KEY,
uuid VARCHAR(255),
data JSONB,
created_at TIMESTAMP
)
```

### EAV Store
```sql
CREATE TABLE object_values (
object_id INT,
attribute VARCHAR(255),
value TEXT,
PRIMARY KEY (object_id, attribute)
)
```

## Key Benefits

1. **Flexibility**
- Choose optimal storage per register
- Match existing infrastructure
- Scale independently
- Custom implementations

2. **Integration**
- Legacy system support
- Multiple databases
- External systems
- Distributed storage

3. **Performance**
- Storage optimization
- Query efficiency
- Caching strategies
- Scalability options

## Best Practices

1. **Storage Selection**
- Consider data structure
- Evaluate query patterns
- Assess volume requirements
- Plan for growth

2. **Configuration**
- Optimize connections
- Set up caching
- Configure backups
- Monitor performance

3. **Management**
- Regular maintenance
- Performance monitoring
- Security updates
- Backup verification

## Related Features

- [Schema Validation](schema-validation.md) - Validate across storage types
- [Audit Trails](audit-trails.md) - Track changes in all stores
- [Content Search](content-search.md) - Search across stores
- [Object Relations](object-relations.md) - Relations across stores
Loading

0 comments on commit 8a7ea7d

Please sign in to comment.