Skip to content

template.md

Latest commit

 

History

History
798 lines (641 loc) · 31.1 KB

template.md

File metadata and controls

798 lines (641 loc) · 31.1 KB

Template for ioBroker Adapter Copilot Instructions

This is the template file that should be copied to your ioBroker adapter repository as .github/copilot-instructions.md.

How to Use This Template

Prerequisites: Ensure you have GitHub Copilot already set up and working in your repository before using this template. If you need help with basic setup, see the Prerequisites & Setup Guide in the main repository.

  1. Copy this entire content
  2. Save it as .github/copilot-instructions.md in your adapter repository
  3. Customize the sections marked with [CUSTOMIZE] if needed
  4. Commit the file to enable GitHub Copilot integration

Note: If downloading via curl, use the sed command to remove the template comment block:

curl -o .github/copilot-instructions.md https://raw.githubusercontent.com/DrozmotiX/ioBroker-Copilot-Instructions/main/template.md
sed -i '/^<!--$/,/^-->$/d' .github/copilot-instructions.md

ioBroker Adapter Development with GitHub Copilot

Version: 0.4.0 Template Source: https://github.com/DrozmotiX/ioBroker-Copilot-Instructions

This file contains instructions and best practices for GitHub Copilot when working on ioBroker adapter development.

Project Context

You are working on an ioBroker adapter. ioBroker is an integration platform for the Internet of Things, focused on building smart home and industrial IoT solutions. Adapters are plugins that connect ioBroker to external systems, devices, or services.

[CUSTOMIZE: Add specific context about your adapter's purpose, target devices/services, and unique requirements]

Testing

Unit Testing

  • Use Jest as the primary testing framework for ioBroker adapters
  • Create tests for all adapter main functions and helper methods
  • Test error handling scenarios and edge cases
  • Mock external API calls and hardware dependencies
  • For adapters connecting to APIs/devices not reachable by internet, provide example data files to allow testing of functionality without live connections
  • Example test structure: 
    describe('AdapterName', () => {
  let adapter;
  
  beforeEach(() => {
    // Setup test adapter instance
  });
  
  test('should initialize correctly', () => {
    // Test adapter initialization
  });
});

Integration Testing

IMPORTANT: Use the official @iobroker/testing framework for all integration tests. This is the ONLY correct way to test ioBroker adapters.

Official Documentation: https://github.com/ioBroker/testing

Framework Structure

Integration tests MUST follow this exact pattern:

const path = require('path');
const { tests } = require('@iobroker/testing');

// Define test coordinates or configuration
const TEST_COORDINATES = '52.520008,13.404954'; // Berlin
const wait = ms => new Promise(resolve => setTimeout(resolve, ms));

// Use tests.integration() with defineAdditionalTests
tests.integration(path.join(__dirname, '..'), {
    defineAdditionalTests({ suite }) {
        suite('Test adapter with specific configuration', (getHarness) => {
            let harness;

            before(() => {
                harness = getHarness();
            });

            it('should configure and start adapter', function () {
                return new Promise(async (resolve, reject) => {
                    try {
                        harness = getHarness();
                        
                        // Get adapter object using promisified pattern
                        const obj = await harness.objects.getObject('system.adapter.brightsky.0');
                        
                        if (!obj) {
                            return reject(new Error('Adapter object not found'));
                        }

                        // Configure adapter properties
                        Object.assign(obj.native, {
                            position: TEST_COORDINATES,
                            createCurrently: true,
                            createHourly: true,
                            createDaily: true,
                            // Add other configuration as needed
                        });

                        // Set the updated configuration
                        harness.objects.setObject(obj._id, obj);

                        console.log('✅ Step 1: Configuration written, starting adapter...');
                        
                        // Start adapter and wait
                        await harness.startAdapterAndWait();
                        
                        console.log('✅ Step 2: Adapter started');

                        // Wait for adapter to process data
                        const waitMs = 15000;
                        await wait(waitMs);

                        console.log('🔍 Step 3: Checking states after adapter run...');
                        
                        // Get all states created by adapter
                        const stateIds = await harness.dbConnection.getStateIDs('your-adapter.0.*');
                        
                        console.log(`📊 Found ${stateIds.length} states`);

                        if (stateIds.length > 0) {
                            console.log('✅ Adapter successfully created states');
                            
                            // Show sample of created states
                            const allStates = await new Promise((res, rej) => {
                                harness.states.getStates(stateIds, (err, states) => {
                                    if (err) return rej(err);
                                    res(states || []);
                                });
                            });
                            
                            console.log('📋 Sample states created:');
                            stateIds.slice(0, 5).forEach((stateId, index) => {
                                const state = allStates[index];
                                console.log(`   ${stateId}: ${state && state.val !== undefined ? state.val : 'undefined'}`);
                            });
                            
                            await harness.stopAdapter();
                            resolve(true);
                        } else {
                            console.log('❌ No states were created by the adapter');
                            reject(new Error('Adapter did not create any states'));
                        }
                    } catch (error) {
                        reject(error);
                    }
                });
            }).timeout(40000);
        });
    }
});

Testing Both Success AND Failure Scenarios

IMPORTANT: For every "it works" test, implement corresponding "it doesn't work and fails" tests. This ensures proper error handling and validates that your adapter fails gracefully when expected.

// Example: Testing successful configuration
it('should configure and start adapter with valid configuration', function () {
    return new Promise(async (resolve, reject) => {
        // ... successful configuration test as shown above
    });
}).timeout(40000);

// Example: Testing failure scenarios
it('should NOT create daily states when daily is disabled', function () {
    return new Promise(async (resolve, reject) => {
        try {
            harness = getHarness();
            
            console.log('🔍 Step 1: Fetching adapter object...');
            const obj = await new Promise((res, rej) => {
                harness.objects.getObject('system.adapter.your-adapter.0', (err, o) => {
                    if (err) return rej(err);
                    res(o);
                });
            });
            
            if (!obj) return reject(new Error('Adapter object not found'));
            console.log('✅ Step 1.5: Adapter object loaded');

            console.log('🔍 Step 2: Updating adapter config...');
            Object.assign(obj.native, {
                position: TEST_COORDINATES,
                createCurrently: false,
                createHourly: true,
                createDaily: false, // Daily disabled for this test
            });

            await new Promise((res, rej) => {
                harness.objects.setObject(obj._id, obj, (err) => {
                    if (err) return rej(err);
                    console.log('✅ Step 2.5: Adapter object updated');
                    res(undefined);
                });
            });

            console.log('🔍 Step 3: Starting adapter...');
            await harness.startAdapterAndWait();
            console.log('✅ Step 4: Adapter started');

            console.log('⏳ Step 5: Waiting 20 seconds for states...');
            await new Promise((res) => setTimeout(res, 20000));

            console.log('🔍 Step 6: Fetching state IDs...');
            const stateIds = await harness.dbConnection.getStateIDs('your-adapter.0.*');

            console.log(`📊 Step 7: Found ${stateIds.length} total states`);

            const hourlyStates = stateIds.filter((key) => key.includes('hourly'));
            if (hourlyStates.length > 0) {
                console.log(`✅ Step 8: Correctly ${hourlyStates.length} hourly weather states created`);
            } else {
                console.log('❌ Step 8: No hourly states created (test failed)');
                return reject(new Error('Expected hourly states but found none'));
            }

            // Check daily states should NOT be present
            const dailyStates = stateIds.filter((key) => key.includes('daily'));
            if (dailyStates.length === 0) {
                console.log(`✅ Step 9: No daily states found as expected`);
            } else {
                console.log(`❌ Step 9: Daily states present (${dailyStates.length}) (test failed)`);
                return reject(new Error('Expected no daily states but found some'));
            }

            await harness.stopAdapter();
            console.log('🛑 Step 10: Adapter stopped');

            resolve(true);
        } catch (error) {
            reject(error);
        }
    });
}).timeout(40000);

// Example: Testing missing required configuration  
it('should handle missing required configuration properly', function () {
    return new Promise(async (resolve, reject) => {
        try {
            harness = getHarness();
            
            console.log('🔍 Step 1: Fetching adapter object...');
            const obj = await new Promise((res, rej) => {
                harness.objects.getObject('system.adapter.your-adapter.0', (err, o) => {
                    if (err) return rej(err);
                    res(o);
                });
            });
            
            if (!obj) return reject(new Error('Adapter object not found'));

            console.log('🔍 Step 2: Removing required configuration...');
            // Remove required configuration to test failure handling
            delete obj.native.position; // This should cause failure or graceful handling

            await new Promise((res, rej) => {
                harness.objects.setObject(obj._id, obj, (err) => {
                    if (err) return rej(err);
                    res(undefined);
                });
            });

            console.log('🔍 Step 3: Starting adapter...');
            await harness.startAdapterAndWait();

            console.log('⏳ Step 4: Waiting for adapter to process...');
            await new Promise((res) => setTimeout(res, 10000));

            console.log('🔍 Step 5: Checking adapter behavior...');
            const stateIds = await harness.dbConnection.getStateIDs('your-adapter.0.*');

            // Check if adapter handled missing configuration gracefully
            if (stateIds.length === 0) {
                console.log('✅ Adapter properly handled missing configuration - no invalid states created');
                resolve(true);
            } else {
                // If states were created, check if they're in error state
                const connectionState = await new Promise((res, rej) => {
                    harness.states.getState('your-adapter.0.info.connection', (err, state) => {
                        if (err) return rej(err);
                        res(state);
                    });
                });
                
                if (!connectionState || connectionState.val === false) {
                    console.log('✅ Adapter properly failed with missing configuration');
                    resolve(true);
                } else {
                    console.log('❌ Adapter should have failed or handled missing config gracefully');
                    reject(new Error('Adapter should have handled missing configuration'));
                }
            }

            await harness.stopAdapter();
        } catch (error) {
            console.log('✅ Adapter correctly threw error with missing configuration:', error.message);
            resolve(true);
        }
    });
}).timeout(40000);

Advanced State Access Patterns

For testing adapters that create multiple states, use bulk state access methods to efficiently verify large numbers of states:

it('should create and verify multiple states', () => new Promise(async (resolve, reject) => {
    // Configure and start adapter first...
    harness.objects.getObject('system.adapter.tagesschau.0', async (err, obj) => {
        if (err) {
            console.error('Error getting adapter object:', err);
            reject(err);
            return;
        }

        // Configure adapter as needed
        obj.native.someConfig = 'test-value';
        harness.objects.setObject(obj._id, obj);

        await harness.startAdapterAndWait();

        // Wait for adapter to create states
        setTimeout(() => {
            // Access bulk states using pattern matching
            harness.dbConnection.getStateIDs('tagesschau.0.*').then(stateIds => {
                if (stateIds && stateIds.length > 0) {
                    harness.states.getStates(stateIds, (err, allStates) => {
                        if (err) {
                            console.error('❌ Error getting states:', err);
                            reject(err); // Properly fail the test instead of just resolving
                            return;
                        }

                        // Verify states were created and have expected values
                        const expectedStates = ['tagesschau.0.info.connection', 'tagesschau.0.articles.0.title'];
                        let foundStates = 0;
                        
                        for (const stateId of expectedStates) {
                            if (allStates[stateId]) {
                                foundStates++;
                                console.log(`✅ Found expected state: ${stateId}`);
                            } else {
                                console.log(`❌ Missing expected state: ${stateId}`);
                            }
                        }

                        if (foundStates === expectedStates.length) {
                            console.log('✅ All expected states were created successfully');
                            resolve();
                        } else {
                            reject(new Error(`Only ${foundStates}/${expectedStates.length} expected states were found`));
                        }
                    });
                } else {
                    reject(new Error('No states found matching pattern tagesschau.0.*'));
                }
            }).catch(reject);
        }, 20000); // Allow more time for multiple state creation
    });
})).timeout(45000);

Key Integration Testing Rules

  1. NEVER test API URLs directly - Let the adapter handle API calls
  2. ALWAYS use the harness - getHarness() provides the testing environment
  3. Configure via objects - Use harness.objects.setObject() to set adapter configuration
  4. Start properly - Use harness.startAdapterAndWait() to start the adapter
  5. Check states - Use harness.states.getState() to verify results
  6. Use timeouts - Allow time for async operations with appropriate timeouts
  7. Test real workflow - Initialize → Configure → Start → Verify States

Workflow Dependencies

Integration tests should run ONLY after lint and adapter tests pass:

integration-tests:
  needs: [check-and-lint, adapter-tests]
  runs-on: ubuntu-latest
  steps:
    - name: Run integration tests
      run: npx mocha test/integration-*.js --exit

What NOT to Do

❌ Direct API testing: axios.get('https://api.example.com') ❌ Mock adapters: new MockAdapter()
❌ Direct internet calls in tests ❌ Bypassing the harness system

What TO Do

✅ Use @iobroker/testing framework ✅ Configure via harness.objects.setObject() ✅ Start via harness.startAdapterAndWait() ✅ Test complete adapter lifecycle ✅ Verify states via harness.states.getState() ✅ Allow proper timeouts for async operations

API Testing with Credentials

For adapters that connect to external APIs requiring authentication, implement comprehensive credential testing:

Password Encryption for Integration Tests

When creating integration tests that need encrypted passwords (like those marked as encryptedNative in io-package.json):

  1. Read system secret: Use harness.objects.getObjectAsync("system.config") to get obj.native.secret
  2. Apply XOR encryption: Implement the encryption algorithm: 
    async function encryptPassword(harness, password) {
    const systemConfig = await harness.objects.getObjectAsync("system.config");
    if (!systemConfig || !systemConfig.native || !systemConfig.native.secret) {
        throw new Error("Could not retrieve system secret for password encryption");
    }
    
    const secret = systemConfig.native.secret;
    let result = '';
    for (let i = 0; i < password.length; ++i) {
        result += String.fromCharCode(secret[i % secret.length].charCodeAt(0) ^ password.charCodeAt(i));
    }
    return result;
}
  3. Store encrypted password: Set the encrypted result in adapter config, not the plain text
  4. Result: Adapter will properly decrypt and use credentials, enabling full API connectivity testing

Demo Credentials Testing Pattern

  • Use provider demo credentials when available (e.g., demo@api-provider.com / demo)
  • Create separate test file (e.g., test/integration-demo.js) for credential-based tests
  • Add npm script: "test:integration-demo": "mocha test/integration-demo --exit"
  • Implement clear success/failure criteria with recognizable log messages
  • Expected success pattern: Look for specific adapter initialization messages
  • Test should fail clearly with actionable error messages for debugging

Enhanced Test Failure Handling

it("Should connect to API with demo credentials", async () => {
    // ... setup and encryption logic ...
    
    const connectionState = await harness.states.getStateAsync("adapter.0.info.connection");
    
    if (connectionState && connectionState.val === true) {
        console.log("✅ SUCCESS: API connection established");
        return true;
    } else {
        throw new Error("API Test Failed: Expected API connection to be established with demo credentials. " +
            "Check logs above for specific API errors (DNS resolution, 401 Unauthorized, network issues, etc.)");
    }
}).timeout(120000); // Extended timeout for API calls

README Updates

Required Sections

When updating README.md files, ensure these sections are present and well-documented:

  1. Installation - Clear npm/ioBroker admin installation steps
  2. Configuration - Detailed configuration options with examples
  3. Usage - Practical examples and use cases
  4. Changelog - Version history and changes (use "## WORK IN PROGRESS" section for ongoing changes following AlCalzone release-script standard)
  5. License - License information (typically MIT for ioBroker adapters)
  6. Support - Links to issues, discussions, and community support

Documentation Standards

  • Use clear, concise language
  • Include code examples for configuration
  • Add screenshots for admin interface when applicable
  • Maintain multilingual support (at minimum English and German)
  • When creating PRs, add entries to README under "## WORK IN PROGRESS" section following ioBroker release script standard
  • Always reference related issues in commits and PR descriptions (e.g., "solves #xx" or "fixes #xx")

Mandatory README Updates for PRs

For every PR or new feature, always add a user-friendly entry to README.md:

  • Add entries under ## **WORK IN PROGRESS** section before committing
  • Use format: * (author) **TYPE**: Description of user-visible change
  • Types: NEW (features), FIXED (bugs), ENHANCED (improvements), TESTING (test additions), CI/CD (automation)
  • Focus on user impact, not technical implementation details
  • Example: * (DutchmanNL) **FIXED**: Adapter now properly validates login credentials instead of always showing "credentials missing"

Documentation Workflow Standards

  • Mandatory README updates: Establish requirement to update README.md for every PR/feature
  • Standardized documentation: Create consistent format and categories for changelog entries
  • Enhanced development workflow: Integrate documentation requirements into standard development process

Changelog Management with AlCalzone Release-Script

Follow the AlCalzone release-script standard for changelog management:

Format Requirements

  • Always use ## **WORK IN PROGRESS** as the placeholder for new changes
  • Add all PR/commit changes under this section until ready for release
  • Never modify version numbers manually - only when merging to main branch
  • Maintain this format in README.md or CHANGELOG.md:
# Changelog

<!--
  Placeholder for the next version (at the beginning of the line):
  ## **WORK IN PROGRESS**
-->

## **WORK IN PROGRESS**

-   Did some changes
-   Did some more changes

## v0.1.0 (2023-01-01)
Initial release

Workflow Process

  • During Development: All changes go under ## **WORK IN PROGRESS**
  • For Every PR: Add user-facing changes to the WORK IN PROGRESS section
  • Before Merge: Version number and date are only added when merging to main
  • Release Process: The release-script automatically converts the placeholder to the actual version

Change Entry Format

Use this consistent format for changelog entries:

  • - (author) **TYPE**: User-friendly description of the change
  • Types: NEW (features), FIXED (bugs), ENHANCED (improvements)
  • Focus on user impact, not technical implementation details
  • Reference related issues: "fixes #XX" or "solves #XX"

Example Entry

## **WORK IN PROGRESS**

- (DutchmanNL) **FIXED**: Adapter now properly validates login credentials instead of always showing "credentials missing" (fixes #25)
- (DutchmanNL) **NEW**: Added support for device discovery to simplify initial setup

Dependency Updates

Package Management

  • Always use npm for dependency management in ioBroker adapters
  • Keep dependencies minimal and focused
  • Regularly update dependencies to latest stable versions
  • Use npm audit to check for security vulnerabilities
  • Before committing, ensure package.json and package-lock.json are in sync by running npm install

Dependency Best Practices

  • Prefer built-in Node.js modules when possible
  • Use @iobroker/adapter-core for adapter base functionality
  • Avoid deprecated packages
  • Document any specific version requirements

JSON-Config Admin Instructions

Configuration Schema

When creating admin configuration interfaces:

  • Use JSON-Config format for modern ioBroker admin interfaces
  • Provide clear labels and help text for all configuration options
  • Include input validation and error messages
  • Group related settings logically
  • Example structure: 
    {
  "type": "panel",
  "items": {
    "host": {
      "type": "text",
      "label": "Host address",
      "help": "IP address or hostname of the device"
    }
  }
}

Admin Interface Guidelines

  • Use consistent naming conventions
  • Provide sensible default values
  • Include validation for required fields
  • Add tooltips for complex configuration options
  • Ensure translations are available for all supported languages (minimum English and German)
  • Write end-user friendly labels and descriptions, avoiding technical jargon where possible

Best Practices for Dependencies

HTTP Client Libraries

  • Preferred: Use native fetch API (available in Node.js 18+)
  • Alternative: Use node-fetch for older Node.js versions
  • Avoid: axios unless specific features are required (reduces bundle size)

Example with fetch:

try {
  const response = await fetch('https://api.example.com/data');
  if (!response.ok) {
    throw new Error(`HTTP ${response.status}: ${response.statusText}`);
  }
  const data = await response.json();
} catch (error) {
  this.log.error(`API request failed: ${error.message}`);
}

Other Dependency Recommendations

  • Logging: Use adapter built-in logging (this.log.*)
  • Scheduling: Use adapter built-in timers and intervals
  • File operations: Use Node.js fs/promises for async file operations
  • Configuration: Use adapter config system rather than external config libraries

Error Handling

Adapter Error Patterns

  • Always catch and log errors appropriately
  • Use adapter log levels (error, warn, info, debug)
  • Provide meaningful, user-friendly error messages that help users understand what went wrong
  • Handle network failures gracefully
  • Implement retry mechanisms where appropriate
  • Always clean up timers, intervals, and other resources in the unload() method

Example Error Handling:

try {
  await this.connectToDevice();
} catch (error) {
  this.log.error(`Failed to connect to device: ${error.message}`);
  this.setState('info.connection', false, true);
  // Implement retry logic if needed
}

Timer and Resource Cleanup:

// In your adapter class
private connectionTimer?: NodeJS.Timeout;

async onReady() {
  this.connectionTimer = setInterval(() => {
    this.checkConnection();
  }, 30000);
}

onUnload(callback) {
  try {
    // Clean up timers and intervals
    if (this.connectionTimer) {
      clearInterval(this.connectionTimer);
      this.connectionTimer = undefined;
    }
    // Close connections, clean up resources
    callback();
  } catch (e) {
    callback();
  }
}

Code Style and Standards

  • Follow JavaScript/TypeScript best practices
  • Use async/await for asynchronous operations
  • Implement proper resource cleanup in unload() method
  • Use semantic versioning for adapter releases
  • Include proper JSDoc comments for public methods

CI/CD and Testing Integration

GitHub Actions for API Testing

For adapters with external API dependencies, implement separate CI/CD jobs:

# Tests API connectivity with demo credentials (runs separately)
demo-api-tests:
  if: contains(github.event.head_commit.message, '[skip ci]') == false
  
  runs-on: ubuntu-22.04
  
  steps:
    - name: Checkout code
      uses: actions/checkout@v4
      
    - name: Use Node.js 20.x
      uses: actions/setup-node@v4
      with:
        node-version: 20.x
        cache: 'npm'
        
    - name: Install dependencies
      run: npm ci
      
    - name: Run demo API tests
      run: npm run test:integration-demo

CI/CD Best Practices

  • Run credential tests separately from main test suite
  • Use ubuntu-22.04 for consistency
  • Don't make credential tests required for deployment
  • Provide clear failure messages for API connectivity issues
  • Use appropriate timeouts for external API calls (120+ seconds)

Package.json Script Integration

Add dedicated script for credential testing:

{
  "scripts": {
    "test:integration-demo": "mocha test/integration-demo --exit"
  }
}

Practical Example: Complete API Testing Implementation

Here's a complete example based on lessons learned from the Discovergy adapter:

test/integration-demo.js

const path = require("path");
const { tests } = require("@iobroker/testing");

// Helper function to encrypt password using ioBroker's encryption method
async function encryptPassword(harness, password) {
    const systemConfig = await harness.objects.getObjectAsync("system.config");
    
    if (!systemConfig || !systemConfig.native || !systemConfig.native.secret) {
        throw new Error("Could not retrieve system secret for password encryption");
    }
    
    const secret = systemConfig.native.secret;
    let result = '';
    for (let i = 0; i < password.length; ++i) {
        result += String.fromCharCode(secret[i % secret.length].charCodeAt(0) ^ password.charCodeAt(i));
    }
    
    return result;
}

// Run integration tests with demo credentials
tests.integration(path.join(__dirname, ".."), {
    defineAdditionalTests({ suite }) {
        suite("API Testing with Demo Credentials", (getHarness) => {
            let harness;
            
            before(() => {
                harness = getHarness();
            });

            it("Should connect to API and initialize with demo credentials", async () => {
                console.log("Setting up demo credentials...");
                
                if (harness.isAdapterRunning()) {
                    await harness.stopAdapter();
                }
                
                const encryptedPassword = await encryptPassword(harness, "demo_password");
                
                await harness.changeAdapterConfig("your-adapter", {
                    native: {
                        username: "demo@provider.com",
                        password: encryptedPassword,
                        // other config options
                    }
                });

                console.log("Starting adapter with demo credentials...");
                await harness.startAdapter();
                
                // Wait for API calls and initialization
                await new Promise(resolve => setTimeout(resolve, 60000));
                
                const connectionState = await harness.states.getStateAsync("your-adapter.0.info.connection");
                
                if (connectionState && connectionState.val === true) {
                    console.log("✅ SUCCESS: API connection established");
                    return true;
                } else {
                    throw new Error("API Test Failed: Expected API connection to be established with demo credentials. " +
                        "Check logs above for specific API errors (DNS resolution, 401 Unauthorized, network issues, etc.)");
                }
            }).timeout(120000);
        });
    }
});

[CUSTOMIZE: Add any adapter-specific coding standards or patterns here]