Skip to content

Releases: streetsidesoftware/cspell

v9.7.0

23 Feb 14:52

Choose a tag to compare

Features

feat: Substitution Part 4 - enable substitutions during document check (#8630)

feat: Substitution Part 4 - enable substitutions during document check (#8630)

Pull request overview

This PR enables substitution-based text transformations during document spell checking, allowing configured text patterns to be replaced before validation occurs.

Changes:

  • Added substitution transformer support to the text validation pipeline
  • Enhanced SubstitutionTransformer to handle both string and MappedText inputs with source map merging
  • Refactored settingsToValidateOptions to explicitly map all validation option fields

feat: Substitution Part 3 (#8616)

feat: Substitution Part 3 (#8616)

Pull request overview

This PR updates the SourceMap encoding/handling to use relative span-length pairs (instead of absolute offset pairs) so transformations can be composed more reliably, while updating affected parsers, mappers, and tests across the monorepo.

Changes:

  • Redefines SourceMap documentation/usage to represent relative [srcSpanLen, dstSpanLen] segments (with non-linear segment semantics).
  • Introduces new SourceMap utilities in cspell-lib and refactors TextMap/mapping code and tests to use them.
  • Updates TypeScript grammar parsing/mappers to emit relative maps and adjusts fixtures/tests accordingly.

[!CAUTION]
Internal breaking: SourceMaps are now span lengths instead of being offsets. This makes them invariant during translation, making concatenation, and slicing much easier.
A minor version change was chosen instead of a major version change since it was only used with the experimental TypeScript parser.


feat: Substitution Part 2 (#8599)

feat: Substitution Part 2 (#8599)

Pull request overview

This pull request is part of a "Substitution" feature series (Part 2), focusing on reorganizing type definitions and adding utility functions for text transformation and source mapping. The main purpose is to consolidate Parser-related types and introduce new mapping utilities needed for the substitution feature.

Changes:

  • Reorganizes type definitions by moving MappedText, Range, and SourceMap from root-level exports to the Parser directory structure
  • Adds new source map manipulation functions (mapOffsetToSource, mapRangeToSource, mergeSourceMaps) to support text transformation workflows
  • Introduces API declaration file generation for documentation purposes via separate tsdown build configurations

feat: Support text substitutions prior to spell checking - part 1 (#8592)

feat: Support text substitutions prior to spell checking - part 1 (#8592)

Text Substitution Support Part 1

Steps towards supporting text substitutions prior to spell checking by introducing a Transform layer (range/mapping + substitution) and wiring it into document validation configuration.

It is not yet added to the normal spell checking flow. That will happen in a later PR.

Changes:

  • Extend types + JSON schema to support substitution definitions keyed by name and export Range for reuse.
  • Add cspell-lib Transform module (TextRange/TextMap/parsedText mapping + SubstitutionTransformer) and update validators/tests to use it.
  • Export GTrie/GTrieNode from cspell-trie-lib to support efficient substitution matching.

Related Issues


Fixes

fix: Publish 9.7 (#8634)

fix: Publish 9.7 (#8634)

Fix issue with test packages having the same name preventing publishing.


fix: Support ESLint 10 (#8620)

fix: Support ESLint 10 (#8620)


fix: Prepare to support substitutions (#8584)

fix: Prepare to support substitutions (#8584)

Pull request overview

Adds initial type/schema/plumbing in cspell-types and cspell-lib to support upcoming β€œsubstitutions” configuration fields (definitions + per-document substitutions) throughout config merging, exported field lists, and JSON schema.

Changes:

  • Introduces Substitutions types and wires them into CSpellSettingsDef and the public @cspell/cspell-types index exports.
  • Adds substitutionDefinitions / substitutions to config field registries and config merge behavior.
  • Updates JSON schemas (package + root) and adjusts API snapshot / settings sanitization to recognize the new fields.

fix: cspell-rpc - reduce the size of an RPC result. (#8574)

fix: cspell-rpc - reduce the size of an RPC result. (#8574)

Pull request overview

This PR updates the cspell-lib RPC spell-checking surface to reduce payload sizes by introducing RPC-specific result/issue types and returning a slimmer result from spellCheckDocumentRPC.

Changes:

  • Switch spellCheckDocumentRPC to return a new SpellCheckFileResultRPC that omits large fields (notably document text and several result fields).
  • Introduce ValidationResultRPC / ValidationIssueRPC and a mapper (toValidationIssueRPC) to reduce issue payload.
  • Update the cspell-rpc client/server communication test expectations for the new RPC result shape.

fix: Add convertToBtrie method to trie-lib (#8562)

fix: Add convertToBtrie method to trie-lib (#8562)

Pull request overview

This PR adds utilities to decode dictionary/trie files (including .gz) and convert them into the binary .btrie format, then integrates that conversion into the dictionary bundler plugin so bundled dictionaries can be inlined as (optionally compressed) BTrie data for faster load times.

Changes:

  • Add decodeFile / convertToBTrie APIs to cspell-trie-lib and export them from the public index.
  • Add encodeITrieToBTrie convenience encoder for converting an ITrie directly to BTrie bytes.
  • Extend cspell-dictionary-bundler-plugin to optionally convert/inline dictionaries as .btrie (and optionally gzip them), with updated snapshots/tests.

refactor: Use standard dispose (#8559)

refactor: Use standard dispose (#8559)


refactor: Experiment with bundling default dictionaries (#8556)

refactor: Experiment with bundling default dictionaries (#8556)

Pull request overview

This PR experiments with bundling default dictionaries for CSpell, adding infrastructure to bundle dictionary configuration files into JavaScript modules using a custom plugin. The changes introduce a new build pipeline for the cspell-bundled-dicts package but intentionally leave the bundling functionality disabled through configuration.

Changes:

  • Added dictionary bundler plugin integration to cspell-bundled-dicts with tsdown configuration (currently disabled via /no-match/ pattern)
  • Enhanced CSpellDictionaryBundler class with debug logging support and module import resolution
  • Updated package.json exports to include new bundled output format while removing compatibility-related files

fix: Enable reading dictionaries from cspell-vfs (#8553)

fix: Enable reading dictionaries from cspell-vfs (#8553)

Pull request overview

This PR enables dictionaries bundled into the cspell-vfs: virtual filesystem to be registered into the runtime VFS so dictionary loading can read them. It adds fixtures/tests to validate the behavior.

Changes:

  • Register settings.vfs entries into the active VirtualFS during config load/merge.
  • Add a default in-memory cspell-vfs: provider to cspell-io’s createVirtualFS, and export CSPELL_VFS_PROTOCOL.
  • Add bundling/build + fixtures and a docValidator test that loads a bundled config using cspell-vfs: dictionary content.

fix: Work on a bundler for CSpell Dictionaries (#8532)

fix: Work on a bundler for CSpell Dictionaries (#8532)

Pull request overview

This PR introduces a new @cspell/dictionary-bundler-plugin package intended to inline/bundle CSpell dictionary/config assets for consumption across multiple bundlers (Rollup/Vite/Webpack/Rspack/Rolldown/Farm/esbuild), and updates workspace metadata and dependency lockfiles accordingly.

Changes:

  • Added new packages/cspell-dictionary-bundler-plugin package with build config (tsdown), TypeScript config, plugin entrypoints, core bundling logic, and Vitest coverage (fixtures + snapshots).
  • Updated repo-level Node engine requirement and pnpm lockfile to reflect new dependencies and resolutions.
  • Updated workspace and repo dictionary wordlist to recognize new terms used by the package/dependencies.

fix: Add support for MemVfs to cspell-io (#8543)

fix: Add support for MemVfs to cspell-io (#8543)

Pull request overview

This PR refactors cspell-io’s VirtualFS internals (exports/import paths) and introduces foundational pieces for an in-memory VirtualFS provider (β€œMemVfs”), along with new shared error/capability utilities and updated tests.

Changes:

  • Consolidates VirtualFS public exports under src/VirtualFS/index.ts and updates imports across VirtualFS code/tests.
  • Extracts shared VirtualFS concerns into new modules (errors.ts, capabilities.ts, CFileType.ts, CVfsStat.ts) and updates `WrappedProviderFs...
Read more

v9.6.4

04 Feb 08:55

Choose a tag to compare

Fixes

fix: add --no-dictionary option to lint command (#8514)

fix: add --no-dictionary option to lint command (#8514)

Pull request overview

This PR adds the --no-dictionary option to the lint command and refactors the prefixCollect helper function for reuse across commands.

Changes:

  • Moved prefixCollect function from commandTrace.ts to commandHelpers.ts for reuse
  • Added --no-dictionary option to the lint command (hidden, as an alias for --disable-dictionary)
  • Added comprehensive tests for the new option including --dictionary=!words and --no-dictionary=words syntax

fix: Correctly report blocked dictionaries (#8506)

fix: Correctly report blocked dictionaries (#8506)


v9.6.3

02 Feb 18:21

Choose a tag to compare

Fixes

fix: Add `engines` setting (#8491)

fix: Add engines setting (#8491)

Pull request overview

This PR adds a new engines configuration field that allows dictionary authors and configuration authors to specify the minimum version of cspell (or other engines) required for compatibility. This is informational only and does not enforce versioning.

Changes:

  • Added CompatibleEngineVersions interface and SemVersionPredicate type to define engine version requirements
  • Added engines field to FileSettings interface and all related configuration structures
  • Updated JSON schemas to include the new engines definitions
  • Updated snapshot tests to reflect the new configuration field

refactor: consolidate reporting to file. (#8481)

refactor: consolidate reporting to file. (#8481)


fix: Move cspell-lib/rpc into its own package (#8468)

fix: Move cspell-lib/rpc into its own package (#8468)

Pull request overview

This PR refactors the RPC implementation by moving it from packages/cspell-lib/src/rpc into a standalone package @cspell/rpc. This improves modularity and allows the RPC library to be used independently of cspell-lib.

Changes:

  • Created new @cspell/rpc package with complete RPC implementation (client, server, protocol, types, utilities)
  • Updated cspell-lib to import from @cspell/rpc instead of local ./rpc directory
  • Updated workspace configuration and dependency management in pnpm-lock.yaml

fix: Add types for cspell-vfs (#8451)

fix: Add types for cspell-vfs (#8451)

Pull request overview

This PR adds TypeScript type definitions for the CSpell Virtual File System (cspell-vfs) feature, enabling users to declare virtual files in their configuration that can be referenced via cspell-vfs:// URLs.

Changes:

  • Added comprehensive TypeScript type definitions for the CSpell VFS feature
  • Added vfs configuration field to CSpell settings
  • Updated JSON schemas to reflect the new types

Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8492)

fix: Workflow Bot -- Update Dictionaries (main) (#8492)

Update Dictionaries (main)

Summary

 packages/cspell-bundled-dicts/package.json |  8 +++---
 pnpm-lock.yaml                             | 44 ++++++++++++++++++------------
 2 files changed, 31 insertions(+), 21 deletions(-)

v9.6.2

26 Jan 12:12

Choose a tag to compare

Fixes

fix: Conditionally compress and build bTrie (#8437)

fix: Conditionally compress and build bTrie (#8437)

Pull request overview

This PR refactors the compilation process to conditionally compress dictionary files and build bTrie files based on the compression settings. The main changes separate compression and bTrie generation into distinct steps and update dependency path handling to be relative to the checksum file location.

Changes:

  • Separated dictionary compression into a dedicated step that runs after building the dictionary
  • Modified bTrie generation to use the appropriate source file (compressed or uncompressed) based on target configuration
  • Updated dependency path resolution to make paths relative to the checksum file directory instead of the root directory

Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8440)

fix: Workflow Bot -- Update Dictionaries (main) (#8440)

Update Dictionaries (main)

Summary

 packages/cspell-bundled-dicts/package.json |  4 ++--
 pnpm-lock.yaml                             | 24 ++++++++++++------------
 2 files changed, 14 insertions(+), 14 deletions(-)

v9.6.1

26 Jan 05:38

Choose a tag to compare

Fixes

fix: Move performance monitoring into its own package (#8431)

fix: Move performance monitoring into its own package (#8431)

Pull request overview

This PR extracts performance monitoring functionality from multiple packages into a new dedicated package @cspell/cspell-performance-monitor. The new package provides a centralized, conditionally-enabled performance measurement system using the standard Performance API.

Changes:

  • Created a new @cspell/cspell-performance-monitor package with improved performance measurement functions that can be conditionally enabled/disabled
  • Migrated and removed duplicate performance measurement code from cspell-lib, cspell-dictionary, and cspell-trie-lib packages
  • Updated performance measurement calls to use the new disposable pattern (using _ = measurePerf()) that leverages TypeScript's explicit resource management

fix: Add NotifyEmitter (#8423)

fix: Add NotifyEmitter (#8423)

Pull request overview

This pull request adds a new NotifyEmitter class to provide a reusable event notification system based on the Disposable pattern. The PR refactors the RPC server test to use this new event emitter pattern instead of a custom AsyncMessageQueue implementation, resulting in cleaner and more maintainable code.

Changes:

  • Introduces NotifyEmitter class with NotifyEvent pattern for event handling with automatic cleanup via Symbol.dispose
  • Adds helper functions notifyEventToPromise and notifyEventOnce for working with notify events
  • Creates MessagePortEvents wrapper class to adapt MessagePort events to the NotifyEmitter pattern
  • Refactors server.test.ts to use the new event system, removing the custom AsyncMessageQueue implementation
  • Adds result?: unknown field to RPCResponse interface for more consistent type handling
  • Exports new notify-related types and functions through the public API

refactor: Adjust RPC interface to use a config (#8420)

refactor: Adjust RPC interface to use a config (#8420)

Pull request overview

Refactors the RPC client/server APIs (and cspell RPC wrappers) to accept a single configuration object, and updates worker integration and tests accordingly.

Changes:

  • Update RPCClient/RPCServer (and CSpellRPCClient/CSpellRPCServer) constructors/factories to accept { port, ...options } config objects.
  • Adjust RPC tests and worker usage to match the new config-based API.
  • Update cspell-lib package entrypoint to dist/index.* and add src/index.ts re-export.

refactor: Refactor CLI to make it easier to spell check files in parallel (#8387)

refactor: Refactor CLI to make it easier to spell check files in parallel (#8387)

Pull request overview

This PR refactors the CLI file processing architecture to improve processing speed. The main improvements include extracting file processing logic into a dedicated module, adding sequence tracking for file processing, fixing regex lastIndex state sharing issues, and preparing infrastructure for optional worker pool parallelization (currently disabled).

Changes:

  • Extracted file processing logic from lint.ts into a new processFiles.ts module with improved batch processing
  • Added sequence tracking interfaces (FileToProcess, ProcessPrefetchFileResult) to maintain file processing order
  • Fixed regex lastIndex state sharing bugs by creating new RegExp instances before use
  • Added worker pool and RPC infrastructure to cspell-api.ts (currently disabled via feature flags)
  • Updated performance reporting to track both accumulated processing time and total elapsed time

refactor: Refactor settings a bit (#8385)

refactor: Refactor settings a bit (#8385)

Pull request overview

This pull request refactors settings handling in the CSpell codebase. The main focus is on improving the serialization of settings objects by adding circular reference detection and handling to walkToJSONObj, along with introducing new utility functions for working with settings that need to be serialized (e.g., for IPC via MessageChannel).

Changes:

  • Enhanced walkToJSONObj to handle circular references using a WeakMap-based visited tracking system
  • Added toCSpellSettingsWithSourceTrace and toCSpellSettingsWithOutSourceTrace functions to convert settings to serializable formats
  • Refactored processFile to accept and use userSettings parameter instead of directly accessing configInfo.config

refactor: Refactor internal settings (#8382)

refactor: Refactor internal settings (#8382)

Pull request overview

This pull request refactors the internal settings structure by reorganizing code from Models/CSpellSettingsInternalDef.js into a new Settings/internal/ directory structure. This improves code organization by collocating internal settings definitions with other settings-related code.

Changes:

  • Created new Settings/internal/ directory with split files: CSpellSettingsInternalDef.ts, InternalDictionaryDef.ts, and DictionarySettings.ts (with improved private field syntax)
  • Updated all import paths across the codebase from Models/CSpellSettingsInternalDef to Settings/internal/ or Settings/index
  • Added new utility function walkToJSONObj for JSON serialization (with comprehensive tests) and __getOriginalDefinition() method for accessing original dictionary definitions

fix: report errors in spelling result. (#8381)

fix: report errors in spelling result. (#8381)

Pull request overview

This PR enhances error reporting in spell check results by including configuration and dictionary errors directly in the SpellCheckFileResult object. Previously, errors were only extracted from configuration during the lint process; now they're also captured from the spell checking itself.

Changes:

  • Added configErrors and dictionaryErrors fields to SpellCheckFileResult interface
  • Refactored error reporting to extract errors from spell check results instead of re-computing them from configuration
  • Created a new cspell-api module to abstract spell checking implementation
  • Added comprehensive test coverage with fixture files demonstrating error scenarios

fix: Add CSpell Worker Pool (#8379)

fix: Add CSpell Worker Pool (#8379)

Pull request overview

This pull request adds a CSpell Worker Pool implementation to manage multiple spell-checking workers efficiently, along with significant improvements to the RPC infrastructure for better readiness tracking.

Changes:

  • Added CSpellWorkerPool class to manage a pool of spell-checking workers with configurable sizing based on CPU cores
  • Simplified the worker implementation by removing custom status tracking in favor of RPC-based readiness checks
  • Enhanced RPC client/server with ready request/response mechanism, allowing clients to wait for server initialization
  • Renamed Resolver utility class to Future with improved state tracking (added isRejected property)

refactor: Clean Up CSpell Worker (#8372)

refactor: Clean Up CSpell Worker (#8372)


fix: Add CSpellRPCServer and CSpellRPCClient classes (#8366)

fix: Add CSpellRPCServer and CSpellRPCClient classes (#8366)

Pull request overview

This pull request adds RPC (Remote Procedure Call) infrastructure to enable remote spell checking through CSpell. The implementation includes a generic RPC client/server framework, CSpell-specific RPC adapters, and worker thread support for isolated spell checking operations.

Changes:

  • Move the RPC framwork from cspell to cspell-lib
  • Adds complete RPC framework with message-based client/server communication over MessagePort
  • Implements CSpell-specific RPC server and client classes for remote spell checking
  • Introduces worker thread support for isolated spell checking operations
  • Adds utility functions for cloning and sanitizing settings for safe RPC transmission

fix: Work on adding support for Workers (#8363)

fix: Work on adding support for Workers (#8363)

Pull request overview

This pull request adds RPC (Remote Procedure Call) infrastructure to support Workers in the cspell package. The implementation provides a client-server communication framework using MessagePort for inter-process or inter-worker communication.

Changes:

  • Added RPC client and server implementations with support for request/response patterns, cancellation, and error handling
  • Implemented type-safe protocol definitions with automatic Promise wrapping for async operations
  • Created comprehensive test coverage for client, server, and integration scenarios

Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8433)

fix: Workflow Bot -- Update Dictionaries (main) (#8433)

Update Dictionaries (main)

Summary

 .../MartinThoma/LaTeX-examples/report.yaml         |  9 +---
 .../MartinThoma/LaTeX-examples/snapshot.txt        |  7 +--
 packages/cspell-bundled-dicts/package.json         | 10 ++--
 packages/cspell/src/__snapshots__/app.test.ts.snap |  6 +--
 pnpm-lock.yaml                                     | 57 ++++++++++++----------
 5 files changed, 42 insertions(+), 47 deletions(-)

v9.6.0

12 Jan 12:06

Choose a tag to compare

Features

feat: ESLint-plugin -- Distinguish between Forbidden, Misspelled, and Unknown words. (#8337)

feat: ESLint-plugin -- Distinguish between Forbidden, Misspelled, and Unknown words. (#8337)

Pull request overview

This PR updates the eslint-plugin to report different types of spelling issues with improved categorization and messaging. The changes include upgrading ESLint from v8 to v9, introducing new issue severity levels (Forbidden, Misspelled, Unknown, Hint), and adding a report option to control the level of reporting.

Changes:

  • Upgraded ESLint dependency from v8.57.1 to v9.39.2 across all fixtures and packages
  • Introduced new severity level "Misspelled" to distinguish typos with suggestions from truly unknown words
  • Added report option ('all', 'simple', 'typos', 'flagged') to control which types of issues are reported
  • Enhanced error messages to include suggestions directly in the message text

feat: ESLint-plugin -- Add `report` option to be able to control the reporting level (matching the CLI): "all", "simple", "typos", "flagged". (#8273)

feat: ESLint-plugin -- Add report option to be able to control the reporting level (matching the CLI): "all", "simple", "typos", "flagged". (#8273)

Pull request overview

This PR adds support for unknown word reporting options to the cspell ESLint plugin, allowing users to control which types of spelling issues are reported (all unknown words, simple typos, common typos, or only flagged words).

Changes:

  • Added a new report option to the ESLint plugin configuration with values: 'all', 'simple', 'typos', and 'flagged'
  • Implemented mapping from the report option to CSpell's unknownWords setting
  • Added unit tests for the new report filtering functionality

This PR contains the changes required to resolve cspell ESLint plugin: support typos-only reporting (CLI --report typos / unknownWords report-common-typos) #8261


feat: support storing suggestions in trie (#8228)

feat: support storing suggestions in trie (#8228)

Pull request overview

This PR adds support for storing word suggestions directly in trie data structures, enabling dictionaries to provide preferred spelling suggestions for words. The implementation introduces a new suggestionPrefix field (default :) to TrieInfo and a hasPreferredSuggestions characteristic flag.

Key Changes:

  • Adds suggestion storage capability to trie structures with a new suggestionPrefix field
  • Removes the isCaseAware field from TrieInfo (minor breaking change since it was no longer used)
  • Implements suggestion parsing logic in dictionary parsers to handle formats like word:suggestion and word->suggestion

Details

Add support to have suggestion in normal word lists:

Note: : or -> separators can be used in the word list.

Examples:

colour:color # colour is allowed, but color is preferred.
:colour:color # this is just a suggestion to use `color` instead of `colour`.
:colour->color # same as `:colour:color`
!colour:color # `colour` will be flagged and `color` will be suggested as the fix.

You can add more than one suggestion, but auto-fix won't be possible since
the spell checker cannot determine which one you want.

!incase:encase, in case # two suggestions

feat: Add Support reading / writing bTrie files (#8204)

feat: Add Support reading / writing bTrie files (#8204)

Pull request overview

This pull request adds support for reading and writing binary Trie (bTrie) files, which is a more efficient binary format for storing trie data structures. The changes make the cspell-trie-lib package more platform-neutral by replacing Node.js-specific APIs (like Buffer and os.endianness()) with Web standard APIs (Uint8Array, TextEncoder/TextDecoder, and a custom endianness detection function).

Changes:

  • Implemented a new binary format specification system with encoder/decoder classes
  • Added bTrie file format support with proper endianness handling
  • Made cspell-trie-lib platform-neutral by removing Node.js dependencies from production code
  • Added new CLI command btrie to cspell-tools for generating bTrie files
  • Enhanced readers in cspell-tools to handle multiple file formats including bTrie

Fixes

refactor: extract processFile from lint.ts (#8344)

refactor: extract processFile from lint.ts (#8344)


fix: Improve word lookup performance when not found (#8330)

fix: Improve word lookup performance when not found (#8330)

Pull request overview

This PR improves word lookup performance when words are not found by optimizing how the spell checker generates different word forms for dictionary lookups. The key optimization is to avoid expensive Unicode normalization and replacement mapping operations when they're not needed.

Changes:

  • Converted mapWord from required to optional on SpellingDictionary interface, using undefined for dictionaries that don't need word mapping
  • Added early-exit optimization for ASCII words to skip Unicode normalization
  • Added test-before-apply optimization for replacement mappers to avoid expensive operations when the word won't match
  • Enhanced dictionary logging to track cache misses and improved the public API for logging functions
  • Added comprehensive tests including integration test with real German dictionary

When a word is not found, the spell checker tries many different forms to find the word. This can be expensive if a lot of words are not found.


fix: Improve loading time of dictionaries (#8318)

fix: Improve loading time of dictionaries (#8318)

Pull request overview

This pull request aims to improve the loading time of dictionaries by optimizing the internal data structures and sorting algorithms used in the trie builder. The changes focus on:

  • Removing unused timer utility functions and tests
  • Simplifying performance measurement infrastructure
  • Optimizing node sorting by avoiding unnecessary Uint32Array conversions
  • Adding an optional batch sorting feature for dictionary parsing
  • Generalizing types to work with both regular arrays and Uint32Arrays

fix: Reduce loading time of dictionaries. (#8316)

fix: Reduce loading time of dictionaries. (#8316)


fix: Support loading btrie dictionaries in cspell-lib (#8311)

fix: Support loading btrie dictionaries in cspell-lib (#8311)

Pull request overview

This PR adds support for loading btrie (binary trie) dictionaries in cspell-lib, enabling more efficient dictionary storage and loading. The changes also include enhancements to the table utility for better formatting and a refactoring of performance measurement tracking to use a hierarchical structure.

  • Added btrie path resolution and loading support in DictionarySettings and DictionaryLoader
  • Enhanced the table utility with column alignment, indentation, and title support
  • Refactored performance measurements to use a hierarchical tree structure instead of flat key-based tracking
  • Improved performance reporting with color coding and additional metrics

fix: Improve loading performance a bit. (#8309)

fix: Improve loading performance a bit. (#8309)


fix: Prepare tools to generate btrie files. (#8298)

fix: Prepare tools to generate btrie files. (#8298)


fix: Optimize btrie files when building (#8285)

fix: Optimize btrie files when building (#8285)

Have CSpell Tools optimize the Trie while building. We are trying to reduce the size of the file.


fix: Add `btrie` path to dictionary definition (#8284)

fix: Add btrie path to dictionary definition (#8284)

Make it possible for a dictionary to specify a btrie path in addition to the normal path.


fix: Work towards support prefixes when walking ITrie (#8276)

fix: Work towards support prefixes when walking ITrie (#8276)


fix: Fix CStyleHexValue to handle BigInt values. (#8282)

fix: Fix CStyleHexValue to handle BigInt values. (#8282)

Example: 0xffff_ffff_ffffn


refactor: Remove FastTrieBlob (#8267)

refactor: Remove FastTrieBlob (#8267)


fix: Make endian required when encoding a StringTable (#8265)

fix: Make endian required when encoding a StringTable (#8265)


refactor: Removed FastTrieBlob part 1 (#8266)

refactor: Removed FastTrieBlob part 1 (#8266)

FastTrieBlob served it purpose. It is now time for it to be retired since it is too difficult to maintain in parallel with TrieBlob.


fix: Support string prefixes when walking nodes (#8259)

fix: Support string prefixes when walking nodes (#8259)


refactor: Document TrieBlob format (#8256)

refactor: Document TrieBlob format (#8256)

  • Document TrieBlob format
  • Add TribBlob node optimizer to make DAWGs
  • Test out StringTables
  • Remove some dead code

fix: adjust error message (#8249)

fix: adjust error message (#8249)


fix: Add StringTable and refactor BinaryFormat (#8243)

fix: Add StringT...

Read more

v9.5.0

16 Dec 07:56

Choose a tag to compare

Features

feat: Report errors with nested config files (#8155)

feat: Report errors with nested config files (#8155)

fixes: #7837

Errors in nested configuration files were ignored. This lead to unexpected behavior.

Note: this is a fix, but since it can impact existing installs, it is marked as a feature change.


fix: Enable cts config support (#8152)

fix: Enable cts config support (#8152)

Add support for .cts config files.

Update docs with ts, mts, and cts config.


Fixes

fix: Add cspell-tools option keepUncompressed (#8163)

fix: Add cspell-tools option keepUncompressed (#8163)


fix: Improve CSpell Tools (#8160)

fix: Improve CSpell Tools (#8160)

Generate both compressed and non-compressed files when compress: true. This allows cspell-dicts to keep track of changes and have compressed files at the same time.


fix: Make sure `cspell` exports `defineConfig ` for CommonJS (#8156)

fix: Make sure cspell exports defineConfig for CommonJS (#8156)


Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8149)

fix: Workflow Bot -- Update Dictionaries (main) (#8149)

Update Dictionaries (main)

Summary

 .../snapshots/AdaDoom3/AdaDoom3/report.yaml        |   7 +-
 .../snapshots/AdaDoom3/AdaDoom3/snapshot.txt       |   7 +-
 .../snapshots/TheAlgorithms/Python/report.yaml     |   5 +-
 .../snapshots/TheAlgorithms/Python/snapshot.txt    |   5 +-
 .../snapshots/dart-lang/sdk/report.yaml            |   8 +-
 .../snapshots/dart-lang/sdk/snapshot.txt           |   4 +-
 .../snapshots/django/django/report.yaml            |  13 +--
 .../snapshots/django/django/snapshot.txt           |  44 ++++----
 .../iluwatar/java-design-patterns/report.yaml      |   3 +-
 .../iluwatar/java-design-patterns/snapshot.txt     |   3 +-
 .../snapshots/ktaranov/sqlserver-kit/report.yaml   |   4 +-
 .../snapshots/ktaranov/sqlserver-kit/snapshot.txt  |   3 +-
 .../snapshots/neovim/nvim-lspconfig/report.yaml    |   5 +-
 .../snapshots/neovim/nvim-lspconfig/snapshot.txt   |   3 +-
 .../snapshots/slint-ui/slint/report.yaml           |   8 +-
 .../snapshots/slint-ui/slint/snapshot.txt          |   7 +-
 .../snapshots/sveltejs/svelte/report.yaml          |   3 +-
 .../snapshots/sveltejs/svelte/snapshot.txt         |   3 +-
 packages/cspell-bundled-dicts/package.json         |  20 ++--
 pnpm-lock.yaml                                     | 120 +++++++++++----------
 20 files changed, 115 insertions(+), 160 deletions(-)

v9.4.0

01 Dec 19:48

Choose a tag to compare

Features

feat: Add option to skip large files (#8040)

feat: Add option to skip large files (#8040)

New command line option: --max-file-size <value>
The value can be <number>[units], number with optional units.
Supported units:

  • K, KB - value * 1024
  • M, MB - value * 2^20
  • G, GB - value * 2^30

There is also a new setting called maxFileSize. At the moment, it only takes numbers in bytes.

This allows for setting a global value and overrides based upon glob settings.

Note: the command line value will override any value found in the configuration files.


Fixes

fix: turn off gpg when publishing for now (#8104)

fix: turn off gpg when publishing for now (#8104)


fix: Turnoff signing for now (#8102)

fix: Turnoff signing for now (#8102)


fix: Sign tags and commits when publishing (#8096)

fix: Sign tags and commits when publishing (#8096)


fix: Clean up verbose output (#8053)

fix: Clean up verbose output (#8053)


fix: allow string based maxFileSize (#8047)

fix: allow string based maxFileSize (#8047)

  • Support allowing string based maxFileSize in config files.
  • Report the number of skipped files.

Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8091)

fix: Workflow Bot -- Update Dictionaries (main) (#8091)

Update Dictionaries (main)

Summary

 packages/cspell-bundled-dicts/package.json |  2 +-
 pnpm-lock.yaml                             | 11 ++++++++---
 2 files changed, 9 insertions(+), 4 deletions(-)

fix: Workflow Bot -- Update Dictionaries (main) (#8080)

fix: Workflow Bot -- Update Dictionaries (main) (#8080)

Update Dictionaries (main)

Summary

 .../snapshots/django/django/report.yaml            |  25 +--
 .../snapshots/django/django/snapshot.txt           |  20 +-
 .../snapshots/php/php-src/report.yaml              | 237 ++++++++++-----------
 .../snapshots/php/php-src/snapshot.txt             |   7 +-
 packages/cspell-bundled-dicts/package.json         |  10 +-
 pnpm-lock.yaml                                     |  64 +++---
 6 files changed, 163 insertions(+), 200 deletions(-)

fix: Workflow Bot -- Update Dictionaries (main) (#8058)

fix: Workflow Bot -- Update Dictionaries (main) (#8058)

Update Dictionaries (main)

Summary

 integration-tests/snapshots/slint-ui/slint/report.yaml  | 11 +++--------
 integration-tests/snapshots/slint-ui/slint/snapshot.txt |  7 +------
 packages/cspell-bundled-dicts/package.json              |  2 +-
 pnpm-lock.yaml                                          | 12 ++++++------
 4 files changed, 11 insertions(+), 21 deletions(-)

fix: Workflow Bot -- Update Dictionaries (main) (#8036)

fix: Workflow Bot -- Update Dictionaries (main) (#8036)

Update Dictionaries (main)

Summary

 .../snapshots/TheAlgorithms/Python/report.yaml     |  7 +--
 .../snapshots/TheAlgorithms/Python/snapshot.txt    |  5 +--
 .../snapshots/eslint/eslint/report.yaml            |  6 +--
 .../snapshots/eslint/eslint/snapshot.txt           |  3 +-
 .../microsoft/TypeScript-Website/report.yaml       |  5 +--
 .../microsoft/TypeScript-Website/snapshot.txt      |  3 +-
 .../snapshots/slint-ui/slint/report.yaml           |  7 ++-
 .../snapshots/slint-ui/slint/snapshot.txt          |  3 +-
 packages/cspell-bundled-dicts/package.json         |  8 ++--
 pnpm-lock.yaml                                     | 51 ++++++++++++----------
 10 files changed, 46 insertions(+), 52 deletions(-)

v9.3.2

15 Nov 16:43

Choose a tag to compare

Fixes

fix: Add Zig programming language dictionary (#7998)

fix: Add Zig programming language dictionary (#7998)


fix: Search for TypeScript config files. (#7997)

fix: Search for TypeScript config files. (#7997)

TypeScript files were allowed, but would not be automatically found.


Dictionary Updates

fix: Workflow Bot -- Update Dictionaries (main) (#8004)

fix: Workflow Bot -- Update Dictionaries (main) (#8004)

Update Dictionaries (main)

Summary

 .../snapshots/ktaranov/sqlserver-kit/report.yaml   |  4 +--
 .../snapshots/ktaranov/sqlserver-kit/snapshot.txt  |  5 ++-
 packages/cspell-bundled-dicts/package.json         |  6 ++--
 pnpm-lock.yaml                                     | 36 +++++++++++-----------
 4 files changed, 24 insertions(+), 27 deletions(-)

v9.3.1

12 Nov 15:04

Choose a tag to compare

Fixes

fix: Support Deno (#7966)

fix: Support Deno (#7966)

Deno

This should enable running:

deno run -A cspell lint -c ./cspell.config.mts src

Bun

I tried bun, but it failed with:

image

WIP

Note: this is a work in progress. Deno does not support NodeJS's Buffer for decoding compressed binary data. I'll investigate a workaround.

Current issue:
image

It also blows up with encountering yarn blobs, but that is to be expected.