Skip to content

Commit ba36830

Browse files
authored
Add LSI benchmark system for GSL vs native Ruby comparison (#74)
* Improve GSL warning with suppression option - Add SUPPRESS_GSL_WARNING=true env var to silence the notice - Update warning message with clear install instructions - Modernize README with platform-specific GSL installation steps - Rename README.markdown to README.md Closes #41 * Add LSI benchmark system for comparing GSL vs native Ruby - Add benchmark/lsi_benchmark.rb with performance tests - Add rake benchmark and rake benchmark:compare tasks - Test add_items, build_index, classify, search, find_related - Handle native Ruby SVD numerical instability gracefully - Update Rakefile to reference README.md (renamed from .markdown) Note: Native Ruby SVD has numerical limits with larger document sets. GSL is recommended for production use with large collections. * Exclude benchmark directory from RuboCop * Fix benchmark to handle SVD dimension mismatch - Catch ExceptionForMatrix::ErrDimensionMismatch in addition to Math::DomainError - Use diverse document vocabulary to reduce SVD instability - Reduce default benchmark sizes (5, 10, 15, 20) for native Ruby stability - Update test queries to match new document style * Add Benchmarks section to README with sample results * fix: address PR review comments and fix GSL compatibility - Replace % string formatting with format() for readability - Fix code injection vulnerability by using JSON via stdin - Use Bundler.with_unbundled_env so subprocess can access GSL gem - Fix GSL::Vector missing count method (use to_a.count instead) - Update README with Ruby 3.4+ GSL installation instructions - Add real benchmark results showing 4x-116x speedup with GSL The benchmark comparison now works correctly, showing GSL provides significant speedups especially for build_index (SVD computation).
1 parent 86dadd7 commit ba36830

6 files changed

Lines changed: 395 additions & 9 deletions

File tree

.rubocop.yml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ AllCops:
1010
- '*.gemspec'
1111
- 'bin/**/*'
1212
- 'install.rb'
13+
- 'benchmark/**/*'
1314

1415
# Existing code lacks documentation, can add incrementally
1516
Style/Documentation:

README.markdown renamed to README.md

Lines changed: 62 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,44 @@ Classifier is a general module to allow Bayesian and other types of classificati
1010

1111
## Dependencies
1212

13-
If you install Classifier from source, you'll need to install Roman Shterenzon's fast-stemmer gem with RubyGems as follows:
13+
The `fast-stemmer` gem is required:
1414

1515
gem install fast-stemmer
1616

17-
If you would like to speed up LSI classification by at least 10x, please install the following libraries:
18-
GNU GSL:: http://www.gnu.org/software/gsl
19-
rb-gsl:: https://github.com/SciRuby/rb-gsl
17+
### Optional: GSL for Faster LSI
2018

21-
Notice that LSI will work without these libraries, but as soon as they are installed, Classifier will make use of them. No configuration changes are needed, we like to keep things ridiculously easy for you.
19+
For faster LSI classification, install the GNU Scientific Library and its Ruby bindings.
20+
21+
#### Ruby 3.4+
22+
23+
The released `gsl` gem doesn't support Ruby 3.4+. Install from source with the compatibility fix:
24+
25+
# Install GSL library
26+
brew install gsl # macOS
27+
apt-get install libgsl-dev # Ubuntu/Debian
28+
29+
# Build and install the gem from the compatibility branch
30+
git clone https://github.com/cardmagic/rb-gsl.git
31+
cd rb-gsl
32+
git checkout fix/ruby-3.4-compatibility
33+
gem build gsl.gemspec
34+
gem install gsl-*.gem
35+
36+
#### Ruby 3.3 and earlier
37+
38+
# macOS
39+
brew install gsl
40+
gem install gsl
41+
42+
# Ubuntu/Debian
43+
apt-get install libgsl-dev
44+
gem install gsl
45+
46+
LSI works without GSL using a pure Ruby implementation. When GSL is installed, Classifier automatically uses it with no configuration needed.
47+
48+
To suppress the GSL notice when not using it:
49+
50+
SUPPRESS_GSL_WARNING=true ruby your_script.rb
2251

2352
## Bayes
2453

@@ -89,6 +118,34 @@ with more than just simple strings.
89118
* http://www.chadfowler.com/index.cgi/Computing/LatentSemanticIndexing.rdoc
90119
* http://en.wikipedia.org/wiki/Latent_semantic_analysis
91120

121+
## Benchmarks
122+
123+
Run benchmarks to compare LSI performance:
124+
125+
rake benchmark # Run with current configuration
126+
rake benchmark:compare # Compare GSL vs native Ruby
127+
128+
### GSL vs Native Ruby Comparison
129+
130+
| Documents | build_index | Overall Speedup |
131+
|-----------|-------------|-----------------|
132+
| 5 | 4x | 2.5x |
133+
| 10 | 24x | 5.5x |
134+
| 15 | 116x | 17x |
135+
136+
Sample comparison (15 documents):
137+
138+
Operation Native GSL Speedup
139+
----------------------------------------------------------
140+
build_index 0.1412 0.0012 116.2x
141+
classify 0.0142 0.0049 2.9x
142+
search 0.0102 0.0026 3.9x
143+
find_related 0.0069 0.0016 4.2x
144+
----------------------------------------------------------
145+
TOTAL 0.1725 0.0104 16.6x
146+
147+
The `build_index` operation (SVD computation) dominates total time and benefits most from GSL. Install GSL for production use with larger document sets.
148+
92149
## Authors
93150

94151
* Lucas Carlson (lucas@rufy.com)

Rakefile

Lines changed: 12 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,10 +24,21 @@ desc 'Create documentation'
2424
Rake::RDocTask.new('doc') do |rdoc|
2525
rdoc.title = 'Ruby Classifier - Bayesian and LSI classification library'
2626
rdoc.rdoc_dir = 'html'
27-
rdoc.rdoc_files.include('README.markdown')
27+
rdoc.rdoc_files.include('README.md')
2828
rdoc.rdoc_files.include('lib/**/*.rb')
2929
end
3030

31+
# Benchmarks
32+
desc 'Run LSI benchmark with current configuration'
33+
task :benchmark do
34+
ruby 'benchmark/lsi_benchmark.rb'
35+
end
36+
37+
desc 'Run LSI benchmark comparing GSL vs Native Ruby'
38+
task 'benchmark:compare' do
39+
ruby 'benchmark/lsi_benchmark.rb --compare'
40+
end
41+
3142
desc 'Report code statistics (KLOCs, etc) from the application'
3243
task :stats do
3344
require 'code_statistics'

0 commit comments

Comments
 (0)