feat: Implement function-based tokenizer API

[isaacvando]

Ticket(s) Closed

• Closes #

What

• Adds a test covering all tokenizer variants explicitly
• Introduces a function-based tokenizer API for queries and index creation
• Switches to the same python schema-compat script from the SQLAlchemy repo instead of using a bespoke ruby one.
  • The ruby had some bugs and was letting some symbols through without showing up in the source.

Why

How

Tests

[codecov]

Codecov Report

❌ Patch coverage is 94.87179% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.34%. Comparing base (17ec036) to head (c0cb1a2).
⚠️ Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/parade_db/migration_helpers.rb	81.81%	2 Missing ⚠️
lib/parade_db/index.rb	92.30%	1 Missing ⚠️
lib/parade_db/tokenizer.rb	98.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #84      +/-   ##
==========================================
+ Coverage   90.15%   90.34%   +0.19%     
==========================================
  Files          19       20       +1     
  Lines        2396     2403       +7     
  Branches      721      710      -11     
==========================================
+ Hits         2160     2171      +11     
+ Misses        236      232       -4     

Flag	Coverage Δ
rails-paradedb	90.34% <94.87%> (+0.19%)	⬆️
rb3.2	90.62% <94.87%> (+0.19%)	⬆️
rb3.3	90.62% <94.87%> (+0.19%)	⬆️
rb3.4	90.34% <94.87%> (+0.19%)	⬆️
rb4.0	90.34% <94.87%> (+0.19%)	⬆️
rl7.2	90.34% <94.87%> (+0.19%)	⬆️
rl8.1	90.34% <94.87%> (+0.19%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
lib/parade_db.rb	98.11% <100.00%> (+0.03%)	⬆️
lib/parade_db/arel/builder.rb	94.09% <100.00%> (-1.57%)	⬇️
lib/parade_db/index.rb	89.36% <92.30%> (+6.47%)	⬆️
lib/parade_db/tokenizer.rb	98.00% <98.00%> (ø)
lib/parade_db/migration_helpers.rb	84.60% <81.81%> (-1.14%)	⬇️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[philippemnoel]

Why are we ignoring this tokenizer?

[philippemnoel]

As discussed in Slack, instead of this we should add support for edge_ngram and test all tokenizers

[philippemnoel]

As d