Skip to content

Commit d946efe

Browse files
committed
Use suffixed index names by default to prevent alias creation conflicts
Updated index creation logic to use a suffixed index name (e.g., `users_<timestamp-suffix>`) by default. Previously, during index reset, the old index (e.g., `users`) was deleted, and a new suffixed index (e.g., `users_<timestamp-suffix>`) was created and aliased to the old index name. This caused a race condition, where another job could recreate the unsuffixed `users` index before the alias was applied, leading to a conflict when trying to create the alias.
1 parent 259089f commit d946efe

File tree

11 files changed

+99
-19
lines changed

11 files changed

+99
-19
lines changed

CHANGELOG.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,13 @@
11
# Changelog
22

3+
## master (unreleased)
4+
5+
### New Features
6+
7+
### Changes
8+
9+
* [#973](https://github.com/toptal/chewy/pull/973): **(Potentially Breaking)** Indexes now use suffixed physical names from first creation (e.g., `users_1700000000000`) with an unsuffixed alias (`users`), instead of switching to this layout only after an index reset. This reduces the risk of a race condition during index reset. Existing unsuffixed indexes are not auto-migrated and continue to work, but direct ES operations that expect unsuffixed concrete index names may need updates. To migrate existing unsuffixed indexes to the alias-based layout, perform a one-time `chewy:reset` in a controlled window (pause jobs/writes that can recreate the unsuffixed index name). ([@dmeremyanin][])
10+
311
## 8.0.0 (2026-02-25)
412

513
### New Features
@@ -827,6 +835,7 @@
827835
[@davekaro]: https://github.com/davekaro
828836
[@dck]: https://github.com/dck
829837
[@dm1try]: https://github.com/dm1try
838+
[@dmeremyanin]: https://github.com/dmeremyanin
830839
[@dmitry]: https://github.com/dmitry
831840
[@dnd]: https://github.com/dnd
832841
[@DNNX]: https://github.com/DNNX

lib/chewy/index.rb

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -197,6 +197,16 @@ def scopes
197197
public_methods - Chewy::Index.public_methods
198198
end
199199

200+
# Generates an automatic suffix for index names. By default, the method
201+
# returns a timestamp-based suffix, using the current time in milliseconds (e.g., 1638947285000).
202+
#
203+
# It is used to differentiate indexes for zero-downtime deployments.
204+
#
205+
# @return [Object] an object that can be cast to a string (e.g., integer, timestamp, etc.)
206+
def auto_suffix
207+
(Time.now.to_f * 1000).round
208+
end
209+
200210
def settings_hash
201211
_settings.to_hash
202212
end

lib/chewy/index/actions.rb

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -7,12 +7,13 @@ module Actions
77
extend ActiveSupport::Concern
88

99
module ClassMethods
10-
# Checks index existance. Returns true or false
10+
# Checks index existance. Supports suffixes. Returns true or false
1111
#
1212
# UsersIndex.exists? #=> true
13+
# UsersIndex.exists?('11-2024') #=> false
1314
#
14-
def exists?
15-
client.indices.exists(index: index_name)
15+
def exists?(suffix = nil)
16+
client.indices.exists(index: index_name(suffix: suffix))
1617
end
1718

1819
# Creates index and applies mappings and settings.

lib/chewy/index/import/routine.rb

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -67,7 +67,10 @@ def create_indexes!
6767
Chewy::Stash::Journal.create if @options[:journal] && !Chewy.configuration[:skip_journal_creation_on_import]
6868
return if Chewy.configuration[:skip_index_creation_on_import]
6969

70-
@index.create!(**@bulk_options.slice(:suffix)) unless @index.exists?
70+
suffix = @bulk_options[:suffix] || @index.auto_suffix
71+
index_exists = @bulk_options[:suffix] ? @index.exists?(suffix) : @index.exists?
72+
73+
@index.create!(suffix) unless index_exists
7174
end
7275

7376
# The main process method. Converts passed objects to the bulk request body,

lib/chewy/rake_helper.rb

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -279,7 +279,7 @@ def create_missing_indexes!(output: $stdout, env: ENV)
279279
next
280280
end
281281

282-
index.create!
282+
index.create!(index.auto_suffix)
283283

284284
output.puts "#{index.name} index successfully created"
285285
end
@@ -336,7 +336,9 @@ def human_duration(seconds)
336336

337337
def reset_one(index, output, parallel: false)
338338
output.puts "Resetting #{index}"
339-
index.reset!((Time.now.to_f * 1000).round, parallel: parallel, apply_journal: journal_exists?)
339+
340+
suffix = index.auto_suffix
341+
index.reset!(suffix, parallel: parallel, apply_journal: journal_exists?)
340342
end
341343

342344
def warn_missing_index(output)

lib/chewy/stash.rb

Lines changed: 9 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,20 @@ module Chewy
66
#
77
# @see Chewy::Index::Specification
88
module Stash
9-
class Specification < Chewy::Index
10-
index_name 'chewy_specifications'
11-
9+
class Index < Chewy::Index
1210
default_import_options journal: false
1311

12+
# Disable automatic suffixes for specification and journal indexes
13+
def self.auto_suffix; end
14+
end
15+
16+
class Specification < Index
17+
index_name 'chewy_specifications'
18+
1419
field :specification, type: 'binary'
1520
end
1621

17-
class Journal < Chewy::Index
22+
class Journal < Index
1823
index_name 'chewy_journal'
1924

2025
# Loads all entries since the specified time.
@@ -51,8 +56,6 @@ def self.for(*something)
5156
scope
5257
end
5358

54-
default_import_options journal: false
55-
5659
field :index_name, type: 'keyword'
5760
field :action, type: 'keyword'
5861
field :references, type: 'binary'

spec/chewy/index/actions_spec.rb

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,18 @@
1515
before { DummiesIndex.create }
1616
specify { expect(DummiesIndex.exists?).to eq(true) }
1717
end
18+
19+
context do
20+
before { DummiesIndex.create('2024') }
21+
specify { expect(DummiesIndex.exists?).to eq(true) }
22+
specify { expect(DummiesIndex.exists?('2024')).to eq(true) }
23+
end
24+
25+
context do
26+
before { DummiesIndex.create('2024', alias: false) }
27+
specify { expect(DummiesIndex.exists?).to eq(false) }
28+
specify { expect(DummiesIndex.exists?('2024')).to eq(true) }
29+
end
1830
end
1931

2032
describe '.create' do

spec/chewy/index/import_spec.rb

Lines changed: 27 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,13 +32,38 @@ def subscribe_notification
3232

3333
describe 'index creation on import' do
3434
let(:dummy_city) { City.create }
35+
let(:dummy_auto_suffix) { (Time.now.to_f * 1000).round }
36+
37+
before do
38+
allow(Chewy::Index).to receive(:auto_suffix).and_return(dummy_auto_suffix)
39+
end
3540

3641
specify 'lazy (default)' do
37-
expect(CitiesIndex).to receive(:exists?).and_call_original
38-
expect(CitiesIndex).to receive(:create!).and_call_original
42+
expect(CitiesIndex).to receive(:exists?).with(no_args).and_call_original
43+
expect(CitiesIndex).to receive(:create!).with(dummy_auto_suffix).and_call_original
44+
CitiesIndex.import(dummy_city)
45+
end
46+
47+
specify 'lazy when index is already created' do
48+
CitiesIndex.create!
49+
expect(CitiesIndex).to receive(:exists?).with(no_args).and_call_original
50+
expect(CitiesIndex).not_to receive(:create!)
3951
CitiesIndex.import(dummy_city)
4052
end
4153

54+
specify 'lazy when index is already created and suffix is given' do
55+
CitiesIndex.create!(dummy_auto_suffix)
56+
expect(CitiesIndex).to receive(:exists?).with(dummy_auto_suffix).and_call_original
57+
expect(CitiesIndex).not_to receive(:create!)
58+
CitiesIndex.import(dummy_city, suffix: dummy_auto_suffix)
59+
end
60+
61+
specify 'lazy when suffix is given' do
62+
expect(CitiesIndex).to receive(:exists?).with(dummy_auto_suffix).and_call_original
63+
expect(CitiesIndex).to receive(:create!).with(dummy_auto_suffix).and_call_original
64+
CitiesIndex.import(dummy_city, suffix: dummy_auto_suffix)
65+
end
66+
4267
specify 'lazy without objects' do
4368
expect(CitiesIndex).not_to receive(:exists?)
4469
expect(CitiesIndex).not_to receive(:create!)

spec/chewy/index_spec.rb

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -246,6 +246,16 @@ def self.by_id; end
246246
specify { expect(subject.specification).to equal(subject.specification) }
247247
end
248248

249+
describe '.auto_suffix' do
250+
subject { stub_index(:documents) }
251+
252+
around do |spec|
253+
Timecop.freeze { spec.run }
254+
end
255+
256+
specify { expect(subject.auto_suffix).to eq((Time.now.to_f * 1000).round) }
257+
end
258+
249259
context 'index call inside index', :orm do
250260
before do
251261
stub_index(:cities) do

spec/chewy/search/request_spec.rb

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -755,12 +755,13 @@
755755
expect(subject.limit(5).source(:name).pluck(:id, :age)).to eq([[1, 10], [2, 20], [3, 30], [4, 40], [5, 50]])
756756
end
757757
specify do
758+
index_name = ProductsIndex.indexes[0]
758759
expect(subject.limit(5).pluck(:_index, :name)).to eq([
759-
%w[products Name1],
760-
%w[products Name2],
761-
%w[products Name3],
762-
%w[products Name4],
763-
%w[products Name5]
760+
[index_name, 'Name1'],
761+
[index_name, 'Name2'],
762+
[index_name, 'Name3'],
763+
[index_name, 'Name4'],
764+
[index_name, 'Name5']
764765
])
765766
end
766767

0 commit comments

Comments
 (0)