Skip to content

Commit bb22fcc

Browse files
committed
refactor: simplify batch_slice_columns column-narrowing (#186)
Replace the per-call memoized index array and cursor with a plain reset: batch_import restores the full header set at the start of each batch, so batch_slice_columns just computes indices from the current headers and slices. Successive calls still compose and multi-batch still works, with far less state.
1 parent 8629ee0 commit bb22fcc

1 file changed

Lines changed: 11 additions & 25 deletions

File tree

lib/active_admin_import/importer.rb

Lines changed: 11 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -82,29 +82,15 @@ def batch_replace(header_key, options)
8282
# end
8383
#
8484
def batch_slice_columns(slice_columns)
85-
@batch_slices ||= []
86-
@batch_slice_cursor ||= 0
87-
88-
# Indices are computed once (on the first batch) against the columns present
89-
# *at this point in the call chain*, then replayed for later batches. Keeping
90-
# one entry per call lets successive calls narrow the result of the previous
91-
# one instead of re-applying the first call's indices (issue #186).
92-
indexes = @batch_slices[@batch_slice_cursor]
93-
unless indexes
94-
columns = headers.values
95-
indexes = columns.each_index.select { |i| columns[i].in?(slice_columns) }
96-
@batch_slices[@batch_slice_cursor] = indexes
97-
# slice CSV headers
98-
@headers = headers.to_a.values_at(*indexes).to_h unless indexes.empty?
99-
end
100-
@batch_slice_cursor += 1
101-
85+
columns = headers.values
86+
indexes = columns.each_index.select { |i| columns[i].in?(slice_columns) }
10287
return csv_lines if indexes.empty?
10388

104-
# slice CSV values
105-
csv_lines.map! do |line|
106-
line.values_at(*indexes)
107-
end
89+
# @headers is reset to the full set at the start of every batch (see
90+
# #batch_import), so each call narrows the previous call's result and every
91+
# batch slices the same way — calling this more than once now composes (#186).
92+
@headers = headers.to_a.values_at(*indexes).to_h
93+
csv_lines.map! { |line| line.values_at(*indexes) }
10894
end
10995

11096
def values_at(header_key)
@@ -138,7 +124,7 @@ def prepare_headers
138124
headers = self.headers.present? ? self.headers.map(&:to_s) : yield
139125
@headers = Hash[headers.zip(headers.map { |el| el.underscore.gsub(/\s+/, '_') })].with_indifferent_access
140126
@headers.merge!(options[:headers_rewrites].symbolize_keys.slice(*@headers.symbolize_keys.keys))
141-
@headers
127+
@full_headers = @headers
142128
end
143129

144130
def run_callback(name)
@@ -147,9 +133,9 @@ def run_callback(name)
147133

148134
def batch_import
149135
batch_result = nil
150-
# Restart the batch_slice_columns pipeline so its memoized indices are
151-
# replayed from the first call for every batch (see #batch_slice_columns).
152-
@batch_slice_cursor = 0
136+
# Every batch re-parses full-width rows, so restore the full header set
137+
# before slicing; batch_slice_columns then narrows it the same way each time.
138+
@headers = @full_headers.dup if @full_headers
153139
@resource.transaction do
154140
run_callback(:before_batch_import)
155141
batch_result = resource.import(headers.values, csv_lines, import_options)

0 commit comments

Comments
 (0)