-
Notifications
You must be signed in to change notification settings - Fork 2
Open
Description
3D convolutions at large resolutions use cuFFT internally and gives the following error. Reverting this PR fixes it since in that PR we loop over each variable and convolve instead of using convolutions for the entire 3D array. Creating this issue so that I'll remember to look into this in a couple of weeks.
Start: total_mass, total_energy: 152577474.45441937, 28337000790599.57
Signal 8 received by node 0, process 3158851 (thread 7fd8e8a20000) - obtaining backtrace
Signal 8 received by process 3158851 (thread 7fd8e8a20000) at: stack trace: 16 frames
[0] = void cunumeric::cufft_convolution<double, 3>(Legion::FieldAccessor<(legion_privilege_mode_t)268435463, double, 3, long long, Realm::AffineAccessor<double, 3, long long>, false>, Legion::FieldAccessor<(legion_privilege_mode_t)1, double, 3, long long, Realm::AffineAccessor<double, 3, long long>, false>, Legion::FieldAccessor<(legion_privilege_mode_t)1, double, 3, long long, Realm::AffineAccessor<double, 3, long long>, false>, Realm::Rect<3, long long> const&, Realm::Rect<3, long long> const&, Realm::Rect<3, long long> const&) at unknown file:0 [00007fd8945c5324]
[1] = void cunumeric::ConvolveImpl<(cunumeric::VariantKind)2>::operator()<(legate::Type::Code)11, 3, (void*)0>(cunumeric::ConvolveArgs&) const at unknown file:0 [00007fd8945c5b66]
[2] = void cunumeric::convolve_template<(cunumeric::VariantKind)2>(legate::TaskContext&) at unknown file:0 [00007fd894565dba]
[3] = cunumeric::ConvolveTask::gpu_variant(legate::TaskContext) at unknown file:0 [00007fd894565fd6]
[4] = legate::detail::task_wrapper(void (*)(legate::TaskContext), legate_core_variant_t, std::optional<std::basic_string_view<char, std::char_traits<char> > >, void const*, unsigned long, void const*, unsigned long, Realm::Processor) at unknown file:0 [00007ffdcc9d77fb]
[5] = void legate::LegateTask<cunumeric::ConvolveTask>::task_wrapper_<&cunumeric::ConvolveTask::gpu_variant, (legate_core_variant_t)2>(void const*, unsigned long, void const*, unsigned long, Realm::Processor) at unknown file:0 [00007fd8927b6d1b]
[6] = Realm::Cuda::GPUProcessor::execute_task(unsigned int, Realm::ByteArrayRef const&) at unknown file:0 [00007ffff53d8a0d]
[7] = Realm::Task::execute_on_processor(Realm::Processor) at unknown file:0 [00007ffff53c5872]
[8] = Realm::KernelThreadTaskScheduler::execute_task(Realm::Task*) at unknown file:0 [00007ffff53c58f9]
[9] = Realm::Cuda::GPUTaskScheduler<Realm::KernelThreadTaskScheduler>::execute_task(Realm::Task*) at unknown file:0 [00007ffff541e2d7]
[10] = Realm::ThreadedTaskScheduler::scheduler_loop() at unknown file:0 [00007ffff53c3ec3]
[11] = Realm::ThreadedTaskScheduler::scheduler_loop_wlock() at unknown file:0 [00007ffff53c4430]
[12] = Realm::KernelThread::pthread_entry(void*) at unknown file:0 [00007ffff53c8325]
[13] = start_thread at unknown file:0 [00007ffff2999608]
[14] = __clone at unknown file:0 [00007ffff4b5a132]
[15] = unknown symbol at unknown file:0 [ffffffffffffffff]
Metadata
Metadata
Assignees
Labels
No labels