Add support for device vectors through a workaround #23
Conversation
tjhei
left a comment
tjhei
left a comment
There was a problem hiding this comment.
Can you add a test or two please?
| assert_dof_handler(dof_handler); | ||
|
|
||
| const unsigned int min_level = dst.min_level(); | ||
| const unsigned int min_level = transfer.min_level(); |
There was a problem hiding this comment.
Oops, this is a mistake, this must have changed in the commits since I made the PR last time. I copied and pasted the whole file from my previous PR over and it must have overwritten this change. Should have double checked, I restored this to its current version.
| LinearAlgebra::distributed::Vector<Number, dealii::MemorySpace::Host>; | ||
|
|
||
| MGTransferMatrixFree( | ||
| const MGLevelObject<MGTwoLevelTransfer<dim, VectorTypeHost>> &mg_transfers, |
There was a problem hiding this comment.
It is a bit surprising that we initialize with host vectors here, right? That would mean if we make this work on device later, user code needs to change?
There was a problem hiding this comment.
Yes this is a good point but rather annoying to work around. These inputs are used to initialize the underlying MGTransferMatrixFree class that is defined on the host which does the "workaround" part of this. To switch these to be device elements we would need some method by which we could convert them to equivalent objects on the host which I'm not sure how easy or feasible that is.
| dst_host; | ||
| MGLevelObject<VectorTypeHost> src_host(src.min_level(), src.max_level()); | ||
|
|
||
| copy_to_host(dst_host, dst); |
There was a problem hiding this comment.
This is not safe to remove directly because "dst.copy_locally_owned_data_from()" inside copy_from_mg called on the host transfer complains that dst is inappropriately sized. I believe this is because it is initialized but never partitioned correctly and the copy_to_host() call is accomplishing this.
Once I partitioned the vector this was safe to remove.
| VectorTypeHost dst_host; | ||
| VectorTypeHost src_host; | ||
|
|
||
| copy_to_host(dst_host, dst); |
| VectorTypeHost dst_host; | ||
| VectorTypeHost src_host; | ||
|
|
||
| copy_to_host(dst_host, dst); |
RyanMoulday
force-pushed
the
mgtransfermf_device_workaround
branch
from
4f6e401 to
b5f1aaf
Compare
…to host to execute transfer before sending back to device.
RyanMoulday
force-pushed
the
mgtransfermf_device_workaround
branch
from
b5f1aaf to
934ea03
Compare
| // ------------------------------------------------------------------------ | ||
|
|
||
|
|
||
| // Check MGTransferMatrixFree by comparison with MGTransferPrebuilt on a |
There was a problem hiding this comment.
Update this comment to include device
|
|
||
| // build host reference | ||
| MGTransferMatrixFree<dim, Number, dealii::MemorySpace::Host> | ||
| transfer_host(mg_constrained_dofs); |
There was a problem hiding this comment.
It looks like you are using local smoothing multi grid here. Not wrong, but portable MatrixFree is currently only doing global coarsening.
2 participants
No description provided.