larger geometry in the pipflow_preinlet case

[mojtabaAmiraslanpour]

Hi,

I ma trying to use my custom geomtry in the pipeflow_with_preinlet case. It works fine when my geometry is small i.e. D = 5 um, but when I change it to something bigger like D = 20 um (not big actually) it stuck forever in the boundary creation step. Any ideas?

 _   _   ____   __  __   _____    ___   ____   __     __
( )_( ) ( ___) (  \/  ) (  _  )  / __) ( ___) (  )   (  )
 ) _ (   )__)   )    (   )(_)(  ( (__   )__)   )(__   )(__
(_) (_) (____) (_/\/\_) (_____)  \___) (____) (____) (____)
                         v.2.6

(Stl preinlet) (Geometry) reading and voxelizing STL file cui_coarseMesh.stl
(main) Voxelisation is done. Resulting domain parameters are:
Size of the multi-block:     641-by-159-by-42
Number of atomic-blocks:     67
Smallest atomic-block:       25-by-26-by-42
Largest atomic-block:        26-by-27-by-42
Number of allocated cells:   1.92217 million
Fraction of allocated domain: 44.9043 percent

(Voxelizer) (Warning) particleEnvelopeSize not given, setting to 25
(Stl preinlet) (Parameters) setting lbm parameters
(HemoCell) System parameters:
         dx:    5e-07
         dt:    1e-07
         dm:    1.28125e-16
         dN:    6.40625e-09
         tau:   1.82
         nu_lbm:        0.44
         u_lb_max:      0
         f_limit:       0.00780488
(PreInlets) creating preInlet
(PreInlet::preInletFromSlice) preInletFromSlice called without setting up a lattice first
(Stl preinlet) (Fluid) Initializing Palabos Fluid Field
(HemoCell) (PreInlet) Using default domain management.
(HemoCell) Using default sparse domain management.
(Stl preinlet) (Fluid) Setting up boundaries in Palabos Fluid Field

[gzavo]

Hi @mojtabaAmiraslanpour,

I can't tell it for sure from this, but my first guess would be along the message (PreInlet::preInletFromSlice) preInletFromSlice called without setting up a lattice first. This usually means you want the simulation to create the pre-inlet based on the main domain, however, the main domain is not defined yet. I suggest checking the order of setting things up in your code.

[mojtabaAmiraslanpour]

@gzavo

Actually, the case is fine for the smaller cases. Also, this is the same order of functions used in the curvedflow_with_preinlet example.

There is also another fact. When I set blockSize to -1 it would take like one hour to pass this point in my middle size geometry and I am fine with it, but the thing is, setting that blockSize would change the fraction of the allocated domain to 100% which is not desirable for me. When I use 25 for the blockSize, it would be stuck forever.

[mojtabaAmiraslanpour]

I changed the order of functions and again it is stuck. However the previous warning is not there:

(HemoCell) (Config) reading config.xml
 _   _   ____   __  __   _____    ___   ____   __     __
( )_( ) ( ___) (  \/  ) (  _  )  / __) ( ___) (  )   (  )
 ) _ (   )__)   )    (   )(_)(  ( (__   )__)   )(__   )(__
(_) (_) (____) (_/\/\_) (_____)  \___) (____) (____) (____)
                         v.2.6

(Stl preinlet) (Geometry) reading and voxelizing STL file curved_tube.stl
(main) Voxelisation is done. Resulting domain parameters are:
Size of the multi-block:     643-by-159-by-43
Number of atomic-blocks:     67
Smallest atomic-block:       25-by-27-by-43
Largest atomic-block:        26-by-27-by-43
Number of allocated cells:   1.9771 million
Fraction of allocated domain: 44.973 percent

(Voxelizer) (Warning) particleEnvelopeSize not given, setting to 25
(Voxelizer) (Warning) There are fewer blocks (67) than processors (74), you are wasting CPUs!
(Stl preinlet) (Parameters) setting lbm parameters
(HemoCell) System parameters:
         dx:    5e-07
         dt:    1e-07
         dm:    1.28125e-16
         dN:    6.40625e-09
         tau:   1.82
         nu_lbm:        0.44
         u_lb_max:      0
         f_limit:       0.00780488
(Stl preinlet) (Fluid) Initializing Palabos Fluid Field
(HemoCell) Using default domain management.
(PreInlets) creating preInlet
(Stl preinlet) (Fluid) Setting up boundaries in Palabos Fluid Field

It gets stuck here to be more precise:

  hlog << "(Stl preinlet) (Fluid) Setting up boundaries in Palabos Fluid Field" << endl;
  boundaryFromFlagMatrix(hemocell.lattice, flagMatrix, hemocell.partOfpreInlet);

[mojtabaAmiraslanpour]

To be honest my main concern is that why you have always used blockSize = -1 for all of your examples? This will not remove the unnecessary atomic blocks from the domain and demand more computational costs. In my case, it gets 2.5X more expensive.

[mojtabaAmiraslanpour]

OK, now more updates. It seems not to have any relation to the blockSize and what so ever. It shows everything is good and the fraction of allocation is decreased to like 40% in my case when using blockSize = 25. The culprit is the boundaryFromFlagMatrix function:

namespace hemo {
void boundaryFromFlagMatrix(plb::MultiBlockLattice3D<T,DESCRIPTOR> * fluid, plb::MultiScalarField3D<int> * flagMatrix, bool partOfpreInlet) {
  plb::Box3D domain2 = flagMatrix->getBoundingBox();
  for (int x  = domain2.x0-1 ; x <= domain2.x1+1 ; x++) {
   for (int y  = domain2.y0-1 ; y <= domain2.y1+1 ; y++) {
    for (int z  = domain2.z0-1 ; z <= domain2.z1+1 ; z++) {
      if ((x == domain2.x0-1 || y == domain2.y0-1 || z == domain2.z0-1 || x == domain2.x1+1 || y == domain2.y1+1 || z == domain2.z1+1) && !partOfpreInlet ) {
        defineDynamics(*fluid,x,y,z,new plb::BounceBack<T,DESCRIPTOR>(1.));
      }
    }
   }
  }
  
  for (int x  = domain2.x0 ; x <= domain2.x1 ; x++) {
   for (int y  = domain2.y0 ; y <= domain2.y1 ; y++) {
    for (int z  = domain2.z0 ; z <= domain2.z1 ; z++) {
      if (flagMatrix->get(x,y,z) == 0 && !partOfpreInlet) {
        defineDynamics(*fluid,x,y,z,new plb::BounceBack<T,DESCRIPTOR>(1.));
      }
    }
   }
  }
}

}

And I can't understand why this function is applying the bounceBack BC to all nodes in the outer bounding box? and more importantly why it works on 100% fraction allocation and not the one with 40%?

[mojtabaAmiraslanpour]

So I managed to replace that function with the function that is used in the pipeflow case for defining BounceBack on nodes:

  hlog << "(Stl preinlet) (Fluid) Setting up boundaries in Palabos Fluid Field" << endl;
  if (!hemocell.partOfpreInlet) {
        defineDynamics(*hemocell.lattice, *flagMatrix, (*hemocell.lattice).getBoundingBox(), new BounceBack<T, DESCRIPTOR>(1.), 0);
      }

But I don't get the boundary:

I am just guessing that maybe there is a conflict between the main multiblock lattice and the preinlet lattice because they are pretty much mixed in the code.

Also, another issue is the block decomposition of the pre-inlet domain. It produces blocks with a high aspect ratio. I don't know why this happens and how I can control it.

[mojtabaAmiraslanpour]

I tried adding the pre-inlet code step by step. Putting every other thing aside, it will crash if I add hemocell.preInlet->preInletFromSlice(Direction::Xpos,slice); after hemocell.initializeLattice(voxelizedDomain.get()->getMultiBlockManagement()); strange.