Ghost Dofs incorrectly defined as internal on home rank.

[farg967]

This problem is seen when running monodomain_2D_HH in parallel.

I have narrowed the main cause of the problem down to DOMAIN_MAPPINGS_NODES_DOFS_CALCULATE, line 4554

IF(NUMBER_OF_DOMAINS==1) THEN

should be

IF(MAX_NUMBER_DOMAINS==1) THEN

This bug causes boundary nodes/dofs one element away from the boundary plane between computational domains to be incorrectly defined as internal dofs. I think the problem originates from a double up on the meaning of "boundary". In some places, "boundary" means it is another domains ghost. In other places, "boundary" means it is on the physical boundary plane between ranks. Unless advised otherwise I will use "boundaryPlane" for the latter from now on.

There is a bit more to the fix than the above line, which I am working on atm.

Note: This will only be a temporary fix, as NODES_DOFS_CALCULATE will be done locally soon.

[farg967]

Fixed, with above commit.

[chrispbradley]

Hi Finbar, just reopening the issue as it gets closed when the pull request containing the commit is pulled into the main trunk rather than when you do the commit.

[lorenzo-mechbau]

I further modified the routine DOMAIN_MAPPINGS_NODES_DOFS_CALCULATE and pushed it in the branch I am currently working on, FV_merged. Now the outcome is correct also for a number of ranks >2, keeping Finbar's improvements.

Naming convention: All nodes can be either internal or involved in communication, that is nodes on boundary elements. The latter can be in turn either boundary nodes, defined as LOCAL_DOMAIN_BOUNDARY, i.e. ghost-send, or ghost nodes (ghost-receive). Additionally, some of the nodes on boundary elements are boundary plane nodes, that is they can be found on the physical boundary plane between ranks, as Finbar wrote.

See also the project: https://github.com/users/lorenzo-mechbau/projects/1