u_dis.mat format has silently changed between v3.1.0 and develop

[jaemolihm]

Previously, if some low-energy bands were outside the outer window, u_dis.mat were shifted to start with nonzero entry. Now, they are not shifted and keep the original band index.

For example, if there are 5 bands and only band 2, 3, 4 are inside the outer window and 1 and 5 are outside the window, the u_dis.mat file stores

• v3.1.0 : [A, B, C, 0, 0]
• develop: [0, A, B, C, 0]

The relevant code is (note the commented line)

wannier90/src/plot.F90

Lines 2445 to 2460 in 0137a03

	open (newunit=matunit, file=trim(seedname)//'_u_dis.mat', form='formatted')
	write (matunit, *) header
	write (matunit, *) num_kpts, num_wann, num_bands
	do nkp = 1, num_kpts
	utmp = 0.d0
	ioff = dis_manifold%nfirstwin(nkp)
	nbw = dis_manifold%ndimwin(nkp)
	utmp(ioff:ioff + nbw - 1, :) = u_matrix_opt(1:nbw, :, nkp)
	write (matunit, *)
	write (matunit, '(f15.10,sp,f15.10,sp,f15.10)') kpt_latt(:, nkp)
	!write (matunit, '(f15.10,sp,f15.10)') ((u_matrix_opt(i, j, nkp), i=1, num_bands), j=1, num_wann)
	write (matunit, '(f15.10,sp,f15.10)') ((utmp(i, j), i=1, num_bands), j=1, num_wann)
	end do
	close (matunit)
	deallocate (utmp)
	endif

See http://github.com/TRIQS/dftkit/issues/1 for details and for an example where this causes a problem in a Wannier90 interface code.

I like the new ordering much better, but since it is a breaking change, I think one should revert to the old behavior.

Instead, I suggest that we create a new output, seedname_v.mat, which writes V = U_dis * U, with the new index ordering. Then we can suggest the interface packages to migrate to v.mat, while keeping the old interface u_dis.mat + u.mat for some time until we deprecate and remove it.

In addition, if the variational disentanglement is implemented in the future, it directly computes V instead of U_dis and U, so writing V is more natural.

[JeromeCCP9]

This is indeed a nasty change, because it was not intended!

I totally agree that the old behaviour should be restored, as an urgent bugfix!, and then the new output can be added later?

Thank you for tracking this down, Jaemo!

[JeromeCCP9]

Ahh, let's discuss this: git blame shows that this code is from me, but it's quite a specific change and I strongly suspect it was brought in from some other commit. I'd like to research a little to find where this came from. J.

[thenoursehorse]

I have also run into problems with this change with the latest develop commit c1ada05. I also am not sure if it works as intended (to match band indices of u_dis.mat to the full KS bands), at least with my toolchain.

The test case I'm checking against has 104 total bands, with approx 34 bands in dis_win, sitting somewhere in the middle of all the KS bands. When summing over all wanniers and all kpoints, all the weight is still packed, from band index 0 -> approx 33. Some weight is missing, and, e.g., there is appreciable weight in band index 104 (which has to be a bug). So it is approximately the v3.1.0 packed format, but clearly wrong.

Commenting out the new develop format and using v3.1.0 format (still same develop commit c1ada05) gives the expected old packed format.

Separately, the sorting logic won't work if u_dist.mat is obtained with dis_froz_proj = .true.. But that likely requires tracking each KS band index that gives non-zero weight at each k-point, with a big penalty to memory. Either way, maybe that should be flagged somewhere.

I also think outputting a v.mat would be helpful.

[jaemolihm]

Hi @JeromeCCP9, just checking in on the status, will this issue be worked on before the 4.0 release?

[JeromeCCP9]

Yes, I flag this for 4.0