Reputation: 13
I am trying to use PETSc to solve some equations, and find a weird result of MatGetLocalSize()
the test code is:
#include <petscmat.h>
int main(int argc,char **args)
{
Mat A;
PetscErrorCode ierr;
PetscInt m, n;
PetscMPIInt rank;
PetscInitialize(&argc,&args,(char *)0, "");
ierr = MPI_Comm_rank(PETSC_COMM_WORLD,&rank);CHKERRQ(ierr);
ierr = MatCreate(PETSC_COMM_WORLD,&A);CHKERRQ(ierr);
ierr = MatSetSizes(A,PETSC_DECIDE,PETSC_DECIDE,20,20);CHKERRQ(ierr);
ierr = MatSetFromOptions(A);CHKERRQ(ierr);
PetscCall(MatSetUp(A));
ierr = MatAssemblyBegin(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
ierr = MatAssemblyEnd(A,MAT_FINAL_ASSEMBLY);CHKERRQ(ierr);
PetscCall(MatGetLocalSize(A, &m, &n));
printf("rank=%d, local_rows=%d, local_cols=%d\n", rank, m, n);
PetscCall(MatDestroy(&A));
ierr = PetscFinalize();
return 0;
}
after compiling it, I run it in different processors via mpirun, such as:
$ mpirun -n 1 ./main.x
rank=0, local_rows=20, local_cols=20
$ mpirun -n 2 ./main.x
rank=0, local_rows=10, local_cols=10
rank=1, local_rows=10, local_cols=10
$ mpirun -n 3 ./main.x
rank=0, local_rows=7, local_cols=7
rank=1, local_rows=7, local_cols=7
rank=2, local_rows=6, local_cols=6
Apparently, a 20x20 matrix cannot be distributed as two 10x10 sub-matrix, or two 7x7 with one 6x6
However, if I add the set-value statements and get-value diagnose statments, it seems that columns of sub-matrix for each processorshould be 20 either (which corresponds to the rule I found: row-wise distribution of matrices in PETSc), the diagnostic code is below:
... ...
for (i = 0; i < 20; ++i) {
for (j = 0; j < 20; ++j) {
PetscScalar value = i + j;
ierr = MatSetValue(A, i, j, value, INSERT_VALUES); CHKERRQ(ierr);
}
}
... ...
if (rank == 0){
PetscCall(MatGetValue(A, 0, 19, &psc1));
PetscCall(PetscPrintf(PETSC_COMM_WORLD, "Value: %d\n", (int)psc1));
}
The code succcessfully print the value, while the local columns remain 10 for two processors
Therefore, I suspect that MatGetLocalSize()
get a wrong local columns, but why? is it a bug, or feature?
Upvotes: 0
Views: 13