An Error when Calling torch.sparse.sum and .to_dense(): Could not run 'aten::to_dense' with arguments from the 'CPU' backend

Question

I met a problem when calling torch.sparse.sum(). The original code is as follows:

class RGCN_Layer(nn.Module):
    """ A Relation GCN module operated on documents graphs. """

    def __init__(self, in_dim, mem_dim, num_layers, relation_cnt=8):
        super().__init__()
        self.layers = num_layers
        self.device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
        self.mem_dim = mem_dim
        self.relation_cnt = relation_cnt
        self.in_dim = in_dim

        self.dropout = 0.2
        # self.in_drop = nn.Dropout(self.dropout)
        self.gcn_drop = nn.Dropout(self.dropout)

        # gcn layer
        self.W_0 = nn.ModuleList()
        self.W_r = nn.ModuleList()
        # for i in range(self.relation_cnt):
        for i in range(relation_cnt):
            self.W_r.append(nn.ModuleList())

        for layer in range(self.layers):
            input_dim = self.in_dim if layer == 0 else self.mem_dim
            self.W_0.append(nn.Linear(input_dim, self.mem_dim).to(self.device))
            for W in self.W_r:
                W.append(nn.Linear(input_dim, self.mem_dim).to(self.device))


    def forward(self, nodes, adj):
        """
        
        :param nodes:  batch_size * num_event * num_event
        :param adj:  batch_size * 8 * num_event * num_event
        :return:
        """
        # gcn_inputs = self.in_drop(nodes)
        gcn_inputs = nodes

        maskss = []
        denomss = []
        for batch in range(adj.shape[0]):
            masks = []
            denoms = []
            for i in range(self.relation_cnt):
                denom = torch.sparse.sum(adj[batch, i], dim=1).to_dense()
                t_g = denom + torch.sparse.sum(adj[batch, i], dim=0).to_dense()
                mask = t_g.eq(0)
                denoms.append(denom.unsqueeze(1))
                masks.append(mask)
            denoms = torch.sum(torch.stack(denoms), 0)
            denoms = denoms + 1
            masks = sum(masks)
            maskss.append(masks)
            denomss.append(denoms)
        denomss = torch.stack(denomss) # 40 * 61 * 1

        # sparse rgcn layer
        for l in range(self.layers):
            gAxWs = []
            for j in range(self.relation_cnt):
                gAxW = []

                bxW = self.W_r[j][l](gcn_inputs)
                for batch in range(adj.shape[0]):

                    xW = bxW[batch]  # 255 * 25
                    AxW = torch.sparse.mm(adj[batch][j], xW)  # 255, 25
                    # AxW = AxW/ denomss[batch][j]  # 255, 25
                    gAxW.append(AxW)
                gAxW = torch.stack(gAxW)
                gAxWs.append(gAxW)
            gAxWs = torch.stack(gAxWs, dim=1)
            # print("denomss", denomss.shape)
            # print((torch.sum(gAxWs, 1) + self.W_0[l](gcn_inputs)).shape)
            gAxWs = F.relu((torch.sum(gAxWs, 1) + self.W_0[l](gcn_inputs)) / denomss)  # self loop
            gcn_inputs = self.gcn_drop(gAxWs) if l < self.layers - 1 else gAxWs

        return gcn_inputs, maskss

However, when executing to the line denom = torch.sparse.sum(adj[batch, i], dim=1).to_dense(), an error occurred as follows:

RuntimeError: Could not run 'aten::to_dense' with arguments from the 'CPU' backend. 'aten::to_dense' is only available for these backends: [MkldnnCPU, SparseCPU, BackendSelect, Named, AutogradOther, AutogradCPU, AutogradCUDA, AutogradXLA, AutogradPrivateUse1, AutogradPrivateUse2, AutogradPrivateUse3, Tracer, Autocast, Batched, VmapMode].

The error told me that the code cannot be executed on the CPU platform; however, I noticed the CUDA is also not listed in the supported list. In addition, after checking the library, the execution on CPU and CUDA might be supported. Could someone help me to deal with this problem? Thanks!

Answer 1

the function torch.sparse.sum returns a dense tensor if you sum over all of the sparse dimensions. And since you're slicing the adjacency matrix first and then summing, you're calling the method .to_dense() on a dense tensor.

torch.arange(10).to_dense() # will give same error