What's wrong with my code implementing RSA?

Question

I've only been learning about cryptography for the last 3 weeks and coding in Java for about a year. I'm working on a practice project that is supposed to generate public and private key pairs, encrypt plaintext using public key, and decrypt ciphertext using private key. I don't get any compilation errors when I try to run it, but I'm not getting any output either.

I'm thinking that I'm not using the right variables or call methods where they belong, but the more I try to make heads and tails of what's going on, the more confused I become. Can anyone take a look and point out the issue(s)?

Sorry for the janky mess. Note: The comments are all instruction that I've been given.

package project4;


import java.math.BigInteger; // for generating prime numbers and for large integer arithematics
import java.util.Base64;     // for encoding of byte array to string and decode string to byte array
import java.util.Random;     // need this class when generating random prime numbers

/*
 * This class implements the RSA algorithm for public-key cryptography using BigInteger class.
 */
public class PKC {
    // modulus = p * q
    // private_key = public_key^-1 mod (p-1)*(q-1)
    BigInteger p, q, public_key, private_key, modulus;
    Random rnd= new Random();   
    
    


    // initialize p and q with two prime numbers of 1024 bits long using BigInteger's constructor
    // the likelihood that p or q is not a prime number is <= 2^-10
    // 
    // also initialize modulus as p * q
    PKC() { 
        
        p = BigInteger.probablePrime(1024, rnd);
        q = p.nextProbablePrime();
        modulus = p.multiply(q); 
        
    }
    

    // initialize the public and private key pair
    // the public key is a 20 bit long prime number (probability of not prime is <= 2^-10
    // private key is the multiplicative inverse of the public key modulo (p-1)*(q-1) --- public BigInteger modInverse(BigInteger modulus)
    void genKeyPair() {
        public_key = BigInteger.probablePrime(20, rnd);
        private_key = public_key.modInverse(modulus);   
        
    }
    
    
    // encrypt a plaintext message as a string using the public key 
    // return the ciphertext as a string (using Base64 encoding)
    
    //string --> bytearray --> biginteger --> bytearray --> encode --> string
    String encrypt(String m) {
        
        return Base64.getEncoder().encodeToString(m.getBytes());

    }
    
    // decrypt a ciphertext using the private key 
    // return the plaintext as string
    //string --> decode --> bytearray --> biginteger --> decrypt --> biginteger --> bytearray --> string
    String decrypt(String c) {
        
        return new String(Base64.getDecoder().decode(c));
        
    }
    
    // encrypt a plaintext message as a BigInteger using the public key
    BigInteger encrypt(BigInteger m) {
        
        return m.modPow(public_key, modulus);
        
    }
    
    // decrypt a ciphertext message as a BigInteger using the private key
    BigInteger decrypt(BigInteger c) {
        
        return c.modInverse(modulus);
    }
    
    public static void main(String[] args) {
        
        PKC pkc = new PKC(); 
        pkc.genKeyPair();
        System.out.printf("p: %d\n\nq: %d\n\npublic key: %d\n\nprivate key: %d\n\nmodulus: %d\n\n", 
                pkc.p, pkc.q, pkc.public_key, pkc.private_key, pkc.modulus); 
        
        BigInteger m = new BigInteger("438709437431787");
        BigInteger c = pkc.encrypt(m);
        
        System.out.printf("plaintext: %d\n\nciphertext: %d\n\ndecrypted text: %d\n\n", m, c, pkc.decrypt(c));
        
        String plaintext = "test message";
        String ciphertext = pkc.encrypt(plaintext);
        
        System.out.printf("plaintext: %s\n\nciphertext: %s\n\ndecrypted text: %s\n\n", plaintext, ciphertext, pkc.decrypt(ciphertext));
    }
}

Answer 1

Here are some of the obvious problems:

PKC() {      
   p = BigInteger.probablePrime(1024, rnd);
   q = p.nextProbablePrime();

The two prime factors quite near to each other; this makes pq quite easy to factor. This doesn't affect the correctness; it does affect the security.

void genKeyPair() {
   public_key = BigInteger.probablePrime(20, rnd);

This does not take any precautions to ensure that the generated public exponent is relatively prime to (p-1)(q-1), which is necessary for the next step to succeed (well, if you did the next step correctly

// private key is the multiplicative inverse of the public key modulo (p-1)*(q-1) --- public BigInteger modInverse(BigInteger modulus)

 void genKeyPair() {
   ...
   private_key = public_key.modInverse(modulus); 

This code does not correspond to the comment; the generated private key is incorrect

BigInteger decrypt(BigInteger c) {     
    return c.modInverse(modulus);
}

This is not the correct decryption procedure.