BadPaddingException in RSA Encrryption

Question

I got a BadPaddingException in my RSA Encryption Program. I dont know why it occurs though.

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.security.InvalidKeyException;
import java.security.KeyPair;
import java.security.KeyPairGenerator;
import java.security.NoSuchAlgorithmException;
import java.security.PrivateKey;
import java.security.PublicKey;

import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.CipherInputStream;
import javax.crypto.CipherOutputStream;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.SecretKey;


public class Crypter
{


    private static final String text="Hallo";


    public static byte[] encryptObject(Nachricht msg,PublicKey pubkey) //verschlüsselt das Objekt im CipherStream
    {
        Cipher cipher;
        try
        {
            cipher = Cipher.getInstance("RSA");
            cipher.init(Cipher.ENCRYPT_MODE, pubkey);

            ByteArrayOutputStream bos = new ByteArrayOutputStream();
            CipherOutputStream cos = new CipherOutputStream(bos, cipher); 
            ObjectOutputStream oos = new ObjectOutputStream(cos); 
            oos.writeObject(msg); 
            oos.flush();

            byte[] encryptedBytes = bos.toByteArray();  
            return encryptedBytes;

            //weiss nicht was ich zurückgeben soll 
            //den ObjectOutputStream
            //oder die verschlüsselten Bytes


            //byte[] encryptedBytes = bos.toByteArray();  

        } catch (NoSuchAlgorithmException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (NoSuchPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (InvalidKeyException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return null;

    }

    public static Nachricht decryptObject(PrivateKey privKey,byte[] encryptedBytes) //entschlüsselt das Object
    {
        Nachricht  msg=null;
        try
        {
            Cipher cipher = Cipher.getInstance("RSA");

            cipher.init(Cipher.DECRYPT_MODE, privKey);
            ByteArrayInputStream bin = new ByteArrayInputStream(encryptedBytes);
            CipherInputStream cin = new CipherInputStream(bin, cipher);
            ObjectInputStream in = new ObjectInputStream(cin);
            return (Nachricht) in.readObject();


        } catch (InvalidKeyException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (NoSuchAlgorithmException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (NoSuchPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IOException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (ClassNotFoundException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }



        return null;
    }




    //verschlüsselt Text
    public static byte[] encryptBytes(PublicKey key, byte[] plain)
    {
        byte[] chiffre=null;

        try
        {
            Cipher cipher=Cipher.getInstance("RSA");
            cipher.init(Cipher.ENCRYPT_MODE, key);

            chiffre= cipher.doFinal(plain);

            return chiffre; //verschlüsseltes byte[]

        } catch (NoSuchAlgorithmException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (NoSuchPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (InvalidKeyException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IllegalBlockSizeException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (BadPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return null; 

    }
    //entschlüsselt Text
    public static byte[] decryptBytes(PrivateKey key, byte[] chiffre)
    {

        try
        {
            Cipher cipher=Cipher.getInstance("RSA");
            cipher.init(cipher.DECRYPT_MODE,key);

            byte[] btext= cipher.doFinal(chiffre);
            return btext;


        } catch (NoSuchAlgorithmException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (NoSuchPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (InvalidKeyException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (IllegalBlockSizeException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        } catch (BadPaddingException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        return null; //er meckert sonst....
    }



    public static void main(String [] args)
    {


        try
        {


            //KeyPair und Keys managen
            KeyPairGenerator keyPairGen = KeyPairGenerator.getInstance("RSA");
            keyPairGen.initialize(1024);
            KeyPair keypair=keyPairGen.generateKeyPair();
            PrivateKey privKey=keypair.getPrivate();
            PublicKey pubKey=keypair.getPublic();

            //Text->Byte
            //Byte->Text
            System.out.println("\n\nTest");
            String testtext= "h";
            byte[] testbyte=testtext.getBytes();
            System.out.println(testbyte);
            testtext=new String(testbyte);
            System.out.println(testtext);
            System.out.println("\n\n");

            //Text->Byte[] text
            //byte[] text->encrypt
            //byte[]encrypt-> decrypt
            //byte[] decrypt->String
            System.out.println("Ausgangstext:");
            System.out.println(text);
            System.out.println("1)Text als byte[] unverschlüsselt");
            byte[] bytetext=text.getBytes(); //in byte[] umgewandelt
            System.out.println( bytetext); //bytetext anzeigen lassen
            System.out.println("2) byte[] verschlüsselt");
            byte[] encrypt=encryptBytes(pubKey,bytetext); //verschlüsseln des byte[]
            System.out.println( encrypt); //ausgeben lassen
            System.out.println("3) byte[] entschlüsselt");
            byte[]decrypt=decryptBytes(privKey,encrypt); //entschlüsseln lassen
            String text=new String(decrypt);
            System.out.println(text);
            //String s = new String(bytes);

            //Object->byte[]
            //byte[] object-> byte[] encrypt
            //byte[] encrypt -> byte[]decrypt
            //byte[] decrypt -> object
            System.out.println("Ausgangsobjekt:");
            Nachricht  msg=new Nachricht();
            msg.setNachricht("Hallo");
            //verschlüsseln des Objekts
            byte[] vObject=encryptObject(msg,pubKey);
            System.out.println("Test ");
            Nachricht neuemsg=decryptObject(privKey,vObject);
            System.out.println("Test2");
            String nachricht=neuemsg.getNachricht();
            System.out.println(nachricht);










        } catch (NoSuchAlgorithmException e)
        {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }






    }

}

I am grateful for any help or tips! The functione I wrote, which are handling String encryption are working properly as displayed in the mainfunction. I dont know what to add. The code passes the "Test" sysouts but also throws some Exceptions. I hope you know whats missing or what I have to add to make it work.

Answer 1

Well, the direct cause of your problem is that you forget to call this method on the ObjectOutputStream (which closes all other streams including the CipherOutputStream) with:

oos.close();

From the JavaDocs for CipherOutputStream.close:

This method invokes the doFinal method of the encapsulated cipher object, which causes any bytes buffered by the encapsulated cipher to be processed.

so with some ciphers maybe flush causes some bytes to encrypt but does not add the padding at the end.

But there are lots of other things that can also cause grief:

1. You don't specify which padding should be used, so this could be different on other Java installations, see for some suggestions: JavaDoc for Cipher or for example this question
2. If your object gets a little larger (or you use a different padding that takes more space) the result of the encryption is an array of 0 bytes, without any exceptions being thrown. See also: this question that suggests to encrypt with a symmetric cipher like AES and only encrypt the AES key with RSA. (Or double the keysize and prepare for 7 or 8 times more CPU time each time the keysize doubles, see: this question for a little more info )
3. You should be aware that String.toBytes() and String(byte[]) both use a default encoding which may be different on other computers, please make sure to specify for example java.nio.charset.StandardCharSets.UTF_8

Free advice: If you are going to exchange your 'Nachricht' (message) to another computer and want something really secure, fast, simple (to a certain extent) and fool proof I recommend you use TLS (or DTLS), it is based on 25 years of research and practice.