OpenSSL 1.0.2 with FIPS 2.0.16

Question

I want to understand the behavior when OpenSSL 1.0.2 is integrated with FIPS module 2.0.16. OpenSSL 1.0.2 has inbuild FIPS module, however we are building OpenSSL 1.0.2 with FIPS 2.0.16. could you please help to understand, when we set the FIPS mode using the function FIPS_mode_set(1) will it load OpenSSL fips related algorithms or FIPS module (2.0.16) related algorithms.

Answer 1

Subscribing to what others stated in comments: don't use older (unsupported) OpenSSL versions (not in production, at least)!
Check [OpenSSL.Docs]: ossl-guide-migration, or [OpenSSL-Library]: Downloads which states (emphasis is mine):

All older versions (including 1.1.1, 1.1.0, 1.0.2, 1.0.0 and 0.9.8) are now out of support and should not be used.

Strictly answering to your question, you are correct. FIPS algorithms are implemented in a separate module (FIPSCanister) which is built, included (and plugged-in) in OpenSSL's libraries (LibCrypto). More details can be found at [GitHub]: CristiFati/Prebuilt-Binaries - (master) Prebuilt-Binaries/OpenSSL/Resources/FIPSUserGuide-2.0.pdf.
As a side note, in the repository that the (above) .pdf is part of, various OpenSSL builds can be found. Here's an eloquent (command-line) example based on OpenSSL 1.0.2u (built with OpenSSL-FIPS 2.0.16):

• Win (10 pc064) - OpenSSL pc064:

  [cfati@CFATI-5510-0:e:\Work\Dev\StackExchange\StackOverflow\q079054445]> sopr.bat
  ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
  
  [prompt]>
  [prompt]> openssl
  'openssl' is not recognized as an internal or external command,
  operable program or batch file.
  
  [prompt]> set PATH=%PATH%;c:\Program Files\OpenSSL\OpenSSL-FIPS\1.0.2u\bin
  
  [prompt]> openssl version -a
  OpenSSL 1.0.2u-fips  20 Dec 2019
  Sun Nov 24 08:17:44 2024
  VC-WIN64A
  options:  bn(64,64) rc4(16x,int) des(idx,cisc,2,long) idea(int) blowfish(idx)
  compiler: cl  /MD /Ox -DOPENSSL_THREADS  -DDSO_WIN32 -DOPENSSL_USE_BUILD_DATE=1 -W3 -Gs0 -Gy -nologo -DOPENSSL_SYSNAME_WIN32 -DWIN32_LEAN_AND_MEAN -DL_ENDIAN -DUNICODE -D_UNICODE -D_CRT_SECURE_NO_DEPRECATE -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -I$(FIPSDIR)/include -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DWHIRLPOOL_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DOPENSSL_USE_APPLINK -I. -DOPENSSL_NO_RC5 -DOPENSSL_NO_MD2 -DOPENSSL_NO_SSL2 -DOPENSSL_NO_KRB5 -DOPENSSL_FIPS -DOPENSSL_NO_JPAKE -DOPENSSL_NO_WEAK_SSL_CIPHERS -DOPENSSL_NO_STATIC_ENGINE  
  OPENSSLDIR: "c:\Program Files\OpenSSL\OpenSSL-FIPS\1.0.2u/ssl"
  
  [prompt]>
  [prompt]> openssl md5 "e:\kkt.txt"
  MD5(e:\kkt.txt)= 2a3695ff36298659e4a73db8ae73f2ba
  
  [prompt]>
  [prompt]> set OPENSSL_FIPS=1
  
  [prompt]> openssl md5 "e:\kkt.txt"
  Error setting digest md5
  47812:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:.\fips\utl\fips_md.c:180:
  

• Nix (Ubuntu 20 pc064 (WSL)) - OpenSSL pc032:

  (qaic-env) [cfati@cfati-5510-0:/mnt/e/Work/Dev/StackExchange/StackOverflow/q079054445]> ~/sopr.sh
  ### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
  
  [064bit prompt]>
  [064bit prompt]> /usr/local/pc032/openssl/openssl-fips/1.0.2u/bin/openssl version -a
  OpenSSL 1.0.2u-fips  20 Dec 2019
  built on: Sat Nov 23 21:30:34 2024
  platform: linux-generic32
  options:  bn(64,32) rc4(ptr,char) des(idx,cisc,16,long) idea(int) blowfish(ptr)
  compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -m32 -O3 -fomit-frame-pointer -Wall -I/mnt/e/Work/Dev/CristiFati/Builds/Win/OPSWopenssl/work/fips/install/linux/pc032/openssl-fips/2.0.16/include
  OPENSSLDIR: "/usr/local/pc032/openssl/openssl-fips/1.0.2u/ssl"
  [064bit prompt]>
  [064bit prompt]> /usr/local/pc032/openssl/openssl-fips/1.0.2u/bin/openssl md5 /mnt/e/kkt.txt
  MD5(/mnt/e/kkt.txt)= 2a3695ff36298659e4a73db8ae73f2ba
  [064bit prompt]>
  [064bit prompt]> OPENSSL_FIPS=1 /usr/local/pc032/openssl/openssl-fips/1.0.2u/bin/openssl md5 /mnt/e/kkt.txt
  Error setting digest md5
  4154218164:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180:
  

I worked extensively in this area, you might find the following items (and references) interesting:

• [SO]: OpenSSL FIPS_mode_set not working in Python cryptography library (@CristiFati's answer)

• [SO]: RHEL8: FIPS Object Module (fipscanister) existing? (@CristiFati's answer)