DER, ASN.1 and Crypto formatting

We need ways to distribute our public keys, private keys and digital certificates in a portable format. One of the most common forms is…

DER, ASN.1 and Crypto formatting

One of the great things about cryptography is the way we have managed to migrate our methods. This migration is often required when we introduce new methods (such as with ECC) or where methods are deprecated (such as for DES). We thus have ways to define new methods and which can be easily interpreted by applications. At the core of this is the DER format for defining our cryptography.

We need ways to distribute our public keys, private keys and digital certificates in a portable format. One of the most common forms is Distinguished Encoding Rules (DER) encoding of ASN.1 (Abstract Syntax Notation One). Overall it is a truly binary representation of the encoded data.

The other common format is PEM, and which converts the binary encoding into a text readable format. With PEM we can encode cryptographic information in a Base64 ASCII format and with plain-text headers and footers of “ — — -BEGIN RSA PRIVATE KEY — — -” and “ — — -END RSA PRIVATE KEY — — -”, whereas with DER we have binary format.

This article will look at the DER format and has code to decode a hex string and into its contents. Overall ASN.1 is used to define abstract types and values. One of the most basic types is SEQUENCE and is an ordered collection of one or more types. In DER, SEQUENCE is identified with a tag of “30”, and followed by a byte value for the length of the object defined. The other common types are OBJECT IDENTIFIER (and which has a tag of “06”) and BIT STRING (and which has a tag of “03”).

The object identifier tag is used to define the cryptography methods used. An example identifier for ECC encryption is “1.2.840.10045.2.1”, and where 1 is OSI, 2 is member body, 840 is US (ANSI), and 10045 is “ansi-X9–62”, and “2” is key type [1]. Other common algorithms are: “1.2.840.113549.1.1.1” (X509 RSA), “1.2.840.10040.4.1” (X509 Digital Signature Standard -DSS), and “1.2.840.10046.2.1” (Diffie-Hellman — DH). The following is an example of the hex sequence for an object ID, and where we have the “06” tag, followed by an identifier for seven bytes (“07”), and then the Object ID of seven bytes (“2a8648ce3d0201”):

06 07 2a8648ce3d0201  # Object ID -  7 bytes long: 1.2.840.10045.2.1 (ECC)

We can also define the curve type in the object identifier, and where we have the form of iso(1), member-body(2), us(840), ansi-X9–62(10045), curves(3), prime(1). For example, 1.2.840.10045.3.1.7 defines ECDSA P-256. Other examples are SECP192R1 (“1.2.840.10045.3.1.1”), SECP224R1 (“1.3.132.0.33”), SECP256K1 (“1.3.132.0.10”), SECP256R1 (“1.2.840.10045.3.1.7”), SECP384R1 (“1.3.132.0.34”), SECP521R1 (“1.3.132.0.35”), and BRAINPOOLP256R1 (“1.3.36.3.3.2.8.1.1.7”). An example where we have an identifier (“06”), followed by the number of bytes identifier (“08”) and Object ID of eight bytes (“2a8648ce3d030107”):

06 08 2a8648ce3d030107  # Object ID -  8 bytes long: 1.2.840.10045.3.1.7 (ECDSA P256)

For the “03” tag, we define a bitstream for keys. In the following, we have “03”, followed by the number of bytes (66 bytes) for the keys, and then the keys are defined after this (64 bytes):

03 42 # Bit stream - 0x42 (66 bytes long)
0004 # Identifies public key
2927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838 # Identifies public key x co-ordinate
c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e # Identifies public key y co-ordinate

An example hex string for a DER format for ECC public keys is:

3059301306072a8648ce3d020106082a8648ce3d030107034200042927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513

We can then break down with:

30 59 # Sequence length 0x59 -  91 bytes long
30 13 # Sequence length 0x13 -  21 bytes long
06 07 2a8648ce3d0201  # Object ID -  7 bytes long - 1.2.840.10045.2.1 (ECC)
06 08 2a8648ce3d030107  # Object ID -  8 bytes long - 1.2.840.10045.3.1.7 (ECDSA P256)
03 42 # Bit stream - 0x42 (66 bytes long)
0004 # Identifies public key
2927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838 # Identifies public key x co-ordinate
c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e # Identifies public key y co-ordinate

In OpenSSL, we can convert from DER to PEM with:

openssl x509 -inform der -in mycert.der -out mycert.pem

and:

openssl x509 -outform der -in mycert.pem -out mycert.der

An example of a public key is:

The code used is [here]:

import sys,binascii

from Crypto.Util.asn1 import DerObjectId,DerBitString, DerSequence

der="3059301306072a8648ce3d020106082a8648ce3d030107034200042927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e"

if (len(sys.argv)>1):
  der=str(sys.argv[1])

# 30 59 
# 30 13 
# 06 07 2a8648ce3d0201
# 06 08 2a8648ce3d030107
# 03 42
# 0004
#  2927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838
#  c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e
pkx=""
pky=""
ptr=0
der_str=DerSequence()
der_str.decode(binascii.unhexlify(der))

res=der_str[:]
print ("==Sequence== Length",len(der_str[0]))

for r in res:
  r=binascii.hexlify(r).decode()
  print ("Found: ",r)
  if (r[0:2]=="30"):
    print (" ==Sequence==")
    der_str=DerSequence()
    der_str.decode(binascii.unhexlify(r))
    res=der_str[:]
    for i in range(0,len(der_str[:])):
      r=binascii.hexlify(res[i]).decode()
      print ("String: ",len(r))
      if (r[0:2]=="06"):
        print ("   Obj ID tag (06)")
        der_str=DerObjectId()
        der_str.decode(binascii.unhexlify(r))
        print ("    ID algorithm: ",der_str.value)
  elif (r[0:2]=="03"):
    print ("   Obj ID tag (03)")
    der_str=DerBitString()

    der_str.decode(binascii.unhexlify(r))
    bits =binascii.hexlify(der_str.value)
    print ("    Bit value: ",binascii.hexlify(der_str.value))
    pkx=bits[2:64+2].decode()
    pky=bits[64+2:128+2].decode()
    print (f"   Public key: ({pkx},{pky}")

# "wx" : "2927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838",
# "wy" : "00c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e"

A sample run [here]:

==Sequence== Length 21
Found:  301306072a8648ce3d020106082a8648ce3d030107
 ==Sequence==
String:  18
   Obj ID tag (06)
    ID algorithm:  1.2.840.10045.2.1
String:  20
   Obj ID tag (06)
    ID algorithm:  1.2.840.10045.3.1.7
Found:  034200042927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e
   Obj ID tag (03)
    Bit value:  b'042927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e'
   Public key: (2927b10512bae3eddcfe467828128bad2903269919f7086069c8c4df6c732838,c7787964eaac00e5921fb1498a60f4606766b3d9685001558d1a974e7341513e

Here is the code:

sigs3
A Python repl by billbuchananreplit.com

References

[1] RFC 3279, Algorithms and Identifiers for the Internet X.509 Public Key Infrastructure Certificate and Certificate Revocation List (CRL) Profile [here]

A list of object IDs:

Hashing:
MD2	1.2.840.113549.2.2
MD5	1.2.840.113549.2.5
SHA-1	1.3.14.3.2.26
SHA-224	2.16.840.1.101.3.4.2.4
SHA-256	2.16.840.1.101.3.4.2.1
SHA-394	2.16.840.1.101.3.4.2.2
SHA-512	2.16.840.1.101.3.4.2.3

Public key:

RSA Encryption	1.2.840.113549.1.1.1
DSA	1.2.840.10040.4.1
Diffie-Hellman (dhPublicNumber)	1.2.840.10046.2.1
ECC (ecPublicKey)  	1.2.840.10045.2.1  
md2WithRsaEncryption	1.2.840.113549.1.1.2

Signatures:
md5WithRsaEncryption	1.2.840.113549.1.1.4
sha1WithRsaEncryption	1.2.840.113549.1.1.5
sha224WithRsaEncryption  	1.2.840.113549.1.1.14
sha256WithRsaEncryption	1.2.840.113549.1.1.11
sha384WithRsaEncryption	1.2.840.113549.1.1.12
sha512WithRsaEncryption	1.2.840.113549.1.1.13
dsaWithSha1	1.2.840.10040.4.3
dsaWithSha224	2.16.840.1.101.3.4.3.1
dsaWithSha256	2.16.840.1.101.3.4.3.2
ecdsaWithSha1  	1.2.840.10045.4.1  
ecdsaWithSha224	1.2.840.10045.4.3.1
ecdsaWithSha256	1.2.840.10045.4.3.2
ecdsaWithSha384	1.2.840.10045.4.3.3
ecdsaWithSha512	1.2.840.10045.4.3.4

Password Base Encryption Algorithms:
pbeWithMd2AndDesCbc	1.2.840.113549.1.5.1
pbeWithMd5AndDesCbc	1.2.840.113549.1.5.3
pbeWithSha1AndDesCbc	1.2.840.113549.1.5.10
pbeWithMd2AndRc2Cbc	1.2.840.113549.1.5.4
pbeWithMd5AndRc2Cbc	1.2.840.113549.1.5.6
pbeWithSha1AndRc2Cbc	1.2.840.113549.1.5.11
pbeWithSha1And40BitRc2Cbc	1.2.840.113549.1.12.1.6
pbeWithSha1And128BitRc2Cbc	1.2.840.113549.1.12.1.5
pbeWithSha1And40BitRc4	1.2.840.113549.1.12.1.2
pbeWithSha1And128BitRc4	1.2.840.113549.1.12.1.1
pbeWithSha1And3DesCbc	1.2.840.113549.1.12.1.3

Symmetric Encryption Algorithms:
DES CBC	1.3.14.3.2.7
3DES CBC	1.2.840.113549.3.7
RC2	1.2.840.113549.3.2
ArcFour	1.2.840.113549.3.4
AES CBC 128  	2.16.840.1.101.3.4.1.2
AES CBC 256	2.16.840.1.101.3.4.1.42

x.500 Distinguished Name Attributes:
name	2.5.4.41
surname	2.5.4.4
given name	2.5.4.42
initials	2.5.4.43
generation qualifier	2.5.4.44
common name	2.5.4.3
locality name	2.5.4.7
state or province name	2.5.4.8
organization name	2.5.4.10
organizational unit name	2.5.4.11
title	2.5.4.12
dnQualifier	2.5.4.46
country name	2.5.4.6
email address	1.2.840.113549.1.9.1
domain component	0.9.2342.19200300.100.1.25
street address	2.5.4.9
postal code	2.5.4.17
mail	0.9.2342.19200300.100.1.3
serial number              	2.5.4.5

ECC names:
secp192r1	1.2.840.10045.3.1.1
secp224r1	1.3.132.0.33
secp256r1	1.2.840.10045.3.1.7
secp384r1	1.3.132.0.34
secp521r1	1.3.132.0.35
brainpoolP160r1	1.3.36.3.3.2.8.1.1.1
brainpoolP192r1	1.3.36.3.3.2.8.1.1.3
brainpoolP224r1	1.3.36.3.3.2.8.1.1.5
brainpoolP256r1	1.3.36.3.3.2.8.1.1.7
brainpoolP320r1	1.3.36.3.3.2.8.1.1.9
brainpoolP384r1	1.3.36.3.3.2.8.1.1.11
brainpoolP512r1	1.3.36.3.3.2.8.1.1.13