What’s Magic In Cybersecurity?

Well, it’s those special magic numbers that give the game away
Photo by Artem Maltsev on Unsplash

What’s Magic In Cybersecurity?

Well, it’s those special magic numbers that give the game away

Well, in cybersecurity and digital forensics, a magic number is a special way of investigating a file type (such as an image or a document), without looking at the file extension. In this way we can scan documents in network connections or on disks in a fast way, and where we only have to sample the first (or last) few bytes to discover certain types of files. An intruder may hide a file type with a different file extension, such as to change an .EXE to a .PDF, and avoid virus scanners and intrusion detection systems (IDSs). With magic number detection, we can dive into the file and look for certain patterns of data.

Magic numbers

Sometimes we need to scan a disk at a low level and determine the files that are contained on a disk. One method of determining the files is to look for standard signatures, normally using standard sequences at the start of the file. I’ve tried to gather as many of these signatures as possible for key file types (see Table 1) [here]. For example, an Abobe Illustrator file should start with the hex sequence of 0x25, 0x50, 0x44, 0x46 (which is the ASCII characters of %PDF), and which shows that it is a standard PDF file. If we scan a disk and find this signature, it may thus be an Illustrator file.

Table 1: Magic Numbers

PNG File

PNG files provide high-quality vector and bitmapped graphic formats. They have a magic number of 0x89 0x50 0x4E 0x47 0x0D 0x0A 0x1A 0x0A. The following gives a sample listing for a real PNG file:

https://www.asecuritysite.com/magic/png?file=bg.png

The starting part of the file shows the magic number:

[00000000] 89 50 4E 47 0D 0A 1A 0A   .PNG....
[00000008] 00 00 00 0D 49 48 44 52   ....IHDR
[00000016] 00 00 00 F3 00 00 00 C3   ........
[00000024] 08 06 00 00 00 57 8C 27   .....W.'
[00000032] 92 00 00 00 04 67 41 4D   .....gAM
[00000040] 41 00 00 AF C8 37 05 8A   A....7..
[00000048] E9 00 00 00 19 74 45 58   .....tEX

A demonstration of this is given in:

GIF file

The GIF file format uses a file signature of 0x47 0x49 0x46 0x38 0x39 0x61 (GIF89a) in the first few bytes of the file. After this, the key fields are then Width (16 bits), Height (16 bits), Packed (8 bits), Color Index (8 bits) and Aspect (8 bits), followed by a colour table of 256 24-bit colours. This means that GIF files have a good resolution of the colour of a pixel, but only have 256 different colours, which limits their scope. For example, it is not good for photographs, as these typically need thousands of colours.

A sample analysis is:

http://asecuritysite.com/magic/gif?file=cat01_with_hidden_text.gif

which analyses this image:

An example header is then:

[00000000] 47 49 46 38 39 61 64 00   GIF89ad.
[00000008] 55 00 E6 00 00 FF FF FF   U.......
[00000016] F7 F7 F6 F1 F4 F2 EE EE   ........
[00000024] EF E7 E7 E7 E1 E4 E6 DF   ........

It should be noted that I have added a covert message into the colour table (which will only affect a few pixels — where a few pixels change their colour):

[00000048] A1 CC CC CC C4 C8 CC 68   .......h
[00000056] 65 6C 6C 6F C0 D1 C6 84   ello....
[00000064] C0 BF BD BD BB B8 B8 B6   ........

A presentation on this is at:

PKZIP File

The PKZIP file format is used to compress files, and, potentially encrypt them. It can be identified with the magic number of 0x504B0304 at the start of the file, followed by a fairly standard structure format of:

Version: 14 00
 General purpose bit flag: 02 00
 Compression method: 08 00
 File last modification time: 80 9D
 File last modification date: 6C 39
 CRC: DA4DB80F
 Compessed size: 90010000
 Uncompressed size: 27060000
 File name length: 0900
 Extra field length: 0000
 Filename: anim.xaml

The following shows an example with a real file:

http://www.asecuritysite.com/magic/zip?file=anim.zip

where we see the following at the start of the file:

[00000000] 50 4B03 04 14 00 02 00    PK......
[00000008] 08 00 80 9D 6C 39 DA 4D   ....l9.M

A presentation is here:

An interesting fact is that Office 2010 files, such DOCX, XLSX, and so on, in an XML format, which has a PKZIP compressed format. This can be seen with:

http://asecuritysite.com.com/magic/docx?file=hello.docx

[00000000] 50 4B03 04 14 00 06 00    PK......
[00000008] 08 00 00 00 21 00 09 24   ....!..$
[00000016] 87 82 81 01 00 00 8E 05   ........
[00000024] 00 00 13 00 08 02 5B 43   ......[C
[00000032] 6F 6E 74 65 6E 74 5F 54   ontent_T
[00000040] 79 70 65 73 5D 2E 78 6D   ypes].xm
[00000048] 6C 20 A2 04 02 28 A0 00   l....(..

Table 1: Magic file numbers

Graphics formats

Here are a few other magic numbers:

  • JPEG Analysis. JPEGs. This provides an analysis of JPEG files.
  • GIF Analysis. GIFs. This provides an analysis of GIF files.
  • TIF Analysis. TIFs. This provides an analysis of TIF files.
  • PNG Analysis. PNGs. This provides an analysis of PNG files.
  • BMP Analysis. BMPs. This provides an analysis of BMP files.
  • PSD Analysis. PSDs. This provides an analysis of PSD files.
  • WMF Analysis. WMF. This provides an analysis of WMF files.
  • Adobe Illustrator Analysis. AI. This provides an analysis of AI files.
  • Adobe In Design Analysis. INDD. This provides an analysis of INDD files.
  • MIDI file. MIDI. This provides an analysis of MID files.
  • ICO file. ICO. This provides an analysis of ICO files.
  • PS file. PS. This provides an analysis of PS files.
  • EPS file. EPS. This provides an analysis of EPS files.
  • VSD file. VSD. This provides an analysis of Microsft Visio (VSD) files.

Video and audio formats

Here are a few other magic numbers:

  • MP3 Analysis. MP3s. This provides an analysis of MP3 files.
  • AVI Analysis. AVIs. This provides an analysis of AVI files.
  • Flash SWF Analysis. SWFs. This provides an analysis of Flash files.
  • Flash FLV Analysis. FLVs. This provides an analysis of FLV files.
  • MP4 Analysis. MP4. This provides an analysis of MP4 files.
  • MOV Analysis. MOV. This provides an analysis of MOV files.
  • WMV Analysis. WMV. This provides an analysis of WMV files.
  • WMA Analysis. WMA. This provides an analysis of WMA files.
  • JPEG 2000 Analysis. JPEG 2. This provides an analysis of JPEG 2000 files.
  • WAV Analysis. WAV. This provides an analysis of WAV files.

Zips, Archives and Library files

Here are a few other magic numbers:

  • ZIP/Archive Analysis. ZIPs. This provides an analysis of ZIP/Archive files.
  • GZ Analysis. GZ. This provides an analysis of GZip files.
  • Tar Analysis. Tar. This provides an analysis of TAR files.
  • MSI Analysis. MSI. This provides an analysis of MSI files.
  • OBJ Analysis. OBJ. This provides an analysis of OBJ files.
  • DLL Analysis. DLL. This provides an analysis of DLL files.
  • CAB Analysis. CAB. This provides an analysis of CAB files.
  • EXE Analysis. EXE. This provides an analysis of EXE files.
  • RAR Analysis. RAR. This provides an analysis of RAR files.
  • SYS Analysis. SYS. This provides an analysis of SYS files.
  • HLP Analysis. HLP. This provides an analysis of HLP files.
  • VMDK Analysis. VMDK. This provides an analysis of VMDK files (VMWare Virtual Disks).
  • PST Analysis. PST. This provides an analysis of PST files (Outlook Email File).
  • JAR Analysis. JAR. This provides an analysis of JAR files.
  • SLN Analysis. SLN. This provides an analysis of Microsoft SLN files.
  • Class Analysis. Class. This provides an analysis of Java Class files.
  • Zlib. Zlib. This provides an analysis of Zlib files.

Documents

Here are a few other magic numbers:

  • PDF Analysis. PDFs. This provides an analysis of PDF files.
  • DOC Analysis. DOCs. This provides an analysis of DOC files.
  • RTF Analysis. RTFs. This provides an analysis of RTF files.
  • XLS Analysis. XLSs. This provides an analysis of XLS files.
  • PPT Analysis. PPTs. This provides an analysis of PPT files.
  • MMP File. MMP. This provides an analysis of MMP files.
  • DOCX Analysis. DOCX. This provides an analysis of DOCX files.
  • XLSX Analysis. XLSX. This provides an analysis of XLSX files.
  • PPTX Analysis. PPTX. This provides an analysis of PPTX files.
  • MDB Analysis. MDB. This provides an analysis of MDB files.
  • Outlook Message Analysis. MSG. This provides an analysis of MSG files.

Disk Image

Here is another magic number:

  • Raw disk format. RAW. This provides an analysis of the RAW disk format.

Conclusions

And so, if you’re into cybersecurity, magic numbers are just magic! Also, if an adversary erases the magic number, the file will still work, but there will still be tell-tale signs of a file type (such as colour maps, video tags, and so on).

Finally, let’s see if you can make a million on magic numbers:

https://asecuritysite.com/tests/fun?sortBy=forensics01