How it works and what we know.
© Written mid 2021 by Teufelchen.
Last updated on Wed 29. Sep 2021
The original Xbox, published in 2001, has a standard IDE harddrive in the range of 8 to 10 GB. It utilises the FATX filesystem which is closely related to FAT16/32, which are all developed by Microsoft[1]. While FAT16/32 is generally well understood and documented, the FATX filesystem remains proprietary and has not been documented by Microsoft. There have been multiple attempts to reverse engineer it by the open source community[2] but none of them are complete, neither are they up to date. Most of the original authors have also left the xbox-scene, further hampering the development of modern FATX tooling. Last, none of the documentation is straightforward enough to allow for easy understanding of the inner workings of the FATX filesystem.
In this document, I will try to explain in detail how FATX works as simple as possible. Additionally, I will also attempt to test what knowledge of FATX is correct by testing and collecting evidence.
We know for fact that FATX is used on the internal harddrive of the original Xbox.
It is also used for the memory cards that can be inserted into the Xbox-controllers.
You can also find a derivation of FATX in the Xbox 360 system, its also called FATX but it is not clear if / how much they are compatible.
The hard disk drive of the Xbox features five partitions as well as a config area. This area is right at the start of the hdd and is 512 KiB in size. It's mainly used to store network and xbox live related configuration. It does not store a partition table - the partition offsets are directly hardcoded in the xbox kernel.
On modded consoles (both with softmods and modchips) this is often not the case. Instead, modified consoles often alter the kernel to use a partition table stored on the hdd.
In the graphics below, you can see an overview over the diffrent partitions and their sizes. More information about the xbox hdd can be found in the xboxdevwiki.net.
The FATX filesystem is build upon three diffrent sections. At the start is the superblock (0x0000 - 0x0FFF). It contains necessary metadata, such as the cluster size or the FATX identifier. It is followed by the FAT, which starts at 0x1000 and its size depends on the size of the partition. The rest of the partition is the actual data area, including directories and meta data.
The superblock provides basic, necessary information about the partition to the user (read: the OS, the fs-driver, what ever software interacts with it). It is always 4 KiB in size.
First offset | Last offset | Size | Name |
---|---|---|---|
0x0000 | 0x0003 | 4 bytes | Identifier |
0x0004 | 0x0007 | 4 bytes | SerialNumber |
0x0008 | 0x000B | 4 bytes | Cluster size |
0x000C | 0x000F | 4 bytes | Root dir / first cluster |
0x0010 | 0x0FFF | 4082 bytes | Unknown & Unused bytes |
The identifier is the first data you will encounter when reading in a FATX partition. It is used to identify a given partition as a FATX partition. Written in ASCII, it encodes "FATX". To verify a given image contains indeed a FATX filesystem, hexdump can be used:
hexdump -n 4 partition.img
0000000 46 41 54 58
0000004
This is a verified fact.
The serial number can be used to distinguish between partitions and should be unique.
It is send at time of formating using the systemcall KeQuerySystemTime()
.
hexdump -s 4 -n 4 partition.img
0000004 e0 d4 f9 22
0000008
The serial number has been researched an verified by aerosoul.
The cluster size determines how many sectors are used to build one cluster. A sector is 512 bytes. For the xbox hdd, the only valid value is 32
, resulting in a cluster size of 16 KiB (32x512 bytes = 16384 bytes
).
hexdump -e \"%i\" -s 0x000C -n 4 partition.img
32
This is almost proven. While the xbox has never used / produced a diffrent cluster size, no attempts have been made to feed the xbox with a partition using a diffrent cluster size.
Additionally, the Xbox Memory Units (XMU) are said to use 16 sectors per cluster, but I can't confirm that at the time of writing.
This field tells on which cluster the root dir of the filesystem is located. It is always set to 1
.
hexdump -e \"%i\" -s 12 -n 2 partition.img
1
This was verified by aerosoul.
The superblock is always 4 KiB in size. Most of the time, all extra bytes are set to zero. There are how ever some exceptions but it is not known what they are and what they encode.
On XMUs the nickname of the device is saved inside the superblock.
The File Allocation Table keeps track which cluster is used to store data and which one are free. It also sometimes called "Cluster chain map" as clusters can be "chained" in it, to allow storing files that are bigger than one single cluster. Each table entry represents one cluster on the disk. You can thereby calculate the number of entries of the FAT by dividing the size of the partition by the size of a cluster (always 16 KiB): 750MiB / 16 KiB = 48000
.
The values of a FAT entry can represent that the cluster is unused (free), part of a cluster chain, end of a cluster chain or that this cluster should not be used at all (i.e. bad sector on the harddrive).
On FATX16, one entry is two bytes wide, on FATX32 four bytes. Up until 65525 clusters, FATX16 is used otherwise FATX32.
The switching point for the FATX16-to-FATX32 transition is derivated from the normal FAT16/32 transition and is known to work. How ever, no precise testing has been done.
FATX16 | FATX32 | Name | Purpose |
---|---|---|---|
0x0000 | 0x00000000 | FREE | Marks this cluster as free |
0x0001 | 0x00000001 | ??? | Unknown / Unused |
0x0002 - 0xffef | 0x00000002 - 0xffffffef | CHAIN | This cluster is part of a chain |
0xfff0 - 0xfff6 | 0xfffffff0 - 0xfffffff6 | CHAIN | On regular FAT reserved - can be part of a chain on FATX |
0xfff7 | 0xfffffff7 | BAD | This cluster is probably bad |
0xfff8 | 0xfffffff8 | ROOT | This is the very first cluster of the FAT |
0xfff9 - 0xfffe | 0xfffffff9 - 0xfffffffe | ??? | Unknown / Unused |
0xffff | 0xffffffff | EOC | This cluster is the end of a chain |
There is a lot of uncertainty in the values provided in the table above. They should work as expected but are not yet proven.
Example FATX16 FAT. It shows the chaining of clusters. Blue (hex offset=1) marks the start of the fat, turquoise (3 & 4) are a file spanning two clusters. Yellow (6, 15, 16, 17, 1e) is a fragmented file and red (9 - 10) being a bigger file.
Size of the FAT, where entry size is 2 bytes in FATX16 and 4 in FATX32:
\[ FAT_{size} = {Partition\ size\ in\ bytes \over cluster\ size} * Entry_{size} \]
The size should always be rounded up to the nearest 4096 byte boundary.
I.e. the partition is 750MiB, using FATX16, cluster size is always 16KiB:
\[ FAT_{size} = {786.432.000\ bytes\over 16384\ bytes} * 2\ bytes = 96.000\ bytes \]
\[ boundary\ offset = 4096 - (96.000\ \%\ 4096) = 2304 \]
\[ FAT_{size,\ rounded} = FAT_{size} + boundary\ offset = 98304\ bytes \]
\[ Offset(Entry_{Id}) = (Entry_{Id} - 1) * Entry_{size} + Superblock_{size} \]
I.e. get offset for cluster 713 on FATX16:
\[
Offset(713) = (713 - 1) * 2 + 4\ KiB = 5520
\]
\[ Offset(Entry_{Id}) = (Entry_{Id} - 1) * Cluster_{size} + Superblock_{size} + FAT_{size} \]
I.e. get data offset for cluster 713 on FATX16, using 16 KiB clusters on a 750 MiB partition:
\[ Offset(713) = (713 - 1) * 16\ KiB + 4\ KiB + 98304\ bytes = 712 * 16\ KiB + 100\ KiB = 11.492\ KiB \]
The data area (= all clusters) stores all your files, directories and meta data. Any given cluster can either be a list of items[3] or raw data of a file.
A given item in such a list can either specify a directory or a file. As the stopping of the list is encode inside the items, we will look at them first.
An item is a 64 bytes long object, which stores meta data for a file or a directory. In both cases, the string field for the filename
is not null terminated. Instead, the first field filename size
holds the length of the name (max 42 chars). An other important field is the first cluster
. It contains the id/number of the FAT entry which specifies the cluster chain belonging to this item. It is important to realise that it is the reference to the FAT entry, not the cluster number itself, otherwise, the chaining of clusters could be missed. Every item also has a FAT-attributes
field, which is one byte long and contains eight binary flags (bit zero to seven).
Deleted items are marked by setting the first byte (filename size
) to 0xe5
.
Offset | Size | Description |
---|---|---|
0 | 1 | filename size |
1 | 1 | FAT-attributes |
2 | 42 | filename ASCII, padded with 0xff |
44 | 4 | first cluster |
48 | 4 | file size |
52 | 2 | Creation time |
54 | 2 | Creation date |
56 | 2 | Modification time |
58 | 2 | Modification date |
60 | 2 | Access time |
62 | 2 | Access date |
The flags in the FAT-attributes
field
Bit No. | Description |
---|---|
0 | Read only |
1 | Hidden |
2 | Maybe: System |
3 | Maybe: Volume label |
4 | Directory |
5 | Maybe: Archive |
6 | Maybe: Device |
7 | Reserved |
Everything marked with "Maybe" is just that, a guess. Do not rely on it.
A directory entry (sometimes called subdirectory) is a regular item entry where the file size
field is ignored and the bit no. 4 (mask 0x10) Directory
in the FAT-attributes
is set. As usual, the field first cluster
holds the FAT entry reference. Jumping to the referenced cluster(chain) must yield a new item list.
A regular file has bit no. 4 (mask 0x10) in the FAT-attributes
set to zero. The file size
field holds the exact file size in bytes. Concatenate the cluster chain starting at first cluster
should give a combined byte size bigger or equal to file size
. All excess bytes can be dropped.
Given the size of an item and the size of a cluster, there is a maximum list length of 16 KiB / 64 bytes = 256
items per cluster. As clusters can be chained, this can be extendet. However, it is not known if the Xbox supports that.
Every list is terminated by an item that is filled with 0x00
(64 times), giving you an maximum of 255 usable item entrys per list an a single cluster.
It is also possible to use 0xff
to fill an item as a list termination, it's unkwown which way is the intended way.
Thanks for reviewing this document english-wise, Luke Usher 🥰.
Thanks for your extended technical feedback, aerosoul 🥰.
Microsoft does not take all credit for FAT. There were several other companies that also adapted and improved upon FAT, helping making it the defacto standard it is today. Read up [https://en.wikipedia.org/wiki/File_Allocation_Table] for further details. ↩︎
i.e. This writing by Michael Steil: https://web.archive.org/web/20100617022009/http://www.xbox-linux.org/wiki/Differences_between_Xbox_FATX_and_MS-DOS_FAT or this one by Lucien Murray-Pitts and Andrew de Quincey https://web.archive.org/web/20020617181617/http://www.tardis.ed.ac.uk:80/~lucien/computing/projects/xbox/XBOX-disk-layout.htm ↩︎
Most of the time these are called "Directory entry list" or similar. How ever, I consider that to be confusing, hence renaming that to "item list". ↩︎