Amazing difference on two different files systems

Back in March 2008 I bought a new external drive, copied my WampServer directory over to it, and came back to a strange sight: the same files, but a wildly different “size on disk”. My first thought was that something had gone wrong during the copy. Checking the file counts showed both copies were complete — the difference was the file system. The old copy sat on a FAT32 volume; the new one was NTFS with a 4 KB allocation unit size.

Every file system allocates disk space in fixed-size units called clusters (or allocation units). A file that doesn’t fill its last cluster wastes the remainder, and that waste is called slack space. The larger the cluster, the more space every small file wastes on average — and a web development stack like WampServer is full of small files. With 16 KB clusters, ten thousand small files can easily waste a couple of hundred megabytes; with 4 KB clusters, far less.

Default cluster sizes in Windows

Windows picks the cluster size from the volume size at format time. The defaults below come from Microsoft’s documentation (KB 140365, now archived):

• NTFS: 4 KB for anything between 2 GB and 2 TB — which covers essentially every volume you will ever format on a PC.
• FAT32: 4 KB up to 8 GB, 8 KB from 8–16 GB, and 16 KB from 16–32 GB (the largest FAT32 volume Windows will format).
• exFAT: 32 KB for 256 MB–32 GB, and 128 KB above 32 GB.

That’s the whole mystery: the old drive used FAT32 with big clusters, the new one NTFS with 4 KB clusters, and “size on disk” reflected the difference. The actual file sizes were identical.

What this means today

• Accept the default allocation unit size when formatting — Windows’ defaults are sensible for nearly every workload. For the background, see Microsoft’s overview of Windows file systems.
• For external drives, prefer NTFS (Windows-only use) or exFAT (cross-platform). FAT32 can’t hold files larger than 4 GB, and Windows won’t format volumes above 32 GB as FAT32.
• Directories with many tiny files — node_modules is the 2020s equivalent of a WampServer folder — will always show a much larger “size on disk” than their logical size. That’s slack space, not a copy error.