Typed Data, Bytes & FFI

By the end of this lesson you will be able to explain why raw byte buffers (Uint8List, ByteData and friends) exist alongside List<int>, correctly predict what happens when a value overflows a fixed-width slot, read and write multi-byte numbers in either byte order, spot and use aliased views over the same memory, encode/decode text and binary formats by hand, and describe — with appropriate hedging about platform and package boundaries — how dart:ffi lets Dart call into C code.

1. Why typed data: boxed ints vs packed bytes

A List<int> is like a row of full-size lockers: every locker is big enough to hold a small note OR a forwarding slip pointing at a bigger box elsewhere, and nobody ever checks how big the number you hand it is. A Uint8List is a completely different object: a single strip of tiny, fixed-size compartments glued edge to edge, each one EXACTLY one byte (8 bits) wide, holding a raw number from 0 to 255 and nothing else — no forwarding slips, no room for anything bigger.

Every value you've stored in a List<int> so far lives in a normal, general-purpose slot: word-sized (able to hold a small integer directly, called a "Smi" internally, or a reference to a bigger boxed number), with no fixed range. Typed data — the classes in dart:typed_data like Uint8List, Int32List and Float64List — throws that flexibility away on purpose: every element is packed edge-to-edge as raw bytes of one fixed width and one fixed interpretation (unsigned/signed integer, or floating point). That trade gives you three things a boxed list can't: a predictable, compact memory layout (exactly N bytes per element, no per-element overhead), the exact byte layout a file format, network protocol, image, or C library expects, and — because there's no boxing/unboxing — real speed for large numeric buffers.

Here is the whole family at once, with the exact size of each element and of a few lists (every line below is real output):

And the memory comparison with List<int>, for one million values:

The first time most people meet typed data, they assume it behaves like List<int> with "extra strictness" — e.g. that storing an out-of-range value throws an error. It doesn't. Every fixed-width integer typed-data list truncates silently: it keeps only the low bits that fit and throws the rest away, with no exception and no warning. If your values might exceed the element type's range, you must check that yourself before storing.

Try it yourself with different values — including ones you'd guess would be an error:

Wrapping versus clamping. A normal integer typed list wraps (keeps the low bits). Uint8ClampedList is the one exception: it clamps, pinning anything below 0 to 0 and anything above 255 to 255 — the behaviour image code usually wants. Floats round instead:

Dart's own documentation describes this precisely: storing a value into an integer typed-data list stores the value's two's-complement bit pattern, truncated (or, for signed lists, sign-extended) to that element's width. That's why -1 stored into a Uint8List slot reads back as 255 — -1's two's-complement bit pattern is all-1s, and the low 8 bits of "all 1s" is exactly the unsigned byte value 255. It's the same underlying bits every time; only how the element type CHOOSES to read those bits (as signed or unsigned) changes what number you see.

The two's-complement reading from the card above, as code (toUnsigned(8) / toSigned(8) do the same reinterpretation in one call):

One more rule: a typed list has a fixed length. It can be read and written at any valid index, but it can never grow or shrink, and a bad index throws a RangeError (it does not wrap around):

Input size → what's feasible: up to ~10⁴ values → a plain List<int> is fine; 10⁶ bytes → Uint8List is 1 MB against roughly 4–8 MB for a List<int> (one reference-sized slot per element: 4 bytes with compressed pointers, 8 without); 10⁸ samples → typed data (100 MB vs ~400–800 MB) is the only sensible choice.

List<int>: general-purpose, no size limit, never truncates, some per-element overhead. Typed data (Uint8List, Int16List, Uint32List, Float64List, …): fixed-width, packed, exact byte layout, silently truncates/reinterprets out-of-range values — you own the range-checking.

2. ByteData, views over one buffer, endianness, aliasing

Picture one long shelf of labelled cubbyholes — that's a ByteBuffer, a raw block of bytes with no opinion about what's stored in it. A Uint8List, an Int32List and a ByteData are three different PAIRS OF READING GLASSES you can put on to look at that SAME shelf: one glasses-pair sees single bytes, another groups every 4 cubbyholes into one 32-bit number, a third lets you ask for any width and byte order on demand. Swap glasses and the shelf underneath hasn't moved an inch — you're just interpreting the same bytes differently.

Before the views, here is every ByteData method family in action (each call takes a byte offset; wider reads take an Endian):

ByteData is the most flexible typed-data view: instead of one fixed element type, it gives you explicit methods — getUint8, setInt16, getUint32, setFloat64, and so on — each taking a byte offset and, for anything wider than one byte, an Endian (byte order) argument. Endian.big writes the most-significant byte first (at the lowest address); Endian.little writes the least-significant byte first. The two produce completely different byte sequences for the exact same number.

Mixing up byte order is one of the most common real bugs in networking and file-format code: a value written as big-endian but read back as little-endian (or vice versa) doesn't crash — it just silently produces a different, wrong number. Most network protocols (TCP/IP headers, most binary file formats like PNG and ZIP-central-directory fields) use big-endian, historically called "network byte order"; most desktop/mobile CPUs default to little-endian internally. Always be explicit — never rely on Endian.host (the current platform's native order) for anything that leaves the process, gets saved to disk, or is read by another machine.

Because Uint8List, Int32List, ByteData and friends are all just different "glasses" over the same underlying ByteBuffer, you can build several views over ONE buffer — and a write through any one of them is instantly visible through every other view of the same bytes. This sharing is called aliasing:

Uint8List.sublistView(buffer, start, end) and ByteData.sublistView(buffer, start, end) create a view over a SUBRANGE of an existing typed-data object's buffer, again with zero copying — this is the standard way to hand a slice of a larger buffer (e.g. "the payload of this network packet, skipping its header") to code that expects its own typed-data list. Contrast this with Uint8List.fromList(source), which allocates a brand-new buffer and copies every byte in — after that call, the two lists share nothing, and mutating one never affects the other.

Input size → what's feasible: every ByteData read or write is O(1), so a file with 10⁶ fields parses in well under a second; views never copy, so slicing a 10⁸-byte buffer is O(1) while sublist() or fromList() is O(n).

Same bytes, different view = aliasing: writes through any view are visible through every other view of those bytes, instantly, with no synchronization needed (and no thread-safety either — that's a separate concern). Same bytes, but built with .fromList(...) = an independent copy. Confusing the two is a classic source of "I mutated A and B changed too!" bugs — or, just as often, "I mutated the view but the original list never changed!" bugs when a copy was made where a view was expected.

3. Encoding text: utf8, base64, hex

A Dart String is text; a Uint8List is raw bytes. Encoding is the translation step between the two — exactly like converting a spoken sentence into Morse code (bytes) so it can travel down a wire, and decoding is translating the Morse code back into words.

dart:convert ships this translation for the two most common byte-facing encodings: UTF-8 (utf8.encode/utf8.decode) turns text into bytes and back, using 1 byte per character for plain ASCII and up to 4 bytes for other Unicode characters; Base64 (base64.encode/base64.decode) turns arbitrary BYTES into plain-ASCII TEXT (so binary data can be safely embedded inside things that only understand text, like JSON strings or email bodies) by regrouping every 3 bytes' 24 bits into four 6-bit chunks, each mapped to one of 64 printable characters.

A common mix-up: treating Base64 as encryption or compression. It is neither — it's purely a byte-to-text RESHAPING with no secrecy (anyone can decode it instantly) and it actually makes data slightly BIGGER (roughly 4 bytes of Base64 text for every 3 bytes of input, plus up to 2 padding = characters at the end). Its only job is "make these bytes safe to put somewhere that expects text."

A hex dump — formatting each byte as exactly two hexadecimal digits — is the simplest possible byte-to-text view, and the one you'll see most often in debuggers and protocol docs, because each pair of hex digits maps to exactly one byte with no grouping math required.

Input size → what's feasible: utf8, base64 and hex are all O(n): 10⁶ characters take milliseconds. Base64 output is ⌈n/3⌉·4 characters, so a 10 MB file becomes ~13.3 MB of text; keep large payloads as bytes whenever the channel allows it.

utf8 ⇄ bytes: for TEXT. base64 ⇄ text: for reshaping arbitrary BYTES so text-only channels can carry them, at a ~33% size cost. Hex dump: a human-readable debugging view of raw bytes, two characters per byte, not meant to be sent anywhere.

4. Binary protocols: headers and bit-packed flags

Think of a binary file format like a PNG or a ZIP archive as a form with fixed-position boxes to fill in: the first few bytes are always a "magic" signature so a reader can instantly reject a file that clearly isn't the right kind, then a version number, then a length telling the reader exactly how many more bytes of payload follow. There's no punctuation, no field names in the bytes themselves — the READER'S CODE is the only thing that knows "bytes 0-3 mean this, byte 4 means that."

Parsing a binary header is mechanical once you know the layout: read the magic bytes and compare them, read fixed-width fields at fixed offsets (using ByteData/sublistView for anything wider than a byte, with the byte order the format specifies), and refuse to continue the moment something doesn't match — never guess.

Real formats are strict about validating BEFORE trusting: check the magic first, check the buffer is long enough for the fields you're about to read before you read them (reading past the end of a real native buffer in other languages is undefined behaviour; Dart's typed-data lists throw a RangeError instead, which is safer but still a crash you should prevent with an explicit length check), and only then interpret the payload. Skipping validation is exactly how a malformed or malicious file causes a crash — or worse, a misinterpreted read — deep inside a parser.

The other classic binary-protocol trick is bit-packing: cramming several independent yes/no flags into the individual BITS of one integer, instead of spending a whole byte (or a whole field) per flag. | (OR) turns a bit on without disturbing any other bit, & with a bitwise-complement (& ~bit) turns one bit off without disturbing any other, and & (AND) alone tests whether a bit is on.

Varints, zigzag and checksums

Many binary formats (Protocol Buffers, WebAssembly, DWARF) store integers as varints (the LEB128 scheme): each byte carries 7 payload bits plus its top bit (0x80) as a "more bytes follow" flag, so small numbers cost one byte. A number with b significant bits needs ⌈b / 7⌉ bytes — at most 9 for a non-negative 64-bit int:

Negative numbers break that: in two's complement −1 has every bit set, so it would need 10 bytes. Zigzag encoding interleaves signs — 0, −1, 1, −2, 2, … become 0, 1, 2, 3, 4, … — using (n << 1) ^ (n >> 63), so small magnitudes stay small:

A checksum detects accidental corruption. Adler-32 keeps two running sums modulo 65521: a = (a + byte) mod 65521 and b = (b + a) mod 65521, starting from a = 1, b = 0; the result is (b << 16) | a. It catches random damage, but is not secure against someone deliberately editing the data:

Input size → what's feasible: parsing one header is O(1); a file of 10⁶ chunks is O(chunks) if each chunk is a view; a varint is at most 9 bytes; Adler-32 over 10⁷ bytes is one pass (~0.1 s).

Takeaway. A binary header is "read fixed fields at fixed offsets, validate as you go." Bit-packed flags are "one int, many independent yes/no answers, manipulated with OR / AND / AND-NOT / XOR." Both techniques trade a little bit of manual bookkeeping for a much smaller, exactly-specified byte layout — which is the entire point of choosing typed data over a friendlier but larger general-purpose structure.

5. dart:ffi: calling C from Dart

FFI stands for Foreign Function Interface — a formal, guarded doorway between Dart's managed, garbage-collected world and a completely different country next door: raw, unmanaged native (C) memory, where nothing is tracked, nothing is bounds-checked, and nothing is ever freed for you automatically. Every trip across that doorway (a pointer going out, a pointer coming back) is a promise you're making by hand.

dart:ffi is a core SDK library (no extra package needed) for calling into native C-compatible libraries directly from Dart — this is how Flutter/Dart code can reuse existing C, C++ (via a C-compatible wrapper) or Rust (via its C ABI) libraries instead of rewriting them. Its main pieces: Pointer<T> (a raw address, annotated with the native type it points at — Pointer<Int32>, Pointer<Utf8>, etc.), Struct (a Dart class whose field layout is defined to match a C struct's memory layout exactly, byte for byte, including padding), and DynamicLibrary (loads a native shared library — DynamicLibrary.open('libfoo.so') for an external file, or DynamicLibrary.process()/DynamicLibrary.executable() to look up symbols already loaded into the current process — then .lookupFunction<CSig, DartSig>('name') hands you a callable Dart function backed by the real native one).

Current Dart also offers a second, newer way to declare a single native function: mark an external Dart function with the @Native<CSig>(symbol: 'cFunctionName') annotation instead of manually calling DynamicLibrary.lookupFunction — the tool-chain resolves the symbol and generates the call for you at compile time. It's the recommended shape for a small, fixed set of native functions known ahead of time; DynamicLibrary + lookupFunction is still what you reach for when the library path or the exact symbols to call are only known at runtime (plugin-style loading). Both are core dart:ffi, not a separate package.

More native-memory building blocks, all running for real: a native array indexed like C, a Struct whose byte layout (including padding) you can print, and a wrapper class that owns its memory:

The calloc/free animation above is reference only, because it needs package:ffi, which this lesson's check program does not install. The panels around it use only core dart:ffi and the C library's own abs, strlen, malloc and free through DynamicLibrary.process(); they were run on macOS (Dart 3.11, arm64) and rely on those symbols being visible in the process, which is a platform fact (true on macOS and Linux), not a guarantee on every platform. The one thing that IS a hard guarantee, on every platform: dart:ffi's native memory is never scanned or freed by the Dart garbage collector. calloc/free themselves live in a separate, commonly-paired package, package:ffi (not core dart:ffi) — and every calloc needs exactly one matching free: skip it and you leak that memory for the life of the process; call it twice, or touch the pointer afterwards, and you've triggered undefined behaviour, the same class of bug C programmers have always had to avoid by hand.

Input size → what's feasible: one FFI call is cheap but not free, so hand C a pointer to a whole buffer and let it loop, rather than making 10⁷ calls that each pass one byte; native memory is not garbage-collected, so 10⁶ un-freed 1 KB blocks leak a gigabyte.

package:ffigen is a build-time code generator: point it at a real C header file (a .h), and it auto-generates the matching Dart Pointer/Struct/DynamicLibrary bindings for you, so you don't hand-transcribe function signatures and struct layouts (and risk a padding/alignment mistake that corrupts memory silently). It's the recommended approach for any non-trivial native library; hand-writing bindings, as shown above, is really only for a handful of functions. Exactly which C constructs ffigen supports, and its exact CLI/config shape, are tool details that evolve across versions — treat "ffigen exists and generates bindings from headers" as the stable idea to remember.

Dart's GC manages the Dart heap only. Anything you get from calloc/malloc/a native allocator is YOUR responsibility, start (allocate) to finish (free), exactly once each. ffigen is the practical way to get correct bindings for anything beyond a toy example — don't hand-write C struct layouts if you can generate them instead.

6. When to reach for typed data

Typed data earns its extra rigidity whenever the SHAPE of your data is fixed by something outside your program: a file format, a network wire format, a hardware/driver boundary, a native library's C struct, or simply a huge amount of same-typed numbers where per-element boxing overhead would be wasteful. If you're just holding "some integers my Dart code will iterate over," a plain List<int> remains the right default — it's more flexible (growable, no truncation surprises) and, for small collections, the tiny extra memory per element rarely matters.

Use case	Typical choice	Why
Decoded image pixels	`Uint8List` (or `Uint32List` for packed RGBA)	Exactly matches the byte layout image codecs and the GPU expect
Raw PCM audio samples	`Int16List`/`Float32List`	Fixed sample width, huge volume — boxing every sample would be far too slow/large
Network packets / file formats	`ByteData` + views	Exact, explicit byte order and offsets — the format's spec IS the layout
Cryptographic hashing/ciphers	`Uint8List`	Algorithms are specified in terms of raw byte/bit operations, not language-level ints
Calling a C library	`dart:ffi` `Pointer`/`Struct`	The C ABI has no concept of a Dart object — only raw memory layout
"A list of numbers my Dart logic uses"	plain `List<int>`/`List<double>`	No externally-fixed layout to match; flexibility and safety win

Input size → what's feasible: up to ~10⁴ values → List<int>; 10⁶ or more same-typed values, or any layout fixed by a file/wire format/C library → typed data.

Quiz

Interview questions

Cheat sheet

Term	Meaning
`Uint8List` / `Int8List`	1 byte per element, unsigned 0-255 / signed -128..127; silently truncates/reinterprets out-of-range stores
`Uint16List`/`Int32List`/`Float64List`, …	Same idea, fixed width per element (2/4/8 bytes), packed contiguously
`ByteBuffer`	The raw block of bytes underneath one or more typed-data views
`ByteData`	Explicit get/set methods for any width + `Endian`, over one `ByteBuffer`
`Endian.big` / `.little`	Most-significant-byte-first vs least-significant-byte-first storage order
`X.sublistView(buf, a, b)`	Zero-copy view over a subrange of an existing buffer — aliases the same bytes
`Uint8List.fromList(list)`	Allocates a NEW buffer and copies values in — independent from the source
Aliasing	Two+ views over the same bytes; a write through one is visible through all of them
`utf8.encode`/`decode`	Text ⇄ UTF-8 bytes
`base64.encode`/`decode`	Arbitrary bytes ⇄ plain-ASCII text (not encryption, not compression)
Hex dump	Each byte shown as 2 hex digits — a debugging view, not a wire format
Varint (LEB128)	Variable-length integer encoding: 7 payload bits + 1 continuation bit per byte
Zigzag encoding	Maps signed ints to small unsigned ones (0,-1,1,-2,2,… → 0,1,2,3,4,…) so varint stays compact for negatives
Adler-32 / CRC-32	Fast checksums for detecting accidental data corruption (not cryptographically secure)
`Pointer<T>`	A raw native address, typed by what it points at; no bounds checking
`DynamicLibrary`	Loads/looks up a native shared library and its exported functions
`@Native<CSig>(symbol: '…')`	Declares a single `external` Dart function bound to a native symbol at compile time — the modern alternative to manual `lookupFunction` for a fixed, known function
`NativeFinalizer`	Runs a native free callback when a Dart wrapper object is GC'd — a safety net, never a substitute for an explicit `dispose()`/`free()`
`calloc`/`free` (package:ffi)	Native memory allocation outside the Dart GC — every calloc needs exactly one free
ffigen	Generates Dart FFI bindings from a C header, avoiding hand-written struct-layout mistakes