#include <name>, the library inside the binary

mc carries its standard library and its own source inside the executable, compressed, and serves them through the angle-bracket form of #include. That is what makes one downloaded binary the whole toolchain: no checkout, no include path, no install step.

// expect-exit: 0
// expect-stdout: 42
#include <sys>
#include <prelude>

i64 main() {
    i64 n = 0;
    while (n < 42) { n = n + 1; }
    putnum(n);
    write(1, "\n", 1);
    return 0;
}

<name> is served by the bundle or it is an error — there is no fallback to the working directory, on purpose: <name> means "a library that is not in my tree", and the answer must be a function of (this binary, this project's lock, the installed packages) and of nothing else.

$ mc prog.mc -o prog.o
prog.mc:1: unknown bundled include: no/such/module

#include "path" is unchanged: the includer's own directory first, then each [include].paths root in order. Angle brackets are libraries, quotes are my files.

Since M44 the bundle is the SECOND of three places a <name> can come from. Before it comes a package mc.lock pins, which is why a project may override a bundled name — [deps] float = "1.3.0" makes <float> that tree instead of this blob, pinned by a content hash and re-checked on every build. After it comes the installed copy of the compiler's own package, which a binary that carries the blob never reaches. A project with no [deps] never leaves the blob, and its output is byte for byte what it was. See packages.md § 2.

The catalogue #

The manifest is tools/bundle.list, one NAME<TAB>PATH per line, sorted by name: 83 entries, plus mc/bundle_data, which is regenerated on demand (see below). Those are the names <...> accepts.

The system layer — pick exactly one #

O_RDONLY/O_WRONLY/O_CREAT/O_TRUNC live in each system layer, not in <io>, because they are per-system values (O_CREAT is 0x200 on macOS, 0x40 on Linux and 0x100 in the Windows layer, where the flags are not passed to the system at all).

The language layer #

The compiler itself #

<mc/core> is the whole compiler minus exactly one function, void user_init(). Including it and supplying that function is a taught compiler (hooks.md).

<mc/version>, and why the version is in the bundle (M44) #

src/version.mc is one function, mc_version(), returning a string literal: 0.0.0-dev in the repository and the tag in a released binary, rewritten by scripts/set-version.sh in the release job and never committed. It is bundled for the same reason <mc/host> is resolved per binary: a taught compiler must report the version of the binary that built it. mc --exe over <mc/host> + <mc/core> + a module produces a compiler out of the blob, so if the blob's copy said 0.0.0-dev a release binary would build compilers that lie about their version — and, from M44's install step, look in the wrong ~/.mc/libs/mc/v<version>/. That is why scripts/set-version.sh's second act is make bundle, and why scripts/check-bundle.sh guards the sentinel. See ../ci.md § Versioning.

<mc/host>, the one name that is not an entry #

<mc/core> is host-neutral: it says nothing about posix_spawnp, the environment, the O_* values or which (os, arch) pair the binary is. That comes from a host file, and a compiler needs exactly one of them, included before the core.

<mc/host> is how a source asks for "the one that matches whichever mc is compiling me". It is resolved in src/core_bundle.mc (host_bundle_open) to host_include(), which each host file answers for itself — mc/host_macos, mc/host_linux_aarch64, mc/host_linux_x86_64, mc/host_windows_aarch64 or mc/host_windows_x86_64 — and the entry it lands on is what the once-only include list records, so writing <mc/host> and <mc/host_macos> in the same program includes the file once, not twice.

#include <mc/host>
#include <mc/core>

void user_init() { }

That is a complete compiler, and it is exactly what mc build generates for a [compiler] section (../build.md). See ../guide/90-linux-host.md for what the host layer answers.

<float> — f32 and f64 (M24) #

A LIBRARY, not a compiler feature. The stock mc has no floats: nothing here is in lib/user_default.mc, and a float program is built by a taught compiler the way examples/api and examples/lang are.

The generality proofs (M24 step 2) #

Three modules the core has never heard of, each with an empty git diff src/.

examples/avx/ is the third and is not bundled: it is an example directory with its own README, and it teaches one AVX instruction by its encoding.

The demonstrations #

Everything make check-surface wires up is bundled too, so the demos can be reproduced from a downloaded binary with no checkout:

The parts of the core (M41) #

<mc/core> is the sum of seven parts, each a bundled name of its own. A recreated compiler names the parts it wants and writes its own main(); src/core.mc is literally those seven plus src/main.mc, and scripts/check-parts.sh compiles both spellings and cmps the two objects, so this table cannot drift from the code.

The two files M41 split out are bundled under their own names too: <mc/objmodel> (src/objmodel.mc, the section/symbol/relocation model every writer reads) and <mc/cli> (src/cli.mc, mc_main()). <mc/macho> is now the Mach-O writer alone.

<mc/core_sandbox> (M43) needs nothing but <mc/core_min> and the host file: every system call it issues goes through host_syscall6() and is named by an SN_* index of <mc/sysno>, whose per-architecture number tables are <mc/sysno_linux_aarch64> and <mc/sysno_linux_x86_64> — the two files that also carry sys6, the raw shim. Those three are bundled because a Linux host file includes one of them, and mc build writes #include <mc/host> for a taught compiler.

Spelled out, the whole compiler is:

#include <mc/host>
#include <mc/core_min>
#include <mc/core_machines>
#include <mc/core_writers>
#include <mc/core_build>
#include <mc/core_bundle>
#include <mc/core_pkg>
#include <mc/core_sandbox>
#include <mc/main>
#include <user_default>

and docs/guide/98-recreating-the-compiler.md is what each omitted line costs, in bytes and in capability.

<mc/core_pkg> (M44) is the one part that names another: it #includes core_build.mc, because resolving a dependency needs the driver that reads mc.toml. The split is deliberate and it is where the milestone's debloat lives — the READ side of packages ([deps], mc.lock, the tree hash, #include <pack/file.mc>, the refusals) is <mc/core_build>'s src/deps.mc, so a compiler WITHOUT <mc/core_pkg> still builds a project from its lock and its deps/ tree. That compiler has no fetcher, no registry, no MVS and no lock writer at all, and it advertises neither mc pkg nor mc update: the usage IS the subcommand table. tests/pkg/nopkg.mc is that compiler, and scripts/check-pkg.sh builds the vendored fixture project with it.

A part stands on <mc/core_min> alone, and scripts/check-parts.sh compiles each of the six optional parts on top of the minimal one to say so. It is not free: four names had to move when M41 landed, because the full assembly hides a cross-part dependency completely. tm_cat and tm_num_str went from src/toml.mc to src/arena.mc (src/cli.mc needs the first for --include= and src/backend_coff.mc the second for a long section name); MODE_755 went from src/backend_exe.mc to src/arena.mc (src/driver.mc uses it for mkdir -p); and R_X86_PC32/R_X86_PLT32 went from src/machine_x86_64.mc to src/objmodel.mc, which is where a relocation kind the machine emits and the writer maps belongs. None of the four changed a byte of generated code — they are a #define and three leaf functions.

The naming rule. A bundle name's last path component is the file's basename, because a relative #include inside a bundled file resolves by joining and then, failing that, by last component (see below), and tools/bundle.mc refuses a manifest where two entries share one. That is why the parts are core_min and not core-min: src/core_min.mc cannot be reached as core-min, so a hyphenated bundle name would be a name nothing inside the bundle could include.

Your own bundle #

A debloated compiler can keep #include <name> with its own, much smaller library, at zero core lines. tools/bundle.mc is four includes — <mc/arena>, <lz>, <mc/bundle_data>, <mc/bundle> — over a manifest of NAME<TAB>PATH lines, and all four are bundled names. So:

1. write the same four-line tool against those names and build it with mc --exe;
2. run it over your own manifest to generate your own bundle_data.mc;
3. have your compiler include that file plus <mc/bundle> instead of <mc/core_bundle>, and call lex_set_bundle(&bundle_open) from its main().

src/bundle.mc needs nothing but BUNDLE_COUNT and the two arrays the generator emits. The one thing you give up is <mc/host>: that name is resolved by host_bundle_open in src/core_bundle.mc, which is four lines you copy if you want it.

Relative includes inside a bundled file #

The bundle is flat: mc/lex is one name, not a directory and a file. But src/core.mc still says #include "arena.mc" and src/driver.mc still says #include "../lib/prelude.mc", and both must keep working when the including file is itself bundled. So the lexer joins and normalises the name the usual way, drops a trailing .mc, and looks the result up; if that misses, it retries with the last path component.

tools/bundle.mc refuses a manifest in which two entries share a last component, so that fallback can never be ambiguous.

A bundled file is pushed onto the same #include stack a real one uses, with the bundled name in place of a path — which is what a diagnostic then shows:

syntax_demo_test:10: type expected at top level

mc/bundle_data, the file the bundle cannot contain #

src/core.mc includes bundle_data.mc, and src/bundle_data.mc is the bundle. It cannot be inside itself: its own bytes would change the bytes it contains. So it is the one name that is regenerated on demand — bundle_find("mc/bundle_data") answers with index BUNDLE_COUNT, and the file is written out again from the blob and index already in memory, by the very same bundle_emit that tools/bundle.mc uses. One definition of the format, so the two cannot drift.

Since M21.5 bundle_emit has two modes, and the difference is arena, not format:

An array initializer costs one AST node per element, so spelling the 180 KB blob out cost ~22 500 nodes — 2.3 MB of arena — in every taught compiler that includes <mc/core>. As #embed it costs exactly one node (N_BLOB). mc/bundle.bin is therefore the second name that is not in the blob (index BUNDLE_BIN = BUNDLE_COUNT + 1): it is bundle_blob, served straight out of the compiler's own data with no inflate and no copy, rounded up to a multiple of 8 so both forms declare a global of the same size.

The disk copy keeps the u64 form because the frozen stage0/lex.c has no embed in its dir_names[] and build/mc0 src/mc.mc is the seed step of make mc1. Both forms produce the same object, which is what makes the split safe — and what check-standalone measures.

scripts/check-standalone.sh proves the consequence in the strongest form available: a compiler built from #include <mc/core> + #include <user_default>, in an empty directory, compiles src/mc.mc into an object byte for byte identical to build/mc2.o.

The format, and regenerating it #

src/bundle_data.mc is generated source, checked in. It holds one blob — the NUL-terminated names first, in manifest order, then each LZ77 stream — and one flat index with four values per entry (name offset, stream offset, compressed size, real size). Names live in the blob rather than as string literals because every literal counts against the C seed's budget, and the index is one array rather than four for the same reason.

make bundle          # regenerate src/bundle_data.mc from tools/bundle.list
make check-bundle    # prove the checked-in copy is exactly what comes out

make bundle is the only way that file is ever written. Run it whenever a lib/*.mc or a core module changes, before make bootstrap — make check runs check-bundle first precisely so that a stale bundle fails with a message naming make bundle, instead of failing later as a mysterious fixed-point difference.

The bundle is inside the fixed point: mc1 and mc2 both carry it, and the objects they produce must still be identical. See ../guide/70-bootstrap.md.

#embed and the bundle #

#embed NAME "path" [lz] (directives.md) reads its payload the same way its includer was read: from disk for a real file, and from the bundle when the directive was written inside a bundled <name> include. <embed_demo> and <embed_demo.txt> are in the manifest exactly so that this path is exercised from a binary with no checkout.

Edit this page