File & Data Persistence

Files & data in a C++ canister are persisted as follows:

What	Files	Data in static memory	Other data
canister install	deleted	deleted	deleted
canister reinstall	deleted	deleted	deleted
canister upgrade	saved	deleted	deleted
update call	updated	updated	unchanged
query call	unchanged	unchanged	unchanged

Under the hood, the following happens:

• Files are stored in the canisters stable memory, by the file system of ic-wasi-polyfill
• The C++ static memory segment is using the canisters Orthogonal Persistence

To see it in action, please review the following demos:

• files: A demo of the virtual file system: open/close/read/write, ascii & binary, upload/download
• counter: An Orthogonal Persistence demo for uint64_t
• counters: An Orthogonal Persistence demo for std::vector
• counter4me: An Orthogonal Persistence demo for std::unordered_map<std::string, uint64_t>
• memory: Orthogonal Persistence of C++ Data Structures on the Internet Computer: A Study

Writing a file over several calls

A file that is written over several calls, e.g. uploaded in chunks, must be reopened without emptying it. An std::ofstream opened with std::ios::out alone, like fopen "w", truncates the file:

// The first chunk: create the file, or empty it
std::ofstream file(path, std::ios::binary);

// Every later chunk: keep what is there, and write at the offset
std::fstream file(path, std::ios::binary | std::ios::in | std::ios::out);
file.seekp(offset);
file.write(chunk.data(), chunk.size());

To add at the end instead, open it with std::ios::app.

icpp-pro 7.0.0rc3 and higher truncate like POSIX. Before, a truncated file kept its earlier chunks in stable memory, and a write at an offset made them visible again, so code that reopened a file with std::ios::out alone appeared to work.

Upgrade hooks

A canister can export canister_pre_upgrade and canister_post_upgrade. The IC calls canister_pre_upgrade on the old wasm right before an upgrade replaces it, and canister_post_upgrade on the new wasm right after.

Construct an IC_API with the entry point of the hook, just like in any other endpoint:

void canister_pre_upgrade() WASM_SYMBOL_EXPORTED("canister_pre_upgrade");
void canister_post_upgrade() WASM_SYMBOL_EXPORTED("canister_post_upgrade");

// Called on the OLD wasm, right before an upgrade replaces it.
// A trap here aborts the upgrade, and the old wasm stays installed.
void canister_pre_upgrade() {
  IC_API ic_api(CanisterPreUpgrade{std::string(__func__)}, false);

  UpgradeHistory history = load_history();
  history.pre_upgrade_count += 1;
  history.last_pre_upgrade_caller = ic_api.get_caller().get_text();
  save_history(history);
}

// Called on the NEW wasm, right after the upgrade installed it.
void canister_post_upgrade() {
  IC_API ic_api(CanisterPostUpgrade{std::string(__func__)}, false);

  UpgradeHistory history = load_history();
  history.post_upgrade_count += 1;
  save_history(history);
}

Keep in mind:

• A trap in canister_pre_upgrade aborts the upgrade, and the old wasm stays installed. Keep the hook small.
• Use icpp-pro 6.1.0 or higher. Older releases trap when you construct an IC_API inside canister_pre_upgrade.
• An upgrade has no caller message to reply to: do not call to_wire in either hook, nor from_wire in canister_pre_upgrade, which receives no argument.
• Data on the Wasm heap is deleted by an upgrade, files are saved (see the table above). Persist what a hook must hand over in a file.
• Do not use the raw stable memory API: the files live in stable memory.

The full example is the files demo.