Tutorial Datasets

Tutorial Datasets#

The quantem.widget.datasets module provides small, named examples for tutorials, smoke tests, and Colab notebooks. Choose a named example; image and scan previews also take an explicit size. The loader handles Hugging Face paths, cache layout, and calibration details.

from quantem.widget.datasets import (
    gold_session,
    show1d_ducky,
    show2d_gold,
    show3d_gold,
    show4dstem_gold,
    showpdf_ta,
    showpdf_ta_ground_truth,
)

Images and stacks come back as quantem core Dataset2d and Dataset3d (show2d_gold, show3d_gold), and the 4D-STEM scan as every 4D-STEM file reads (read_4dstem): a quantem.gpu.io.Dataset4dstemGPU when quantem.gpu is installed and a GPU is present, else a quantem core Dataset4dstem (show4dstem_gold). showpdf_ta returns a quantem core Dataset4dstem with its detector calibrated in 1/Å, and showpdf_ta_ground_truth the (r, g) arrays of its atomic model’s g(r). On an Intel Mac or Windows on ARM, which cannot install quantem core, the widget’s stand-in carries the same array, name, sampling, units, metadata, shape, ndim and dtype. show1d_ducky and gold_session return folders.

show1d_ducky() selects one collection and has no size argument. For helpers that expose a size selector:

Size	Intended use
`small`	Documentation, Colab, CI smoke tests, quick first view
`medium`	Better visual detail for local notebooks
`large`	Local workstation review
`full`	Full available tutorial source where practical

For Show1D, the synthetic Ducky example is a file-backed monitor run.

Ducky simulation tutorial: Georgios Varnavides, Trademark ducky dataset (October 23, 2025). The viewer files contain subsequent reconstruction results for the synthetic example.

from quantem.widget import Show1D
from quantem.widget.datasets import show1d_ducky

run = show1d_ducky()
widget = Show1D.from_monitor_file(
    run / "show1d_monitor.jsonl",
    title="Synthetic Ducky reconstructions (tutorial: Georgios Varnavides)",
    x_label="frame",
    y_label="final loss",
    log_scale=False,
)
widget

Shared inputs and revisions#

Each helper downloads one input family from bobleesj/quantem-data, then makes its documented preview locally. The default is immutable revision 5925e4b49b2ca28ff82d6d5a6b9a8bfea9edfb20, so later folder changes cannot silently alter an example.

Tutorial / helper	Canonical input	Local operation
Show2D, widget export / `show2d_gold`	`haadf/gold_haadf_npy`	Explicit image stride; sampling multiplied by that stride
Show3D / `show3d_gold`	Same HAADF input	Moving crops for demonstration, not measured time/depth frames
Show4DSTEM / `show4dstem_gold`	`4dstem/gold_128_npy_bin8`	Scan stride only; detector sampling and uint16 counts preserved
Image readers, folder watching / `gold_session`	`widget/gold_session`	Existing 26 EMDs; original acquisition names
Show1D, reconstruction comparison / `show1d_ducky`	`widget/ducky`	Monitor JSONL and every relative snapshot path retained
ShowPDF / `showpdf_ta`, `showpdf_ta_ground_truth`	`4dstem/ta_amorphous_sim`	None; simulated scan and its model’s g(r), pinned at the revision that adds them

The helpers use the canonical paths above. An explicit revision must contain those paths; loaders do not search retired folders or another revision. To reproduce an older folder layout, use its matching software and data revision. The Ducky helper returns its one monitor collection. The gold session has one published size (small); no additional measured data is generated.

small describes the returned preview, not necessarily the network transfer: the shared HAADF source is about 33.55 MB and the reduced diffraction source is about 18.88 MB. Show2D and Show3D download the same HAADF file once into the Hub cache. force_download=True explicitly refreshes it; cache_dir selects a cache location.

For general retrieval, quantem.data owns qualified IDs and revisions:

import quantem.data as qdata

folder = qdata.download(
    "4dstem/gold_128_npy_bin8",
    revision="5925e4b49b2ca28ff82d6d5a6b9a8bfea9edfb20",
)

This qualified/revision API requires the matching updated quantem.data implementation. It is prepared locally; no compatible wheel release or Colab installation is claimed here. Existing friendly helpers use the pinned canonical inputs without that new resolver. Native detector acquisitions are opened with quantem.gpu.io.load; the Ducky monitor and EMD image session are different input types, not raw 4D-STEM acquisitions.

Show3DSlices, ShowCIF and Plot2D generate their small teaching inputs locally. Keep those recipes in the software repository. Use Hugging Face for reusable input data, numeric metadata, checksums, credit and minimal data cards. Keep notebook code, helpers, training overrides, checkpoints and rendered reports in their owning software or private results repository. This migration reuses existing public inputs; it adds no new scientific data or private files. See Contribute tutorial data.

Diffraction example#

The ShowDiffraction notebook uses a labeled deterministic synthetic magnetite control. No measured Fe3O4 pattern is published, so there is no download helper for that missing input. Open your own measured image with ShowDiffraction.

Pair distribution function example#

showpdf_ta() and showpdf_ta_ground_truth() read 4dstem/ta_amorphous_sim/ (data.npy, meta.json and ground_truth_gr.npz) at revision fe907499a3a2c1f827c8827c0b3472c5b08f47bf, the dataset commit that added that folder (the next commit after the default revision above); pass revision to read another. The scan is a simulation of an atomic model, not a measured acquisition.

Reference#

Tutorial inputs: real acquisitions and synthetic reconstruction examples.

Image and scan loaders take a friendly size name and return a calibrated quantem dataset (or a folder of files), so the tutorials never ship data in git: a quantem core Dataset2d/Dataset3d for images and stacks, and for the 4D-STEM scan a quantem.gpu.io.Dataset4dstemGPU on a GPU or a quantem core Dataset4dstem elsewhere (the widget’s stand-in where quantem core cannot install).

quantem.widget.datasets.gold_session(*, size: str = 'small', cache_dir: str | Path | None = None, revision: str | None = None, force_download: bool = False, verbose: bool = True) → Path#

Download the public gold HAADF session for image-reader tutorials.

The pinned default uses widget/gold_session. The one measured session is downloaded intact; size is retained for existing notebook calls.

Parameters:

size – Tutorial size, normally "small".
cache_dir – Download cache, dataset revision and refresh controls.
revision – Download cache, dataset revision and refresh controls.
force_download – Download cache, dataset revision and refresh controls.
verbose – Print the number of downloaded EMD files.

Returns:

Cached folder containing the public EMD images.

Return type:

Path

Examples

>>> folder = gold_session()
>>> images = sorted(folder.glob("*.emd"))

quantem.widget.datasets.show1d_ducky(*, cache_dir: str | Path | None = None, revision: str | None = None, force_download: bool = False, verbose: bool = True) → Path#

Download reconstruction snapshots for the synthetic Ducky example.

The returned folder contains show1d_monitor.jsonl and snapshot .npy files. Open it with quantem.widget.Show1D.from_monitor_file().

The original Trademark ducky dataset simulation tutorial is by Georgios Varnavides (October 23, 2025). These files contain subsequent reconstruction results, not measured detector data.

Parameters:

cache_dir – Optional Hugging Face cache directory.
revision – Hugging Face dataset revision. The default pins the documented inputs; an explicit revision is never mixed with files from another revision.
force_download – If True, ask Hugging Face Hub to refresh the cached files.
verbose – If True, print a short dataset summary.

Returns:

Local folder containing the Show1D monitor run.

Return type:

Path

quantem.widget.datasets.show2d_gold(*, size: str = 'small', cache_dir: str | Path | None = None, revision: str | None = None, force_download: bool = False, verbose: bool = True)#

Load the gold HAADF Show2D tutorial image by friendly size name.

The source image is downloaded once from haadf/gold_haadf_npy; every size but "full" keeps every Nth pixel of it (stride 8, 4, 2) and says so in one line. The same source is reused by show3d_gold().

quantem.widget.datasets.show3d_gold(*, size: str = 'small', cache_dir: str | Path | None = None, revision: str | None = None, force_download: bool = False, verbose: bool = True)#

Load the gold HAADF Show3D tutorial stack by friendly size name.

The stack is built from moving crops of the shared gold HAADF tutorial source instead of storing a second copy of the image for Show3D. Every size but "full" crops from every Nth pixel and says so in one line.

quantem.widget.datasets.show4dstem_gold(*, size: str = 'small', cache_dir: str | Path | None = None, revision: str | None = None, force_download: bool = False, verbose: bool = True)#

Load the gold 4D-STEM Show4DSTEM tutorial scan by friendly size name.

The source scan is stored once under 4dstem/gold_128_npy_bin8; "small" and "medium" keep every 4th or 2nd scan position and say so in one line. The file is read as every 4D-STEM file is (read_4dstem): a quantem.gpu.io.Dataset4dstemGPU that stays on the GPU when quantem.gpu and a GPU are present, else a quantem core Dataset4dstem.

quantem.widget.datasets.showpdf_ta(*, cache_dir: str | Path | None = None, revision: str | None = 'fe907499a3a2c1f827c8827c0b3472c5b08f47bf', force_download: bool = False, verbose: bool = True)#

Load the simulated amorphous Ta 4D-STEM scan used by the ShowPDF tutorial.

The scan is downloaded from 4dstem/ta_amorphous_sim: a 20 x 20 scan of 307 x 307 float32 patterns simulated from a 1,024,000-atom amorphous Ta model, with the detector calibrated in 1/Å (k = 1/d). The model’s number density is 1,024,000 / 268.6053³ atoms/Å³.

Parameters:

cache_dir – Optional Hugging Face cache directory.
revision – Hugging Face dataset revision; defaults to the pinned one.
force_download – If True, ask Hugging Face Hub to refresh the cached files.
verbose – If True, print a short dataset summary.

Returns:

The calibrated scan, a quantem core Dataset4dstem (the widget’s stand-in where quantem core cannot install).

Return type:

Dataset4dstem

quantem.widget.datasets.showpdf_ta_ground_truth(*, cache_dir: str | Path | None = None, revision: str | None = 'fe907499a3a2c1f827c8827c0b3472c5b08f47bf', force_download: bool = False)#: g(r) of the atomic model behind showpdf_ta: (r, g) arrays, r in Å.

Tutorial Datasets

Contents

Tutorial Datasets#

Shared inputs and revisions#

Diffraction example#

Pair distribution function example#

Reference#