--- library_name: transformers license: mit pipeline_tag: text-generation tags: - qfs - fixture - random-init - cpu - reproducibility - random-architecture-fixture - deepseek_v4 - tiny --- # DeepSeek V4: a tiny toolchain fixture A small, independently initialized **random-weight reference model** for learning, debugging, and reproducing QFS capture workflows. It is a base/root fixture, not a quantized child. **These weights are untrained, not an assistant, and not a language-quality benchmark.** Generated text has no useful semantic quality. No upstream trained weights or training data are implied by an architecture name. ## Weight lineage This checkpoint was initialized directly for an architecture test. It was **not fine-tuned from a production model** and does not inherit that model's trained weights. Architecture/configuration lineage is documented separately from weight lineage. Browse the [Random Architecture Fixtures collection](https://huggingface.co/collections/malaiwah/qfs-random-architecture-fixtures-6a9f071c3f740e024ce15b72). A random base may also serve as the common source for an aligned quantization family. ## At a glance | Property | Observed value | |---|---| | Artifact role | Base/reference model; capture role `root` | | Serialized top-level weight files | 814,660 bytes (0.777 MiB); metadata, tokenizer and evidence excluded | | Generated parameters | 339,037 | | Included vision parameters | 0; inclusion is not vision-quality evidence | | Saved semantic tensors | 112 (parameters and buffers are not interchangeable) | | Fixture vocabulary | 272 tokens; independent byte tokenizer, not upstream vocabulary | | CPU stack | Python 3.12; Torch 2.11.0+cpu; Transformers 5.16.1; two Torch threads in the recorded capture workflow | | Remote model code | Explicit immutable consent required; see below | Size is serialized artifact size, **not** runtime RAM or a capacity/performance guarantee. Packed-array element counts are not model parameter counts. ## What is actually included Complete native-primitives text class with explicit reviewed per-instance RMSNorm arithmetic correction: weighted normalization in FP32 followed by final cast to input dtype. This is not bit-identical native Transformers execution or the complete upstream release. No vision encoder/aligner, DSpark, MTP, original FP8/FP4 storage execution, full-size numerical alignment, or million-token claim. HCA rate is intentionally 16 rather than production 128; CSA rate remains 4 and sliding window is 8 rather than 128. Random weights are not a quality benchmark. The repository file inventory includes `config.json`, `generation_config.json`, `tokenizer.json`, `tokenizer_config.json`, `build-manifest.json`, `support.json`, `requirements-cpu.txt`. Exact build/runtime/license inventories are linked below. A file may describe historical provenance without being an executable entry point. ## Good community uses—and boundaries - Learn how to download pinned artifacts, seal a small synthetic token panel, capture hidden states and replay a full vocabulary head. - Debug model-family adapters, strict tensor loading, storage decoders, or reproducibility tooling without downloading a production-sized checkpoint. - Reproduce a narrowly scoped result, report an adapter/reader regression, and retain the source, panel and runtime identities needed to explain it. **Not established:** trained-model accuracy; useful instruction following; quantizer optimization quality; original production-weight compatibility; GPU/NPU/serving-kernel parity; cross-hardware determinism; long-context behavior outside the recorded panel; throughput or paid-compute admission. ### Original-release limitations - **Forbidden workaround:** Never remove quantization_config/expert_dtype from real original-format checkpoint bytes, suppress conversion errors, or allow random initialization/mismatched tensors to obtain a nominal pass. - **Native bf16 route:** Use explicit pinned corrected-runtime override for complete BF16 text checkpoints; unmodified native5.16.1 fails at FP32-retained RMSNorm/BF16Linear. For an original config without auto_map, selected code repository supplies authenticated auto_map ONLY; original config/quantization/weight bytes remain unchanged. A BF16 derivative must retain converter/version/input/output hashes and is not original-format execution proof. - **Native source coverage:** Installed finegrained_fp8 integration includes packed FP4 and FP8 experts/linear dispatch through CUDA DeepGEMM or Triton; this CPU BF16 workflow does not measure those paths. - **Published storage:** - **Activation scheme:** dynamic - **Expert dtype:** fp4 - **Fmt:** e4m3 - **Quant method:** fp8 - **Scale fmt:** ue8m0 - **Weight block size:** 128; 128 - **Qualification required:** Actual pinned full-checkpoint loading without missing/mismatched/conversion errors, exact scope for unused speculative/vision tensors, supported original-format GPU kernels and independent captured evidence. No original large weights are downloaded or rented compute used here. - **Quantizer plan bug:** hf_capture --drop-parallel-plan neutralizes unused single-process TP/EP plans to bypass the HF5.16.1 quantizer plan rewrite defect. This changes in-memory plans, not checkpoint quantization metadata/bytes, and is recorded by QFS. ## Model family, root dataset and evidence This model is the **base/root of its own random fixture family**, not a reproduction of the trained upstream model. The [fidelity-root dataset repository](https://huggingface.co/datasets/malaiwah/deepseek-v4-tiny-fidelity-root-v1) is the family reference location. A link is not a claim that registration or publication has completed. The [historical CPU evidence bundle](https://huggingface.co/datasets/malaiwah/deepseek-v4-tiny-cpu-repro-v1/tree/6b750c0e06573009e25bf6c70028c362c37138d9) retains two independent captures at `first/` and `repeat/`, plus `comparison/`. Its repository root is a **receipt bundle, not a single canonical QFS dataset**. Those recorded same-machine/same-stack forced comparisons reported 0.0 nats on 252 synthetic scored positions; that is not a trained-quality result or a transferable hardware floor. The original detailed receipt and caveats remain authoritative. ## Runtime requirements and safe local reproduction ### Replay the published evidence without loading a model This uses QFS's existing **NumPy FP64 comparator in a Torch-free environment**. It reads the stored hidden states and each side's own head. No model forward, remote model code, GPU, upload or registry mutation is involved. Runtime receipts name the actual backend; last-bit differences from another FP64 implementation are not a new quality claim. Synthetic exact controls remain zero. Set `QFS` to a reviewed Quant Fidelity Suite checkout and use Bash: ```bash : "${QFS:?Set QFS to your reviewed QFS checkout}" WORK=$(mktemp -d) export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2 python3.12 -m venv "$WORK/replay-env" "$WORK/replay-env/bin/pip" install 'numpy==2.5.3' 'huggingface-hub==1.30.0' "$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \ --revision f53b204091c988ce4a2161af81886f3018745559 --include 'roots/deepseek-v4/**' --local-dir "$WORK/evidence" "$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \ "$WORK/evidence/roots/deepseek-v4/first" --verify-tensors "$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" verify \ "$WORK/evidence/roots/deepseek-v4/repeat" --verify-tensors "$WORK/replay-env/bin/python" "$QFS/bin/fidelity_dataset.py" compare \ --reference "$WORK/evidence/roots/deepseek-v4/first" \ --candidate "$WORK/evidence/roots/deepseek-v4/repeat" \ --out "$WORK/replayed" --device cpu --replay-device numpy --replay-dtype float32 \ --vocab-chunk 8192 --verify-tensors --self-compare --force-compute ``` Reconstructed format comparisons are intentionally **advisory** and normally return exit code 2 while writing a valid receipt. Inspect that receipt; do not silence refusals or interpret an advisory result as a production-quality ranking. ### Capture the actual checkpoint Capture uses a separate pinned Torch CPU environment. The input below is the **original token-panel format**, not a sealed capture's internal `panel/` folder. The native source, tokenizer and model revisions remain explicit. Set `AUTHOR` to your own HF handle; the dataset repository argument is attribution only and nothing is uploaded by these commands. ```bash : "${AUTHOR:?Set AUTHOR to your Hugging Face handle}" "$WORK/replay-env/bin/hf" download malaiwah/deepseek-v4-tiny-random-bf16 --revision 185723bec10ee57db86e8c269b3c6a811122a0ca --local-dir "$WORK/model" "$WORK/replay-env/bin/hf" download malaiwah/deepseek-v4-tiny-random-bf16 --revision 185723bec10ee57db86e8c269b3c6a811122a0ca --local-dir "$WORK/source" "$WORK/replay-env/bin/hf" download malaiwah/qfs-fixture-root-captures-v1 --repo-type dataset \ --revision f53b204091c988ce4a2161af81886f3018745559 --include 'requirements-capture.txt' --include 'roots/deepseek-v4/input-panel/**' \ --local-dir "$WORK/inputs" python3.12 -m venv "$WORK/capture-env" "$WORK/capture-env/bin/pip" install -r "$WORK/inputs/requirements-capture.txt" "$WORK/capture-env/bin/python" "$QFS/bin/fidelity_dataset.py" architectures prepare \ --architecture deepseek-v4 --model-dir "$WORK/model" \ --model-repository malaiwah/deepseek-v4-tiny-random-bf16 --model-revision 185723bec10ee57db86e8c269b3c6a811122a0ca \ --panel "$WORK/inputs/roots/deepseek-v4/input-panel" --tokenizer-root "$WORK/source" \ --author "$AUTHOR" --dataset-repository "$AUTHOR/deepseek-v4-tiny-random-bf16-capture" \ --dataset-id "fidelity--$AUTHOR.deepseek-v4-tiny-random-bf16" --out "$WORK/workflow" \ --trust-remote-code --code-repository malaiwah/deepseek-v4-tiny-random-bf16 --code-revision 4f9dcfcbabdbe336f5f83e836dd8fac7138dda23 ``` Inspect `workflow.json`. This launcher executes its exact capture/verification commands and selects the Torch-free interpreter only for the final comparison: ```bash export OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 OPENBLAS_NUM_THREADS=2 "$WORK/replay-env/bin/python" - "$WORK/workflow/workflow.json" <<'PY' import json, subprocess, sys workflow = json.load(open(sys.argv[1])) for step in workflow["commands"]: argv = list(step["argv"]) if step["step"] == "compare": argv[0] = sys.executable result = subprocess.run(argv) if result.returncode: raise SystemExit(result.returncode) PY ``` Use the actual root at [`malaiwah/deepseek-v4-tiny-fidelity-root-v1@c86180f1844f932b2e685eeeed96b633e4c54cfb`](https://huggingface.co/datasets/malaiwah/deepseek-v4-tiny-fidelity-root-v1/tree/c86180f1844f932b2e685eeeed96b633e4c54cfb). The [community collection](https://huggingface.co/collections/malaiwah/qfs-test-fixtures-and-reproducible-captures-6a9efbe7e77c03e879c7cde4) groups models and captures. Custom code, where required above, is explicitly pinned and executed only after your consent; hash verification is provenance, **not a sandbox**. ## Licensing and detailed provenance - **Fixture weights and workflow:** MIT - **Native runtime:** Transformers5.16.1, Apache-2.0 dependency; native config/model source SHA256 recorded during build. Derived deepseek_v4_qfs.py retains Apache notices and LICENSE-APACHE-2.0, not MIT. - **Remote code:** Fixture captures explicitly opt in to reviewed deepseek_v4_qfs.py from model repository at immutable code SHA; code closure verified by QFS before execution. Generator may import the reviewed local wrapper directly. Weights/tokenizer remain local; HF network permitted for pinned code verification, not entirely offline capture. - **Synthetic panel:** MIT; authored text, no upstream training data - **Upstream text and vision:** MIT; Copyright (c)2023 DeepSeek; pinned originals retained at stage as lineage-LICENSE and vision-lineage-LICENSE
Immutable provenance, historical cards, source inventories and full caveats - Model/card snapshot used while authoring: [malaiwah/deepseek-v4-tiny-random-bf16@185723bec10ee57db86e8c269b3c6a811122a0ca](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/tree/185723bec10ee57db86e8c269b3c6a811122a0ca). - [Full preserved earlier model card](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/README.md), SHA-256 `d034249beb37f995c2d33769830f0ffd5a2331fd795d5f07ceb6d85360117cfa`. This is an immutable historical record, including original build pins, measured resource tables, command logs, limitations and failed-attempt provenance. Historical publisher commands are **not** part of the local-use recipe above. - [build-manifest.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/build-manifest.json). - [support.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/support.json). - [qfs-source.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/qfs-source.json). - [evidence-summary.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/evidence-summary.json). - [native-verification.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/native-verification.json). - [regeneration-verification.json](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/regeneration-verification.json). - [LICENSE-APACHE-2.0](https://huggingface.co/malaiwah/deepseek-v4-tiny-random-bf16/blob/185723bec10ee57db86e8c269b3c6a811122a0ca/LICENSE-APACHE-2.0). Selected original provenance fields (full tensor/component evidence remains in the linked inventories): ```json { "compress_rates": { "compressed_sparse_attention": 4, "heavily_compressed_attention": 16 }, "compress_ratio_selectors": [ 0, 0, 4, 128 ], "generation": "Complete native FP32 random initialization with seeded hash-router buffers; native BF16 conversion retaining native-required FP32 state.", "generator_sha256": "0801acbd80e6eab6d7891c384c8fd7e3778f7e4e92ad75a7fff79831adb237ab", "layer_types": [ "sliding_attention", "sliding_attention", "compressed_sparse_attention", "heavily_compressed_attention" ], "lineage": { "relationship": "text architecture lineage only; no upstream weights copied", "repository": "deepseek-ai/DeepSeek-V4-Flash-0731", "revision": "7872f01b1d1fe23eabc4c98b48bffcef5a386062" }, "native_fp32_exceptions": [ "model.layers.0.self_attn.sinks", "model.layers.0.self_attn.q_a_norm.weight", "model.layers.0.self_attn.kv_norm.weight", "model.layers.0.input_layernorm.weight", "model.layers.0.post_attention_layernorm.weight", "model.layers.0.attn_hc.fn", "model.layers.0.attn_hc.base", "model.layers.0.attn_hc.scale", "model.layers.0.ffn_hc.fn", "model.layers.0.ffn_hc.base", "model.layers.0.ffn_hc.scale", "model.layers.1.self_attn.sinks", "model.layers.1.self_attn.q_a_norm.weight", "model.layers.1.self_attn.kv_norm.weight", "model.layers.1.input_layernorm.weight", "model.layers.1.post_attention_layernorm.weight", "model.layers.1.attn_hc.fn", "model.layers.1.attn_hc.base", "model.layers.1.attn_hc.scale", "model.layers.1.ffn_hc.fn", "model.layers.1.ffn_hc.base", "model.layers.1.ffn_hc.scale", "model.layers.2.self_attn.sinks", "model.layers.2.self_attn.q_a_norm.weight", "model.layers.2.self_attn.kv_norm.weight", "model.layers.2.self_attn.compressor.position_bias", "model.layers.2.self_attn.compressor.kv_norm.weight", "model.layers.2.self_attn.compressor.indexer.position_bias", "model.layers.2.self_attn.compressor.indexer.kv_norm.weight", "model.layers.2.input_layernorm.weight", "model.layers.2.post_attention_layernorm.weight", "model.layers.2.attn_hc.fn", "model.layers.2.attn_hc.base", "model.layers.2.attn_hc.scale", "model.layers.2.ffn_hc.fn", "model.layers.2.ffn_hc.base", "model.layers.2.ffn_hc.scale", "model.layers.3.self_attn.sinks", "model.layers.3.self_attn.q_a_norm.weight", "model.layers.3.self_attn.kv_norm.weight", "model.layers.3.self_attn.compressor.position_bias", "model.layers.3.self_attn.compressor.kv_norm.weight", "model.layers.3.mlp.gate.e_score_correction_bias", "model.layers.3.input_layernorm.weight", "model.layers.3.post_attention_layernorm.weight", "model.layers.3.attn_hc.fn", "model.layers.3.attn_hc.base", "model.layers.3.attn_hc.scale", "model.layers.3.ffn_hc.fn", "model.layers.3.ffn_hc.base", "model.layers.3.ffn_hc.scale", "model.norm.weight", "model.hc_head.hc_fn", "model.hc_head.hc_base", "model.hc_head.hc_scale" ], "parameter_count": 339037, "scope": "Complete native-primitives text class with explicit reviewed per-instance RMSNorm arithmetic correction: weighted normalization in FP32 followed by final cast to input dtype. This is not bit-identical native Transformers execution or the complete upstream release. No vision encoder/aligner, DSpark, MTP, original FP8/FP4 storage execution, full-size numerical alignment, or million-token claim. HCA rate is intentionally 16 rather than production 128; CSA rate remains 4 and sliding window is 8 rather than 128. Random weights are not a quality benchmark.", "seed": 20260907, "versions": { "safetensors": "0.8.0", "tokenizers": "0.23.2", "torch": "2.11.0+cpu", "transformers": "5.16.1" }, "vision_parameter_count": 0, "reviewed_code_pin": { "repository": "malaiwah/deepseek-v4-tiny-random-bf16", "revision": "4f9dcfcbabdbe336f5f83e836dd8fac7138dda23" } } ```