Scripts used to diagnose and repair the defects reported in dandi/helpdesk#154, across all 8 assets of dandiset 000044.
Every edit is an in-place HDF5 attribute or single-element write. No dataset is resized, and no sample data — position, spike times, or ephys — is modified.
Investigation notes for dandi/helpdesk#209.
Confirmed, root cause identified, and it is exactly one file out of 1518. The affected asset is a
stub_test=True conversion of the conversion script's hardcoded test session that was uploaded during
initial development and then made permanently invisible to the batch converter by its own
already-done check.
| | |
A reproducible pipeline that takes every primary research article published in a journal over a date window, parses its Data availability and Code availability statements, and then actually tries to resolve every dataset, repository and accession they point to. Run in depth on Nature Neuroscience (scraped from nature.com, including paywalled articles) and across four neuroscience journals via PMC full text.
It exists because of a single observation: a 2026 Nature Neuroscience paper cites a Zenodo DOI for its imaging software that was never registered, so the link 404s. That is
| Reference output, suite2p main (90be895) vs the rfft2 branch. | |
| 300 frames of 512x512, CPU (Apple Silicon, 10 cores), torch 2.6.0. | |
| $ python verify_rfft2_registration.py compare old.npz new.npz | |
| key identical max diff shape | |
| -------------------------------------------------------------- | |
| nr_frames False 4.000e+01 (300, 512, 512) | |
| nr_meanImg False 1.338e-01 (512, 512) | |
| nr_off0 True 0.000e+00 (300,) |
These scripts record the output of rastermap's sorting and compare it across two
commits. They accompany a change to rastermap/sort.py that speeds up the
travelling salesman sort, and they support the claim that the output is
bit-identical rather than approximately equal.
The comparison needs a recording from each commit:
| # analyze data from https://dandiarchive.org/dandiset/001753/0.260220.1838 | |
| """Visualize intracellular electrophysiology data from an NWB file.""" | |
| import h5py | |
| import numpy as np | |
| import matplotlib | |
| matplotlib.use("Agg") | |
| import matplotlib.pyplot as plt | |
| from matplotlib import cm |
| import numpy as np | |
| import json | |
| import base64 | |
| def _add_dataset_to_rfs( | |
| rfs: dict, | |
| shape: list[int], | |
| dtype: np.dtype, | |
| dset_name: str, |
| from typing import Tuple, Type, Any, Union, Optional | |
| from pydantic import BaseModel | |
| import numpy as np | |
| def get_shape(data): | |
| """ | |
| Get the shape of various data structures: NumPy array, h5py dataset, Zarr dataset, or a nested list of arbitrary depth. | |
| Parameters |
