The Fornax A entry in srclist_pumav3_EoR0LoBES_EoR1pietro_CenA-GP_2023-11-07 over-predicts
visibility amplitude on the baselines MWA calibration uses. Fornax A carries about 46% of the
di-calibrate weight at --uvw-min 30l, so that error sets the flux scale for the whole field.
Multiplying the seven Fornax A components by about 0.8 and changing nothing else:
| original model | rescaled |
|---|
ghcr.io/d3v-null/rapthor-mwa is a lean (no Jupyter/conda) Spack build of the
LOFAR Rapthor direction-dependent
calibration and imaging pipeline with MWA support compiled in:
| component | version | MWA notes |
|---|---|---|
| rapthor | 2.1.20260630 (py-rapthor +mwa) |
patched: DP3/WSClean pointed at the MWA beam, facet-beam options dropped, flagged-tile and solar-model robustness fixes (list below) |
| DP3 | 6.6.20260819 | applybeam.coefficients_path / solve.coefficients_path = /opt/mwa_full_embedded_element_pattern.h5 |
An investigation into why hyperdrive peel leaves a large coherent residual on MWA EoR1 data,
and what actually fixes it. Short version: the field catalogue is fine, GLEAM-X DR2 confirms it,
hyperdrive's shapelet code is correct, and the Fornax A model over-predicts visibility amplitude
on the baselines MWA calibration uses by about 29%. Rescaling it cuts the coherent residual by a
quarter in amplitude, which is 44% in power.
Found 2026-08-21 while validating the SWF-22 Rapthor Marimo demo
(swf22_rapthor_broker_demo.py) against pilot_resource=slurm on a
long-lived dev cluster (deploy-ubuntu, ~9 days uptime at the time). None of
these are specific to that notebook — they block any job requesting
pilot_resource=slurm on an affected cluster. SWF-22's default was changed to
kubernetes to work around this; see its lib module for the pointer back
here.
About this gist. Gists are flat, so the companion files below are prefixed (
script-,manifest-,benchmon-) copies of the repository tree atdocs/oke-demo/{scripts,manifests,benchmon}/on theoke-demo-2026-08-27branch ofska-src-api-deployment-stack. A path written asdocs/oke-demo/scripts/remove-site1-ms.shbelow isscript-remove-site1-ms.shhere.
Federating a second HTCondor pilot pool — Kueue + job-gateway on Oracle's
managed OKE — with a schedd/collector that lives somewhere else entirely.
Status (2026-08-14): working end to end. A job submitted on the VM is matched to a pilot running in OKE, its sandbox is transferred across the boundary, the payload executes on an Oracle Linux node, and stdout comes back:
A client POSTs a job to the Computing Broker, which authorizes via PAPI, exchanges tokens via AAPI, and intersects SCAPI (site capability) with DMAPI (data location) into a candidate set of sites — it never picks the final site itself. The broker submits exactly one run leader (a condor-native local-universe job — this replaced the old Toil/WES stack entirely) to the central HTCondor pool. The run leader fans the job out as a vanilla-universe payload cluster. Independently, each site's Job Gateway is a pure pilot factory: it spawns pilot pods (Kubernetes, Kueue-gated) or Slurm jobs (via slurmrestd + Apptainer) sized to idle demand. When a pilot joins the pool, HTCondor's negotiator matches it to an idle task — this match, not the broker or the gateway, is what decides where a job actually runs ("late binding"). The pilot then runs the payload u
This didn't work but I'm documenting it here in case I want to try it again.
take calibration solutions, fit linear model, update cable lengths in metafits before a second round of preprocessing.
singularity shell --cleanenv --bind /data/curtin_mwaeor/src/mwa-demo/demo:/demo --bind $PWD --workdir $PWD /data/curtin_mwaeor/singularity/mwa-demo_autos_cuda12.5.1.sif
/demo/82_calfit.py \
--metafits /data/curtin_mwaeor/dev/1274010302/raw/1274010302.metafits \
--solns /data/curtin_mwaeor/dev/1274010302/hyp_soln_1274010302_kr_75-1667l_src1k_500it_sdc50_el36.692.fits \
--name hyp_soln --out-dir /data/curtin_mwaeor/dev/1274010302/ \