Skip to content

Instantly share code, notes, and snippets.

@s-sajid-ali

s-sajid-ali/petsc-ksp-crash-debug-mode-asan-checkstack

Created February 23, 2023 00:32
Show Gist options
  • Save s-sajid-ali/455b3982d47a31bff9e7ee211dd43991 to your computer and use it in GitHub Desktop.
Save s-sajid-ali/455b3982d47a31bff9e7ee211dd43991 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
petsc-ksp-crash-debug-mode-asan-checkstack
Propagator: turn 10/inf., time = 0.521s, macroparticles = (1024) / ()
=================================================================
==5750==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7fffefea4650 at pc 0x7fffe7b84009 bp 0x7fffffff64b0 sp 0x7fffffff64a8
READ of size 4 at 0x7fffefea4650 thread T0
#0 0x7fffe7b84008 in VecLockReadPush /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/rvector.c:3824
#1 0x7fffedc9b91e in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:439
#2 0x7fffec938bbf in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#3 0x7fffec93a6a5 in KSPSolve_PREONLY /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/preonly/preonly.c:21
#4 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#5 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#6 0x7fffecf06c52 in PCApply_BJacobi_Singleblock /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/bjacobi/bjacobi.c:607
#7 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#8 0x7fffec938bbf in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#9 0x7fffec93a6a5 in KSPSolve_PREONLY /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/preonly/preonly.c:21
#10 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#11 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#12 0x7fffed9e3086 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:28
#13 0x7fffed9e7c14 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:84
#14 0x7fffed9e7c14 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:84
#15 0x7fffeda08244 in PCApply_MG_Internal /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:611
#16 0x7fffeda096f3 in PCApply_MG /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:633
#17 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#18 0x7fffec64924f in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#19 0x7fffec689957 in KSPConvergedDefault /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/iterativ.c:1487
#20 0x7fffecbe3807 in KSPGMRESCycle /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:138
#21 0x7fffecbe91ed in KSPSolve_GMRES /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:228
#22 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#23 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#24 0x7ffff64d0050 in solve(SubcommCtx&, GlobalCtx&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd_utils.cc:413
#25 0x7ffff64aab97 in Space_charge_3d_fd::apply_bunch(bunch_t<double>&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:249
#26 0x7ffff64a7b28 in Space_charge_3d_fd::apply_impl(Bunch_simulator&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:149
#27 0x7ffff6cfd203 in Operator::apply(Bunch_simulator&, double, Logger&) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so+0x51c203)
#28 0x7ffff6cf53d7 in Step::apply(Bunch_simulator&, Logger&) const /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/step.cc:56
#29 0x7ffff6dfb47f in Propagator::do_step(Bunch_simulator&, Step&, int, int, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:112
#30 0x7ffff6dfc4c9 in Propagator::propagate(Bunch_simulator&, Logger&, int) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:455
#31 0x5d2602 in run(Fodo_cxx_options) /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:170
#32 0x5d2d5f in main /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:189
#33 0x7fffe50e9554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#34 0x5cf998 (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/examples/fodo_cxx/fodo_cxx+0x5cf998)
0x7fffefea4650 is located 0 bytes to the right of global variable 'petscstack' defined in '/wclustre/accelsim/sajid/packages/petsc/src/sys/error/pstack.c:5:12' (0x7fffefea4040) of size 1552
SUMMARY: AddressSanitizer: global-buffer-overflow /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/rvector.c:3824 in VecLockReadPush
Shadow bytes around the buggy address:
0x10007dfcc870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007dfcc8c0: 00 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9
0x10007dfcc8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9
0x10007dfcc8f0: f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc900: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc910: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
=================================================================
==5750==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7fffefea4650 at pc 0x7fffe7b84282 bp 0x7fffffff64b0 sp 0x7fffffff64a8
READ of size 4 at 0x7fffefea4650 thread T0
#0 0x7fffe7b84281 in VecLockReadPush /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/rvector.c:3824
#1 0x7fffedc9b91e in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:439
#2 0x7fffec938bbf in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#3 0x7fffec93a6a5 in KSPSolve_PREONLY /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/preonly/preonly.c:21
#4 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#5 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#6 0x7fffecf06c52 in PCApply_BJacobi_Singleblock /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/bjacobi/bjacobi.c:607
#7 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#8 0x7fffec938bbf in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#9 0x7fffec93a6a5 in KSPSolve_PREONLY /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/preonly/preonly.c:21
#10 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#11 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#12 0x7fffed9e3086 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:28
#13 0x7fffed9e7c14 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:84
#14 0x7fffed9e7c14 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:84
#15 0x7fffeda08244 in PCApply_MG_Internal /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:611
#16 0x7fffeda096f3 in PCApply_MG /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:633
#17 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#18 0x7fffec64924f in KSP_PCApply /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:378
#19 0x7fffec689957 in KSPConvergedDefault /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/iterativ.c:1487
#20 0x7fffecbe3807 in KSPGMRESCycle /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:138
#21 0x7fffecbe91ed in KSPSolve_GMRES /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:228
#22 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#23 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#24 0x7ffff64d0050 in solve(SubcommCtx&, GlobalCtx&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd_utils.cc:413
#25 0x7ffff64aab97 in Space_charge_3d_fd::apply_bunch(bunch_t<double>&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:249
#26 0x7ffff64a7b28 in Space_charge_3d_fd::apply_impl(Bunch_simulator&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:149
#27 0x7ffff6cfd203 in Operator::apply(Bunch_simulator&, double, Logger&) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so+0x51c203)
#28 0x7ffff6cf53d7 in Step::apply(Bunch_simulator&, Logger&) const /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/step.cc:56
#29 0x7ffff6dfb47f in Propagator::do_step(Bunch_simulator&, Step&, int, int, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:112
#30 0x7ffff6dfc4c9 in Propagator::propagate(Bunch_simulator&, Logger&, int) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:455
#31 0x5d2602 in run(Fodo_cxx_options) /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:170
#32 0x5d2d5f in main /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:189
#33 0x7fffe50e9554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#34 0x5cf998 (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/examples/fodo_cxx/fodo_cxx+0x5cf998)
0x7fffefea4650 is located 0 bytes to the right of global variable 'petscstack' defined in '/wclustre/accelsim/sajid/packages/petsc/src/sys/error/pstack.c:5:12' (0x7fffefea4040) of size 1552
SUMMARY: AddressSanitizer: global-buffer-overflow /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/rvector.c:3824 in VecLockReadPush
Shadow bytes around the buggy address:
0x10007dfcc870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007dfcc8c0: 00 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9
0x10007dfcc8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9
0x10007dfcc8f0: f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc900: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc910: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Propagator: turn 11/inf., time = 0.593s, macroparticles = (1024) / ()
=================================================================
==5750==ERROR: AddressSanitizer: global-buffer-overflow on address 0x7fffefea4650 at pc 0x7fffe74528c3 bp 0x7fffffff67b0 sp 0x7fffffff67a8
READ of size 4 at 0x7fffefea4650 thread T0
#0 0x7fffe74528c2 in PetscStackCopy /wclustre/accelsim/sajid/packages/petsc/src/sys/error/pstack.c:178
#1 0x7fffe7396915 in PetscTrMallocDefault /wclustre/accelsim/sajid/packages/petsc/src/sys/memory/mtr.c:216
#2 0x7fffe738eb89 in PetscMallocA /wclustre/accelsim/sajid/packages/petsc/src/sys/memory/mal.c:418
#3 0x7fffe73fb83d in PetscFunctionListDLAllPush_Private /wclustre/accelsim/sajid/packages/petsc/src/sys/dll/reg.c:191
#4 0x7fffe73ff3a0 in PetscFunctionListCreate_Private /wclustre/accelsim/sajid/packages/petsc/src/sys/dll/reg.c:261
#5 0x7fffe740014a in PetscFunctionListAdd_Private /wclustre/accelsim/sajid/packages/petsc/src/sys/dll/reg.c:299
#6 0x7fffe6e5b83c in PetscObjectComposeFunction_Petsc /wclustre/accelsim/sajid/packages/petsc/src/sys/objects/inherit.c:691
#7 0x7fffe6e5f770 in PetscObjectComposeFunction_Private /wclustre/accelsim/sajid/packages/petsc/src/sys/objects/inherit.c:814
#8 0x7fffe7e05069 in VecDestroy_Seq /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/impls/seq/bvec2.c:734
#9 0x7fffe7a56754 in VecDestroy /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/vector.c:528
#10 0x7fffec593751 in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:810
#11 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#12 0x7fffed9e3086 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:28
#13 0x7fffeda08244 in PCApply_MG_Internal /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:611
#14 0x7fffeda096f3 in PCApply_MG /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:633
#15 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#16 0x7fffedcb3778 in PCApplyBAorAB /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:715
#17 0x7fffecbdc819 in KSP_PCApplyBAorAB /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:414
#18 0x7fffecbe4272 in KSPGMRESCycle /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:147
#19 0x7fffecbe91ed in KSPSolve_GMRES /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:228
#20 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#21 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#22 0x7ffff64d0050 in solve(SubcommCtx&, GlobalCtx&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd_utils.cc:413
#23 0x7ffff64aab97 in Space_charge_3d_fd::apply_bunch(bunch_t<double>&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:249
#24 0x7ffff64a7b28 in Space_charge_3d_fd::apply_impl(Bunch_simulator&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:149
#25 0x7ffff6cfd203 in Operator::apply(Bunch_simulator&, double, Logger&) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so+0x51c203)
#26 0x7ffff6cf53d7 in Step::apply(Bunch_simulator&, Logger&) const /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/step.cc:56
#27 0x7ffff6dfb47f in Propagator::do_step(Bunch_simulator&, Step&, int, int, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:112
#28 0x7ffff6dfc4c9 in Propagator::propagate(Bunch_simulator&, Logger&, int) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:455
#29 0x5d2602 in run(Fodo_cxx_options) /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:170
#30 0x5d2d5f in main /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:189
#31 0x7fffe50e9554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#32 0x5cf998 (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/examples/fodo_cxx/fodo_cxx+0x5cf998)
0x7fffefea4650 is located 0 bytes to the right of global variable 'petscstack' defined in '/wclustre/accelsim/sajid/packages/petsc/src/sys/error/pstack.c:5:12' (0x7fffefea4040) of size 1552
SUMMARY: AddressSanitizer: global-buffer-overflow /wclustre/accelsim/sajid/packages/petsc/src/sys/error/pstack.c:178 in PetscStackCopy
Shadow bytes around the buggy address:
0x10007dfcc870: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc880: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc890: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x10007dfcc8c0: 00 00 00 00 00 00 00 00 00 00[f9]f9 f9 f9 f9 f9
0x10007dfcc8d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x10007dfcc8e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 f9 f9 f9
0x10007dfcc8f0: f9 f9 f9 f9 04 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc900: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
0x10007dfcc910: f9 f9 f9 f9 00 f9 f9 f9 f9 f9 f9 f9 00 f9 f9 f9
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
=================================================================
==5750==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x61b000000028 at pc 0x7fffe7393984 bp 0x7fffffff6c80 sp 0x7fffffff6c78
READ of size 4 at 0x61b000000028 thread T0
#0 0x7fffe7393983 in PetscMallocValidate /wclustre/accelsim/sajid/packages/petsc/src/sys/memory/mtr.c:122
#1 0x7fffe7398309 in PetscTrFreeDefault /wclustre/accelsim/sajid/packages/petsc/src/sys/memory/mtr.c:278
#2 0x7fffe7e053ff in VecDestroy_Seq /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/impls/seq/bvec2.c:736
#3 0x7fffe7a56754 in VecDestroy /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/interface/vector.c:528
#4 0x7fffec593751 in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:810
#5 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#6 0x7fffed9e3086 in PCMGMCycle_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:28
#7 0x7fffeda08244 in PCApply_MG_Internal /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:611
#8 0x7fffeda096f3 in PCApply_MG /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/impls/mg/mg.c:633
#9 0x7fffedc9c06b in PCApply /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:441
#10 0x7fffedcb3778 in PCApplyBAorAB /wclustre/accelsim/sajid/packages/petsc/src/ksp/pc/interface/precon.c:715
#11 0x7fffecbdc819 in KSP_PCApplyBAorAB /wclustre/accelsim/sajid/packages/petsc/include/petsc/private/kspimpl.h:414
#12 0x7fffecbe4272 in KSPGMRESCycle /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:147
#13 0x7fffecbe91ed in KSPSolve_GMRES /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/impls/gmres/gmres.c:228
#14 0x7fffec59829b in KSPSolve_Private /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:897
#15 0x7fffec59fa88 in KSPSolve /wclustre/accelsim/sajid/packages/petsc/src/ksp/ksp/interface/itfunc.c:1069
#16 0x7ffff64d0050 in solve(SubcommCtx&, GlobalCtx&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd_utils.cc:413
#17 0x7ffff64aab97 in Space_charge_3d_fd::apply_bunch(bunch_t<double>&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:249
#18 0x7ffff64a7b28 in Space_charge_3d_fd::apply_impl(Bunch_simulator&, double, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/collective/space_charge_3d_fd.cc:149
#19 0x7ffff6cfd203 in Operator::apply(Bunch_simulator&, double, Logger&) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so+0x51c203)
#20 0x7ffff6cf53d7 in Step::apply(Bunch_simulator&, Logger&) const /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/step.cc:56
#21 0x7ffff6dfb47f in Propagator::do_step(Bunch_simulator&, Step&, int, int, Logger&) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:112
#22 0x7ffff6dfc4c9 in Propagator::propagate(Bunch_simulator&, Logger&, int) /wclustre/accelsim/sajid/packages/synergia2/src/synergia/simulation/propagator.cc:455
#23 0x5d2602 in run(Fodo_cxx_options) /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:170
#24 0x5d2d5f in main /wclustre/accelsim/sajid/packages/synergia2/examples/fodo_cxx/fodo_cxx.cc:189
#25 0x7fffe50e9554 in __libc_start_main (/lib64/libc.so.6+0x22554)
#26 0x5cf998 (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/examples/fodo_cxx/fodo_cxx+0x5cf998)
0x61b000000028 is located 88 bytes to the left of 1536-byte region [0x61b000000080,0x61b000000680)
allocated by thread T0 here:
#0 0x7ffff75f4ca8 in operator new(unsigned long) ../../../../gcc-12.2.0/libsanitizer/asan/asan_new_delete.cpp:95
#1 0x7ffff1db88da in std::__new_allocator<std::__detail::_State<char> >::allocate(unsigned long, void const*) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0x1198da)
#2 0x7ffff1da8d01 in std::allocator_traits<std::allocator<std::__detail::_State<char> > >::allocate(std::allocator<std::__detail::_State<char> >&, unsigned long) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0x109d01)
#3 0x7ffff1d98e77 in std::_Vector_base<std::__detail::_State<char>, std::allocator<std::__detail::_State<char> > >::_M_allocate(unsigned long) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0xf9e77)
#4 0x7ffff1d8f7f8 in void std::vector<std::__detail::_State<char>, std::allocator<std::__detail::_State<char> > >::_M_realloc_insert<std::__detail::_State<char> >(__gnu_cxx::__normal_iterator<std::__detail::_State<char>*, std::vector<std::__detail::_State<char>, std::allocator<std::__detail::_State<char> > > >, std::__detail::_State<char>&&) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0xf07f8)
#5 0x7ffff1d8a3c6 in std::__detail::_State<char>& std::vector<std::__detail::_State<char>, std::allocator<std::__detail::_State<char> > >::emplace_back<std::__detail::_State<char> >(std::__detail::_State<char>&&) /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/vector.tcc:123
#6 0x7ffff1d8788d in std::vector<std::__detail::_State<char>, std::allocator<std::__detail::_State<char> > >::push_back(std::__detail::_State<char>&&) /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/stl_vector.h:1294
#7 0x7ffff1d804bf in std::__detail::_NFA<std::__cxx11::regex_traits<char> >::_M_insert_state(std::__detail::_State<char>) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0xe14bf)
#8 0x7ffff1d80c73 in std::__detail::_NFA<std::__cxx11::regex_traits<char> >::_M_insert_alt(long, long, bool) (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0xe1c73)
#9 0x7ffff1d7c036 in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_M_disjunction() /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex_compiler.tcc:103
#10 0x7ffff1d8b29a in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_M_atom() /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex_compiler.tcc:329
#11 0x7ffff1d87b4a in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_M_term() /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex_compiler.tcc:133
#12 0x7ffff1d807a5 in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_M_alternative() (/wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/utils/kokkos/core/src/libkokkoscore.so.4.0+0xe17a5)
#13 0x7ffff1d7bdf4 in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_M_disjunction() /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex_compiler.tcc:91
#14 0x7ffff1d79f8d in std::__detail::_Compiler<std::__cxx11::regex_traits<char> >::_Compiler(char const*, char const*, std::locale const&, std::regex_constants::syntax_option_type) /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex_compiler.tcc:76
#15 0x7ffff1d78dd1 in std::__cxx11::basic_regex<char, std::__cxx11::regex_traits<char> >::_M_compile(char const*, char const*, std::regex_constants::syntax_option_type) /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex.h:789
#16 0x7ffff1d78079 in std::__cxx11::basic_regex<char, std::__cxx11::regex_traits<char> >::basic_regex(char const*, std::regex_constants::syntax_option_type) /srv/software/gnu12/12.2.0/include/c++/12.2.0/bits/regex.h:453
#17 0x7ffff1d74f75 in __static_initialization_and_destruction_0 /wclustre/accelsim/sajid/packages/synergia2/src/synergia/utils/kokkos/core/src/impl/Kokkos_Command_Line_Parsing.cpp:32
#18 0x7ffff1d751bf in _GLOBAL__sub_I_Kokkos_Command_Line_Parsing.cpp /wclustre/accelsim/sajid/packages/synergia2/src/synergia/utils/kokkos/core/src/impl/Kokkos_Command_Line_Parsing.cpp:264
#19 0x7ffff7dea9c2 in _dl_init_internal (/lib64/ld-linux-x86-64.so.2+0xf9c2)
#20 0x7fffffffbf37 ([stack]+0x20f37)
SUMMARY: AddressSanitizer: heap-buffer-overflow /wclustre/accelsim/sajid/packages/petsc/src/sys/memory/mtr.c:122 in PetscMallocValidate
Shadow bytes around the buggy address:
0x0c367fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff7fd0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff7fe0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff7ff0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c367fff8000: fa fa fa fa fa[fa]fa fa fa fa fa fa fa fa fa fa
0x0c367fff8010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff8020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff8030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff8040: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c367fff8050: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
[0]PETSC ERROR: PetscMallocValidate: error detected in VecDestroy_Seq() at /wclustre/accelsim/sajid/packages/petsc/src/vec/vec/impls/seq/bvec2.c:736
[0]PETSC ERROR: Memory at address 0x61b000000000 is corrupted
[0]PETSC ERROR: Probably write before beginning of or past end of array
[0]PETSC ERROR: Last intact block [id=12063(16)] at address 0x61b000000650 allocated in PetscFunctionListDLAllPush_Private() at /wclustre/accelsim/sajid/packages/petsc/src/sys/dll/reg.c:191
[wcwn003:05750] *** Process received signal ***
[wcwn003:05750] Signal: Aborted (6)
[wcwn003:05750] Signal code: (-6)
[wcwn003:05750] [ 0] /wclustre/accelsim/sajid/packages/spack/opt/spack/linux-scientific7-ivybridge/gcc-12.2.0/openmpi-4.1.4-focezot2jvs37syrsqec3xyhi7ot7fap/lib/libopen-pal.so.40(+0x733f7)[0x7fffe44c83f7]
[wcwn003:05750] [ 1] /lib64/libpthread.so.0(+0xf630)[0x7ffff1649630]
[wcwn003:05750] [ 2] /lib64/libc.so.6(gsignal+0x37)[0x7fffe50fd387]
[wcwn003:05750] [ 3] /lib64/libc.so.6(abort+0x148)[0x7fffe50fea78]
[wcwn003:05750] [ 4] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(PetscMallocValidate+0xd4a)[0x7fffe739421b]
[wcwn003:05750] [ 5] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(PetscTrFreeDefault+0xad8)[0x7fffe739830a]
[wcwn003:05750] [ 6] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x171a400)[0x7fffe7e05400]
[wcwn003:05750] [ 7] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(VecDestroy+0x1832)[0x7fffe7a56755]
[wcwn003:05750] [ 8] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x5ea8752)[0x7fffec593752]
[wcwn003:05750] [ 9] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(KSPSolve+0xb46)[0x7fffec59fa89]
[wcwn003:05750] [10] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x72f8087)[0x7fffed9e3087]
[wcwn003:05750] [11] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x731d245)[0x7fffeda08245]
[wcwn003:05750] [12] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x731e6f4)[0x7fffeda096f4]
[wcwn003:05750] [13] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(PCApply+0x1fd7)[0x7fffedc9c06c]
[wcwn003:05750] [14] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(PCApplyBAorAB+0x880b)[0x7fffedcb3779]
[wcwn003:05750] [15] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x64f181a)[0x7fffecbdc81a]
[wcwn003:05750] [16] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(KSPGMRESCycle+0x5261)[0x7fffecbe4273]
[wcwn003:05750] [17] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(KSPSolve_GMRES+0xd1b)[0x7fffecbe91ee]
[wcwn003:05750] [18] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(+0x5ead29c)[0x7fffec59829c]
[wcwn003:05750] [19] /wclustre/accelsim/sajid/packages/petsc/debug/lib/libpetsc.so.3.018(KSPSolve+0xb46)[0x7fffec59fa89]
[wcwn003:05750] [20] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/collective/libsynergia_collective.so(_Z5solveR10SubcommCtxR9GlobalCtx+0xa4c)[0x7ffff64d0051]
[wcwn003:05750] [21] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/collective/libsynergia_collective.so(_ZN18Space_charge_3d_fd11apply_bunchER7bunch_tIdEdR6Logger+0x2ef0)[0x7ffff64aab98]
[wcwn003:05750] [22] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/collective/libsynergia_collective.so(_ZN18Space_charge_3d_fd10apply_implER15Bunch_simulatordR6Logger+0xb0f)[0x7ffff64a7b29]
[wcwn003:05750] [23] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so(_ZN8Operator5applyER15Bunch_simulatordR6Logger+0xba)[0x7ffff6cfd204]
[wcwn003:05750] [24] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so(_ZNK4Step5applyER15Bunch_simulatorR6Logger+0x2d0)[0x7ffff6cf53d8]
[wcwn003:05750] [25] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so(_ZN10Propagator7do_stepER15Bunch_simulatorR4StepiiR6Logger+0x17a)[0x7ffff6dfb480]
[wcwn003:05750] [26] /wclustre/accelsim/sajid/packages/synergia2/build_cpu/src/synergia/simulation/libsynergia_simulation.so(_ZN10Propagator9propagateER15Bunch_simulatorR6Loggeri+0x5d0)[0x7ffff6dfc4ca]
[wcwn003:05750] [27] ./fodo_cxx[0x5d2603]
[wcwn003:05750] [28] ./fodo_cxx[0x5d2d60]
[wcwn003:05750] [29] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fffe50e9555]
[wcwn003:05750] *** End of error message ***
--------------------------------------------------------------------------
Primary job terminated normally, but 1 process returned
a non-zero exit code. Per user-direction, the job has been aborted.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that process rank 0 with PID 0 on node wcwn003 exited on signal 6 (Aborted).
--------------------------------------------------------------------------
[Thread 0x7ffff37ce700 (LWP 5748) exited]
[Thread 0x7ffff3fcf700 (LWP 5747) exited]
[Thread 0x7ffff2fcd700 (LWP 5749) exited]
[Inferior 1 (process 5735) exited with code 0206]
(gdb)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment