gistfile1.md

Known Causes

1. Duplicate steps/fields within a single job

   • haven't confirmed any problems resulting from field-sharing yet
   • results from adding the same template multiple times
   • same DB row, but different jobBaseStepIds

2. Shared steps/fields between clone/duplicate jobs and their originals/sources

   • haven't confirmed any problems resulting from field-sharing yet
   • results from JobController#CreateFromOffline
   • same DB row, and also same jobBaseStepId
     • wait, is it the same DB row???

3. In JobController#CreateFromOffline, MaterializeJobOp is sent before services.JobRefWork() and services.ResolveReferences()

   • haven't confirmed problems arising, but it could be the reason why job_creation_spec.rb fails only on CreateFromOffline
     • MaterializeJobOp sending: https://github.com/parsable/mothership/blob/master/controllers/job_controller.go#L1782
     • JobRefWork https://github.com/parsable/mothership/blob/master/controllers/job_controller.go#L1864
     • StoreResolvedReferences https://github.com/parsable/mothership/blob/master/controllers/job_controller.go#L1870

---

Known Problems

in services.ResolveReferences(), duplicate-within-job steps overwrite one-another's job-resolved maps

• because we don't properly update a JSON column -- we only over-write it
• solution is to add a map-refresher function that should be called before each update
  • makes the refereshing explicit, so there are no side-effects to the update
• one example of where to put the refresher: right before this: https://github.com/parsable/mothership/blob/master/services/reference.go#L330

Suspected Problems

when thrifting duplicate-within-job steps, they share pointers to wrappers, and overwrite each others changes

• haven't fully explored this -- don't know which pointers are the crucial ones
• so far, efforts to fix it have instead broken it badly (nil pointer dereferences, etc)
• looking at ToThriftStringWrapper()

the JobController.AddTemplates action doesn't send references in its event

• currently trying to change the steps sent with the event, so they have references

---