-
-
Save sjl/4438002 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ehazlett sjl: hey have a sec? | |
| sjl ehazlett: sure | |
| ehazlett sjl: i'm trying to migrate the old testdata instance to the latest production | |
| sjl ok | |
| ehazlett sjl: when i try to migrate i'm getting the "ghost migrations" error | |
| sjl fun | |
| ehazlett sjl: http://d.pr/i/k7N | |
| ehazlett sjl: sorry for the terrible formatting | |
| sjl ehazlett: what exactly are you trying to do again? | |
| sjl the production branch doesn't have team migrations 16+ | |
| ehazlett sjl: i'm trying to update a database from rev 646137e to the latest production (maggie's test instance) | |
| sjl it looks like you're trying to take an instance that was running on the dev branch and migrate it to the production branch | |
| ehazlett sjl: well shit | |
| ehazlett sjl: figures | |
| sjl what you'd need to do is migrate *backwards* to the last migration that's present in production first, *before* you change branches | |
| sjl because after you switch the branch, south doesn't know how to undo those migrations, since they're not on disk any more | |
| sjl (this is all assuming those migrations actually do contain the backwards migrations, but I'm usually pretty good at making sure we do that) | |
| sjl databases are bullshit | |
| ehazlett sjl: ugh | |
| sjl yeah that's an accurate depiction of my thoughts too | |
| nicksergeant let's just all save everything to plain text files | |
| ehazlett LOL | |
| nicksergeant need to login? upload your identity file and we'll try and find it on-disk | |
| ehazlett ha | |
| nicksergeant log in* | |
| sjl ugh, stupid rochester, stop snowing | |
| nicksergeant sjl: nevar! | |
| ehazlett sjl: ok so i'm a south newb -- could you spare 5 min to educate me? | |
| sjl ehazlett: sure | |
| sjl ehazlett: so the basic idea is that a single "migration" describes a change in the schema of the database | |
| sjl ehazlett: so let's say I add a "foo" field to a Django model | |
| ehazlett sjl: yep | |
| sjl ehazlett: the migration actually contains two parts, a "forward" part that says "add column foo to table whatever" | |
| sjl ehazlett: and a "backward" part that says "remove column foo from table whatever" | |
| ehazlett sjl: got it | |
| sjl so it describes how to go from the original DB schema to the new one, and how to go from the new one back to the original | |
| sjl each migration is stored in its own file on disk. that's those 0115_add_blah_blah.py files in the migrations/ folder for each app | |
| ehazlett sjl: yep i've seen those | |
| sjl there are also things called data migrations. in general you use them to just modify data, but conceptually they're not any different than the normal ones | |
| ehazlett sjl: ok | |
| sjl all migrations take the DB from one state to another | |
| sjl whether that means adding/removing a field, or capitalizing some data or something, doesn't matter | |
| sjl DB + migration = new DB | |
| sjl make sense so far? | |
| ehazlett sjl: yep | |
| sjl ok | |
| sjl so normally this is pretty straightforward to work with. You have your local database, and you need to add a field to a model | |
| sjl You do that by creating the migration. How exactly that happens isn't important right now. | |
| ehazlett sjl: got it | |
| sjl But you end up with a new file like 0116_add_field_foo.py | |
| sjl and that file contains two parts: the forward part (that adds the field) and the backwards part | |
| ehazlett sjl: ok | |
| sjl Now, south itself also has a table in your database for its own data | |
| sjl It stores the current "state" for each app in there | |
| ehazlett sjl: ok | |
| sjl so something like "the teams app is currently at migration 0114_..." | |
| ehazlett sjl: gotcha | |
| sjl when you tell south to migrate an app, it look at the current state of the DB, and it looks at the files in your migrations folder for that app | |
| ehazlett sjl: ah and that's how it decides where it needs to be? | |
| sjl so in our example, if you started out at migration 0115 and created 0116 to add your field, south would know that it needed to run the SQL for the forwards part of 0116 | |
| ehazlett sjl: got it | |
| sjl and once that finished it would update its own table to record that the teams app is now at 0116 | |
| sjl now, you can also tell south to migrate to a specific target, instead of just "whatever is the newest" | |
| ehazlett sjl: ok | |
| sjl so you can say "manage.py migrate teams 0112" for example | |
| sjl south will look in its table to see where it currently is (which is 0116 in our example) | |
| sjl so it says okay, I'm at 116 and need to get to 112, so I need to go backwards | |
| sjl so first it runs the backwards part of 0116 | |
| ehazlett sjl: ah ok -- i wondered how it went back | |
| sjl then it runs the backwards part of 0115 | |
| sjl then it runs the backwards part of 0114 | |
| sjl then it runs the backwards part of 0113 | |
| sjl then it's done | |
| ehazlett sjl: ah ok | |
| sjl so far, so good | |
| sjl right? | |
| ehazlett sjl: yep | |
| sjl ehazlett: ok, so this still is pretty straightforward. you do your local development, maybe creating a few migrations along the way | |
| sjl let's say 0116 0117 and 0118 | |
| sjl now you push those to github, and as part of your deployment process you tell south to migrate to the latest point | |
| sjl so south migrates your server or whatever up to 0118 and you're all good | |
| sjl that's the way things are *supposed* to work | |
| ehazlett :) | |
| sjl ok, so where this gets tricky is when you have concurrent development happening | |
| sjl i.e.: more that one line of development happening at the same time | |
| sjl which may or may not involve git branching | |
| sjl Let's use a single branch as an example for now | |
| ehazlett ok | |
| sjl let's say we're in a repo with just one master branch and that's it | |
| sjl and nick and I are working on two different things | |
| sjl let's say that in the morning when we start, there are 10 migrations for the app we're going to work on | |
| sjl so our server knows it's at migration 10, and our local servers are at the same place | |
| sjl so now we start working. nick creates a migration 11 to add a field foo on his local machine | |
| sjl while he's doing that, I create a migration (also 11, because nick hasn't pushed his yet) to add field bar on my machine | |
| sjl each of us tells south to migrate, and in both cases there's no issue (yet). south runs the sql for the forwards part of my 11 on my machine, and his 11 on his machine | |
| ehazlett yep | |
| sjl now let's say nick finishes first and pushes to github | |
| sjl so the master branch on github now contains the first 10 migrations we started with, plus 11N(ick) | |
| ehazlett start the chaos | |
| sjl now when I pull from github, I get 11N | |
| sjl so now my migrations folder contains 11S and 11N (plus the first 10, of course) | |
| sjl understand so far? | |
| ehazlett yep | |
| sjl ok, so at this point there are two things that can happen with I try to run "python manage.py migrate myapp" | |
| sjl the key point here is that south runs migrations in filename order | |
| ehazlett yeah | |
| sjl which normally is sane, because there's a number at the beginning of the filename | |
| sjl 0001_... 0002_... etc | |
| sjl but now that we have two with the same number, it's going to look at whatever the rest of the name happens to be to decide the order | |
| sjl If I'm lucky, at this point South will explode when it tries to migrate. This would happen when Nick's file comes *before* mine | |
| sjl Because my DB has ... 8 9 10 11S applied | |
| ehazlett yeah | |
| sjl and South looks at the files and sees that it should be ... 8 9 10 11N 11S | |
| sjl it can't just apply the 11N on top of 11S because that would be in the wrong order | |
| ehazlett but it would for 11S on nick's right? | |
| sjl so at that point South will just throw up its hands and say "fuck it, I'm out, good luck" | |
| sjl that make sense? | |
| ehazlett yeah | |
| sjl ok, so let's think about the opposite case: my filename happens to come before nick's | |
| sjl so the order South thinks it needs is 8 9 10 11S 11N | |
| sjl and my current database is at 8 9 10 11S | |
| sjl in this case, South will happily just apply nick's migration on top of mine | |
| ehazlett ok | |
| sjl (just like any other new migration) | |
| sjl this is the case that will make you drink | |
| ehazlett lol | |
| sjl because now everything seems fine | |
| sjl but what happens when you push to the server? | |
| ehazlett wouldn't it work also if it was at 10? | |
| ehazlett 9 10 11S 11N | |
| sjl yes | |
| sjl but remember that nick pushed, and maybe deployed | |
| ehazlett ah | |
| sjl what would happen if he deployed? | |
| ehazlett so it's fubar'd | |
| sjl right | |
| ehazlett 9 10 11N - no 11S | |
| sjl now we're at the first case again, only this time it's the *server* that's borked | |
| sjl (and Nick's machine once he pulls down the new migration) | |
| ehazlett got it | |
| ehazlett so does he have to migrate 10, then migrate? | |
| sjl that's a way to manually fix things -- tell south to migrate back to 10 (which unapplies one of the 11's) and then tell it to migrate forwards to the latest, which will apply the 11's in the correct order | |
| ehazlett ok | |
| nicksergeant I disprove of the use of my name in this horrific story | |
| nicksergeant disapprove* | |
| ehazlett haha | |
| sjl lol | |
| sjl be glad you're not the one typing it out :P | |
| nicksergeant :) | |
| sjl though this keyboard sounds so awesome I don't really mind at all | |
| nicksergeant lol | |
| nicksergeant don't forget to add the part where Nick drinks a fifth of vodka and runs his car into a ditch intentionally | |
| sjl ehazlett: ok, 2m break and then we'll dive further into the rabbit hole | |
| * ehazlett thx sjl for typing and apologizes to his neighbors for the extra machine gun | |
| ehazlett sjl, sure thx! | |
| nicksergeant lol | |
| nicksergeant sjl: what keyboard did you switch to this week? | |
| sjl nicksergeant: a unicomp one | |
| sjl (buckling spring like the original model Ms) | |
| nicksergeant sjl: one of these? http://pckeyboard.com/page/category/UKBD | |
| sjl yeah, the spacesaver m in white with blank keycaps | |
| nicksergeant sjl: I like the feel of the mechanicals but my office is right next to the bedroom and it's insanely loud when the wife is sleeping | |
| sjl heh | |
| sjl my cat doesn't seem to mind | |
| nicksergeant lol | |
| nicksergeant the profit margins on those kb's must be insanely high | |
| nicksergeant they look like they were constructed in the 40s | |
| nicksergeant and they're selling for $100 | |
| sjl I dunno, the keys are pretty complicated | |
| sjl there's an individual spring in each key | |
| nicksergeant ah | |
| sjl ehazlett: ok, ready to continue? | |
| ehazlett sjl: yep | |
| sjl ok, so the next important thing to understand is that South migrations aren't quite as simple as I've been making them out to be | |
| ehazlett ok | |
| sjl there are actually three parts to each migration | |
| sjl the first two I mentioned | |
| sjl 1. a description of how to go "forwards" from the previous DB state to the new one | |
| sjl 2. a description of how to go "backwards" from the new state to the previous one | |
| sjl but there's one more thing that's included in the file | |
| sjl 3. a "frozen" state definition | |
| sjl this is a Python dictionary that describes the models as they stand *after* the forwards migration | |
| ehazlett ok | |
| sjl so each migration not only includes forwards/backwards definitions, but also a full description of what the tables look like after that forwards part takes effect | |
| sjl you can think of it kind of like a checksum -- "this is what the DB should look like after the forwards part is done" | |
| sjl does that make sense? | |
| ehazlett yep | |
| sjl ok, so now think back to our 9 10 11S 11N example | |
| sjl let's assume we migrate backwards to 10 on the server and then migrate forwards to get all the stuff applied in the correct order | |
| ehazlett ok | |
| sjl is there anything wrong with this scenario? | |
| ehazlett doesn't seem to be but i'm not sure what south uses those "checksum dicts" for | |
| ehazlett if it compares the state before/after it would fail | |
| sjl let's assume my migration added a field "foo" and nick's added a field "bar" | |
| sjl and let's say the model had fields "a" and "b" to start with | |
| ehazlett ok | |
| sjl so 10's frozen state would say something like "model: a b" | |
| sjl 11S's frozen state would say "model: a b foo" | |
| sjl what would 11N say? | |
| ehazlett model: a b bar | |
| sjl right | |
| sjl and what does the database *actually* look like? | |
| ehazlett model: a b -- right? at 10? | |
| sjl yep | |
| sjl and when you migrate forwards to apply 11s and 11n? | |
| sjl assuming 11s gets ordered before 11n alphabetically | |
| ehazlett assuming model: a b foo bar | |
| sjl right | |
| sjl but now the frozen state in the current migration says a b bar | |
| ehazlett ugh | |
| sjl in this particular case, we could probably slip this by without too much trouble, since we're just adding fields | |
| sjl but things get a lot more complicated when you're removing or renaming columns | |
| ehazlett sjl: ok | |
| sjl ehazlett: ok, so we've now hit a roadblock and are in a place where we don't want to be. to fix this we need to rewind time to the point where Nick has just pushed his migration to github and deployed it to the server | |
| ehazlett sjl: ok | |
| sjl ehazlett: so now the server has 9 10 11n and my machine has 9 10 11s | |
| ehazlett ok | |
| sjl now I pull from GitHub and notice that I've got two 11's: 11N and 11S | |
| sjl at this point it's my responsibility to stop and fix this mess before moving on, otherwise we're going to hit the brokenness | |
| sjl any guesses on how I should do that? | |
| ehazlett hmm no -- renaming the file wouldn't have the right frozen state, right? | |
| ehazlett re-do the migration? | |
| sjl renaming the file would still have the incorrect frozen state, that's right | |
| sjl re-do how? | |
| ehazlett or i guess you could manually edit the frozen state | |
| ehazlett if it's just a dict | |
| sjl ok, but *which* one would you manually edit? | |
| sjl remember that I currently have 9 10 11S applied, and now I just pulled in 11N | |
| ehazlett 11N since it's been deployed? | |
| sjl if it's been deployed, you probably don't want to change it | |
| ehazlett ah yeah | |
| sjl 11S only exists on my laptop, so it's the safest one to modify, right? | |
| ehazlett yep | |
| sjl ok, so let's say you can manually edit the dict to contain the correct state (you usually don't want to do this but for now that's fine) | |
| sjl what exactly do you need to do? | |
| sjl 1. edit the state dict | |
| sjl what's the next step? | |
| * ehazlett blank stare | |
| sjl heh | |
| sjl let's assume we want to avoid having migrations with the same number, because it's confusing as hell | |
| sjl having each migration have its own unique number makes it really easy to tell what goes where | |
| ehazlett yeah | |
| sjl so the obvious next step is to rename 11S to 12, right? | |
| ehazlett yeah | |
| sjl except that's going to break something | |
| ehazlett dammit | |
| sjl heh | |
| sjl think about what south thinks is applied on my local database | |
| sjl 9 10 11S | |
| sjl if I rename the file with 11S to 12, what happens when I try to migrate? | |
| ehazlett it would try to re-run it? | |
| sjl in theory, yes, it would try to run 11N (which is fine) and then 12, but 12 has already been applied | |
| ehazlett yep | |
| sjl but in practice south says "wait a second, there's this 11S thing in the database, and I can't find a file on disk for that. wtf?" | |
| sjl so what do we need to do? | |
| sjl (note: at this point "sqlreset the whole fucking thing and get on with your life" is a valid answer, but let's go through the "right" way just for the exercise) | |
| ehazlett lol | |
| ehazlett migrate 10, then migrate? | |
| sjl OK, so what does south do when you say "migrate 10" | |
| ehazlett shit | |
| sjl it looks at the current stack of "applied" migrations: 9 10 11S | |
| sjl then what? | |
| ehazlett it would try to go backwards from 11s right? | |
| sjl yes | |
| ehazlett dammit | |
| sjl and where is the information on how to go backwards stored? | |
| ehazlett so migrate back 10, then rename? | |
| sjl ding ding ding | |
| ehazlett what a CF | |
| sjl you have to migrate backwards *first* so that the backwards instructions are in the correct locations | |
| sjl *then* rename your file to sit on top of the ones you've pulled | |
| sjl *then* migrate forwards again | |
| ehazlett got it | |
| sjl it has to be exactly in that order, as you've seen | |
| ehazlett yeah | |
| sjl also you need to make sure the frozen state is correct for every case | |
| ehazlett this is a nightmare | |
| sjl sometimes it's easy to manually edit the dicts, other times it might be easier to just recreate the migration from scratch | |
| ehazlett got it | |
| sjl heh, we're still not all the way down the rabbit hole | |
| ehazlett ha | |
| sjl the final bit of pain is when you throw git branches into the mix | |
| sjl let's assume that staging is at migration 10, and we want to add a new migration on dev | |
| sjl no problem, we add migration 11, push to github, etc etc | |
| ehazlett oh in which they exist at a certain point but then disappear when switching ? | |
| ehazlett k | |
| sjl now say we need to fix a ticket on staging. how do you get your local machine back in the "staging" state? | |
| ehazlett well i would nuke the db and re-sync from staging | |
| sjl haha | |
| ehazlett but that's probably not what you were looking for ;) | |
| sjl yeah, let's do it the right way for now | |
| ehazlett migrate back to the common migration? | |
| sjl before or after you git checkout staging? | |
| ehazlett before to get before the new migrations | |
| sjl right | |
| sjl those migrations need to exist because you need the backwards parts of them | |
| sjl now | |
| ehazlett yep | |
| sjl what happens when, in order to solve your staging ticket, you need to create a migration? | |
| ehazlett well shit -- vodka? | |
| ehazlett jk | |
| ehazlett create the migration on staging with next number, push, merge to dev and re-do your local migration as before? | |
| ehazlett er wait, that wouldn't work if it's been pushed / deployed | |
| sjl heh | |
| sjl good, good, let the hate flow through you | |
| ehazlett ffs | |
| honza --vodka? | |
| ehazlett i'm full of hate now | |
| sjl heh | |
| ehazlett i don't know -- so dev is now at 11S and pushed ; staging is at 12 and pushed | |
| ehazlett wtf | |
| sjl at this point there's no simple solution | |
| sjl what you need to do is rename/modify the migrations that are on the dev branch to sit on top of the new one from the staging branch, as before | |
| ehazlett yeah because if you create a new migration on dev it would presumably fuck up staging | |
| ehazlett but what if they've been pushed? | |
| sjl but you're also going to need to tell everyone else about it, and fix it on the server, by migrating back to the common point and then forwards like you need to do locally | |
| ehazlett got it | |
| sjl but remember | |
| sjl in order to migrate backwards...? | |
| ehazlett you have to do it before touching the current migrations | |
| sjl right | |
| sjl so they'll need to migrate backwards to a common point (which you should probably tell them instead of making them look it up), THEN pull your changes, THEN migrate forwards | |
| sjl in that order | |
| sjl and the same thing needs to happen on the server | |
| sjl BUT, when you merge in the other direction, dev -> staging, everything will go cleanly and smoothly | |
| sjl and the same for staging -> prod | |
| ehazlett got it | |
| ehazlett and you still have to rename as before right? | |
| sjl yes | |
| ehazlett in order to keep staging happy | |
| ehazlett got it | |
| sjl exactly | |
| sjl so there you have it | |
| ehazlett awesome | |
| sjl there are some other painful edge cases having to do with data migrations, but you probably don't need to know about those (yet) | |
| sjl so let's stop there | |
| sjl untangle your brain | |
| ehazlett i'm sure i'll have questions, but thanks a ton! | |
| ehazlett exactly | |
| sjl so now you should be able to understand your original problem |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment