Created
July 2, 2013 04:14
-
-
Save hub-cap/5906732 to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 1:01 PM imsplitbit ok | |
| 1:01 PM imsplitbit Clustering API | |
| 1:01 PM imsplitbit ready? go | |
| 1:02 PM vipul can you please paste the link again? | |
| 1:02 PM imsplitbit https://wiki.openstack.org/wiki/Trove-Replication-And-Clustering-API | |
| 1:02 PM SlickNik Thanks! | |
| 1:04 PM KennethWilke lifeless: i dunno if it might help, but the output of my kickstart is at https://gist.github.com/KennethWilke/5878664#file-dib-log-L974 | |
| 1:04 PM lifeless Building elements: base vm guest mysql | |
| 1:04 PM lifeless is the key line | |
| 1:04 PM lifeless so the image looks fine. | |
| 1:05 PM lifeless hub_cap: you should add 'ubuntu' to the explicit list of elements; we only default to ubuntu for backwards comapt. | |
| 1:06 PM hub_cap lifeless: roger | |
| 1:06 PM kevinconway what is the desire to modify /instance/ to contain cluster data rather than exposing that through /cluster/{id}/node{/id} | |
| 1:07 PM imsplitbit well the idea for the <insert name here> attribute field was so that instances which are accessed via the instance api had some sort of clue that they are part of something larger | |
| 1:07 PM imsplitbit otherwise you have to remove them from being operated on at all via /instances | |
| 1:08 PM kevinconway can an instance not be resized independently when it is part of a cluster? | |
| 1:08 PM imsplitbit and it also assists systems builders in making interesting things like network diagrams | |
| 1:08 PM imsplitbit kevinconway: the thought is that yes you should be able to resize a node | |
| 1:08 PM imsplitbit independantly | |
| 1:09 PM kevinconway then why would you need to remove the ability to operate on /instances? | |
| 1:09 PM imsplitbit well you may not | |
| 1:09 PM imsplitbit but it adds intelligence there | |
| 1:10 PM vipul how woudl the user know what actions they can take on an instance vs cluster | |
| 1:10 PM kevinconway but then clustering has this weird side effect on instance resources | |
| 1:10 PM kevinconway it mutates them | |
| 1:10 PM imsplitbit sort of a hint for users to be reminded that this instance is a part of a clustering | |
| 1:10 PM imsplitbit well we envisioned that attributes field would be more general purpose | |
| 1:11 PM imsplitbit but I don't have any specific use cases beyond that just yet | |
| 1:11 PM imsplitbit hub_cap and morris thought it was a good idea | |
| 1:11 PM hub_cap dont u tell me what to do | |
| 1:11 PM kevinconway it seems like it opens the door for any future resource to mutate instance | |
| 1:12 PM hub_cap catches up | |
| 1:12 PM vipul I think to the user, it may be a good thing to let them know that it belongs to something larger | |
| 1:12 PM vipul but agree that mutating it is kinda odd | |
| 1:13 PM imsplitbit I am open to suggestions on how to make that friendlier | |
| 1:13 PM imsplitbit right now there's no way for a user who does /instances get to know if any or all of the returned instances belong to something larger | |
| 1:13 PM kevinconway i think it should be kept in the /cluster/{id}/nodes{id} | |
| 1:13 PM kevinconway right, because they aren't looking at the cluster api | |
| 1:14 PM imsplitbit so programatically they wouldn't know there was a cluster nor that they should check the cluster api | |
| 1:14 PM hub_cap i disagree w/ imsplitbit | |
| 1:14 PM kevinconway didn't they pay for the cluster though? | |
| 1:14 PM hub_cap u should not be able to mutate /instances | |
| 1:14 PM imsplitbit so now if you're writing a systems integration app, e.g. rightscale you have to do a get on /instances and /clusters to piece everything together | |
| 1:15 PM vipul what about requiring that the cluster be created through /instances instead | |
| 1:16 PM vipul maybe allowing multiple instances to be created in a single call? | |
| 1:16 PM kevinconway i'm not sure that fits, clusters aren't really a sub-resource of /instances | |
| 1:16 PM vipul it's just a group of instances... | |
| 1:16 PM imsplitbit right | |
| 1:17 PM vipul otherwise we need to make it so that /instances are differnt from /clusters | |
| 1:17 PM vipul and you can't query one if it's creqtaed by the other | |
| 1:17 PM imsplitbit and then what do you do? replicate the entire instances api for /clusters/{id}/node? | |
| 1:18 PM vipul that will not allow you to do things like add instance to a cluster through /instances | |
| 1:18 PM imsplitbit to allow individual node operations? | |
| 1:18 PM kevinconway no, the /node sub-resource should only expose the node data and operations | |
| 1:18 PM vipul it seems that's the only way to do it if we can't modify instance | |
| 1:18 PM kevinconway resizes, etc. should still go to /instances | |
| 1:18 PM vipul hmm that's confusing | |
| 1:18 PM kevinconway its RESTful | |
| 1:19 PM vipul but then a 'node' is also an instance | |
| 1:19 PM vipul there is a link | |
| 1:19 PM vipul and can be modified in two ways | |
| 1:19 PM kevinconway for example, you can add or remove nodes | |
| 1:19 PM kevinconway but you can only resize instances | |
| 1:19 PM KennethWilke not fully informed here, but it seems it might make sense to have the operations through /clusters but a request for details from /instances would return some data about what cluster its in | |
| 1:20 PM imsplitbit KennethWilke: thats what I was shooting for with the attributes field. albeit a poor choice of name | |
| 1:20 PM hub_cap i dont buy the arguemtn of restful for resizes thru isntances | |
| 1:21 PM SlickNik Why not have all valid node operations happen through /cluster/ ? Under the covers we know it's an instance and can call instance resize, but the users specifically don't have to hit the instance API. | |
| 1:21 PM SlickNik That way all of the cluster operations happen through /cluster. | |
| 1:21 PM hub_cap lets first figure this out | |
| 1:21 PM hub_cap should the cluster be homogenous | |
| 1:21 PM hub_cap id say lets start with "yes" | |
| 1:21 PM hub_cap cuz its easy | |
| 1:21 PM imsplitbit SlickNik: then do we mask the instances from /instances once they're part of a cluster? | |
| 1:21 PM hub_cap and make it more complicated as time goes on | |
| 1:21 PM SlickNik homogenous wrt size? | |
| 1:21 PM hub_cap wrt flavor/disk | |
| 1:21 PM vipul hub_cap: does it have to be? | |
| 1:22 PM imsplitbit well disk size seems pretty hard set | |
| 1:22 PM hub_cap vipul: for rev 1 it is easiest i think | |
| 1:22 PM imsplitbit but size aside from disk seems a clear choice to allow | |
| 1:22 PM hub_cap rememeber im the guy who wants less features for the sake of getting somethign working | |
| 1:22 PM vipul ok, but i could see a user that wants less powerful slaves | |
| 1:22 PM imsplitbit right | |
| 1:22 PM imsplitbit vipul: +1 | |
| 1:22 PM KennethWilke +1 to working | |
| 1:22 PM kevinconway if the goal of making the nodes homogenous is to allow resizes to /cluster then yeah | |
| 1:23 PM vipul ok, v1 it might be ok to require homogeneous | |
| 1:23 PM hub_cap no its to punt on resizes kevinconway | |
| 1:23 PM hub_cap cuz its a whole nuther can of worms | |
| 1:23 PM SlickNik so homogeneity is one thing; but do we want to have cluster flavors? | |
| 1:23 PM hub_cap w/ validation and such | |
| 1:23 PM imsplitbit while I don't disagree with working, very few users will want to double their cost and try the working clusters just to get a reporting slave | |
| 1:23 PM vipul agree imsplitbit | |
| 1:23 PM SlickNik i.e. allow a cluster of all smalls, vs a cluster of all larges, etc. | |
| 1:23 PM hub_cap imsplitbit: sure that makse sense | |
| 1:23 PM hub_cap but lets at least try to break it up into pieces, get it working in versions | |
| 1:24 PM hub_cap as in, if we remove resizes, we remove, say 30% of complexity for v1 | |
| 1:24 PM imsplitbit the top 3 use cases for doing master-slave in mysql is 1. scale reads, 2. backups with no affect to master, 3. reporting slave | |
| 1:24 PM hub_cap and then we do a follow up, v2 w/ it | |
| 1:24 PM vipul is a resize that complex though? | |
| 1:24 PM imsplitbit forcing same size will address all 3 but can be severely cost prohibitive if you're just trying to solve for 2 or 3 | |
| 1:24 PM vipul assuming we know the disk usage, etc for the master | |
| 1:25 PM vipul it's just a bunch of real time chekcing, and we don't resize if it doensn't pass those checks | |
| 1:25 PM imsplitbit I think if you know the disk size of the master then part of a resize validation is that you can't resize to less disk than the master | |
| 1:25 PM kevinconway how does resizing change though? isn't that already a part of /instance? | |
| 1:25 PM imsplitbit and maybe that is a v2 thing | |
| 1:26 PM KennethWilke kevinconway: i wonder this as well | |
| 1:26 PM imsplitbit but I don't think you want to not think about that for the purposes of this discussion | |
| 1:26 PM kevinconway imsplitbit: that makes a good point | |
| 1:26 PM imsplitbit remember we're not talking implementation, just api | |
| 1:26 PM KennethWilke homogeneous sizing would also look silly for mongodb arbiters | |
| 1:27 PM imsplitbit where does everything fit in the api | |
| 1:27 PM kevinconway resizes already fit in /instances | |
| 1:27 PM imsplitbit KennethWilke: good point | |
| 1:27 PM hub_cap sure i dont mean to say lets _not_ do resizes | |
| 1:27 PM hub_cap hell | |
| 1:27 PM hub_cap if we dont disallow it | |
| 1:27 PM hub_cap it will still work | |
| 1:27 PM hub_cap and it might cover our bases for the sake of not worrying about it | |
| 1:27 PM vipul is there a use case wehre you want to resize a cluster vs an instance? | |
| 1:27 PM imsplitbit theres certainly some validation to be done if one wants to resize individual instances | |
| 1:28 PM imsplitbit vipul: I think so | |
| 1:28 PM vipul so then it kinda seems like it has to be in /instances | |
| 1:28 PM kevinconway resizing a cluster is just resizing each instance isn't it? | |
| 1:28 PM imsplitbit if you're solving for read scales and you're about to get featured on Oprah, you may want to /cluster/{id}/resize or something to that effect | |
| 1:28 PM vipul but why shouoldn't there be a /cluster/action/resize or something | |
| 1:28 PM SlickNik well, if we enforce homogeneity, then by definition you cannot resize an instance independent of the cluster. | |
| 1:28 PM hub_cap vipul: there probably should be | |
| 1:29 PM imsplitbit vipul: I think there should. but thats for operations on an entire cluster as a whole | |
| 1:29 PM imsplitbit IMO | |
| 1:29 PM hub_cap and yes kevinconway it is just resizing instances, but ther coud be thought involved in that | |
| 1:29 PM hub_cap if u are resizing down u dont want to resize master first (maybe?) | |
| 1:29 PM vipul right, i think then we have to eventually support both use cases | |
| 1:29 PM KennethWilke there probably should be thought involved in it | |
| 1:29 PM imsplitbit vipul: +1 | |
| 1:29 PM kevinconway *should* | |
| 1:29 PM KennethWilke and the logic for that will vary based on database technology too though, won't it? | |
| 1:29 PM hub_cap aye | |
| 1:29 PM KennethWilke very concerned with that ;) | |
| 1:30 PM hub_cap :) | |
| 1:30 PM KennethWilke definitely need simple to start | |
| 1:30 PM hub_cap so let me propsoe this | |
| 1:30 PM hub_cap if we are going to do /clusters/id/action {resize} | |
| 1:30 PM hub_cap why not make _all_ resizing, even for indiv instances in a cluster, here. | |
| 1:31 PM hub_cap seems like a good place to put all the logic for cluster based resizes | |
| 1:31 PM hub_cap either single or full cluster | |
| 1:31 PM hub_cap brb migrating outside | |
| 1:31 PM kevinconway i disagree, /instance already covers the resize for an instance | |
| 1:31 PM kevinconway if i'm acting on /cluster i expect to modify the /cluster resource not an individual /instance resource | |
| 1:31 PM vipul so /instances resize only works with instances that do not belong to a cluster? | |
| 1:32 PM kevinconway no, they're all still instances | |
| 1:32 PM kevinconway we don't modify /instance | |
| 1:32 PM imsplitbit I agree | |
| 1:32 PM imsplitbit the only stipulation there is how do I as a user programatically know that my instance is a member of a cluster when operating on it through /instance | |
| 1:33 PM imsplitbit or do I not care? | |
| 1:33 PM KennethWilke you shouldn't care for the action of resizing, but that should be in the metadata for the instance imo | |
| 1:33 PM imsplitbit this was one of the reasons we added the attributes field to the instance object | |
| 1:34 PM hub_cap well i guess the question is do we let the customer shoot themself in the foot | |
| 1:34 PM hub_cap (potentially) | |
| 1:34 PM KennethWilke yes | |
| 1:34 PM kevinconway if it's critical to add it to /instance then /instance/{id}/cluster can return a list of clusters its a part of | |
| 1:34 PM imsplitbit hub_cap: always | |
| 1:34 PM KennethWilke they'll do it anyway, make something that's simple and works logically | |
| 1:34 PM imsplitbit +1 | |
| 1:35 PM imsplitbit I believe that at some point, v34 maybe, we'll have the bullet proof cluster but right now we have no idea how users want to use them | |
| 1:35 PM imsplitbit so exposing knobs and buttons seems wise | |
| 1:35 PM SlickNik So what happens in this scenario: The instance is a slave node in a master/slave type cluster config. And you issue a create database / create user in the instance? | |
| 1:35 PM imsplitbit let them break it in intersting ways so we can figure out how they really want to mess with it | |
| 1:36 PM hub_cap let me ask something controversial | |
| 1:36 PM KennethWilke i think it might be confusing if there are two interfaces to resize an individual instance | |
| 1:36 PM SlickNik on the instance* | |
| 1:36 PM hub_cap if we want to prohibit users from shooting themselves in the foot, why not make /instances and /clusters completely different and dont show the /instances that belong to a /cluster | |
| 1:36 PM hub_cap then they _cant_ make mistakes | |
| 1:36 PM kevinconway oh... | |
| 1:36 PM imsplitbit SlickNik: if attributes:{clusterType is something like master slave and there is a read only attribute there then the operation shoudl fail I guess | |
| 1:37 PM KennethWilke hmm, that's an interesting idea | |
| 1:37 PM SlickNik ++ hub_cap | |
| 1:37 PM kevinconway yeah, i kind of like that | |
| 1:37 PM hub_cap /cluster/id/resize {"slave1":"flavor", "slave2":"flavor2"} | |
| 1:37 PM imsplitbit I don't dislike that | |
| 1:37 PM hub_cap or /cluster/id/resize {"flavor3"} //for all | |
| 1:37 PM KennethWilke yeah, i'll give that my support | |
| 1:37 PM imsplitbit but we would *have* to then make them invisible in /instance | |
| 1:37 PM hub_cap correct imsplitbit | |
| 1:37 PM hub_cap then we make our lives easier | |
| 1:37 PM imsplitbit and then replicate all of /instance functionality | |
| 1:37 PM hub_cap well | |
| 1:37 PM SlickNik Well, not replicate | |
| 1:37 PM hub_cap we refactor instances so we can reuse it ;) | |
| 1:37 PM SlickNik but delegate | |
| 1:37 PM kevinconway resizing the nodes individually is the same as instances? | |
| 1:38 PM imsplitbit sure delegate | |
| 1:38 PM SlickNik yup, refactor - not copy. | |
| 1:38 PM hub_cap kevinconway: im not sure what u mean, technically it follows the same code path but its got lots of checks and balances | |
| 1:38 PM imsplitbit but there'd need to be something that handles every normal operation that one would normally do on an instance | |
| 1:38 PM KennethWilke +1 refactor | |
| 1:38 PM kevinconway so are you saying that /instance would never return the ones used in my cluster? | |
| 1:38 PM hub_cap because of the cluster logic | |
| 1:38 PM imsplitbit and redirect it to it's /instance counterpart | |
| 1:38 PM hub_cap kevinconway: correct | |
| 1:38 PM kevinconway ok, that sounds good | |
| 1:38 PM hub_cap they arent instances anymore and not controlled by /instances | |
| 1:38 PM imsplitbit I'm not opposed to that | |
| 1:38 PM hub_cap they are members of a cluster | |
| 1:38 PM kevinconway but would /cluster/id/nodes/id/resize fit better for resizing nodes? | |
| 1:39 PM hub_cap sure | |
| 1:39 PM imsplitbit so you cannot do any instance operation on it whatsoever | |
| 1:39 PM imsplitbit via /instance | |
| 1:39 PM hub_cap correct | |
| 1:39 PM hub_cap u cant even see it via /instances | |
| 1:39 PM imsplitbit I like that | |
| 1:39 PM imsplitbit it makes things simple | |
| 1:39 PM hub_cap well i guess this was less controversial than i thought | |
| 1:39 PM kevinconway oh wait, what happens when i want to upgrade my instance to a cluster? | |
| 1:39 PM vipul what abot an individaul instance resize ? | |
| 1:39 PM hub_cap kevinconway: its no longer in /instances after that | |
| 1:39 PM vipul kevinconway: yep good piont | |
| 1:39 PM imsplitbit stipulating that you hide the instances in /instances | |
| 1:39 PM SlickNik There are potentially a lot of restrictions based on what you can and cannot do to a cluster node (as opposed to a plain ol' instance). | |
| 1:39 PM hub_cap sure thats easy | |
| 1:39 PM KennethWilke would that require a change to /instance? as clusters would then probably have a separate table in the database | |
| 1:40 PM SlickNik Some of these even depend on the type of cluster. | |
| 1:40 PM hub_cap well its gonna be different wrt heat (tables etc...) | |
| 1:40 PM imsplitbit well the cluster table would contain uuids of instances only no? | |
| 1:40 PM SlickNik So IMO it might be a good idea to separate out the concepts. | |
| 1:40 PM hub_cap implementation | |
| 1:40 PM imsplitbit yah | |
| 1:40 PM imsplitbit ok back to api | |
| 1:40 PM imsplitbit no individual operations on instances via the /instances api | |
| 1:41 PM imsplitbit agreed? | |
| 1:41 PM kevinconway so we take out the cluster data in /instance and remove /instances that are parts of clusters? | |
| 1:41 PM hub_cap #agreed | |
| 1:41 PM vipul how do i resize an individaul instance | |
| 1:41 PM vipul that belongs to a cluster? | |
| 1:41 PM hub_cap vipul: kevinconway mentioned that | |
| 1:41 PM hub_cap /cluster/id/nodes/id/resize | |
| 1:41 PM imsplitbit /clusters/id/node/id/action/resize? | |
| 1:41 PM vipul ah | |
| 1:41 PM kevinconway #agreed | |
| 1:41 PM hub_cap yes action/resize | |
| 1:41 PM SlickNik like /cluster/id/node/id/resize | |
| 1:41 PM hub_cap i HATE actions | |
| 1:41 PM KennethWilke lol | |
| 1:41 PM imsplitbit lol | |
| 1:41 PM hub_cap but its the api as it stands today | |
| 1:41 PM hub_cap we will fix in v2 | |
| 1:41 PM imsplitbit morris is a huge fan of actions | |
| 1:41 PM hub_cap and actions {} will dissapear | |
| 1:41 PM KennethWilke yay | |
| 1:41 PM imsplitbit he keeps pushing me to make actions | |
| 1:41 PM hub_cap i dont give a flying f... | |
| 1:42 PM imsplitbit lol | |
| 1:42 PM KennethWilke hub_cap +1 | |
| 1:42 PM imsplitbit I'm fine with no actions | |
| 1:42 PM KennethWilke useless text! | |
| 1:42 PM imsplitbit I didn't really care for it | |
| 1:42 PM hub_cap :) | |
| 1:42 PM SlickNik I don't like actions, either. | |
| 1:42 PM imsplitbit it was a crackheaded concept to begin with | |
| 1:42 PM hub_cap i think nonoe of us do | |
| 1:42 PM imsplitbit :-D | |
| 1:42 PM hub_cap its from old cloud servers | |
| 1:42 PM hub_cap but | |
| 1:42 PM vipul maybe this was answered too.. what if i created a instance and now i want it to be a node? | |
| 1:42 PM hub_cap we all agree that we keep it in | |
| 1:42 PM KennethWilke vipul: no! | |
| 1:42 PM hub_cap /clusters/superfancyupgrade/instanceid | |
| 1:42 PM KennethWilke you can't! | |
| 1:42 PM KennethWilke stop that! | |
| 1:42 PM KennethWilke lol | |
| 1:43 PM imsplitbit hub_cap: +1 | |
| 1:43 PM hub_cap or | |
| 1:43 PM imsplitbit it's just a part of the post for /clusters | |
| 1:43 PM vipul that's not too resty | |
| 1:43 PM kevinconway ok, next question: when adding a node to the cluster is it PUT /cluster or POST/cluster/id/node | |
| 1:43 PM imsplitbit if you specify a master instance | |
| 1:43 PM hub_cap /clusters/action {upgrade} | |
| 1:43 PM vipul you want to do a PUT on /instances or something | |
| 1:43 PM SlickNik vipul, for v1 maybe bring up cluster based on node. | |
| 1:43 PM KennethWilke so can you add an node to a cluster if it's a different size than the servers in the cluster? | |
| 1:43 PM SlickNik not necessarily make node part of cluster. | |
| 1:44 PM imsplitbit vipul: I think the put makes sense since it's operating and changing an instance but it's confusing to me to put a cluster operation in /instances | |
| 1:44 PM SlickNik kind of like restore a backup gives you a new node. | |
| 1:44 PM SlickNik new instance* | |
| 1:45 PM hub_cap i think we are talking about 2 things at once here | |
| 1:45 PM hub_cap lets finish vipuls thought first | |
| 1:45 PM kevinconway should we talk about more? | |
| 1:45 PM vipul it's a little differnt in that cae the Backup doesn't change.. | |
| 1:45 PM hub_cap before we go on to kevins | |
| 1:45 PM hub_cap vipul: did u get an answer to your q | |
| 1:45 PM SlickNik I agree it's a bit different. | |
| 1:45 PM SlickNik But suggesting that that may be an approach | |
| 1:46 PM vipul i have a instance, and i want to have it join a cluster.. if we make it /cluster/id/node/instanceid or something | |
| 1:46 PM vipul that changes the instance | |
| 1:46 PM vipul so the user should explicitly be doing a PUT on an instance | |
| 1:46 PM vipul to promote it i think | |
| 1:46 PM SlickNik define join the cluster. | |
| 1:46 PM hub_cap i can buy that | |
| 1:46 PM hub_cap create a cluster froma instance SlickNik | |
| 1:46 PM kevinconway i think that ties into my question of adding nodes to the cluster | |
| 1:46 PM KennethWilke they seem related | |
| 1:47 PM kevinconway i would think POST /cluster/id/node | |
| 1:47 PM KennethWilke or one in the same | |
| 1:47 PM imsplitbit hub_cap: /instance/id/superfancyupgrade? | |
| 1:47 PM vipul SlickNik: yea basically making that instance a node in a new cluster | |
| 1:47 PM imsplitbit is that what we're talking about? | |
| 1:47 PM hub_cap lets use actions for upgrade | |
| 1:47 PM imsplitbit omg | |
| 1:47 PM KennethWilke lol | |
| 1:47 PM imsplitbit didn't we just say we're killing that? | |
| 1:47 PM vipul hmm i guess that'll work | |
| 1:47 PM SlickNik so we can create a new cluster with new "nodes" (not the instance) with the same data on the instance. | |
| 1:48 PM vipul is that true? | |
| 1:48 PM vipul that means you'd have to backup, spin up a new node, and restore to that | |
| 1:48 PM vipul i thought that instance would just 'transplant' itself | |
| 1:48 PM imsplitbit yeah if you're bringing an existing dataset that seems logical to reuse that code | |
| 1:48 PM SlickNik It's a matter of implementation. We could go either way. | |
| 1:48 PM KennethWilke i think a transplant makes more sense too | |
| 1:48 PM imsplitbit for making replicas if the cluster type requres it | |
| 1:49 PM KennethWilke would certainly be quicker | |
| 1:49 PM vipul hmm | |
| 1:49 PM vipul i guess what does the user expect when they do 'superfancyupgrade' | |
| 1:49 PM imsplitbit if you take an existing dataset and say make that a master, you'll need to run the backup on that new master to get it's dataset and (in the case of mysql replication) get the binary log name and postion | |
| 1:49 PM vipul do they expect to see that node still in it's current state, as well a replicted copy? | |
| 1:50 PM KennethWilke should there be disruption for adding an instance to a cluster? | |
| 1:50 PM KennethWilke or moving an instance into a cluster rather | |
| 1:50 PM hub_cap i think that depends on what moving it means :) | |
| 1:50 PM SlickNik KennethWilke: Define moving an instance into a cluster. Why would you want to do this? | |
| 1:51 PM hub_cap its waht we are talking about SlickNik | |
| 1:51 PM hub_cap upgrading a single to cluster | |
| 1:51 PM SlickNik The data on the instance would be wiped out, no? | |
| 1:51 PM imsplitbit NO! | |
| 1:51 PM imsplitbit :-) | |
| 1:51 PM imsplitbit never | |
| 1:51 PM kevinconway the instance stays but is hidden from the /instance listings it becomes a /node in a /cluster | |
| 1:51 PM vipul SlickNik: I start small, with a single node Database... eventually need to add read slaves and want to make that single instance database the master | |
| 1:51 PM kevinconway or did i mangle that as well? | |
| 1:51 PM SlickNik maybe I'm misunderstanding. | |
| 1:51 PM imsplitbit there is a real use case for taking an existing dataset and just starting replication to nodes from that dataset | |
| 1:52 PM SlickNik I can see the reason to create a cluster from an instance. | |
| 1:52 PM hub_cap imsplitbit: i thought thats what we would do | |
| 1:52 PM SlickNik I don't see a reason to move an instance into an already existing cluster. | |
| 1:52 PM imsplitbit albeit that's the more complicated use case. it's much easier to make a fresh cluster *thing* and then have the user migrate to it but that's not in the interest of the user | |
| 1:52 PM hub_cap for v1 | |
| 1:52 PM vipul SlickNik: yea i don't think you could | |
| 1:52 PM vipul move an instance to a cluster that already has different data | |
| 1:52 PM hub_cap SlickNik: youre misinterpreting what KennethWilke said i think | |
| 1:52 PM KennethWilke SlickNik: yes, i meant more along the lines of what vipul was talking about | |
| 1:53 PM hub_cap upgrade instance->cluster | |
| 1:53 PM imsplitbit SlickNik: I think the question was "I have a 400GB dataset, migrating that dataset will take large amounts of time | |
| 1:53 PM KennethWilke promote an instance to be the master of a new cluster | |
| 1:53 PM vipul it would have to be a new cluster when that upgrade happens | |
| 1:53 PM imsplitbit can I just start 2 more instances and make them slaves of that 400gb dataset? | |
| 1:53 PM hub_cap imsplitbit: maybe we make an option to "start_fresh" | |
| 1:53 PM imsplitbit I don't disagree with that | |
| 1:53 PM kevinconway imsplitbit: seems like that would be through POST /cluster/id/node | |
| 1:53 PM imsplitbit but not as a default | |
| 1:53 PM hub_cap as in, the custoemr makes the decision to 1) start replicaitng from wherever or 2) restart | |
| 1:53 PM imsplitbit cause a logical user would imagine we aren't deleting their data | |
| 1:53 PM SlickNik ah, okay gotcha KennethWilke. | |
| 1:54 PM KennethWilke SlickNik: it coulda been worded better :p | |
| 1:54 PM imsplitbit kevinconway: but if the cluster doesn't exist? | |
| 1:54 PM imsplitbit I've got one instance | |
| 1:54 PM imsplitbit and I need to make a cluster *thing* | |
| 1:54 PM SlickNik no worries, on the same page now. | |
| 1:54 PM imsplitbit call it master-slave | |
| 1:54 PM imsplitbit with 2 slaves | |
| 1:54 PM imsplitbit so there's nothing to operate on in /cluster/id | |
| 1:54 PM kevinconway POST /cluster; POST /cluster/id/node; POST /cluster/id/node | |
| 1:54 PM imsplitbit how do we make that look? | |
| 1:55 PM vipul hmm, 3 sepaprate calls? | |
| 1:55 PM kevinconway the first creates the cluster and sets the instance as the master node, removes it from the /instance list | |
| 1:55 PM imsplitbit so my api proposal was a post on /cluster with a primaryNode optional attribute that if specified used that node as the starting point | |
| 1:55 PM KennethWilke that makes me think, how will the featureset of the database technology being used be exposed via /cluster | |
| 1:55 PM hub_cap KennethWilke: somewhat yes | |
| 1:56 PM imsplitbit KennethWilke: thats what i was talking about earlier. we'll have to have createdb create user etc on /cluster | |
| 1:56 PM SlickNik throwing something out there. | |
| 1:56 PM imsplitbit throw it! | |
| 1:56 PM SlickNik What if instead of a node, you start out with a backup. | |
| 1:56 PM KennethWilke i don't want your createdb or your create user in my redis! | |
| 1:56 PM SlickNik So the actual data. | |
| 1:56 PM imsplitbit lol | |
| 1:56 PM imsplitbit good point | |
| 1:56 PM KennethWilke lol | |
| 1:57 PM imsplitbit SlickNik: that's definitely another use case | |
| 1:57 PM SlickNik And the cluster is created based on the actual data rather than a node. | |
| 1:57 PM SlickNik instance* | |
| 1:57 PM imsplitbit I think thats use case 3 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment