gistfile1.txt

[1:00 PM] David Baumgold: how about I set it up so that people in https://github.com/Stanford-Online have the same permissions to kick off new jobs that people in https://github.com/edx have?
[1:01 PM] Joe Blaylock: That would be pretty great. Is there any way to make it aware of hashes on our repos too? Because that's another problem (which admittedly the group thing doesn't solve) that makes this complicated.
		Sorry, I'm having trouble being specific and actionable. Let me try again.
		Yes, that sounds awesome. Please do that. In addition, can you also have jenkins fetch from our fork so we can give our hashes?
[1:02 PM] David Baumgold: try logging into Jenkins now: https://jenkins.testeng.edx.org
		when you say "give your hashes", what do you mean?
[1:03 PM] Joe Blaylock: Well, when you build with parameters, you use a SHA1 of a commit object. Right now, jenkins is configured to fetch from edx/edx-platform only, and so the SHA1s that live on our repo are unknown to it.
		I can log in, and I can reach build with parameters. Yay.
[1:04 PM] David Baumgold: ah, I see
		hmm, let me think about a good way to make that happen -- I'm a bit concerned about making Jenkins fetch from your fork every time
		first, because it will make every job take longer
		and second, others may want the same priviledges, which would make every job take *even* longer
[1:05 PM] Joe Blaylock: If you fetch from multiple remotes, all the SHAs go into a big bag together and you can checkout or cherry-pick by SHA without regard to where the commit(ish) originated from. It seems as if there should be a simpler way, but I'm not sure what.
[1:05 PM] David Baumgold: I know that our testeng team is talking with a company called Solano CI to try to set up a system that is more like Travis CI for our repos
[1:05 PM] Joe Blaylock: Oh, could the machinery around build with parameters be changed to take a fully specified path to a commit? i.e., git+https://github.com/Stanford-Online/edx-platform/commit/blahblahblah ?
[1:06 PM] David Baumgold: which would make it easier for forks to hook up their own CI system
[1:06 PM] Joe Blaylock: (optionally. Clearly you wouldn't want to always have to specify the full path.)
		(though that wouldn't be completely awful since you could just cut and paste from github. But still.)
[1:06 PM] David Baumgold: the problem with Travis, as I understand it, is that Travis always always always starts from a base vanilla Ubuntu OS, and installing all the platform requirements takes so long that it times out before it even starts running the tests
[1:06 PM] Joe Blaylock: Yes, we've tried travis and ran into that.
[1:06 PM] David Baumgold: specifying a full path is a really good idea!
[1:07 PM] Joe Blaylock: It's bollocks.
[1:07 PM] David Baumgold: that sounds like something that would be implemented in the Jenkins plugin, though, rather than in one of our scripts
		let me look into that
[1:07 PM] Joe Blaylock: Ok, thank you.
[1:08 PM] Joe Blaylock: We've often mused that it would be cool to have a travis-like thing which brought up a specific AMI rather than a vanilla box, and then you could at least avoid having to rebuild scipy, which is just a bitch.
[1:09 PM] David Baumgold: I believe that's what Solano is trying to do
		https://www.solanolabs.com
[1:09 PM] Joe Blaylock: I thought Jenkins was perl, but this website looks like lightly wrapped Tracker
[1:09 PM] David Baumgold: Jenkins is Java
[1:09 PM] Joe Blaylock: oh, sad. The chances of me rewriting the plugins just went way down.
[1:10 PM] David Baumgold: yeah, same here :(
[1:10 PM] Joe Blaylock: I wonder what Stephen Sanchez is up to...
[1:10 PM] David Baumgold: oh, BTW, since you said you were bringing interns on soon
		as I said, I gave the Stanford-Online organization on github all the perms of the edx organization on github, when it comes to running stuff on Jenkins
[1:11 PM] Joe Blaylock: That's great, thanks.
[1:11 PM] David Baumgold: however, something that's bitten us in the past is that when you add a user to an organization, by default the user is set to *not* show that organizational affiliation on their github profile
[1:11 PM] Joe Blaylock: We'll still have to publicize membership, I assume?
[1:11 PM] David Baumgold: they're secretly a member of the organization
		yes, that's what I was getting at
[1:11 PM] Joe Blaylock: Yeah. :)
[1:11 PM] David Baumgold: if you don't publicize it, Jenkins doesn't know, and it will shut you out
[1:13 PM] Joe Blaylock: right, ok. I just noted that down so I can make sure it ends up in our first-day handout. Which we don't have yet, but I'm feeling optimistic this morning.
[1:13 PM] David Baumgold: :)
		so, the good news is that Jenkins jobs do support running bash scripts at certain points in the build process
		and we already have a "build" script specified in the jenkins job
		which does things like reporting to Github at the start of the build that the build is starting up
		so, theoretically we could extend that to fetch from a remote repo specified in the full path
		and then checkout the repo to the commit hash specified in the path
		the bad news is that there is a field marked "Branches to build", which is set to the value "${COMMIT_SHA}", which is the value defined in the parameterized build
		and I'm not sure what happens if you give it an invalid commit reference
		I'm not a Jenkins wizard here, I just got admin rights on Jenkins awhile ago and no one has taken them back from me yet :D
[1:16 PM] Joe Blaylock: I think the current behavior is just that it errors out with nonzero status and reports in Jenkins that it couldn't find the build target.
[1:16 PM] David Baumgold: that would make sense
[1:16 PM] Joe Blaylock: Because that's what appears to happen when we give our SHAs now. :)
[1:17 PM] David Baumgold: and if so, then it would halt the build there, so it wouldn't get to the build script step
		theoretically, I could see a way to adjust the parameterized build page so there are two fields, instead of one
		the first field would be the existing one: the git commit hash, if that hash is present in our edx fork
[1:17 PM] Joe Blaylock: The build script isn't very long, right? Could you paste it somewhere for me?
[1:18 PM] David Baumgold: https://gist.github.com/singingwolfboy/774ac6b2c6f74c8c649f
		if you want to build a hash that isn't present in our fork, maybe you would leave the first field blank, and put the full path into the second field
		if you leave that branch specifier field empty, it should just checkout the "master" branch of the repo
		and then the build script could parse the value in that second field, and do any fetches and checkouts that it needs to at that point
		it would be nice if we just had one field, and it would do the right thing depending on the context that you put in, but I suspect that would require rewriting the Jenkins plugin
		whereas having two fields like this would allow us to hack around it, essentially
		if we do want to do this, I would definitely need to run it past testeng first
[1:24 PM] Joe Blaylock: I think that would be a good solution for us, and I'd appreciate if you'd check with testeng.
[1:25 PM] David Baumgold: OK, I'll do that