Possible solutions to currently identified problems with Kubernetes governance

PROBLEM: The structure of the project is opaque to newcomers

project structure added to governance repository under Kubernetes GitHub organization
create "Front Desk" inspired by Debian Project to welcome new contributors
create mentorship requirement in contributor ladder

PROBLEM: There is no clear technical escalation path / procedure

adopt RFC process inspired by Rust langage governance
create technical steering committee bootstrapped from "top level OWNERS" as penultimate arbiters with power vested in Kubernetes community to make all final decisions
affirm decision by technical steering committee in case inability of community to make decision
devolve primary decision making and consensus building authority to SIGs
require SIGs to create policies for exemption from RFC process for "minor" changes
establish Technical Lead role for SIGs responsible for final decision within SIG
establish formal escalation from SIG to technical steering committee
deprecate "features" process and repository
place component/area ownership under SIGs, define escalation from component/area owners to SIGs

PROBLEM: There aren't consistent / official decision-making procedures for ~anything: consensus, lazy consensus, consensus-seeking, CIVS voting, etc.

empower the "PM Group" to serve as a policy clearinghouse which operates under consensus-seeking model
- other SIGs chartered under the "PM Group" and or a technical steering committee will make policy decisions related to a particular area (e.g. contributor experience, release management, harassment) but the "PM Group" should reconcile possibly conflicting policy prescriptions
devolve technical decision making from technical steering committee to SIGs which operate under consensus-seeking model unless otherwise specified in their SIG charter
require SIG charter to specify decision making specifics (e.g. TTL, majority thresholds)
require supermajority voting on issues before technical steering committee
affirm all decisions made by PM Group (or delegates) or technical steering committee by lazy consensus

PROBLEM: There are no official processes for adding org members, reviewers, approvers, maintainers, project leaders, etc

define minimum requirements to be added to project repositories
establish "Front Desk" for project wide GitHub administrative tasks
empower SIGs to perform additional GitHub administration tasks independently. Ideally each SIG should have at least one person who is an GitHub Organization owner. Git is a powerful time machine and I feel that we should be generally permissive in granting access to SCM in order to support a "merge then review" development workflow which differs from the "review than merge" workflow currently adopted by the project
require SIGs to specify component/area ownership change rules in SIG charter
require elections for all project leadership positions

PROBLEM: There is no official / regularly meeting body to drive overall technical vision of the project.

establish technical steering committee

PROBLEM: We don't agree on the right level/types of engagement for leaders. Some feel that leaders should be recused from responsibilities such as SIG leadership, while others feel they need to be deeply involved in releases, etc.

split technical, administrative, and product/project roles at SIG level
establish permanent bodies to handle testing and release management; add or reserve at least one seat for release management leader and testing leader on any technical steering committee
- SIG Testing already exists but should be empowered to prescribe policies including blocking submit queue until test failures are addressed
- Establish SIG Release focusing on release management and infrastructure

PROBLEM: There aren't official technical leads for most subareas of the project

formally devolve component/area ownership to SIGs
establish mentorship requirements for project leadership to grow new leaders
split technical, administrative, and product/project roles at SIG level to encourage broader participation

PROBLEM: There is no centralized / authoritative means of resolving non-technical problems on the project, including staffing gaps (engineering, docs, test, release, ...), effort gaps (tragedy of the commons), expertise mismatches, priority conflicts, personnel conflicts, etc.

establish permanent body to handle moderation issues
establish staffing requirements for corporate inclusion in project materials (e.g https://kubernetes.io/partners/) to address gaps
establish body under permanent "PM Group" to aggregate feature requests
allow removal from any role of leadership
establish "bounties" or PR multipliers for contributions to poorly staffed areas of the project

PROBLEM: [In particular, there is] insufficient effort on contributor experience (e.g., github tooling, project metrics), code organization and build processes, documentation, test infrastructure, and test health. Some on the project have argued that there is insufficient backpressure and/or incentives for people employed to deliver customer-facing features to spend time on issues important to overall project health.

establish "bounties" or PR multipliers for contributions to poorly staffed areas of the project
establish body under permanent "PM Group" to aggregate feature requests
establish staffing requirements for corporate inclusion in project materials (e.g https://kubernetes.io/partners/) to address gaps

PROBLEM: [A related issue is] counterbalancing technical- and product-based decision-making.

establish body under permanent "PM Group" to aggregate feature requests
empower permanent bodies focused on release management and testing to directly influence SIG priorities
empower SIGs to set their own priorities with veto power held by technical steering committee
formalize SIG PM representative role as a project rather than product management role

PROBLEM: Visibility across the entire project is lacking.

require basic project management as a precondition of SIG chartering
require PM representative as a precondition of SIG chartering
require PM Group and a "Technical Steering Committee" produce "weather forecast" for Kubernetes
- weather forecast format will hopefully allow for longer term communication of project direction while acknowledging that deadlines slip and no feature is guarenteed to arrive in a particular release
- weather forecast for external consumers of the project could be combined with a monthly development report based on changes actually merged into SCM

PROBLEM: Metrics, metrics, metrics, metrics. We're flying blind.

require metrics on intended effect to be produced before implementing project policy changes
require corporate staffing of body focused on contributor experience
create PR multiplier for contributions for contributor experience

PROBLEM: There is no documented proposal process.

consolidate proposal and feature process into single RFC process inspired by Rust

PROBLEM: There isn't a documented process for advancing APIs through alpha, beta, stable stages of development.

drop stability distinction to two categories unstable|stable with technical steering committee deciding on graduation
require technical steering committee to ungate functionality

PROBLEM: Project technical leaders (de facto or otherwise) are not available via office hours.

require communication SLAs
establish regularly meeting technical steering committee
require SIG office hours as a precondition for charter

PROBLEM: We don't have processes or documentation for onboarding new contributors.

establish "Front Desk" inspired by Debian project to guide new contributors through ladder
establish mentorship requirements for project leaders

PROBLEM: There are no official safeguards to prevent control over the project by a single company.

establish quotas for corporate contributions for all project leadership positions
route corporate engagements through single body, possibly "PM Group"

PROBLEM: Project leadership lacks diversity.

establish "Front Desk" to help guide new contributors through process
term limits for elected project leadership roles but possibly not for "merit" based component/area ownership roles
create permanent engagement body to reach out to underrepresented communities

PROBLEM: There is no conflict of interest policy regarding leading/directing both the open-source project and commercialization efforts around the project.

require project members to act in project interest
disallow corporate affiliation of contributions (e.g. work is done by individuals not corporations)
allow for removal from all elected roles of project leadership
allow for removal from component/area ownership due to conflict of interest

PROBLEM: There is no consistent, documented process for rolling out new processes and major project changes (e.g., requiring two-factor auth, adding the approvers mechanism, moving code between repositories).

empower "PM Group" to make process and project changes

PROBLEM: Nobody has taken responsibility to think about and improve the structure of the project, processes, values, etc. A few people have been working on this part time, but it needs more and more consistent attention given the rate of growth of the project.

consider having a corporate dues collecting umbrella organization sponsor people to improve structure of project
establish corporate staffing requirements for contributor experience body under "PM Group" and technical steering committee

PROBLEM: [We're also] lacking people to drive, implement, communicate, and roll out improvements (and test, measure, rollback, etc.).

require metrics collection as part of policy and process changes
require corporate staffing requirements for contributor experience body under "PM Group" and technical steering committee

PROBLEM: There isn't a sufficiently strong feedback loop between technical contributors/leadership and the PM group.

have "PM Group" and technical steering committee jointly determine release priorities
empower SIGs to draft their own priorities for each release
empower permanent body focused on release management to focus on releasing Kubernetes project continuously to reduce "fear of missing out" when developing features against a time bounded release window
allow "PM Group" and technical steering committee to veto proposed SIG priorities

PROBLEM: Nobody has taken responsibility for legal issues, license validation, trademark enforcement, etc.

use CNCF or create Kubernetes specific umbrella organization to handle legal issues

PROBLEM: Technical conventions/principles are not sufficiently documented.

empower contributor experience body under "PM Group" and technical steering committee to document project wide technical conventions
require changes to technical conventions to follow RFC process
require SIGs to document conventions/principles in their charter

PROBLEM: Development practices and conventions across our repositories are not consistent.

empower technical steering committee to create body focused on standardizing development practices and conventions

PROBLEM: Our communication media are highly fragmented, which makes it hard to understand past decisions.

make source control the canonical source of truth for all past decisions
ensure all policy and process changes also follow RFC process with results committed to source control

I think the Rust RFC process could be a good starting point for formalizing our proposal process.

Will work with @pwittrock on trying to adapt Rust's RFC process to our project

I also think there's an agreement that we need a concept of Technical Leads for SIGs. Whether one or multiple per SIG is TBD.

I can imagine having both a "technical lead" for a single SIG and a "technical director" for multiple SIGs once we scale to needing the additional structure

Comments on lazy consensus vs consensus seeking vs variants:

kubernetes/community#402 (comment)

Needs more work.

I think we could decide that different decision making processes should be followed for different types of decisions and for different decision making bodies. I would think that a decision made by the community at large could follow a voting process whereas most SIGs which contribute to SCM could operate under a consensus seeking model. I think that many policy decisions could largely be made by a lazy consensus model with some fallback mechanism

I think not all "policy" will fall under one SIG. At least PM, contributor experience, testing, scalability, API machinery, release, and governance are likely to own some policies.

Agreed but I also believe that there needs to be a single body responsible for serving as a policy clearinghouse which should delage policy for specific policies to a few cross cutting SIGs

Github permissions are an unworkable disaster. What "github administration" tasks did you have in mind?

Given our development workflow I agree that GitHub permissions are not nearly granular enough. There's a lot of existing art where direct access to SCM is a general right of contributors and I much prefer a "merge then review" model over a "review then merge" model given the latter seems to incur a fair amount of administrative overhead (e.g. conducting timely code reviews) in practice

"establish body under permanent "PM Group" to aggregate feature requests": How do you imagine this would work and how it would differ from the current features process?

I would like user cases to be consolidated under a single body to

aggregate user requests
help guard against commercial conflicts of interest where a Product Manager from Company Y talks directly to a TL or developer also employed by Company Y so there's no transparency for anyone not employed by Company Y as to why a feature is important

In my ideal world I would imagine a relationship between the PM Group + TSC and a SIG to be similar to the relationship between the legislative and executive branches of the US government where the PM Group and TSC decide what's important across the project and the SIG agrees/vetoes the plan and is responsible for execution of the plan. This arrangement would require some mechanism for overriding a SIG veto and enforcing compliance

"empower permanent bodies focused on release management and testing to directly influence SIG priorities": As we've seen with the "flaky test" discussions, there are details that need to be figured out regarding how we make that happen.

Agreed we need to hammer out some details. I think an idea is to give SIG Testing and a future SIG Release the power to make developers feel pain if project wide health drops below some threshold as a "nuclear" option. I don't know yet how to make individual SIGs responsible for working on globally optimizing for project health (e.g. test flakes)

How do you see the "weather forecast" as different from what the PM group has been doing for K8s?

I think it would primarily signal uncertainty of deliverable dates while providing some insights into overall project direction. I think that monthly reports on changes which actually have landed would also be helpful

"require corporate staffing of body focused on contributor experience": Would love to figure out how to make that happen.

Sneak it into a Corporate CLA? 😄

It's not practical to bottleneck all features on TSC approval.

I think SIG ownership of APIs is a reasonable goal, but there are some issues that need to be resolved: kubernetes/community#419 (comment)

I wouldn't want to block users or developers from having access to features but I think the default enablement of features should be centralized. If members of the TSC cannot discharge their responsibility to steer the project we should have mechanisms for communicating no confidence from the community and changing the members.

calebamiles/problems.md