bazel.md

Extremely unclear to me what the difference between some of the @rules_cc and @bazel_tools things are supposed to be:

`@bazel_tools`	`@rules_cc`
`@bazel_tools//tools/cpp:current_cc_toolchain`	`@rules_cc//cc:current_cc_toolchain`
`@bazel_tools//tools/cpp:toolchain_utils.bzl`:`find_cpp_toolchain`	`@rules_cc//cc:find_cc_toolchain.bzl`:`find_cc_toolchain`
`@bazel_tools//tools/cpp:toolchain_type`	`@rules_cc//cc:toolchain_type`

@rules_cc//cc:find_cc_toolchain.bzl:find_cc_toolchain returns a CcToolchainInfo (documented here). @bazel_tools//tools/cpp:toolchain_utils.bzl:find_cpp_toolchain claims it returns a CcToolchainProvider (not officially documented) but it's lying; it returns a CcToolchainInfo

To make things even more confusing @rules_cc has a find_cpp_toolchain in @rules_cc//cc:toolchain_utils.bzl that does the same thing but says its deprecated.

Regardless — somehow even when I do what @rules_cc tells me to do:

cc_rule(
    _whatever_impl_function,
    attrs = {
        "_cc_toolchain": attr.label(
            default = Label("@rules_cc//cc:current_cc_toolchain"),
        ),
    toolchains = [
        "@rules_cc//cc:toolchain_type",
    ],
)

It doesn't work. If I print out ctx.toolchains I can see that it's actually getting passed a @bazel_tools//tools/cpp:current_cc_toolchain.

...

Update: okay I'm kind of dumb. The @rules_cc toolchain_type is just an alias of its @bazel_tools counterpart.

I'm gonna stick with using the @rules_cc versions since we're pulling in @rules_cc anyways and I think @rules_cc is supposed to be the documented/suitable-for-public consumption interface. Also I'm pretty sure the plan was to rename find_cpp_toolchain → find_cc_toolchain and the fact that it's not renamed in @bazel_tools is (I think) a good hint that we shouldn't be using it.

This does not at all explain why the above is broken though.

bargaining

As a workaround, if you use `` instead everything works.

But why!?

arghh

Update two: actually I'm pretty convinced that @rules_cc//cc:find_cc_toolchain.bzl:find_cc_toolchain is in the wrong here and that it will straight-up break if used with @rules_cc's own :toolchain_type when --incompatible_enable_cc_toolchain_resolution becomes the default.

Given that ctx.toolchains is a ToolchainContext (the stubbed out docs are here, actual source code is here and here — the important bit is that it has a set of labels inside of it) what would we expect the following two print statements to emit?

def _whatever_impl_function(ctx):
    print("@bazel_tools//tools/cpp:toolchain_type" in ctx.toolchains)
    print("@rules_cc//cc:toolchain_type" in ctx.toolchains)

Since the @rules_cc toolchain_type is just an alias of the other toolchain_type we'd expect the outputs of the print statements to be the same, always.

And they are.

And yet this code in the definition of @rules_cc//cc:find_cc_toolchain.bzl:find_cc_toolchain:

    if hasattr(cc_common, "is_cc_toolchain_resolution_enabled_do_not_use") and cc_common.is_cc_toolchain_resolution_enabled_do_not_use(ctx = ctx):
        if "//cc:toolchain_type" in ctx.toolchains:
            return ctx.toolchains["//cc:toolchain_type"]
        fail("In order to use find_cc_toolchain, your rule has to depend on C++ toolchain. See find_cc_toolchain.bzl docs for details.")

complains that //cc:toolchain_type isn't in ctx.toolchains.

However if we change the label the above looks for to "@bazel_tools//tools/cpp:toolchain_type" or even "@rules_cc//cc:toolchain_type" it works.

So it's something about ToolchainContext's lookup mechanism not resolving labels that point to an alias within the current workspace.

wait but why did `@bazel_tools//tools/cpp:toolchain_utils.bzl`:`find_cpp_toolchain` work?

Good question! If we peek at its definition:

    if hasattr(cc_common, "is_cc_toolchain_resolution_enabled_do_not_use") and cc_common.is_cc_toolchain_resolution_enabled_do_not_use(ctx = ctx):
        if "@bazel_tools//tools/cpp:toolchain_type" in ctx.toolchains:
            return ctx.toolchains["@bazel_tools//tools/cpp:toolchain_type"]
        fail("In order to use find_cpp_toolchain, you must include the '@bazel_tools//tools/cpp:toolchain_type' in the toolchains argument to your rule.")

We can see that they used an absolute label (not relative to the workspace root — it has @bazel_tools// in the front instead of just //)!

what do

One possible fix is definitely to just change the rules_cc definition to use an absolute label.

But that doesn't really fix the actual problem.

hmmm

Some sleuthing reveals that the code that does the lookup lives in this function and lo and behold:

    if (!containsKey(semantics, key)) {
      // TODO(bazel-configurability): The list of available toolchain types is confusing in the
      // presence of aliases, since it only contains the actual label, not the alias passed to the
      // rule definition.
      throw Starlark.errorf(
          "In %s, toolchain type %s was requested but only types [%s] are configured",
          targetDescription(),
          toolchainTypeLabel,
          requiredToolchainTypes().stream()
              .map(ToolchainTypeInfo::typeLabel)
              .map(Label::toString)
              .collect(joining(", ")));
    }

A TODO about the exact issue we're running into!

Doing a git blame on that snippet leads us to this issue about toolchain type aliases and this PR that fixed that issue.

!

If we're going to try to fix this first we need a minimal test case (this repo goes and grabs LLVM binaries everytime we change anything which makes cycle times terrible):

Minimal Test Case

WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

RULES_CC_VER = "40548a2974f1aea06215272d9c2b47a14a24e556"
RULES_CC_SHA = "cb8ce8a25464b2a8536450971ad1b45ee309491c1f5e052a611b9e249cfdd35d"

http_archive(
    name = "rules_cc",
    urls = ["https://github.com/bazelbuild/rules_cc/archive/{ver}.tar.gz".format(ver = RULES_CC_VER)],
    sha256 = RULES_CC_SHA,
    strip_prefix = "rules_cc-{ver}".format(ver = RULES_CC_VER),
)

foo.bzl:

load("@rules_cc//cc:find_cc_toolchain.bzl", "find_cc_toolchain")

def _foo(ctx):
    find_cc_toolchain(ctx)

foo = rule(
    implementation = _foo,
    attrs = {
        "_cc_toolchain": attr.label(default = Label("@rules_cc//cc:current_cc_toolchain")),
    },
    toolchains = [
        "@rules_cc//cc:toolchain_type",
    ],
)

BUILD:

load(":foo.bzl", "foo")

foo(name = "bar")

Running bazel build //... --incompatible_enable_cc_toolchain_resolution fails.

finding the problem

fin

Turns out there's already an issue in rules_cc about this! We should probably link to it in our PR.

Okay! Now that we've found a bug and have a fix we just need to:

sign the Google CLA
make an issue on bazelbuild/bazel
- give the minimal testcase (say that this is the reccomended setup from @rules_cc)
  - also say that this isn't a problem with @bazel_tools since it uses an absolute thing
- explain what's going on
- say there are two possible fixes
make an issue on bazelbuild/rules_cc linking to the other issue
- mention the other issue
- say it's the same as #74
- say that this is one of the two fixes
open the PR on bazelbuild/bazel
- say it's one of the solutions
- closes the issue
open the PR on bazelbuild/rules_cc
- say it's the other solution
- but that it's worth merging regardless in case ppl use weird rules_cc + bazel combos (i.e. new rules_cc, old bazel version)
- closes the issue + #74
sleep, having successfully shaved this particular yak

Edit: never filed the issue on made the PR but here's a bad fix.

rrbutani/arghhh.md

...