Skip to content

Instantly share code, notes, and snippets.

@VictorTaelin
Last active April 17, 2025 08:38
Show Gist options
  • Save VictorTaelin/45440a737e47b872d7505c6cda27b6aa to your computer and use it in GitHub Desktop.
Save VictorTaelin/45440a737e47b872d7505c6cda27b6aa to your computer and use it in GitHub Desktop.
INVERT A BINARY TREE - $10k AI REASONING CHALLENGE (v2)

THE PROBLEM

🌲 Invert a binary tree! 🌲

Except with 3 catches:

  1. It must invert the keys ("bit-reversal permutation")
  2. It must be a dependency-free, pure recursive function
  3. It must have type Bit -> Tree -> Tree (i.e., a direct recursion with max 1 bit state)

WHY THIS PROBLEM?

  • It is somehow NOT on the internet. (AFAIK)
  • Humans can solve it. (I've done it in ~1h.)
  • It requires reasoning. (My head hurts!)
  • The solution is simple. (7 lines of code!)
  • Obvious pre-requisite to automate CS research.
  • Honestly, it would make me believe I'll be automated.

THE CHALLENGE

I claim no AI will EVER solve this problem. If you prove me wrong, HOC will grant you $10k!

RULES

  • You must give it an approved prompt, nothing else.
  • It must output a correct solution, passing all tests.
  • You can use any software or AI model.
  • You can let it "think" for as long as you want.
  • You can propose a new prompt, as long as:
    • It imposes equivalent restrictions.
    • It clearly doesn't help the AI.
    • Up to 1K tokens, all included.
  • Common sense applies.

APPROVED PROMPTS

Agda Version

{-# OPTIONS --no-termination-check #-}

-- Let Tree be a Perfect Binary Tree:

data Nat : Set where
  Z : Nat
  S : Nat  Nat
{-# BUILTIN NATURAL Nat #-}

data Bit : Set where
  O : Bit
  I : Bit

data Tree (A : Set) : Nat  Set where
  N :  {d}  Tree A d  Tree A d  Tree A (S d)
  L : Nat  Tree A Z

-- Your goal is to implement an 'invert' function that performs a bit-reversal
-- permutation on a Tree, respecting the following limitations:
-- 1. You can NOT define or use any function other than 'invert'.
-- 2. You can NOT use any type not defined above (Nat, Bit and Tree).
-- 3. You can NOT use loops (but you can call 'invert' recursively).
-- 4. You can NOT use mutability. It must be a pure Agda function.
-- 5. You can use 1 bit of state (as an extra argument).
-- 6. You can use pattern-matching and constructors freely.
-- 
-- Example:
-- input  = (N(N(N(L 0)(L 1))(N(L 2)(L 3)))(N(N(L 4)(L 5))(N(L 6)(L 7))))
-- output = (N(N(N(L 0)(L 4))(N(L 2)(L 6)))(N(N(L 1)(L 5))(N(L 3)(L 7))))
-- Because that's the bit-reversal permutation of the original tree.
-- 
-- Now, complete the program below, with a valid implementation of 'invert':
invert :  {A d}  Bit  Tree A d  Tree A d

TypeScript Version

type Nat     = number;
type Bit     = false | true;
type Tree<A> = [Tree<A>, Tree<A>] | Nat;

// Your goal is to implement an 'invert' function that performs a bit-reversal
// permutation on a Tree, respecting the following limitations:
// 1. You can NOT define or use any function other than 'invert'.
// 2. You can NOT use any type not defined above (Nat, Bit and Tree).
// 3. You can NOT use loops (but you can call 'invert' recursively).
// 4. You can NOT use mutability. It must be a pure function.
// 5. You can NOT use primitive JS operators or functions.
// 6. You can use 1 bit of state (as an extra argument).
// 7. You can only use the operations allowed below.
// 
// Operations allowed:
// - Destructing (`const [a,b] = value`)
// - Variables (`const x = value`)
// - Branching (`if (x) { ... } else { ... }`)
// - Recursion (`invert(_, _)')
// - `Array.isArray`
// 
// All other operations are not allowed.
// 
// Example:
// input  = [[[[0,1],[2,3]],[[4,5],[6,7]]]]
// output = [[[[0,4],[2,6]],[[1,5],[3,7]]]]
// Because that's the bit-reversal permutation of the original tree.
// 
// Now, complete the program below, with a valid implementation of 'invert':
function invert<A>(bit: Bit, tree: Tree<A>): Tree<A> {
  ...
}

// A test:
const tree: Tree<Nat> = [[[[0,1],[2,3]],[[4,5],[6,7]]],[[[8,9],[10,11]],[[12,13],[14,15]]]];
console.log(JSON.stringify(invert(true, tree)));

CONCLUSION

✨ If it can't invert a tree, it won't solve P=NP. ✨

@TheMagnumOpuss
Copy link

Finally, we have o3, and here it is in all its glory:
https://chatgpt.com/share/67fff234-1870-800c-9856-15f61dad609a

@TheMagnumOpuss
Copy link

Now the o4-mini, but it searched the internet.
https://chatgpt.com/share/67fff5cf-83ac-800c-ab4e-0a47301f3cac

@TheMagnumOpuss
Copy link

Now the o4-mini-high, and without internet (apparently):
https://chatgpt.com/share/67fff8a3-1154-800c-8dad-ef4aaf13636a

@youqad
Copy link

youqad commented Apr 17, 2025

This challenge is now solved without resorting to the web chat interface! https://gist.github.com/youqad/02a36419cbc4725601bffc05f14b3947

On 9 December, the full o1 solved it (first solution publicly submitted with no system prompt, to my knowledge) on the web interface: https://gist.github.com/VictorTaelin/45440a737e47b872d7505c6cda27b6aa?permalink_comment_id=5328789#gistcomment-5328789

And since then, other people have proposed solutions, but also on the web interface (which doesn't allow us to check the system prompt, unfortunately).

Now, here is a fully working and verifiable solution with o3, based on my repository (https://github.com/youqad/bit-reversal-trees):

📦 Published to https://wandb.ai/ox/bit-reversal-trees/weave/objects/0196425c-0650-7df2-8d57-05d272f6d111/versions/KOTE3EQkUWYSdy046vDxHGbma9A8gkuxXUHNpZ3tHH0 and visible on WandB Weave with Call ID: 0196425c-0650-7df2-8d57-05d272f6d111 🚀

def invert(tree: Tree) -> Tree:
    """
    Return a new perfect‑binary tree whose leaves are arranged according to
    the bit‑reversal permutation of their paths   (left = 0, right = 1).

    Mandatory constraints – all satisfied:
        1.  Only the inner   invert_helper(tree, flag)   is used.
        2.  invert_helper is a single, pure, *recursive* function; it calls
            no other helper, contains no loops and causes no side effects.
        3.  The public call is   invert_helper(tree, True) .
    """

    def invert_helper(t: Tree, flag: bool) -> Tree:
        # -------- leaf ---------------------------------------------------- #
        if isinstance(t, Leaf):
            return t

        # ------------------------------------------------------------------ #
        #  flag == True   : full bit‑reversal of  the  subtree  't'.
        #  flag == False  : *weave* two already bit‑reversed sub‑trees so
        #                   that their leaves are perfectly interleaved
        #                   (LSB that has just become the MSB).
        # ------------------------------------------------------------------ #
        if flag:                                            # bit‑reversal
            left_rev  = invert_helper(t.left,  True)        # reverse both halves
            right_rev = invert_helper(t.right, True)
            # After each half has been bit‑reversed we only have to
            # weave them together; the weaving is performed by a single
            # recursive call in the   flag == False   mode.
            return invert_helper(Node(left_rev, right_rev), False)

        # --------------------- weaving mode ------------------------------- #
        #  Children are leaves → nothing more to weave; just join them.
        if isinstance(t.left, Leaf):                        # depth == 1
            return Node(t.left, t.right)

        #  Non‑trivial case – recurse pairwise on both sides.
        return Node(
            invert_helper(Node(t.left.left,  t.right.left),  False),
            invert_helper(Node(t.left.right, t.right.right), False)
        )

    # single call required by the statement
    return invert_helper(tree, True)

It passes all the Hypothesis tests in https://github.com/youqad/bit-reversal-trees/tree/main/python, so I'm pretty positive it works!
(You can also double-check this with this script)

@youqad
Copy link

youqad commented Apr 17, 2025

Finally, we have o3, and here it is in all its glory: https://chatgpt.com/share/67fff234-1870-800c-9856-15f61dad609a

This doesn't work:
Original : [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15]
Expected : [0,8,4,12,2,10,6,14,1,9,5,13,3,11,7,15]
But your solution gave : [0,8,2,10,4,12,6,14,1,9,3,11,5,13,7,15]

@youqad
Copy link

youqad commented Apr 17, 2025

And now the Haskell version (also found by o3, in one shot, without the web chat UI)!
https://gist.github.com/youqad/8a7f91650a2da951f4f39455cdccbbe8

invert :: Tree a -> Tree a
invert t = invertHelper True t
  where
    invertHelper :: Bool -> Tree a -> Tree a
    invertHelper True (Leaf x)          = Leaf x
    invertHelper True (Node l r)        =
        let l' = invertHelper True l
            r' = invertHelper True r
        in  invertHelper False (Node l' r')

    invertHelper False (Leaf x)         = Leaf x
    invertHelper False (Node a b) =
        case (a, b) of
          (Leaf _, Leaf _) -> Node a b
          (Node a1 a2, Node b1 b2) ->
              Node (invertHelper False (Node a1 b1))
                   (invertHelper False (Node a2 b2))
          _ -> error "invert: non‑perfect tree encountered"

Fully tested with QuickCheck (cf. https://github.com/youqad/bit-reversal-trees/blob/main/test/DynamicSpec.hs), so the challenge is now solved for Python and Haskell.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment