Skip to content

Instantly share code, notes, and snippets.

@pthom
Forked from nevkontakte/CLion_Ninja.md
Last active December 21, 2020 07:50
Show Gist options
  • Save pthom/bf66cc429ee0c7641674 to your computer and use it in GitHub Desktop.
Save pthom/bf66cc429ee0c7641674 to your computer and use it in GitHub Desktop.
Ninja support for CLion IDE

Ninja support for CLion IDE

This script enables Ninja-powered builds in CLion IDE by wrapping around CMake, which it uses. See my blog post for details.

Disclaimer

This script is provided AS IS with no guarantees given or responsibilities taken by the author. This script relies on undocumented features of CLion IDE and may lead to instability of build and/or IDE. Use it on your own risk under WTFPL terms.

Getting started

Supported OS: I've tested this scipt under Ubuntu and python 2.7. I suppose it should work on Mac and Windows too but I've never tested it.

  1. Make sure you have python 2.7 installed.
  2. Download cmake_ninja_wrapper.py and give it execution permission.
  3. Edit REAL_CMAKE variable at the beginning of the script to point at CLion's bundled CMake binary (recommended) or at system CMake.
  4. In CLion, go to Settings → Build, Execution, Deployment → Toolchains
  5. Under "CMake executable" select cmake_ninja_wrapper.py script.

Make sure that CLion successfully detects make program (it should be still /usr/bin/make) and C/C++ compiler. At this point you are done. Go, reload you CMake project and try to build it.

Troubleshooting

In case of any troubles try setting TRACING variable at the beginning of the file to True and re-parsing CMake project in IDE. After that look at /tmp/cmake_wrapper.log for any insights :-)

#!/usr/bin/python
import sys
import os
import subprocess
import shutil
import re
import select
# ---------------------- Configuration section ------------------------------
REAL_CMAKE = "/usr/local/bin/cmake"
TRACING = True
# --------------------------- Code section ----------------------------------
def trace(message, argv = []):
if not TRACING:
return
with open('/tmp/cmake_wrapper.log', 'a') as log:
if not argv == []:
log.write("\n\n")
log.write(message)
if not argv == []:
argv = '"%s"' % ('" "'.join(argv))
log.write("\n\n\t%s\n\tat: %s\n" % (argv, os.getcwd()))
def call_cmake(passing_args):
"""Call real cmake as a subprocess passing it's output both to stdout and trace file."""
passing_args = [REAL_CMAKE] + passing_args
trace("Calling real cmake:", passing_args)
proc = subprocess.Popen(passing_args, stdout=subprocess.PIPE, stderr=subprocess.PIPE, bufsize=0)
while True:
reads = [proc.stdout.fileno(), proc.stderr.fileno()]
ret = select.select(reads, [], [])
for fd in ret[0]:
if fd == proc.stdout.fileno():
line = proc.stdout.readline()
sys.stdout.write(line)
sys.stdout.flush()
trace(line)
if fd == proc.stderr.fileno():
line = proc.stderr.readline()
sys.stderr.write(line)
sys.stderr.flush()
trace(line)
if proc.poll() != None:
break
for line in proc.stdout:
sys.stdout.write(line)
trace(line)
for line in proc.stderr:
sys.stderr.write(line)
trace(line)
return proc.poll()
def is_real_project():
"""Detect if called inside clion private directory."""
cwd = os.getcwd()
result = "clion" in cwd and "cmake" in cwd.lower() and "generated" in cwd
trace("is_real_project cwd=" + cwd + " result=" + str(result))
return True
#return result
class CMakeCache(object):
"""CMake cache management utility"""
def __init__(self, path):
super(CMakeCache, self).__init__()
self.path = path
def alter(self, variable, value):
"""
Change a variable value in CMake cache.
TODO: Add variable if it doesn't already exist
"""
if not os.path.isfile(self.path):
return
with open(self.path, 'r') as cache_file:
cache_data = cache_file.read()
pattern = '%s=.*' % re.escape(variable)
replacement = '%s=%s' % (variable, value)
cache_data = re.sub(pattern, replacement, cache_data)
with open(self.path, 'w') as cache_file:
cache_file.write(cache_data)
def ninjafy(self):
self.alter('CMAKE_GENERATOR:INTERNAL', 'Ninja')
self.alter('CMAKE_MAKE_PROGRAM:FILEPATH', '/usr/local/bin/ninja')
def makefy(self):
self.alter('CMAKE_GENERATOR:INTERNAL', 'Unix Makefiles')
self.alter('CMAKE_MAKE_PROGRAM:FILEPATH', '/usr/bin/make')
def ninjafy_argv(original):
"""Replace Unix Makefiles generator with Ninja"""
found = False
foundG = False
processed = []
next_g = False
for a in original:
if a == '-G':
next_g = True
foundG = True
elif next_g and 'Unix Makefiles' in a:
#a = a.replace('CodeBlocks - Unix Makefiles', 'Ninja')
a = "Ninja"
next_g = False
found = True
processed.append(a)
trace("ninjafy_argv => found = " + str(found) + " FoundG=" + str(foundG))
return processed
trace('Originally called:', sys.argv)
# Enable wrapping logic only when called inside clion private directory.
if not is_real_project():
sys.exit(call_cmake(sys.argv[1:]))
# Check if generator argument was specified
if '-G' in sys.argv:
# Generate Makefile artifacts required by CLion
cache = CMakeCache('CMakeCache.txt')
cache.makefy()
exit_code = call_cmake(sys.argv[1:])
if exit_code != 0:
sys.exit(exit_code)
# Generate Ninja artifacts for actual build
passing_args = ninjafy_argv(sys.argv[1:])
cache.ninjafy()
sys.exit(call_cmake(passing_args))
else:
sys.exit(call_cmake(sys.argv[1:]))
@MaxThroughput
Copy link

Works on Mac! Shortened my compile times by about 60% versus the standard cmake implementation in CLion. Thanks!!

@pthom
Copy link
Author

pthom commented Oct 10, 2018

Happy to help :-)

@SebastianMosiej
Copy link

HI, on windows it fails
Python 3.7. I needed to wrap this script in bat so CLion would finally launch it.
After that it fails on select due to - possibly - blocking request or too many request (or something like that)

C:\Users\sebastian_lenovo\AppData\Local\Temp\cmake_check_environment\_build3370962343596620944>python c:\APPS\cmake_ninja_wrapper.py -G "NMake Makefiles" C:\Users\sebastian_lenovo\AppData\Local\Temp\cmake_check_environment 
Traceback (most recent call last):
  File "c:\APPS\cmake_ninja_wrapper.py", line 140, in <module>
    sys.exit(call_cmake(sys.argv[1:]))
  File "c:\APPS\cmake_ninja_wrapper.py", line 47, in call_cmake
    ret = select.select(reads, [], [])
OSError: [WinError 10093] Albo aplikacja nie wywołała funkcji WSAStartup, albo nie powiodło się wykonanie funkcji WSAStartup

@long1eu
Copy link

long1eu commented Jul 6, 2019

I doesn't work for me on MacOS.

CLion 2019.1.4
Build #CL-191.7479.33, built on May 30, 2019
Licensed to Razvan Lung
Subscription is active until February 21, 2020
For educational use only.
JRE: 1.8.0_202-release-1483-b53 x86_64
JVM: OpenJDK 64-Bit Server VM by JetBrains s.r.o
macOS 10.14.5

Screenshot 2019-07-06 at 15 47 31

Screenshot 2019-07-06 at 16 23 16

@pthom
Copy link
Author

pthom commented Jul 6, 2019

May be try to look at the trace file : see line 21
https://gist.github.com/pthom/bf66cc429ee0c7641674#file-cmake_ninja_wrapper-py-L21
may be look these traces will help you

@long1eu
Copy link

long1eu commented Jul 6, 2019

I've invalidate chaces and restarted. An this is the content:



Originally called:

	"/Users/long1eu/SDK/engine/src/flutter/cmake_ninja_wrapper.py" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin
is_real_project cwd=/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin result=False

Calling real cmake:

	"/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin/cmake/mac/bin/cmake" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin
cmake version 3.14.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).


Originally called:

	"/Users/long1eu/SDK/engine/src/flutter/cmake_ninja_wrapper.py" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin
is_real_project cwd=/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin result=False

Calling real cmake:

	"/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin/cmake/mac/bin/cmake" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin
cmake version 3.14.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).


Originally called:

	"/Users/long1eu/SDK/engine/src/flutter/cmake_ninja_wrapper.py" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin
is_real_project cwd=/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin result=False

Calling real cmake:

	"/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin/cmake/mac/bin/cmake" "-version"
	at: /Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin


Originally called:

	"/Users/long1eu/SDK/engine/src/flutter/cmake_ninja_wrapper.py" "-G" "Unix Makefiles" "/private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment"
	at: /private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306
is_real_project cwd=/private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306 result=False

Calling real cmake:

	"/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin/cmake/mac/bin/cmake" "-G" "Unix Makefiles" "/private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment"
	at: /private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306
cmake version 3.14.3

CMake suite maintained and supported by Kitware (kitware.com/cmake).
-- The C compiler identification is AppleClang 10.0.1.10010046
-- The CXX compiler identification is AppleClang 10.0.1.10010046
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc
-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306
ninjafy_argv => found = True FoundG=True

Calling real cmake:

	"/Users/long1eu/Library/Application Support/JetBrains/Toolbox/apps/CLion/ch-0/191.7479.33/CLion.app/Contents/bin/cmake/mac/bin/cmake" "-G" "Ninja" "/private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment"
	at: /private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306
-- Configuring done
-- Generating done
-- Build files have been written to: /private/var/folders/dv/0_21mfps09b6s257_vm_gh140000gn/T/cmake_check_environment/_build5066260725996948306

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment