Convert pylint output to simplified CodeClimate JSON suitable for Gitlab Code Quality checks

pylint_to_gitlab_codeclimate.py

import hashlib
import html
import json
import sys
from typing import Optional

from pylint.reporters import JSONReporter
from pylint.lint import Run

# map pylint categories to CodeClimate severity
PYLINT_CATEGORY_TO_SEVERITY = {
    "fatal": "blocker",
    "error": "critical",
    "warning": "major",
    "refactor": "minor",
    "convention": "minor",
}


class GitlabCodeClimateReporter(JSONReporter):
    """
    Custom pylint reporter to convert pylint messages into a reduced CodeClimate JSON report
    suitable for Gitlab's Code Quality. Only reports `description`, `fingerprint`, `severity`,
    `location`.

    See:
    1. https://docs.gitlab.com/ee/user/project/merge_requests/code_quality.html#implementing-a-custom-tool.
    2. https://github.com/PyCQA/pylint/blob/master/pylint/reporters/json_reporter.py
    """

    name = "gitlabcodeclimate"

    def display_messages(self, layout: Optional["Section"]) -> None:
        """
        Convert the pylint messages into a reduced CodeClimate report dictionary and dump as JSON.
        """
        codeclimate_dict = [
            {
                "description": html.escape(f"{msg.msg_id}: {msg.msg or ''}", quote=False),
                "severity": PYLINT_CATEGORY_TO_SEVERITY[msg.category],
                "location": {"path": msg.path, "lines": {"begin": msg.line}},
                "fingerprint": hashlib.sha1(
                    (msg.symbol + msg.path + str(msg.line)).encode()
                ).hexdigest(),
            }
            for msg in self.messages
        ]

        print(json.dumps(codeclimate_dict, indent=4), file=self.out)


if __name__ == "__main__":
    Run(sys.argv[1:], reporter=GitlabCodeClimateReporter())

Added severity field information with an arbitrary mapping of pylint categories to CodeClimate severity.

Hi, thanks for the code. I am trying it out. Python 3.7.7 and PyLint 2.11.1, I get the following errors

(dev_venv_py3) C:\Users\nyue\projects\gitlab_python_cicd>python pylint_to_gitlab_codeclimate.py src > gl-code-quality-report.json
Traceback (most recent call last):
  File "pylint_to_gitlab_codeclimate.py", line 67, in <module>
    Run(sys.argv[1:], reporter=GitlabCodeClimateReporter())
  File "C:\Users\nyue\projects\gitlab_python_cicd\dev_venv_py3\lib\site-packages\pylint\lint\run.py", line 375, in __init__
    score_value = linter.generate_reports()
  File "C:\Users\nyue\projects\gitlab_python_cicd\dev_venv_py3\lib\site-packages\pylint\lint\pylinter.py", line 1251, in generate_reports
    self.reporter.display_messages(report_nodes.Section())
  File "C:\Users\nyue\projects\gitlab_python_cicd\dev_venv_py3\lib\site-packages\pylint\reporters\json_reporter.py", line 47, in display_messages
    for msg in self.messages
  File "C:\Users\nyue\projects\gitlab_python_cicd\dev_venv_py3\lib\site-packages\pylint\reporters\json_reporter.py", line 47, in <listcomp>
    for msg in self.messages
AttributeError: 'dict' object has no attribute 'category'

@nyue I had the same issue with Python 3.6.8 and pylint 2.12.2. I found I had to override the handle_message method from the base class, since it was looking for the original JSON output fields as attributes, and not the new, differently-named ones in a dictionary. My version is here: https://gist.github.com/quanterium/53a277cab4136963374cf703dfe399bd

Personally I ended up using pylint-gitlab. It basically worked out of the box, for me. Their README explains usage well.

Even though it stated Python >=3.7, it seems to work fine in 3.6.8.

Thanks for the reports and suggestions @nyue, @quanterium and @ahogen. pylint-gitlab indeed looks like the easiest option but for a lightweight option I've updated the gist so that it now works with Python 3.10 and pylint==2.12.2. As @nyue noted it's easiest now to just override display_messages because this commit makes it now assume messages contains a list of Message.