The basic idea behind RpmGrill project is to run automated tests on Koji packages as well as Fedora stable & testing repositories. RpmGrill analyzes _builds_, not individual rpms: this allows it to detect problems such as compiler warnings (from the build logs) or multilib inconsistencies (by matching 32- and 64-bit rpms) which cannot be identified by examining a single rpm file.
Some of the current tests in RpmGrill suite include:
- Specfile sanity checks, such as detecting non-UTF8 characters and reporting improper use of %macros in specfile changelogs;
- Multilib checks, alerting one of packages which yum/rpm will refuse to install together;
- ELF binary/DSO checks, such as making sure that daemons and shared library files are compiled with RELRO;
- Setuid/Setgid checks, in which a whitelist is hand-maintained with known setxid files and directories (which don't change often), and new appearances are reported;
- Build log tests, in which the Koji build log is scanned for make failures or known high-risk gcc warnings;
- Security-related content checks, such as warning about insecure content in polkit files and cross-referencing Ruby gem packages against a central database of CVEs.
- RpmGrill is designed with the goal of a high S/N ratio. Its developers actively monitor its results, and work to minimize false positives or even disable too-noisy tests.
- We do not force a specific GUI or CLI on the developers / testers / users.
- Instead, we expose a RESTful API around which various reporting / analysis tools can we written.
- We are planning to include useful tests (from other projects like rpmlint) soon enough.
(RpmGrill project ==> checksec-ng + RH's internal RpmGrill project)
- Standard Fedora 19 installation (with rpm, Perl and Python environments installed)
- >= 2 64-bit CPU cores
- >= 4 GB RAM
- 500 GB disk space
- read-only access to Fedora repositories (nfs / local)
- access to fedmsg
- access to / from the internet
- Ed Santiago
- Dhiru Kholia
- How often does the package need to be updated (for instance for security fixes)?
- Not so far
- Do we sometimes have to program our own code to fix things/auth to fas/etc?
- NO
- Is the upstream alive or dead?
- Alive
- Do we have a relationship with upstream where we can ask them to do things for us?
- Yes, we are upstream.
- Is the upstream branch going to be producing bugfixes (or at least, security fixes) to the service for a long time?
- Yes
- How easy is updating?
- "custom packaging" - a simple "git pull" should do it.
- we are trying to get RPMs working
- AutoQA authors themselves recommended not to use AutoQA for automated tests like checksec (for now).
- fedora-review is unsuitable since we need to operate on the same packages as we are shipping.
- The "checksec" component of RpmGrill has been successfully used to detect more than 100 violations of packaging guidelines (bugs have already been filed for these violations).
- "checksec" found multiple packages which had enabled "%global _hardened_build 1" macro but still were not being hardened (e.g trafficserver).
- "checksec" caught hardening bugs even after the maintainers claimed that they had fixed them ;)
I can see that setuid/setgid check is also presented in fedora-review.
Will fedora-review use rpmgrill as part of it?