Skip to content

Instantly share code, notes, and snippets.

@stsquad
Created October 26, 2018 15:39
Show Gist options
  • Save stsquad/c7bba591c949c1e4d1d3af1e75bcecc0 to your computer and use it in GitHub Desktop.
Save stsquad/c7bba591c949c1e4d1d3af1e75bcecc0 to your computer and use it in GitHub Desktop.

QEMU Status Report

Happy Birthday

QEMU is 15 this year

external/nicubunu-Chocolate-birthday-cake.svg

0.1 Release

From: https://www.winehq.org/pipermail/wine-devel/2003-March/015577.html

Hi,

The first release of the QEMU x86 emulator is available at
http://bellard.org/qemu/. QEMU achieves a fast user space Linux x86
emulation on x86 and PowerPC Linux hosts by using dynamic translation.
Its main goal is to be able to run the Wine project on non-x86
architectures.

Fabrice.

Early Milestones

  • April 2003 Savanah goes live
  • June 2003 0.4 initial system emulation
  • April 2005 0.7 cross-compile with linux-user
  • May 2006 0.8 ARM sysemu, MIPS linux-user

Collect Commit History

:header-args+: :var FIRST=”e63c3dc74bfb90e4522d075d0d5a7600c5145745” :header-args+: :var SINCE=”2003-02-01” :header-args+: :var UNTIL=”2018-11-01” :header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”qemu-commit-history.png”)

Commit History

generated/graphs/qemu-commit-history.png

In that time…

  • 1.3 million lines of code
  • 64k commits
  • 1448 contributors

Effort

  • 400 person-years
  • $54 million

This Cycle

:header-args+: :var SINCE=”2017-10-27” :header-args+: :var base=”1a26f46692320f1981c95967e0d5af4443b5f0b1”

generated/misc/prisoner-bike.png

Totals

use warnings;
use strict;

my $sha;
my ($sob, $tb, $rb);
my $unreviewed = 0;

open(my $fh, "-|", "git log --no-merges master $base..") or die "Cannot fork command: $!";
while (<$fh>) {
    chomp;
    if (/^commit (\w+)$/) {
        my $commit = substr $sha, 0, 10;
        if (defined($sha) && ($sob == 1 && !$rb && !$tb)) {
            $unreviewed++;
        }
        $sha = $1;
        $sob = $rb = $tb = 0;
    } elsif (/^\s*Signed-off-by/) {
        $sob++;
    } elsif (/^\s*Reviewed-by/) {
        $rb++;
    } elsif (/^\s*Tested-by/) {
        $tb++;
    }
}
close($fh);

print "$unreviewed";
  • 6908 Commits (8% un-reviewed)
  • 635 merges
  • 77 maintainers (of 123)
  • 351 authors

Collect Growth Areas

Growth Areas

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”dir-changes.png”)

./generated/graphs/dir-changes.png

Collect Bug Counts/CVEs/Coverity Data

from datetime import datetime
import sys
import os
import time

from launchpadlib.launchpad import Launchpad

lp = Launchpad.login_anonymously('kvmf18', 'production', version='devel')
pillar = lp.projects["qemu"]

new_bugs = pillar.searchTasks(created_since=SINCE)
bug_data = []
for bid in new_bugs:
    bug = bid.bug
    tags = " ".join(bug.tags)
    bug_data.append([bug.id, bid.status, bid.importance,
                     bug.users_affected_count, bug.message_count,
                     bug.heat, tags])
cat data/qemu-cves-2017to2018 | sed 's/\\t//g'

Launchpad Bugs

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”lpbugs.png”)

./generated/graphs/lpbugs.png

https://www.qemu.org/contribute/report-a-bug/

CVEs

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”cves.png”)

25 CVEs

generated/graphs/cves.png

Feature Releases

2.1113th December 2017
2.1224th April 2018
3.014th August 2018

Stable Releases

2.10.218th December 2017
2.11.114th February 2018
2.11.226th June 2018
2.12.12nd August 2018

Word Cloud

:header-args+: :var SINCE=”October 28th, 2017” :header-args+: :var cloud=presentation-filepath(file=”cloud-words.txt”)

generated/qemu-wordcloud.png

Features

Hardware assisted virtualization

  • HVF (MacOS)
  • WHPX (Windows)

Spectre

logos/spectre.svg

New Devices

  • RDMA (Remote RAM)
  • EDID
  • SMMUv3 (arm)

TCG Emulation

  • RISC-V
  • aarch64 BE (linux-user)

TCG Internals

  • MTTCG (hppa, i386/x86_64, s390x, xtensa)
    • multi-threaded translation
  • SoftFloat TNG
  • decodetree.py

System Models

  • RISC-V (SiFive U, Spike, Virt)
  • Raspberry Pi 3
  • i.MX6 UltraLite 14x14 EVK

Micro-controller models

  • BBC Microbit
  • ARM MPS2 & MPS2 TZ (ARM IOT Kit)
  • SiFive E

Outreach

  • BBC Microbit/cortex-m0
    • Julia Suvorova (Outreachy)
    • Steffen Görtz (GSoC)
  • Patchew REST API improvements
    • Shubham Jain
  • Qtest Driver Framework
    • Emanuele Esposito
  • Vulkan-ize VirGL
    • Nathan Gauër

Testing

  • –enable-sanitizers
  • Avocado Acceptance
  • TCG Tests
  • QEMU’s Synchronization Profiler (QSP)

Deprecated Features

  • GTK 2/SDL 1
  • Hosts
    • AIX
  • Backends
    • Itanium (ia64), PARisc (hppa)
  • ppcemb

Collect HW Stats

HW Dir Breakdown

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”hw-dir-changes.png”)

./generated/graphs/hw-dir-changes.png

Top Trumps

:header-args+: :var base=”1a26f46692320f1981c95967e0d5af4443b5f0b1”

generated/toptrump.jpg

Collect Author Profile

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”top-authors.png”)
git shortlog -s --no-merges $base.. | sort -rn

This takes a while to run over every author

import os
import re
from subprocess import check_output
from datetime import datetime, timezone
from dateutil import parser as date

now = datetime.now(timezone.utc)

for a in names:
    cmd = "git log --pretty=\"%%ad\" --author=\"%s\" | tail -n 1" % (a)
    first_time = check_output(cmd, shell=True).rstrip()
    first_date = date.parse(first_time)
    # convert to days from now
    xp = (now - first_date).days

    cmd = "git log --pretty=oneline --author=\"%s\" | wc -l" % (a)
    total_commits = int(check_output(cmd, shell=True).rstrip())

    print("%s,%d,%d" % (a, xp, total_commits))
<<setup-matplotlib>>

# Start with the top authors
top_n = 5
y_pos = np.arange(top_n)
plt.barh(y_pos, counts[:top_n], 0.75, color=bar_colors)
plt.xlim(left=0)
plt.xlabel(xlabel)
plt.yticks(y_pos, names[:top_n])

ax = plt.gca()
ax.invert_yaxis()

# add numbers to each bar
txt_off = counts[0] * 0.1
for i,v in enumerate(y_pos):
  ax.text(counts[i] - txt_off, i + 0.125, str(counts[i]), fontweight='bold')

print("Top 3 are: %s with %s" % (names[:3], counts[:3]))
plt.tight_layout()
plt.savefig(graph, transparent=True)
print("Saved graph as %s" % (graph))

Authors

./generated/graphs/top-authors.png

Author Breakdown

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”author-breakdown.png”)

./generated/graphs/author-breakdown.png

Collect Reviews Profile

Reviewers

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”top-reviewers.png”)

./generated/graphs/top-reviewers.png

Collect Testers Profile

Testers

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”top-testers.png”)

./generated/graphs/top-testers.png

Thanks

Extra Slides

Collect Maintainers

60kraxel
40pmaydell
32rth
30dgibson
29kevin
27ehabkost
26armbru
25stefanha
23ericb
22cohuck
18dgilbert
17mst
13vivier
13stefanberger
13mcayland
13cody
13berrange
12jasowang
12famz
10juanquintela
10jnsnow
9xtensa
9awilliam
8gkurz
6elmarco
5thibault
5riscv
5mdroth
4otubo
4maxreitz
4marcel
4edgar
3xanclic
3mjt
3amarkovic
3alistair
2sstabellini-http
2sstabellini
2huth-gitlab
2huth
2borntraeger

Maintainers

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”maintainer-history.png”)

./generated/graphs/maintainer-history.png

Top Level Domains

:header-args+: :var graph=presentation-filepath(subdir=”generated/graphs”, file=”top-tlds.png”)
git log --no-merges --pretty="%b" $base.. |
    grep -v "Message-Id" |
    grep -v "Cc: " |
    grep "<[^@>]*@[^@>]*>" |
    sed 's/.*@//' |
    sed 's/>//' |
    sort |
    uniq -c |
    sort -rn |
    head -n 20

TLD Breakdown

./generated/graphs/top-tlds.png

Ephemera

Helper Functions

Generated Paths

(concat default-directory subdir "/" file)

Setup Matplot Lib

import numpy as np
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from datetime import datetime

years = mdates.YearLocator()   # every year
twoyears = mdates.YearLocator(2)   # every other year
threeyears = mdates.YearLocator(3)   # every 3rd year
months = mdates.MonthLocator()  # every month
threemonths = mdates.MonthLocator()  # every 3rd month
yearsFmt = mdates.DateFormatter('%Y')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment