majek · May 17, 2012 20:39 · MsBradley · Mar 31, 2015
diff --git a/p0f-ssl-2012-05-17.diff b/p0f-ssl-2012-05-17.diff
 diff --git a/build.sh b/build.sh
 index 112e404..2974fdc 100755
 --- a/build.sh
 +++ b/build.sh
 @@ -31,7 +31,7 @@ else
   USE_LIBS="-lpcap $LIBS"
 fi
 
 -OBJFILES="api.c process.c fp_tcp.c fp_mtu.c fp_http.c readfp.c"
 +OBJFILES="api.c process.c fp_tcp.c fp_mtu.c fp_http.c fp_ssl.c readfp.c"
 
 echo "Welcome to the build script for $PROGNAME $VERSION!"
 echo "Copyright (C) 2012 by Michal Zalewski <[email protected]>"
 diff --git a/docs/README b/docs/README
 index ca2d76f..bed471b 100644
 --- a/docs/README
 +++ b/docs/README
 @@ -101,9 +101,9 @@ implementation quirks (e.g. non-zero values in "must be zero" fields).
 The metrics used for application-level traffic vary from one module to another;
 where possible, the tool relies on signals such as the ordering or syntax of
 HTTP headers or SMTP commands, rather than any declarative statements such as
 -User-Agent. Application-level fingerprinting modules currently support HTTP.
 -Before the tool leaves "beta", I want to add SMTP and FTP. Other protocols,
 -such as FTP, POP3, IMAP, SSH, and SSL, may follow.
 +User-Agent. Application-level fingerprinting modules currently support HTTP
 +and SSL. Before the tool leaves "beta", I want to add SMTP and FTP. Other
 +protocols, such as FTP, POP3, IMAP and SSH may follow.
 
 The list of all the measured parameters is reviewed in section 5 later on.
 Some of the analysis also happens on a higher level: inconsistencies in the
 @@ -713,6 +713,63 @@ responses to OPTIONS or POST. That said, it does not seem to be worth the
 effort: the protocol is so verbose, and implemented so arbitrarily, that we are
 getting more than enough information just with a simple GET / HEAD fingerprint.
 
 +== SSL signatures ==
 +
 +P0f is capable of fingerprinting SSL / TLS requests. This can be used
 +to verify if a browser using https is legitimate, and in some cases
 +can leak some details about client operating system.
 +
 +SSL signatures have the following layout:
 +
 +sig = sslver:ciphers:extensions:sslflags
 +
 +  sslver     - two dot-separated integers representing SSL request version,
 +               may be something like 2.0 or 3.1.
 +
 +               NEW SIGNATURES: Copy literally.
 +
 +  ciphers    - comma-separated list of hex values representing SSL
 +               ciphers supported by the client.
 +
 +               NEW SIGNATURES: Review the list and feel free to
 +               substitute parts that bring no information with the
 +               match all sign '*'.  For efficiency avoid using star in
 +               the beginning of the signature.
 +
 +               You may also use '?' to make particular value optional.
 +
 +  extensions - comma-separated list of hex values representing
 +               SSL extensions.
 +
 +               NEW SIGNATURES: Same as for ciphers.
 +
 +  sslflags   - comma-separated list of SSL flags:
 +
 +               v2    - client used an older SSLv2 handshake frame, instead
 +                       of more recent SSLv3 / TLSv1
 +
 +               Flags specific to SSLv3 handshake:
 +
 +               ver   - requested SSL protocol was different on a protocol
 +                       (request) than on a record layer
 +               stime - gmt_unix_time field has unusually was small value,
 +                       most likely time since boot
 +               rtime - gmt_unix_time field has value that is very far off
 +                       local time, most likely it is random
 +               compr - client supports deflate compression
 +
 +               NEW SIGNATURES: Copy literally.
 +
 +Any of these sections except for the 'sslver' may be blank.
 +
 +For a fingerprint to match signature an exact fit must happen - sslver
 +and flags must be exactly equal, ciphers and extensions must match
 +with respect to '*' and '?' signs.
 +
 +Note that the fingerprint is matched against signatures in order. You
 +may take advantage of this and keep broader signatures closer to the
 +bottom of the list.
 +
 == SMTP signatures ==
 
    *** NOT IMPLEMENTED YET ***
 @@ -772,7 +829,7 @@ The following code is also issued by the HTTP module:
   app_srv_lb   - server application signatures change, suggesting load
                  balancing.
 
 -  date         - server-advertised date changes inconsistently.
 +  date         - advertised date changes inconsistently.
 
 Different reasons have different weights, balanced to keep p0f very sensitive
 even to very homogenous environments behind NAT. If you end up seeing false
 diff --git a/docs/ssl-notes.txt b/docs/ssl-notes.txt
 new file mode 100644
 index 0000000..1deeaba
 --- /dev/null
 +++ b/docs/ssl-notes.txt
 @@ -0,0 +1,108 @@
 +
 +SSL fingerprinting
 +==================
 +
 +Not many people realise that the unencrypted SSL / TLS handshake
 +(ClientHello message) reveals some fingerprintable details about the
 +client.
 +
 +For a given application using SSL it is possible to fingerprint
 +underlying SSL library. In many cases it is even possible to pinpoint
 +specific application version.
 +
 +This especially true in the browser world, where the SSL libraries are
 +most often bundled with a release. Additionally, on the web a lot of
 +innovation happens on SSL layer, like SPDY, ECC or 'renegotiation
 +info' extension.
 +
 +Additionally SSL traffic is unharmed by proxies or routers, we can
 +have high confidentiality that the SSL fingerprint does identify a
 +genuine application. This is useful for detecting NAT's.
 +
 +Although initial frames sent from both SSL client and server are
 +similar, only frame sent by the client 'ClientHello', can be passively
 +fingerprinted. The server frame 'ServerHello' doesn't have enough
 +information. If you wish to fingerprint SSL servers an active
 +fingerprinting tool may be more suitable.
 +
 +
 +Fingerprinting SSL is not a new idea, initial work was done in mid
 +2009 by Ivan Ristic:
 +
 + * https://www.ssllabs.com/projects/client-fingerprinting/
 +
 +He was able to collect few dozen interesting signatures:
 +
 + * http://blog.ivanristic.com/2009/07/examples-of-the-information-collected-from-ssl-handshakes.html
 +
 +Unfortunately on his works he seem to have totally ignored the SSL
 +extensions list, which add valuable information to a
 +fingerprint. Especially the ordering of extensions has high value,
 +similarly to the TCP options ordering for TCP/IP stack fingerprinting.
 +
 +
 +SSL handshake (ClientHello message) contains following fields:
 +
 + * SSL record version - browsers usually use version 3, Ivan Ristic
 +   research shows that web crawlers (including wget) still use 2.
 +   We set a flag:
 +     'v2'  - if request is SSL version 2
 +
 + * Requested SSL protocol version - most likely 3.1. Less likely values
 +   are 3.0 and 3.2. SSL crawlers trying invalid versions like 4.2 were
 +   seen in the wild. ('request_version')
 +   We set a flag:
 +     'ver' - if, for SSL version 3, the version on a 'request' layer
 +             is different than on 'record' layer. This behaviour was seen
 +             in Opera.
 +
 + * gmt_unix_time - a timestamp from the client. In the code we may
 +   set one of two flags derived from this value:
 +     'stime' - when the timestamp is unnaturally small (less than one year 
 +               since the epoch). This is behaviour was seen in old Firefox
 +               releases, when gmt_unix_time is set to seconds since boot.
 +     'rtime' - when the timestamp is far off the current value (delta
 +               is bigger than 5 years). Most often this means that gmt_unix_time
 +               field is set to a random value. This was seen in some
 +               SSL crawlers.
 +
 + * 28 random bytes - not very useful. In a debug build a warning will
 +   be printed if the values don't look very randomly.
 +
 + * session_id - a client may choose to resume previous SSL session. Ignored.
 +
 + * cipher_suites - a list of supported encryption algorithms ('ciphers').
 +   For the meaning of the numbers refer to:
 +       http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-3
 +
 + * compression_methods - a list of supported compression methods. There
 +   is only one valid compression method available, so we set a flag:
 +       'compr' - if compression is enabled 
 +
 + * extensions - a list of SSL extensions. Second, after cipher_suites,
 +   major source of data. For exact meaning refer to:
 +        http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xml
 +
 +
 +It's worth noting that SSL extensions contain not only a list of
 +supported extensions but also optional payload for every extension.
 +We ignore the payload as the experiments shown that it isn't an
 +interesting source of data - we can get the same entropy by just
 +looking at the ordering of SSL extensions.
 +
 +
 +A special explanation should be given for '0' - 'server_name' SSL
 +extension. This extension contains a host name of a remote host and is
 +often described as Server Name Indication TLS extension from RFC 3546.
 +
 +Unfortunately RFC forbids passing raw ip addresses in this
 +option. That means this option must not be present if you enter a
 +website using an ip address. For example, browser will valid extension
 +'0' with hostname if you access 'https://localhost' but will not for
 +'https://127.0.0.1'.
 +
 +P0f assumes that this extension is optional and will always prepend it
 +with an optional sign '?'. This is quite optimistic and may not always
 +be a good idea - it is possible the fingerprint when accessing
 +'localhost' is completely different from '127.0.0.1'. Using '?' before
 +this extension will only clutter a signature in such case.
 diff --git a/fp_ssl.c b/fp_ssl.c
 new file mode 100644
 index 0000000..c77dca0
 --- /dev/null
 +++ b/fp_ssl.c
 @@ -0,0 +1,930 @@
 +/* -*-mode:c; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
 +/*
 +  p0f - SSL fingerprinting
 +  -------------------------
 +
 +  Copyright (C) 2012 by Marek Majkowski <[email protected]>
 +
 +  Distributed under the terms and conditions of GNU LGPL.
 +
 +*/
 +
 +#define _FROM_FP_SSL
 +#define _GNU_SOURCE
 +
 +#include <stdio.h>
 +#include <stdlib.h>
 +#include <unistd.h>
 +#include <ctype.h>
 +#include <time.h>
 +
 +#include <netinet/in.h>
 +#include <sys/types.h>
 +
 +#include "types.h"
 +#include "config.h"
 +#include "debug.h"
 +#include "alloc-inl.h"
 +#include "process.h"
 +#include "readfp.h"
 +#include "p0f.h"
 +#include "tcp.h"
 +#include "hash.h"
 +
 +#include "fp_ssl.h"
 +
 +/* Flags for SSL signaturs */
 +struct flag {
 +  char* name;
 +  int name_len;
 +  u32 value;
 +};
 +
 +struct flag flags[] = {{"compr", 5, SSL_FLAG_COMPR},
 +                       {"v2",    2, SSL_FLAG_V2},
 +                       {"ver",   3, SSL_FLAG_VER},
 +                       {"rtime", 5, SSL_FLAG_RTIME},
 +                       {"stime", 5, SSL_FLAG_STIME},
 +                       {NULL, 0, 0}};
 +
 +
 +/* Signatures are stored as flat list. Matching is fast: ssl version
 +   and flags must match exactly, matching ciphers and extensions
 +   usually require looking only at a first few bytes of the
 +   signature. Assuming the signature doesn't start with a star. */
 +
 +static struct ssl_sig_record* signatures;
 +static u32 signatures_cnt;
 +
 +
 +/* Decode a string of comma separated hex numbers into an annotated
 +   u32 array. Exit with success on '\0' or ':'. */
 +
 +static u32* decode_hex_string(const u8** val_ptr, u32 line_no) {
 +
 +  const u8* val = *val_ptr;
 +
 +  u32 rec[SSL_MAX_CIPHERS];
 +  u8 p = 0;
 +
 +  while (p < SSL_MAX_CIPHERS) {
 +
 +    u32 optional = 0;
 +    const u8* prev_val;
 +    u32* ret;
 +
 +    /* State #1: expecting value */
 +
 +    switch (*val) {
 +
 +    case '*':
 +      rec[p++] = MATCH_ANY;
 +      val ++;
 +      break;
 +
 +    case '?':
 +      optional = MATCH_MAYBE;
 +      val ++;
 +      /* Must be a hex digit after question mark */
 +    case 'a' ... 'f':
 +    case '0' ... '9':
 +      prev_val = val;
 +      rec[p++] = (strtol((char*)val, (char**)&val, 16) & 0xFFFFFF) | optional;
 +      if (val == prev_val) return NULL;
 +      break;
 +
 +    default:
 +      /* Support empty list - jump to second state. */
 +      if (p == 0)
 +        break;
 +
 +      return NULL;
 +
 +    }
 +
 +    /* State #2: comma, expecting '\0' or ':' */
 +
 +    switch (*val) {
 +
 +    case ':':
 +    case '\0':
 +      *val_ptr = val;
 +      ret = DFL_ck_alloc((p + 1) * sizeof(u32));
 +      memcpy(ret, rec, p * sizeof(u32));
 +      ret[p] = END_MARKER;
 +      return ret;
 +
 +    case ',':
 +      val ++;
 +      break;
 +
 +    default:
 +      return NULL;
 +
 +    }
 +
 +  }
 +
 +  FATAL("Too many ciphers or extensions in line %u.", line_no);
 +
 +}
 +
 +
 +/* Is u32 list of ciphers/extensions matching the signature?
 +   first argument is record (star and question mark allowed),
 +   second one is an exact signature. */
 +
 +static int match_sigs(u32* rec, u32* sig) {
 +
 +  u8 match_any = 0;
 +  u32* tmp_sig;
 +
 +  /* Iterate over record. */
 +
 +  for (; *rec != END_MARKER && *sig != END_MARKER; rec++) {
 +
 +    /* 1. Exact match, move on */
 +    if ((*rec & ~MATCH_MAYBE) == *sig) {
 +      match_any = 0; sig++;
 +      continue;
 +    }
 +
 +    /* 2. Star, may match anything */
 +    if (*rec == MATCH_ANY) {
 +      match_any = 1;
 +      continue;
 +    }
 +
 +    /* 3. Optional match, not yet fulfilled */
 +    if (*rec & MATCH_MAYBE) {
 +      if (match_any) {
 +        /* Look forward for the value (aka: greedy match). */
 +        for (tmp_sig = sig; *tmp_sig != END_MARKER; tmp_sig++) {
 +          if ((*rec & ~MATCH_MAYBE) == *tmp_sig) {
 +            /* Got it. */
 +            match_any = 0; sig = tmp_sig + 1;
 +            break;
 +          }
 +        }
 +      }
 +      /* Loop succeeded or optional match failed, whatever, go on. */
 +      continue;
 +    }
 +
 +    /* 4. Looking for an exact match after MATCH_ANY */
 +    if (match_any) {
 +      for (; *sig != END_MARKER; sig++) {
 +        if (*rec == *sig) {
 +          sig ++;
 +          break;
 +        }
 +      }
 +      /* Sig is next char after match or END_MARKER */
 +      match_any = 0;
 +      continue;
 +    }
 +
 +    /* 5. Nope, not matched. */
 +    return 1;
 +
 +  }
 +
 +  /* Right, we're after the loop, either rec or sig are set to END_MARKER */
 +
 +  /* Step 1. Roll rec while it has conditional matches.
 +             Sig is END_MARKER if rec is not done.  */
 +  for (;(*rec & MATCH_MAYBE) || *rec == MATCH_ANY; rec ++) {};
 +
 +  /* Step 2. Both finished - hurray. */
 +  if (*rec == END_MARKER && *sig == END_MARKER)
 +    return 0;
 +
 +  /* Step 3. Rec is done and we're in MATCH_ANY mode - hurray. */
 +  if (*rec == END_MARKER && match_any)
 +    return 0;
 +
 +  /* Step 4. Nope. */
 +  return 1;
 +
 +}
 +
 +
 +static void ssl_find_match(struct ssl_sig* ts) {
 +
 +  u32 i;
 +
 +  for (i = 0; i < signatures_cnt; i++) {
 +
 +    struct ssl_sig_record* ref = &signatures[i];
 +    struct ssl_sig* rs = CP(ref->sig);
 +
 +    /* SSL versions must match exactly. */
 +    if (rs->request_version != ts->request_version) continue;
 +
 +    /* Flags - exact match */
 +    if (ts->flags != rs->flags) continue;
 +
 +    /* Extensions match. */
 +    if (match_sigs(rs->extensions, ts->extensions) != 0) continue;
 +
 +    /* Cipher suites match. */
 +    if (match_sigs(rs->cipher_suites, ts->cipher_suites) != 0) continue;
 +
 +    ts->matched = ref;
 +    return;
 +
 +  }
 +
 +}
 +
 +
 +/* Unpack SSLv2 header to a signature. 
 +   -1 on parsing error, 1 if signature was extracted. */
 +
 +static int fingerprint_ssl_v2(struct ssl_sig* sig, const u8* pay, u32 pay_len) {
 +
 +  const u8* pay_end = pay + pay_len;
 +  const u8* tmp_end;
 +
 +  if (pay + sizeof(struct ssl2_hdr) > pay_end) goto too_short;
 +
 +  struct ssl2_hdr* hdr = (struct ssl2_hdr*)pay;
 +  pay += sizeof(struct ssl2_hdr);
 +
 +  if (hdr->ver_min == 2 && hdr->ver_maj == 0) {
 +
 +    /* SSLv2 is actually 0x0002 on the wire. */
 +    sig->request_version = 0x0200;
 +
 +  } else {
 +
 +    /* Most often - SSLv2 header has request version set to 3.x */
 +    sig->request_version = (hdr->ver_maj << 8) | hdr->ver_min;
 +
 +  }
 +
 +
 +  u16 cipher_spec_len = ntohs(hdr->cipher_spec_length);
 +
 +  if (cipher_spec_len % 3) {
 +
 +    DEBUG("[#] SSLv2 cipher_spec_len=%u is not divisable by 3.\n",
 +          cipher_spec_len);
 +    return -1;
 +
 +  }
 +
 +  if (pay + cipher_spec_len > pay_end) goto too_short;
 +
 +  int cipher_pos = 0;
 +  sig->cipher_suites = ck_alloc(((cipher_spec_len / 3) + 1) * sizeof(u32));
 +  tmp_end = pay + cipher_spec_len;
 +
 +  while (pay < tmp_end) {
 +
 +    sig->cipher_suites[cipher_pos++] =
 +      (pay[0] << 16) | (pay[1] << 8) | pay[2];
 +    pay += 3;
 +
 +  }
 +  sig->cipher_suites[cipher_pos] = END_MARKER;
 +
 +
 +  u16 session_id_len = ntohs(hdr->session_id_length);
 +  u16 challenge_len = ntohs(hdr->challenge_length);
 +
 +  if (pay + session_id_len + challenge_len > pay_end) {
 +
 +    DEBUG("[#] SSLv2 frame truncated (but valid)\n");
 +    goto truncated;
 +
 +  }
 +
 +  pay += session_id_len + challenge_len;
 +
 +  if (pay != pay_end) {
 +
 +    DEBUG("[#] SSLv2 extra %u bytes remaining after client-hello message.\n",
 +          pay_end - pay);
 +
 +  }
 +
 +truncated:
 +
 +  sig->extensions = ck_alloc(1 * sizeof(u32));
 +  sig->extensions[0] = END_MARKER;
 +
 +  return 1;
 +
 +
 +too_short:
 +
 +  DEBUG("[#] SSLv2 frame too short.\n");
 +
 +  ck_free(sig->cipher_suites);
 +  ck_free(sig->extensions);
 +
 +  return -1;
 +
 +}
 +
 +
 +/* Unpack SSLv3 fragment to a signature. We expect to hear ClientHello
 + message.  -1 on parsing error, 1 if signature was extracted. */
 +
 +static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
 +                              u32 frag_len, u16 record_version, u32 local_time) {
 +
 +  int i;
 +  const u8* frag_end = fragment + frag_len;
 +
 +  struct ssl3_message_hdr* msg = (struct ssl3_message_hdr*)fragment;
 +  u32 msg_len = (msg->length[0] << 16) |
 +                (msg->length[1] << 8) |
 +                (msg->length[2]);
 +
 +  const u8* pay = (const u8*)msg + sizeof(struct ssl3_message_hdr);
 +  const u8* pay_end = pay + msg_len;
 +  const u8* tmp_end;
 +
 +
 +  /* Record size goes beyond current fragment, it's fine by SSL but
 +     not for us. */
 +
 +  if (pay_end > frag_end) {
 +
 +    DEBUG("[#] SSL Fragment coalescing not supported - %u bytes requested.\n",
 +          pay_end - frag_end);
 +
 +    return -1;
 +
 +  }
 +
 +  if (msg->message_type != SSL3_MSG_CLIENT_HELLO) {
 +
 +    /* Rfc526 says: The handshake protocol messages are presented
 +         below in the order they MUST be sent; sending handshake
 +         messages in an unexpected order results in a fatal error.
 +
 +       I guess we can assume that the first frame must be ClientHello.
 +    */
 +
 +    DEBUG("[#] SSL First message type 0x%02x (%u bytes) not supported.\n",
 +          msg->message_type, msg_len);
 +    return -1;
 +
 +  }
 +
 +
 +  /* ClientHello */
 +
 +
 +  /* Header (34B) + session_id_len (1B) */
 +
 +  if (pay + 2 + 4 + 28 + 1 > pay_end) goto too_short;
 +
 +  sig->request_version = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (sig->request_version != record_version) {
 +    sig->flags |= SSL_FLAG_VER;
 +  }
 +
 +  sig->remote_time = ntohl(*((u32*)pay));
 +  pay += 4;
 +
 +  sig->recv_time = local_time;
 +  s64 drift = ((s64)sig->recv_time) - sig->remote_time;
 +
 +  if (sig->remote_time < 1*365*24*60*60) {
 +
 +    /* Old Firefox on windows uses time since boot */
 +    sig->flags |= SSL_FLAG_STIME;
 +
 +  } else if (abs(drift) > 5*365*24*60*60) {
 +
 +    /* More than 5 years difference - most likely random */
 +    sig->flags |= SSL_FLAG_RTIME;
 +
 +    DEBUG("[#] SSL timer looks wrong: drift=%lld remote_time=%u.\n",
 +          drift, sig->remote_time);
 +
 +  }
 +
 +  /* Random */
 +  u16* random = (u16*)pay;
 +  pay += 28;
 +
 +  for (i = 0; i < 14; i++) {
 +    if (random[i] == 0x0000 || random[i] == 0xffff) {
 +
 +      DEBUG("[#] SSL 0x%04x found in allegedly random blob at offset %i.\n",
 +            random[i], i);
 +      break;
 +
 +    }
 +  }
 +
 +  /* Skip session_id */
 +  u8 session_id_len = pay[0];
 +  pay += 1;
 +
 +  if (pay + session_id_len + 2 > pay_end) goto too_short;
 +
 +  pay += session_id_len;
 +
 +
 +  /* Cipher suites */
 +
 +  u16 cipher_suites_len = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (cipher_suites_len % 2) {
 +
 +    DEBUG("[#] SSL cipher_suites_len=%u is not even.\n", cipher_suites_len);
 +    return -1;
 +
 +  }
 +
 +  if (pay + cipher_suites_len > pay_end) goto too_short;
 +
 +  int cipher_pos = 0;
 +  sig->cipher_suites = ck_alloc(((cipher_suites_len / 2) + 1) * sizeof(u32));
 +  tmp_end = pay + cipher_suites_len;
 +
 +  while (pay < tmp_end) {
 +
 +    sig->cipher_suites[cipher_pos++] = (pay[0] << 8) | pay[1];
 +    pay += 2;
 +
 +  }
 +  sig->cipher_suites[cipher_pos] = END_MARKER;
 +
 +  if (pay + 1 > pay_end) goto truncated;
 +
 +  u8 compression_methods_len = pay[0];
 +  pay += 1;
 +
 +  if (pay + compression_methods_len > pay_end) goto truncated;
 +
 +  tmp_end = pay + compression_methods_len;
 +
 +  while (pay < tmp_end) {
 +
 +    if (pay[0] == 1) {
 +      sig->flags |= SSL_FLAG_COMPR;
 +    }
 +
 +    pay += 1;
 +
 +  }
 +
 +
 +  if (pay + 2 > pay_end) {
 +
 +    /* Extensions are optional in SSLv3. This behaviour was considered
 +       as a flag, but it doesn't bring any entropy. In other words:
 +       noone who is able to send extensions sends an empty list.  An
 +       empty list of extensions is equal to SSLv2 or this branch. */
 +    goto truncated_ok;
 +
 +  }
 +
 +  u16 extensions_len = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (pay + extensions_len > pay_end) goto truncated;
 +
 +  int extensions_pos = 0;
 +  sig->extensions = ck_alloc(((extensions_len / 4) + 1) * sizeof(u32));
 +  tmp_end = pay + extensions_len;
 +
 +  while (pay + 4 <= tmp_end) {
 +
 +    u16 ext_type = (pay[0] << 8) | pay[1];
 +    u16 ext_len  = (pay[2] << 8) | pay[3];
 +    const u8* extension = &pay[4];
 +    pay += 4;
 +
 +    pay += ext_len;
 +
 +    sig->extensions[extensions_pos++] = ext_type;
 +
 +    /* Extension payload sane? */
 +    if (pay > tmp_end) break;
 +
 +    /* Ignore the actual value of the extenstion. */
 +    extension = extension;
 +  }
 +
 +  /* Make sure the terminator is always appended, even if extensions
 +     are malformed. */
 +  sig->extensions = ck_realloc(sig->extensions, (extensions_pos + 1) *
 +                               sizeof(u32));
 +  sig->extensions[extensions_pos] = END_MARKER;
 +
 +  if (pay != tmp_end) {
 +
 +    DEBUG("[#] SSL malformed extensions, %i bytes over.\n",
 +          pay - tmp_end);
 +
 +  }
 +
 +  if (pay != pay_end) {
 +
 +    DEBUG("[#] SSL ClientHello remaining %i bytes after extensions.\n",
 +          pay_end - pay);
 +
 +  }
 +
 +  if (pay_end != frag_end) {
 +
 +    DEBUG("[#] SSL %i bytes remaining after ClientHello message.\n",
 +          frag_end - pay_end);
 +
 +  }
 +
 +  if (0) {
 +truncated:
 +
 +    DEBUG("[#] SSL packet truncated (but valid).\n");
 +
 +  }
 +truncated_ok:
 +
 +  if (!sig->extensions) {
 +    sig->extensions    = ck_alloc(1*sizeof(u32));
 +    sig->extensions[0] = END_MARKER;
 +  }
 +
 +  return 1;
 +
 +
 +too_short:
 +
 +  DEBUG("[#] SSL packet truncated.\n");
 +
 +  ck_free(sig->cipher_suites);
 +  ck_free(sig->extensions);
 +
 +  return -1;
 +
 +}
 +
 +
 +/* Signature - to - string */
 +
 +static u8* dump_sig(struct ssl_sig* sig, u8 fingerprint) {
 +
 +  int i;
 +
 +  static u8* ret;
 +  u32 rlen = 0;
 +
 +#define RETF(_par...) do {                           \
 +    s32 _len = snprintf(NULL, 0, _par);              \
 +    if (_len < 0) FATAL("Whoa, snprintf() fails?!"); \
 +    ret = DFL_ck_realloc_kb(ret, rlen + _len + 1);   \
 +    snprintf((char*)ret + rlen, _len + 1, _par);     \
 +    rlen += _len;                                    \
 +  } while (0)
 +
 +  RETF("%i.%i:", sig->request_version >> 8, sig->request_version & 0xFF);
 +
 +  for (i = 0; sig->cipher_suites[i] != END_MARKER; i++) {
 +    u32 c = sig->cipher_suites[i];
 +    if (c != MATCH_ANY) {
 +      RETF("%s%s%x", (i ? "," : ""),
 +           (c & MATCH_MAYBE) ? "?" : "",
 +           c & ~MATCH_MAYBE);
 +    } else {
 +      RETF("%s*", (i ? "," : ""));
 +    }
 +  }
 +
 +  RETF(":");
 +
 +  for (i = 0; sig->extensions[i] != END_MARKER; i++) {
 +    u32 ext = sig->extensions[i];
 +    if (ext != MATCH_ANY) {
 +      u8 optional = 0;
 +      if (fingerprint && ext == 0) {
 +        optional = 1;
 +      }
 +      RETF("%s%s%x", (i ? "," : ""),
 +           ((ext & MATCH_MAYBE) || optional) ? "?" : "",
 +           ext & ~MATCH_MAYBE);
 +    } else {
 +      RETF("%s*", (i ? "," : ""));
 +    }
 +  }
 +
 +  RETF(":");
 +
 +  int had_prev = 0;
 +  for (i = 0; flags[i].name != NULL; i++) {
 +
 +    if (sig->flags & flags[i].value) {
 +      RETF("%s%s", (had_prev ? "," : ""), flags[i].name);
 +      had_prev = 1;
 +    }
 +
 +  }
 +
 +  return ret;
 +
 +}
 +
 +
 +/* Register new SSL signature. */
 +
 +void ssl_register_sig(u8 to_srv, u8 generic, s32 sig_class, u32 sig_name,
 +                      u8* sig_flavor, u32 label_id, u32* sys, u32 sys_cnt,
 +                      u8* val, u32 line_no) {
 +
 +  struct ssl_sig* ssig;
 +  struct ssl_sig_record* srec;
 +
 +  /* Client signatures only. */
 +  if (to_srv != 1) return;
 +
 +  /* Only "application" signatures supported, no "OS-identifying". */
 +  if (sig_class != -1)
 +    FATAL("OS-identifying SSL signatures not supported, use \"!\" instead "
 +          "of an OS class in line %u.", line_no);
 +
 +  ssig = DFL_ck_alloc(sizeof(struct ssl_sig));
 +
 +  signatures = DFL_ck_realloc(signatures, (signatures_cnt + 1) *
 +                              sizeof(struct ssl_sig_record));
 +
 +  srec = &signatures[signatures_cnt];
 +
 +
 +  int maj = strtol((char*)val, (char**)&val, 10);
 +  if (!val || *val != '.') FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +  int min = strtol((char*)val, (char**)&val, 10);
 +  if (!val || *val != ':') FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +  ssig->request_version = (maj << 8) | min;
 +
 +  ssig->cipher_suites = decode_hex_string((const u8**)&val, line_no);
 +  if (!val || *val != ':' || !ssig->cipher_suites)
 +    FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +  ssig->extensions = decode_hex_string((const u8**)&val, line_no);
 +  if (!val || *val != ':' || !ssig->extensions)
 +    FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +
 +  while (*val) {
 +
 +    int i;
 +    for (i = 0;  flags[i].name != NULL; i++) {
 +
 +      if (!strncmp((char*)val, flags[i].name, flags[i].name_len)) {
 +        ssig->flags |= flags[i].value;
 +        val += flags[i].name_len;
 +        goto flag_matched;
 +      }
 +
 +    }
 +
 +    FATAL("Unrecognized flag in line %u.", line_no);
 +
 +  flag_matched:
 +
 +    if (*val == ',') val++;
 +
 +  }
 +
 +  srec->class_id = sig_class;
 +  srec->name_id  = sig_name;
 +  srec->flavor   = sig_flavor;
 +  srec->label_id = label_id;
 +  srec->sys      = sys;
 +  srec->sys_cnt  = sys_cnt;
 +  srec->line_no  = line_no;
 +  srec->generic  = generic;
 +
 +  srec->sig      = ssig;
 +
 +  signatures_cnt++;
 +
 +}
 +
 +static void score_nat(u8 to_srv, struct packet_flow* f, struct ssl_sig* sig) {
 +
 +  struct ssl_sig_record* m = sig->matched;
 +  struct host_data* hd;
 +
 +  u8  score = 0;
 +  u16 reason = 0;
 +
 +  /* Client request only. */
 +  if (to_srv != 1) return;
 +
 +  hd = f->client;
 +
 +
 +  if (m && m->class_id == -1) {
 +
 +    /* Application signature: we might look at the OS-es mentioned by
 +       the signature, and make sure the OS from http and/or TCP
 +       matches. */
 +
 +    verify_tool_class(to_srv, f, m->sys, m->sys_cnt);
 +
 +  }
 +
 +  if (hd->ssl_remote_time && sig->remote_time &&
 +      (sig->flags & SSL_FLAG_RTIME) == 0) {
 +
 +    /* Time on the client should be increasing monotically */
 +
 +    s64 recv_diff    = ((s64)sig->recv_time) - hd->ssl_recv_time;
 +    s64 remote_diff  = ((s64)sig->remote_time) - hd->ssl_remote_time;
 +
 +    if (remote_diff < recv_diff - SSL_MAX_TIME_DIFF ||
 +        remote_diff > recv_diff + SSL_MAX_TIME_DIFF) {
 +
 +      DEBUG("[#] SSL gmt_unix_time distance too high (%lld in %lld sec).\n",
 +            remote_diff, recv_diff);
 +
 +      score += 4;
 +      reason |= NAT_APP_DATE;
 +
 +    }
 +
 +  }
 +
 +  add_nat_score(to_srv, f, reason, score);
 +
 +}
 +
 +
 +
 +/* Given an SSL client signature look it up and create an observation.  */
 +
 +static void fingerprint_ssl(u8 to_srv, struct packet_flow* f,
 +                            struct ssl_sig* sig) {
 +
 +  /* Client request only. */
 +  if (to_srv != 1) return;
 +
 +  ssl_find_match(sig);
 +
 +  struct ssl_sig_record* m = sig->matched;
 +
 +  start_observation("ssl request", 5, to_srv, f);
 +
 +  if (m) {
 +
 +    /* Found matching signature */
 +
 +    OBSERVF((m->class_id < 0) ? "app" : "os", "%s%s%s",
 +            fp_os_names[m->name_id], m->flavor ? " " : "",
 +            m->flavor ? m->flavor : (u8*)"");
 +
 +    add_observation_field("match_sig", dump_sig(sig->matched->sig, 0));
 +
 +  } else {
 +
 +    add_observation_field("app", NULL);
 +    add_observation_field("match_sig", NULL);
 +
 +  }
 +
 +  if ((sig->flags & (SSL_FLAG_RTIME | SSL_FLAG_STIME)) == 0) {
 +
 +    s64 drift = ((s64)sig->recv_time) - sig->remote_time;
 +    OBSERVF("drift", "%lld", drift);
 +
 +  } else {
 +
 +    add_observation_field("drift", NULL);
 +
 +  }
 +
 +  OBSERVF("remote_time", "%u", sig->remote_time);
 +
 +  add_observation_field("raw_sig", dump_sig(sig, 1));
 +
 +  score_nat(to_srv, f, sig);
 +
 +}
 +
 +
 +/* Examine request or response; returns 1 if more data needed and
 +   plausibly can be read. Note that the buffer is always NULL
 +   terminated. */
 +
 +u8 process_ssl(u8 to_srv, struct packet_flow* f) {
 +
 +  int success = 0;
 +  struct ssl_sig sig;
 +
 +
 +  /* Already decided this flow? */
 +
 +  if (f->in_ssl) return 0;
 +
 +
 +  /* Tracking requests only. */
 +
 +  if (!to_srv) return 0;
 +
 +
 +  u8 can_get_more = (f->req_len < MAX_FLOW_DATA);
 +
 +
 +  /* SSLv3 record is 5 bytes, message is 4 + 38; SSLv2 CLIENT-HELLO is
 +     11 bytes - we try to recognize protocol by looking at top 6
 +     bytes. */
 +
 +  if (f->req_len < 6) return can_get_more;
 +
 +  struct ssl2_hdr* hdr2 = (struct ssl2_hdr*)f->request;
 +  u16 msg_length = ntohs(hdr2->msg_length);
 +
 +  struct ssl3_record_hdr* hdr3 = (struct ssl3_record_hdr*)f->request;
 +  u16 fragment_len = ntohs(hdr3->length);
 +
 +
 +  /* Does it look like top 5 bytes of SSLv2? Most significant bit must
 +     be set, followed by 15 bits indicating record length, which must
 +     be at least 9. */
 +
 +  if ((msg_length & 0x8000) &&
 +      (msg_length & ~0x8000) >= sizeof(struct ssl2_hdr) - 2 &&
 +      hdr2->msg_type == 1 &&
 +      ((hdr2->ver_maj == 3 && hdr2->ver_min < 4) ||
 +       (hdr2->ver_min == 2 && hdr2->ver_maj == 0))) {
 +
 +    /* Clear top bit. */
 +    msg_length &= ~0x8000;
 +
 +    if (f->req_len < 2 + msg_length) return can_get_more;
 +
 +    memset(&sig, 0, sizeof(struct ssl_sig));
 +    sig.flags |= SSL_FLAG_V2;
 +
 +    success = fingerprint_ssl_v2(&sig, f->request, msg_length + 2);
 +
 +  }
 +
 +
 +  /* Top 5 bytes of SSLv3/TLS header? Currently available TLS
 +     versions: 3.0 - 3.3. The rfc disallows fragment to have more than
 +     2^14 bytes. Also length less than 4 bytes doesn't make much
 +     sense. Additionally let's peek the meesage type. */
 +
 +  else if (hdr3->content_type == SSL3_REC_HANDSHAKE &&
 +           hdr3->ver_maj == 3 && hdr3->ver_min < 4 &&
 +           fragment_len > 3 && fragment_len < (1 << 14) &&
 +           f->request[5] == SSL3_MSG_CLIENT_HELLO) {
 +
 +    if (f->req_len < sizeof(struct ssl3_record_hdr) + fragment_len)
 +      return can_get_more;
 +
 +    memset(&sig, 0, sizeof(struct ssl_sig));
 +    u16 record_version = (hdr3->ver_maj << 8) | hdr3->ver_min;
 +
 +    u8* fragment = f->request + sizeof(struct ssl3_record_hdr);
 +
 +    success = fingerprint_ssl_v3(&sig, fragment, fragment_len,
 +                                 record_version,
 +                                 f->client->last_seen);
 +
 +  }
 +
 +  if (success != 1) {
 +
 +    DEBUG("[#] Does not look like SSLv2 nor SSLv3.\n");
 +
 +    f->in_ssl = -1;
 +    return 0;
 +
 +  }
 +
 +
 +  f->in_ssl = 1;
 +
 +  fingerprint_ssl(to_srv, f, &sig);
 +
 +  if (sig.remote_time && !(sig.flags & SSL_FLAG_RTIME)) {
 +    f->client->ssl_remote_time = sig.remote_time;
 +    f->client->ssl_recv_time   = sig.recv_time;
 +  }
 +
 +
 +  ck_free(sig.cipher_suites);
 +  ck_free(sig.extensions);
 +
 +  return 0;
 +
 +}
 diff --git a/fp_ssl.h b/fp_ssl.h
 new file mode 100644
 index 0000000..4e0d57c
 --- /dev/null
 +++ b/fp_ssl.h
 @@ -0,0 +1,123 @@
 +/* -*-mode:c; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
 +/*
 +  p0f - SSL fingerprinting
 +  -------------------------
 +
 +  Copyright (C) 2012 by Marek Majkowski <[email protected]>
 +
 +  Distributed under the terms and conditions of GNU LGPL.
 +
 + */
 +
 +#ifndef _HAVE_FP_SSL_H
 +#define _HAVE_FP_SSL_H
 +
 +#include "types.h"
 +
 +
 +/* Constants */
 +
 +#define MATCH_MAYBE 0x10000000  /* '?' - indicats a single optional match */
 +#define MATCH_ANY   0x20000000  /* '*' - match zero or more elements */
 +#define END_MARKER  0x40000000  /* internal marker */
 +
 +#define SSL_MAX_CIPHERS 128     /* max number of ciphers in a signature */
 +#define SSL_MAX_TIME_DIFF 10    /* remote clock scew limit, in seconds */
 +
 +
 +/* Flags */
 +
 +#define SSL_FLAG_V2    0x0001  /* SSLv2 handshake. */
 +#define SSL_FLAG_VER   0x0002  /* Record version different than ClientHello. */
 +#define SSL_FLAG_RTIME 0x0004  /* weird SSL time, (delta > 5 years), most likely random*/
 +#define SSL_FLAG_STIME 0x0008  /* small SSL time, (absolute value < 1 year)
 +                                  most likely time since reboot for old ff */
 +#define SSL_FLAG_COMPR 0x0010  /* Deflate compression support. */
 +
 +
 +/* SSLv2 */
 +
 +struct ssl2_hdr {
 +
 +  u16 msg_length;
 +  u8 msg_type;
 +  u8 ver_maj;
 +  u8 ver_min;
 +
 +  u16 cipher_spec_length;
 +  u16 session_id_length;
 +  u16 challenge_length;
 +
 +} __attribute__((packed));
 +
 +
 +/* SSLv3 */
 +
 +#define SSL3_REC_HANDSHAKE 0x16    /* 22 */
 +#define SSL3_MSG_CLIENT_HELLO 0x01
 +
 +struct ssl3_record_hdr {
 +
 +  u8 content_type;
 +  u8 ver_maj;
 +  u8 ver_min;
 +  u16 length;
 +
 +} __attribute__((packed));
 +
 +
 +struct ssl3_message_hdr {
 +
 +  u8 message_type;
 +  u8 length[3];
 +
 +} __attribute__((packed));
 +
 +
 +
 +/* Internal data structures */
 +
 +struct ssl_sig_record;
 +
 +struct ssl_sig {
 +
 +  u16 request_version;   /* Requested SSL version (maj << 8) | min */
 +
 +  u32 remote_time;       /* ClientHello message gmt_unix_time field */
 +  u32 recv_time;         /* Actual receive time */
 +
 +  u32* cipher_suites;    /* List of SSL ciphers, END_MARKER terminated */
 +
 +  u32* extensions;       /* List of SSL extensions, END_MARKER terminated */
 +
 +  u32 flags;             /* SSL flags */
 +
 +  struct ssl_sig_record* matched; /* NULL = no match */
 +};
 +
 +struct ssl_sig_record {
 +
 +  s32 class_id;                         /* OS class ID (-1 = user)            */
 +  s32 name_id;                          /* OS name ID                         */
 +  u8* flavor;                           /* Human-readable flavor string       */
 +
 +  u32 label_id;                         /* Signature label ID                 */
 +
 +  u32* sys;                             /* OS class / name IDs for user apps  */
 +  u32  sys_cnt;                         /* Length of sys                      */
 +
 +  u32  line_no;                         /* Line number in p0f.fp              */
 +
 +  u8 generic;                           /* Generic signature?                 */
 +
 +  struct ssl_sig* sig;                  /* Actual signature data              */
 +
 +};
 +
 +void ssl_register_sig(u8 to_srv, u8 generic, s32 sig_class, u32 sig_name,
 +                      u8* sig_flavor, u32 label_id, u32* sys, u32 sys_cnt,
 +		      u8* val, u32 line_no);
 +
 +u8 process_ssl(u8 to_srv, struct packet_flow* f);
 +
 +#endif /* _HAVE_FP_SSL_H */
 diff --git a/p0f.fp b/p0f.fp
 index d02dc5f..b9627f5 100644
 --- a/p0f.fp
 +++ b/p0f.fp
 @@ -903,3 +903,235 @@ sys   = Linux
 sig   = *:Content-Type,X-Content-Type-Options=[nosniff],Date,Server=[sffe]:Connection,Accept-Ranges,Keep-Alive,Connection:
 sig   = *:Date,Content-Type,Server=[gws]:Connection,Accept-Ranges,Keep-Alive:
 sig   = *:Content-Type,X-Content-Type-Options=[nosniff],Server=[GSE]:Connection,Accept-Ranges,Keep-Alive:
 +
 +; =====================
 +; SSL client signatures
 +; =====================
 +
 +[ssl:request]
 +
 +;-----------------
 +; Windows specific
 +;-----------------
 +
 +; Windows NT 5.1, Windows NT 5.2 (XP)
 +label = s:!:any:MSIE or Safari on Windows XP
 +sys   = Windows
 +sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63:ff01:
 +; no MS10-049 applied?
 +sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63::
 +
 +; with some SSL/TLS options tweaked
 +sig   = 3.0:4,5,a,9,64,62,3,6,13,12,63,ff::
 +sig   = 3.0:4,5,a,10080,700c0,30080,9,60040,64,62,3,6,20080,40080,13,12,63,ff::v2
 +sig   = 2.0:10080,700c0,30080,60040,20080,40080,ff::v2
 +
 +
 +; Windows NT 6.0 (Vista)
 +label = s:!:any:MSIE 5.5-6 or Chrome 1-4 or Safari on Windows Vista
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,a,b,ff01:
 +
 +label = s:!:any:MSIE 7.0-9.0 or Chrome 5 on Windows Vista
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,5,a,b,ff01:
 +
 +
 +; Windows NT 6.1 (7)
 +label = s:!:MSIE:7-9 on Windows 7
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b:
 +
 +label = s:!:Safari:on Windows 7
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b:
 +
 +; Windows NT 6.2 ( 8)
 +; 23 usually means NT 6.2
 +label = s:!:MSIE:10 on Windows 8
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b,23:
 +
 +label = s:!:Safari:Safari on Windows 8
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b,23:
 +
 +
 +; ------
 +; Chrome
 +; ------
 +
 +label = s:!:Chrome:6 or newer
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23:compr
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23,3374:compr
 +; 5 is on on windows
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23,3374,5:compr
 +
 +label = s:!:Chrome:degraded to SSLv3.0
 +sys   = Windows,@unix
 +sig   = 3.0:ff,88,87,39,38,84,35,45,44,66,33,32,96,41,4,5,2f,16,13,feff,a::
 +
 +; -------
 +; Firefox
 +; -------
 +
 +label = s:!:Firefox:1.X
 +sys   = Windows,@unix
 +sig   = 3.1:10080,30080,*,40080,39,38,35,*,64,62,3,6::v2
 +sig   = 3.1:39,38,35,*,64,62,3,6::stime
 +
 +label = s:!:Firefox:2.X
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,39,38,c00f,*,c00d,c003,feff,a:?0,a,b:
 +
 +label = s:!:Firefox:3.0-3.5
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,c005,84,35,c007,*,c003,feff,a:?0,a,b,23:
 +
 +label = s:!:Firefox:3.6.X
 +sys   = Windows,@unix
 +sig   = 3.1:ff,c00a,c014,88,87,38,c00f,c005,84,35,39,*,c00d,c003,feff,a:?0,a,b,23:
 +
 +label = s:!:Firefox:4-11
 +sys   = Windows,@unix
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b,23:
 +; with SSLv2 disalbed
 +sig   = 3.1:c00a,c014,88,87,39,38,*,c003,feff,a:?0,ff01,a,b,23:
 +
 +label = s:!:Firefox:11 (TOR)
 +sys   = Windows,@unix
 +; Lack of a single extension (SessionTicket TLS) is not a very strong signal.
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b:
 +
 +label = s:!:Firefox:14 or newer
 +sys   = Windows,@unix
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b,23,3374:
 +
 +; with TLS switched off
 +label = s:!:Firefox:3.6.X or newer
 +sys   = Windows,@unix
 +sig   = 3.0:ff,88,87,39,38,84,35,45,44,33,32,96,41,4,5,2f,16,13,feff,a::
 +
 +
 +; ------
 +; Safari
 +; ------
 +; Safari on old PowerPC box
 +label = s:!:Safari:4.X
 +sys   = Mac OS X
 +sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1::
 +sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1,10080,*,700c0::v2
 +
 +label = s:!:Safari:5.1.2
 +sys   = Mac OS X
 +sig   = 3.1:c00a,c009,c007,c008,c013,*,33,38,39,16,15,14,13,12,11:?0,a,b:
 +
 +label = s:!:Safari:5.1.3 or newer
 +sys   = Mac OS X
 +sig   = 3.1:c00a,c009,c007,c008,c013,*,33,38,39,16,13:?0,a,b:
 +
 +
 +; -------
 +; Android
 +; -------
 +
 +; in http Android is treated as Linux, oh, well...
 +label = s:!:Android:1.5-2.1
 +sys   = Linux
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3::
 +
 +label = s:!:Android:2.3
 +sys   = Linux
 +sig   = 3.1:4,5,2f,33,32,a,16,13,9,15,12,3,8,14,11,ff::
 +
 +label = s:!:Android:3.2
 +sys   = Linux
 +sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,15,12,9,14,11,8,6,3,ff:?0,b,a,23,3374:compr
 +
 +label = s:!:Android:4.X
 +sys   = Linux
 +sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,ff:?0,b,a,23,3374:compr
 +
 +; -----------
 +; iPhone iPad
 +; -----------
 +
 +label = s:!:Safari:iOS 4.X
 +sys   = iOS
 +sig   = 3.1:c00a,c009,c007,*,33,39,16,15,14:?0,a,b:
 +
 +label = s:!:Safari:iOS 5.X
 +sys   = iOS
 +sig   = 3.3:ff,c024,c023,c00a,*,33,39,16:?0,a,b,d:
 +
 +
 +; ------------
 +; Weird Mobile
 +; ------------
 +label = s:!:Opera Mini:11.X
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +sig   = 3.1:ff,39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +
 +label = s:!:HP-tablet:unknown
 +sys   = Touchpad
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4:?0:
 +
 +
 +; -----
 +; Opera
 +; -----
 +
 +label = s:!:Opera:10.x - 11.00
 +sys   = Windows,@unix
 +sig   = 3.2:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,5:ver
 +sig   = 3.3:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5,d:ver
 +
 +label = s:!:Opera:11.52 or newer
 +sys   = Windows,@unix
 +sig   = 3.1:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +sig   = 3.1:ff,6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +
 +; On second connection Opera replies with the last used crypto in a first place I guess
 +label = s:!:Opera:
 +sys   = Windows,@unix
 +sig   = 3.1:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.1:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.2:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.2:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.3:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.3:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +
 +
 +; --------------
 +; Various things
 +; --------------
 +
 +label = g:!:gnutls:
 +sys   = @unix
 +sig   = 3.1:33,16,39,2f,a,35,5,4,32,13,38,66::compr
 +sig   = 3.2:2f,5,4,a,35,32,66,13,38,33,16,39,34,18,1b,3a,3::
 +sig   = 3.3:3c,2f,5,4,a,3d,35,40,32,66,13,6a,38,67,33,16,6b,39,6c,34,18,1b,6d,3a:ff01,d:
 +sig   = 3.1:*,2f,5,4,a,*,35,*,18,1b,3a,*:*:
 +
 +
 +label = g:!:openssl:
 +sys   = @unix
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0:compr
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,23:
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0,23:compr
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0:compr
 +
 +; darwin
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,9a,99,96,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +
 +sig   = 3.1:39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:b,a,23:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:?0,b,a:compr
 +
 +label = s:!:Epiphany:2.X
 +sys   = Linux
 +sig   = 3.0:33,39,16,32,38,13,2f,35,a,5,4::
 diff --git a/process.c b/process.c
 index 3cee1da..66a0b21 100644
 --- a/process.c
 +++ b/process.c
 @@ -36,6 +36,7 @@
 #include "fp_tcp.h"
 #include "fp_mtu.h"
 #include "fp_http.h"
 +#include "fp_ssl.h"
 
 u64 packet_cnt;                         /* Total number of packets processed  */
 
 @@ -1368,6 +1369,7 @@ static void flow_dispatch(struct packet_data* pk) {
       if (!pk->pay_len) return;
 
       need_more |= process_http(to_srv, f);
 +      need_more |= process_ssl(to_srv, f);
 
       if (!need_more) {
 
 diff --git a/process.h b/process.h
 index 5433282..2addf27 100644
 --- a/process.h
 +++ b/process.h
 @@ -135,6 +135,9 @@ struct host_data {
 
   u16 http_resp_port;                   /* Port on which response seen        */
 
 +  u32 ssl_remote_time;                  /* Last client timestamp from SSL     */
 +  u32 ssl_recv_time;                    /* Time drift derived from SSL        */
 +
 };
 
 /* Reasons for NAT detection: */
 @@ -195,6 +198,8 @@ struct packet_flow {
 
   struct http_sig http_tmp;             /* Temporary signature                */
 
 +  s8  in_ssl;                           /* 0 = tbd, 1 = yes, -1 = no          */
 +
 };
 
 extern u64 packet_cnt;
 diff --git a/readfp.c b/readfp.c
 index 34a3cac..bafd466 100644
 --- a/readfp.c
 +++ b/readfp.c
 @@ -29,6 +29,7 @@
 #include "fp_tcp.h"
 #include "fp_mtu.h"
 #include "fp_http.h"
 +#include "fp_ssl.h"
 #include "readfp.h"
 
 static u32 sig_cnt;                     /* Total number of p0f.fp sigs        */
 @@ -270,6 +271,10 @@ static void config_parse_line(u8* line) {
 
       mod_type = CF_MOD_HTTP;
 
 +    } else if (!strcmp((char*)line, "ssl")) {
 +
 +      mod_type = CF_MOD_SSL;
 +
     } else {
 
       FATAL("Unrecognized fingerprinting module '%s' in line %u.", line, line_no);
 @@ -367,6 +372,11 @@ static void config_parse_line(u8* line) {
                           label_id, cur_sys, cur_sys_cnt, val, line_no);
         break;
 
 +      case CF_MOD_SSL:
 +        ssl_register_sig(mod_to_srv, generic, sig_class, sig_name, sig_flavor,
 +                         label_id, cur_sys, cur_sys_cnt, val, line_no);
 +        break;
 +
     }
 
     sig_cnt++;
 diff --git a/readfp.h b/readfp.h
 index 9998e00..6240462 100644
 --- a/readfp.h
 +++ b/readfp.h
 @@ -18,6 +18,7 @@
 #define CF_MOD_TCP           0x00       /* fp_tcp.c                           */
 #define CF_MOD_MTU           0x01       /* fp_mtu.c                           */
 #define CF_MOD_HTTP          0x02       /* fp_http.c                          */
 +#define CF_MOD_SSL           0x03       /* fp_ssl.c                           */
 
 /* Parser states: */
 
diff --git a/p0f-ssl-2012-11-01.diff b/p0f-ssl-2012-11-01.diff
 diff --git a/build.sh b/build.sh
 index d48a969..c9fd6ed 100755
 --- a/build.sh
 +++ b/build.sh
 @@ -31,7 +31,7 @@ else
   USE_LIBS="-lpcap $LIBS"
 fi
 
 -OBJFILES="api.c process.c fp_tcp.c fp_mtu.c fp_http.c readfp.c"
 +OBJFILES="api.c process.c fp_tcp.c fp_mtu.c fp_http.c fp_ssl.c readfp.c"
 
 echo "Welcome to the build script for $PROGNAME $VERSION!"
 echo "Copyright (C) 2012 by Michal Zalewski <[email protected]>"
 diff --git a/docs/README b/docs/README
 index 4981c41..943b9a2 100644
 --- a/docs/README
 +++ b/docs/README
 @@ -101,9 +101,9 @@ implementation quirks (e.g. non-zero values in "must be zero" fields).
 The metrics used for application-level traffic vary from one module to another;
 where possible, the tool relies on signals such as the ordering or syntax of
 HTTP headers or SMTP commands, rather than any declarative statements such as
 -User-Agent. Application-level fingerprinting modules currently support HTTP.
 -Before the tool leaves "beta", I want to add SMTP and FTP. Other protocols,
 -such as FTP, POP3, IMAP, SSH, and SSL, may follow.
 +User-Agent. Application-level fingerprinting modules currently support HTTP
 +and SSL. Before the tool leaves "beta", I want to add SMTP and FTP. Other
 +protocols, such as FTP, POP3, IMAP and SSH may follow.
 
 The list of all the measured parameters is reviewed in section 5 later on.
 Some of the analysis also happens on a higher level: inconsistencies in the
 @@ -713,6 +713,64 @@ responses to OPTIONS or POST. That said, it does not seem to be worth the
 effort: the protocol is so verbose, and implemented so arbitrarily, that we are
 getting more than enough information just with a simple GET / HEAD fingerprint.
 
 +== SSL signatures ==
 +
 +P0f is capable of fingerprinting SSL / TLS requests. This can be used
 +to verify if a browser using https is legitimate, and in some cases
 +can reveal some details about the client's operating system.
 +
 +SSL signatures have the following layout:
 +
 +sig = sslver:ciphers:extensions:sslflags
 +
 +  sslver     - two dot-separated integers representing SSL request version,
 +               may be something like 2.0 or 3.1.
 +
 +               NEW SIGNATURES: Copy literally.
 +
 +  ciphers    - comma-separated list of hex values representing SSL
 +               ciphers supported by the client.
 +
 +               NEW SIGNATURES: Review the list and substitute boring
 +               parts that don't bring newq information with the 'match
 +               all' sign '*'. For efficiency don't use star at the
 +               beginning of the signature.
 +
 +               You may also use '?' to make a single value optional.
 +
 +  extensions - comma-separated list of hex values representing SSL
 +               extensions.
 +
 +               NEW SIGNATURES: Same as for ciphers.
 +
 +  sslflags   - comma-separated list of SSL flags. SSLv2 flags:
 +
 +               v2    - client used an SSLv2 handshake
 +               chlen - challenge data is not 32 bytes long
 +
 +               Flags specific to SSLv3 handshake:
 +
 +               ver   - SSL protocol on a request layer was different
 +                       than on a record layer
 +               stime - gmt_unix_time field has unusually was small value,
 +                       most likely time since boot
 +               rtime - gmt_unix_time field has value that is very far off
 +                       local time, most likely it is random
 +               compr - client supports deflate compression
 +               rand  - supposedly random data doesn't look randomly
 +
 +               NEW SIGNATURES: Copy literally.
 +
 +Any of these sections except for the 'sslver' may be blank.
 +
 +For a fingerprint to match signature an exact fit must happen - sslver
 +and flags must be exactly equal, ciphers and extensions must match
 +with respect to '*' and '?' signs.
 +
 +Note that the fingerprint is matched against signatures in order. You
 +may take advantage of this and keep broader signatures closer to the
 +bottom of the list.
 +
 == SMTP signatures ==
 
    *** NOT IMPLEMENTED YET ***
 @@ -772,7 +830,7 @@ The following code is also issued by the HTTP module:
   app_srv_lb   - server application signatures change, suggesting load
                  balancing.
 
 -  date         - server-advertised date changes inconsistently.
 +  date         - advertised date changes inconsistently.
 
 Different reasons have different weights, balanced to keep p0f very sensitive
 even to very homogenous environments behind NAT. If you end up seeing false
 @@ -903,6 +961,7 @@ including:
   Jeff Weisberg
   Anthony Howe
   Tomoyuki Murakami
 +  Marek Majkowski
 
 If you wish to help, the most immediate way to do so is to simply gather new
 signatures, especially from less popular or older platforms (servers, networking
 diff --git a/docs/ssl-notes.txt b/docs/ssl-notes.txt
 new file mode 100644
 index 0000000..bd7a86b
 --- /dev/null
 +++ b/docs/ssl-notes.txt
 @@ -0,0 +1,142 @@
 +
 +SSL fingerprinting
 +==================
 +
 +The first message of an SSL/TLS connection, ClientHello, contains some
 +data that could be used to identify the client. P0f SSL code creates a
 +fingerprint from this packet.
 +
 +For a given ClientHello message sometimes it is possible to identify
 +underlying SSL library. For applications which include a custom build
 +of an SSL library it may be even possible to pinpoint a specific
 +application version. For example some web browsers do it. Additionally
 +browsers selectively support newly introduced SSL features like SPDY,
 +ECC or 'renegotiation info' extension making it easier to distinguish
 +one from another.
 +
 +SSL connections are usually directly forwarded by proxies without
 +mangling. In most cases SSL fingerprint comes straight from a genuine
 +application and is not affected by NAT or any specific network
 +configuration.
 +
 +Although initial frames sent from both SSL client and the server are
 +similar, only the one sent by the client (ClientHello) can be
 +passively fingerprinted. The packet from server (ServerHello) doesn't
 +contain enough information. If you wish to fingerprint an SSL server
 +an active fingerprinting tool is more suitable [1].
 +
 +Passively fingerprinting SSL is not a new idea, initial work was done
 +in mid 2009 by Ivan Ristic [2]. He was able to collect a few dozen
 +interesting signatures [3]. Unfortunately in his work he had ignored
 +the SSL extensions which add a lot of information. Especially the
 +ordering of extensions has high value, similarly to the TCP options
 +ordering for TCP/IP stack fingerprinting.
 +
 +  [1] http://nmap.org/nsedoc/scripts/ssl-enum-ciphers.html
 +  [2] https://www.ssllabs.com/projects/client-fingerprinting/
 +  [3] http://blog.ivanristic.com/2009/07/examples-of-the-information-collected-from-ssl-handshakes.html
 +
 +References to SSL/TLS specs:
 +
 + * SSL 2.0: http://www.mozilla.org/projects/security/pki/nss/ssl/draft02.html
 + * SSL 3.0: http://tools.ietf.org/html/rfc6101
 + * TLS 1.0: http://tools.ietf.org/html/rfc2246
 + * TLS 1.1: http://tools.ietf.org/html/rfc4346
 + * TLS 1.2: http://tools.ietf.org/html/rfc5246
 +
 +
 +SSL fingerprint structure
 +=========================
 +
 +SSL fingerprint is generated from ClientHello SSL/TLS data. It's a
 +semicolon delimited string with layout as follows:
 +
 +    ssl version : ciphers list : extensions list : flags
 +
 +For example a generated fingerprint might look like:
 +
 +    3.1:39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,ff:23:compr
 +
 +This reads:
 +
 +1) Client requests TLS 1.0 (value 3.1), using SSL 3.0+ record format
 +   (flag 'v2' is _not_ set), with 'request' layer SSL version equal to
 +   'record' layer version (flag 'ver' is _not_ set).
 +2) Client supports ciphers, in order:
 +    39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,ff
 +3) Client used only a single extension: 0x23 (SessionTicket TLS)
 +4) Client supports deflate compression (flag 'compr' is set).
 +5) Random blob looks legitimate (flag 'rand' is _not_ set).
 +6) Unix time reported by the client looks legitimate (flags 'stime'
 +   and 'rtime' are _not_ set).
 +
 +
 +ClientHello message structure
 +=============================
 +
 +SSL client handshake (ClientHello message) contains following fields:
 +
 + * SSL record version - browsers usually use version 3, Ivan Ristic
 +   showed that older clients and web crawlers (including wget) still
 +   use SSL 2 packet format. Flag 'v2' is set in that case.
 +
 + * Requested SSL protocol version - most likely 3.1 (TLS 1.0). Less
 +   likely values are 3.0 (SSL 3.0) and 3.2 (TLS 1.1). SSL crawlers may
 +   try invalid versions like 4.2. We set a flag:
 +
 +     'ver' - if, for SSLv3 / TLS handshake, the version on a 'request'
 +             layer is different than on 'record' layer. This behaviour
 +             was seen in Opera.
 +
 + * Current unix timestamp from the client. In the code we may set one
 +   of two flags derived from this value:
 +
 +     'stime' - when the timestamp is too small (less than one year
 +               since the epoch). This is behaviour was seen in
 +               Firefox 2.X, when this value is set to the number of
 +               seconds since boot.
 +
 +     'rtime' - when the timestamp is far off the current value (delta
 +               is greater than 5 years). Most often this means that
 +               the field is set to a random value. This was seen in
 +               some SSL crawlers.
 +
 + * 28 random bytes - not very useful. A flag is set:
 +
 +      'rand' - when the values don't look very randomly (0x00000000 or
 +               0xffffffff found in the supposedly random block)
 +
 + * session_id - ignored.
 +
 + * cipher_suites - a list of supported encryption algorithms
 +   ('ciphers').  Iana maintains a list of valid values [4].
 +
 + * compression_methods - a list of supported compression
 +   methods. There is only one valid compression method available, we
 +   set 'compr' flag if compression is enabled
 +
 + * extensions - a list of SSL extensions. Second, after
 +   'cipher_suites', major source of entropy. Iana maintains the list
 +   of valid extensions [5].
 +
 +It's worth noting that SSL extensions may contain a payload data for
 +every extension.  We ignore that - it doesn't seem to provide any more
 +entropy than by just looking at the order of SSL extensions.
 +
 +A special explanation should be given for '0' - 'server_name' SSL
 +extension. This extension contains a host name of a remote host and is
 +often described as Server Name Indication TLS extension from RFC 3546.
 +Unfortunately RFC forbids passing raw ip addresses in this
 +option. That means this option must not be present if you enter a
 +website using an ip address. For example, browser will valid extension
 +'0' with hostname if you access 'https://localhost' but will not for
 +'https://127.0.0.1'.
 +
 +P0f assumes that this extension is optional and will always prepend it
 +with an optional sign '?'. This is quite optimistic and may not always
 +be a good idea - it is possible the fingerprint when accessing
 +'localhost' is completely different from '127.0.0.1'. Using '?' before
 +this extension will only clutter a signature in such case.
 +
 +  [4] http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-3
 +  [5] http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xml
 diff --git a/fp_ssl.c b/fp_ssl.c
 new file mode 100644
 index 0000000..6bffca6
 --- /dev/null
 +++ b/fp_ssl.c
 @@ -0,0 +1,974 @@
 +/* -*-mode:c; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
 +/*
 +  p0f - SSL fingerprinting
 +  -------------------------
 +
 +  Copyright (C) 2012 by Marek Majkowski <[email protected]>
 +
 +  Distributed under the terms and conditions of GNU LGPL.
 +
 +*/
 +
 +#define _FROM_FP_SSL
 +#define _GNU_SOURCE
 +
 +#include <stdio.h>
 +#include <stdlib.h>
 +#include <unistd.h>
 +#include <ctype.h>
 +#include <time.h>
 +
 +#include <netinet/in.h>
 +#include <sys/types.h>
 +
 +#include "types.h"
 +#include "config.h"
 +#include "debug.h"
 +#include "alloc-inl.h"
 +#include "process.h"
 +#include "readfp.h"
 +#include "p0f.h"
 +#include "tcp.h"
 +#include "hash.h"
 +
 +#include "fp_ssl.h"
 +
 +/* Flags for SSL signaturs */
 +struct flag {
 +  char* name;
 +  int name_len;
 +  u32 value;
 +};
 +
 +struct flag flags[] = {{"compr", 5, SSL_FLAG_COMPR},
 +                       {"v2",    2, SSL_FLAG_V2},
 +                       {"ver",   3, SSL_FLAG_VER},
 +                       {"rtime", 5, SSL_FLAG_RTIME},
 +                       {"stime", 5, SSL_FLAG_STIME},
 +                       {"rand",  4, SSL_FLAG_RAND},
 +                       {"chlen", 5, SSL_FLAG_CHLEN},
 +                       {NULL, 0, 0}};
 +
 +
 +/* Signatures are stored as flat list. Matching is fast: ssl version
 +   and flags must match exactly, matching ciphers and extensions
 +   usually require looking only at a first few bytes of the
 +   signature. Assuming the signature doesn't start with a star. */
 +
 +static struct ssl_sig_record* signatures;
 +static u32 signatures_cnt;
 +
 +
 +/* Decode a string of comma separated hex numbers into an annotated
 +   u32 array. Exit with success on '\0' or ':'. */
 +
 +static u32* decode_hex_string(const u8** val_ptr, u32 line_no) {
 +
 +  const u8* val = *val_ptr;
 +
 +  u32 rec[SSL_MAX_CIPHERS];
 +  u8 p = 0;
 +
 +  while (p < SSL_MAX_CIPHERS) {
 +
 +    u32 optional = 0;
 +    const u8* prev_val;
 +    u32* ret;
 +
 +    /* State #1: expecting value */
 +
 +    switch (*val) {
 +
 +    case '*':
 +      rec[p++] = MATCH_ANY;
 +      val ++;
 +      break;
 +
 +    case '?':
 +      optional = MATCH_MAYBE;
 +      val ++;
 +      /* Must be a hex digit after question mark */
 +    case 'a' ... 'f':
 +    case '0' ... '9':
 +      prev_val = val;
 +      rec[p++] = (strtol((char*)val, (char**)&val, 16) & 0xFFFFFF) | optional;
 +      if (val == prev_val) return NULL;
 +      break;
 +
 +    default:
 +      /* Support empty list - jump to second state. */
 +      if (p == 0)
 +        break;
 +
 +      return NULL;
 +
 +    }
 +
 +    /* State #2: comma, expecting '\0' or ':' */
 +
 +    switch (*val) {
 +
 +    case ':':
 +    case '\0':
 +      *val_ptr = val;
 +      ret = DFL_ck_alloc((p + 1) * sizeof(u32));
 +      memcpy(ret, rec, p * sizeof(u32));
 +      ret[p] = END_MARKER;
 +      return ret;
 +
 +    case ',':
 +      val ++;
 +      break;
 +
 +    default:
 +      return NULL;
 +
 +    }
 +
 +  }
 +
 +  FATAL("Too many ciphers or extensions in line %u.", line_no);
 +
 +}
 +
 +
 +/* Is u32 list of ciphers/extensions matching the signature?
 +   first argument is record (star and question mark allowed),
 +   second one is an exact signature. */
 +
 +static int match_sigs(u32* rec, u32* sig) {
 +
 +  u8 match_any = 0;
 +  u32* tmp_sig;
 +
 +  /* Iterate over record. */
 +
 +  for (; *rec != END_MARKER && *sig != END_MARKER; rec++) {
 +
 +    /* 1. Exact match, move on */
 +    if ((*rec & ~MATCH_MAYBE) == *sig) {
 +      match_any = 0; sig++;
 +      continue;
 +    }
 +
 +    /* 2. Star, may match anything */
 +    if (*rec == MATCH_ANY) {
 +      match_any = 1;
 +      continue;
 +    }
 +
 +    /* 3. Optional match, not yet fulfilled */
 +    if (*rec & MATCH_MAYBE) {
 +      if (match_any) {
 +        /* Look forward for the value (aka: greedy match). */
 +        for (tmp_sig = sig; *tmp_sig != END_MARKER; tmp_sig++) {
 +          if ((*rec & ~MATCH_MAYBE) == *tmp_sig) {
 +            /* Got it. */
 +            match_any = 0; sig = tmp_sig + 1;
 +            break;
 +          }
 +        }
 +      }
 +      /* Loop succeeded or optional match failed, whatever, go on. */
 +      continue;
 +    }
 +
 +    /* 4. Looking for an exact match after MATCH_ANY */
 +    if (match_any) {
 +      for (; *sig != END_MARKER; sig++) {
 +        if (*rec == *sig) {
 +          sig ++;
 +          break;
 +        }
 +      }
 +      /* Sig is next char after match or END_MARKER */
 +      match_any = 0;
 +      continue;
 +    }
 +
 +    /* 5. Nope, not matched. */
 +    return 1;
 +
 +  }
 +
 +  /* Right, we're after the loop, either rec or sig are set to END_MARKER */
 +
 +  /* Step 1. Roll rec while it has conditional matches.
 +             Sig is END_MARKER if rec is not done.  */
 +  for (;(*rec & MATCH_MAYBE) || *rec == MATCH_ANY; rec ++) {};
 +
 +  /* Step 2. Both finished - hurray. */
 +  if (*rec == END_MARKER && *sig == END_MARKER)
 +    return 0;
 +
 +  /* Step 3. Rec is done and we're in MATCH_ANY mode - hurray. */
 +  if (*rec == END_MARKER && match_any)
 +    return 0;
 +
 +  /* Step 4. Nope. */
 +  return 1;
 +
 +}
 +
 +
 +static void ssl_find_match(struct ssl_sig* ts) {
 +
 +  u32 i;
 +
 +  for (i = 0; i < signatures_cnt; i++) {
 +
 +    struct ssl_sig_record* ref = &signatures[i];
 +    struct ssl_sig* rs = CP(ref->sig);
 +
 +    /* SSL versions must match exactly. */
 +    if (rs->request_version != ts->request_version) continue;
 +
 +    /* Flags - exact match */
 +    if (ts->flags != rs->flags) continue;
 +
 +    /* Extensions match. */
 +    if (match_sigs(rs->extensions, ts->extensions) != 0) continue;
 +
 +    /* Cipher suites match. */
 +    if (match_sigs(rs->cipher_suites, ts->cipher_suites) != 0) continue;
 +
 +    ts->matched = ref;
 +    return;
 +
 +  }
 +
 +}
 +
 +
 +/* Unpack SSLv2 header to a signature.
 +   -1 on parsing error, 1 if signature was extracted. */
 +
 +static int fingerprint_ssl_v2(struct ssl_sig* sig, const u8* pay, u32 pay_len) {
 +
 +  const u8* pay_end = pay + pay_len;
 +  const u8* tmp_end;
 +
 +  if (pay + sizeof(struct ssl2_hdr) > pay_end) goto too_short;
 +
 +  struct ssl2_hdr* hdr = (struct ssl2_hdr*)pay;
 +  pay += sizeof(struct ssl2_hdr);
 +
 +  if (hdr->ver_min == 2 && hdr->ver_maj == 0) {
 +
 +    /* SSLv2 is actually 0x0002 on the wire. */
 +    sig->request_version = 0x0200;
 +
 +  } else {
 +
 +    /* Most often - SSLv2 header has request version set to 3.x */
 +    sig->request_version = (hdr->ver_maj << 8) | hdr->ver_min;
 +
 +  }
 +
 +
 +  u16 cipher_spec_len = ntohs(hdr->cipher_spec_length);
 +
 +  if (cipher_spec_len % 3) {
 +
 +    DEBUG("[#] SSLv2 cipher_spec_len=%u is not divisable by 3.\n",
 +          cipher_spec_len);
 +    return -1;
 +
 +  }
 +
 +  if (pay + cipher_spec_len > pay_end) goto too_short;
 +
 +  int cipher_pos = 0;
 +  sig->cipher_suites = ck_alloc(((cipher_spec_len / 3) + 1) * sizeof(u32));
 +  tmp_end = pay + cipher_spec_len;
 +
 +  while (pay < tmp_end) {
 +
 +    sig->cipher_suites[cipher_pos++] =
 +      (pay[0] << 16) | (pay[1] << 8) | pay[2];
 +    pay += 3;
 +
 +  }
 +  sig->cipher_suites[cipher_pos] = END_MARKER;
 +
 +
 +  u16 session_id_len = ntohs(hdr->session_id_length);
 +  u16 challenge_len = ntohs(hdr->challenge_length);
 +
 +  /* Although SSLv2 states that challenge must be between 16 and 32
 +     bytes long, in practice no other values were recorded. 32 seems
 +     to be more popular than 16. */
 +
 +  if (challenge_len != 32) {
 +
 +    sig->flags |= SSL_FLAG_CHLEN;
 +
 +    if (challenge_len != 16) {
 +      DEBUG("[#] SSLv2 challenge_len %i req_ver=%04x\n", challenge_len,
 +            sig->request_version);
 +    }
 +
 +  }
 +
 +  if (pay + session_id_len + challenge_len > pay_end) {
 +
 +    DEBUG("[#] SSLv2 frame truncated (but valid) req_ver=%04x\n",
 +          sig->request_version);
 +    goto truncated;
 +
 +  }
 +
 +  pay += session_id_len;
 +
 +  u32 i;
 +  u32* challenge = (u32*)pay;
 +
 +  for (i = 0; i < challenge_len/4; i++) {
 +    if (challenge[i] == 0x00000000 || challenge[i] == 0xffffffff) {
 +
 +      sig->flags |= SSL_FLAG_RAND;
 +      break;
 +
 +    }
 +  }
 +
 +
 +  pay += challenge_len;
 +
 +  if (pay != pay_end) {
 +
 +    DEBUG("[#] SSLv2 extra %u bytes remaining after client-hello message.\n",
 +          pay_end - pay);
 +
 +  }
 +
 +truncated:
 +
 +  sig->extensions = ck_alloc(1 * sizeof(u32));
 +  sig->extensions[0] = END_MARKER;
 +
 +  return 1;
 +
 +
 +too_short:
 +
 +  DEBUG("[#] SSLv2 frame too short.\n");
 +
 +  ck_free(sig->cipher_suites);
 +  ck_free(sig->extensions);
 +
 +  return -1;
 +
 +}
 +
 +
 +/* Unpack SSLv3 fragment to a signature. We expect to hear ClientHello
 + message.  -1 on parsing error, 1 if signature was extracted. */
 +
 +static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
 +                              u32 frag_len, u16 record_version) {
 +
 +  int i;
 +  const u8* frag_end = fragment + frag_len;
 +
 +  struct ssl3_message_hdr* msg = (struct ssl3_message_hdr*)fragment;
 +  u32 msg_len = (msg->length[0] << 16) |
 +                (msg->length[1] << 8) |
 +                (msg->length[2]);
 +
 +  const u8* pay = (const u8*)msg + sizeof(struct ssl3_message_hdr);
 +  const u8* pay_end = pay + msg_len;
 +  const u8* tmp_end;
 +
 +
 +  /* Record size goes beyond current fragment, it's fine by SSL but
 +     not for us. */
 +
 +  if (pay_end > frag_end) {
 +    /* I've seen a packet which looked like fragmented, but upon
 +       inspection I noticed that 4th byte of data had been cleared
 +       (high bits of frame length). Weird, ain't? */
 +
 +    DEBUG("[#] SSL Fragment coalescing not supported - %u bytes requested.\n",
 +          pay_end - frag_end);
 +
 +    return -1;
 +
 +  }
 +
 +  if (msg->message_type != SSL3_MSG_CLIENT_HELLO) {
 +
 +    /* Rfc526 says: The handshake protocol messages are presented
 +         below in the order they MUST be sent; sending handshake
 +         messages in an unexpected order results in a fatal error.
 +
 +       I guess we can assume that the first frame must be ClientHello.
 +    */
 +
 +    DEBUG("[#] SSL Message type 0x%02x (%u bytes) is not ClientHello.\n",
 +          msg->message_type, msg_len);
 +    return -1;
 +
 +  }
 +
 +
 +  /* ClientHello */
 +
 +
 +  /* Header (34B) + session_id_len (1B) */
 +
 +  if (pay + 2 + 4 + 28 + 1 > pay_end) goto too_short;
 +
 +  sig->request_version = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (sig->request_version != record_version) {
 +    sig->flags |= SSL_FLAG_VER;
 +  }
 +
 +  sig->remote_time = ntohl(*((u32*)pay));
 +  pay += 4;
 +
 +  s64 drift = ((s64)sig->recv_time) - sig->remote_time;
 +
 +  if (sig->remote_time < 1*365*24*60*60) {
 +
 +    /* Old Firefox on windows uses time since boot */
 +    sig->flags |= SSL_FLAG_STIME;
 +
 +  } else if (abs(drift) > 5*365*24*60*60) {
 +
 +    /* More than 5 years difference - most likely random */
 +    sig->flags |= SSL_FLAG_RTIME;
 +
 +    DEBUG("[#] SSL timer looks random: drift=%lld remote_time=%u.\n",
 +          drift, sig->remote_time);
 +
 +  }
 +
 +  /* Random */
 +  u32* random = (u32*)pay;
 +  pay += 28;
 +
 +  for (i = 0; i < 7; i++) {
 +    if (random[i] == 0x00000000 || random[i] == 0xffffffff) {
 +
 +      sig->flags |= SSL_FLAG_RAND;
 +      break;
 +
 +    }
 +  }
 +
 +  /* Skip session_id */
 +  u8 session_id_len = pay[0];
 +  pay += 1;
 +
 +  if (pay + session_id_len + 2 > pay_end) goto too_short;
 +
 +  pay += session_id_len;
 +
 +
 +  /* Cipher suites */
 +
 +  u16 cipher_suites_len = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (cipher_suites_len % 2) {
 +
 +    DEBUG("[#] SSL cipher_suites_len=%u is not even.\n", cipher_suites_len);
 +    return -1;
 +
 +  }
 +
 +  if (pay + cipher_suites_len > pay_end) goto too_short;
 +
 +  int cipher_pos = 0;
 +  sig->cipher_suites = ck_alloc(((cipher_suites_len / 2) + 1) * sizeof(u32));
 +  tmp_end = pay + cipher_suites_len;
 +
 +  while (pay < tmp_end) {
 +
 +    sig->cipher_suites[cipher_pos++] = (pay[0] << 8) | pay[1];
 +    pay += 2;
 +
 +  }
 +  sig->cipher_suites[cipher_pos] = END_MARKER;
 +
 +  if (pay + 1 > pay_end) goto truncated;
 +
 +  u8 compression_methods_len = pay[0];
 +  pay += 1;
 +
 +  if (pay + compression_methods_len > pay_end) goto truncated;
 +
 +  tmp_end = pay + compression_methods_len;
 +
 +  while (pay < tmp_end) {
 +
 +    if (pay[0] == 1) {
 +      sig->flags |= SSL_FLAG_COMPR;
 +    } else {
 +      if (pay[0] || pay + 1 != tmp_end)
 +        DEBUG("[#] SSL unknown compression method 0x%x at offset %i/%i.\n",
 +              pay[0], tmp_end - pay, compression_methods_len);
 +    }
 +
 +    pay += 1;
 +
 +  }
 +
 +
 +  if (pay + 2 > pay_end) {
 +
 +    /* SSL 3.0 and all versions of TLS allow the ClientHello message
 +       to be truncated after compression_methods and not specify
 +       extensions field at all. Although rarely, this does seem to
 +       occur in real world. Semantically it is no different to sending
 +       an empty list of extensions. In fact, no client was ever
 +       recorded to send an empty list of extensions if the extensions
 +       field was present. In other words - if the extensions field is
 +       empty you can assume the ClientHello packet was truncated after
 +       compression_methods. */
 +    goto truncated_ok;
 +
 +  }
 +
 +  u16 extensions_len = (pay[0] << 8) | pay[1];
 +  pay += 2;
 +
 +  if (pay + extensions_len > pay_end) goto truncated;
 +
 +  int extensions_pos = 0;
 +  sig->extensions = ck_alloc(((extensions_len / 4) + 1) * sizeof(u32));
 +  tmp_end = pay + extensions_len;
 +
 +  while (pay + 4 <= tmp_end) {
 +
 +    u16 ext_type = (pay[0] << 8) | pay[1];
 +    u16 ext_len  = (pay[2] << 8) | pay[3];
 +    const u8* extension = &pay[4];
 +    pay += 4;
 +
 +    pay += ext_len;
 +
 +    sig->extensions[extensions_pos++] = ext_type;
 +
 +    /* Extension payload sane? */
 +    if (pay > tmp_end) break;
 +
 +    /* Ignore the actual value of the extenstion. */
 +    extension = extension;
 +  }
 +
 +  /* Make sure the terminator is always appended, even if extensions
 +     are malformed. */
 +  sig->extensions = ck_realloc(sig->extensions, (extensions_pos + 1) *
 +                               sizeof(u32));
 +  sig->extensions[extensions_pos] = END_MARKER;
 +
 +  if (pay != tmp_end) {
 +
 +    DEBUG("[#] SSL malformed extensions, %i bytes over.\n",
 +          pay - tmp_end);
 +
 +  }
 +
 +  if (pay != pay_end) {
 +
 +    DEBUG("[#] SSL ClientHello remaining %i bytes after extensions.\n",
 +          pay_end - pay);
 +
 +  }
 +
 +  if (pay_end != frag_end) {
 +
 +    DEBUG("[#] SSL %i bytes remaining after ClientHello message.\n",
 +          frag_end - pay_end);
 +
 +  }
 +
 +  if (0) {
 +truncated:
 +
 +    DEBUG("[#] SSL packet truncated (but valid) req_ver=%04x rec_ver=%04x\n",
 +          sig->request_version, record_version);
 +
 +  }
 +truncated_ok:
 +
 +  if (!sig->extensions) {
 +    sig->extensions    = ck_alloc(1*sizeof(u32));
 +    sig->extensions[0] = END_MARKER;
 +  }
 +
 +  return 1;
 +
 +
 +too_short:
 +
 +  DEBUG("[#] SSL packet truncated.\n");
 +
 +  ck_free(sig->cipher_suites);
 +  ck_free(sig->extensions);
 +
 +  return -1;
 +
 +}
 +
 +
 +/* Signature - to - string */
 +
 +static u8* dump_sig(struct ssl_sig* sig, u8 fingerprint) {
 +
 +  int i;
 +
 +  static u8* ret;
 +  u32 rlen = 0;
 +
 +#define RETF(_par...) do {                           \
 +    s32 _len = snprintf(NULL, 0, _par);              \
 +    if (_len < 0) FATAL("Whoa, snprintf() fails?!"); \
 +    ret = DFL_ck_realloc_kb(ret, rlen + _len + 1);   \
 +    snprintf((char*)ret + rlen, _len + 1, _par);     \
 +    rlen += _len;                                    \
 +  } while (0)
 +
 +  RETF("%i.%i:", sig->request_version >> 8, sig->request_version & 0xFF);
 +
 +  for (i = 0; sig->cipher_suites[i] != END_MARKER; i++) {
 +    u32 c = sig->cipher_suites[i];
 +    if (c != MATCH_ANY) {
 +      RETF("%s%s%x", (i ? "," : ""),
 +           (c & MATCH_MAYBE) ? "?" : "",
 +           c & ~MATCH_MAYBE);
 +    } else {
 +      RETF("%s*", (i ? "," : ""));
 +    }
 +  }
 +
 +  RETF(":");
 +
 +  for (i = 0; sig->extensions[i] != END_MARKER; i++) {
 +    u32 ext = sig->extensions[i];
 +    if (ext != MATCH_ANY) {
 +      u8 optional = 0;
 +      if (fingerprint && ext == 0) {
 +        optional = 1;
 +      }
 +      RETF("%s%s%x", (i ? "," : ""),
 +           ((ext & MATCH_MAYBE) || optional) ? "?" : "",
 +           ext & ~MATCH_MAYBE);
 +    } else {
 +      RETF("%s*", (i ? "," : ""));
 +    }
 +  }
 +
 +  RETF(":");
 +
 +  int had_prev = 0;
 +  for (i = 0; flags[i].name != NULL; i++) {
 +
 +    if (sig->flags & flags[i].value) {
 +      RETF("%s%s", (had_prev ? "," : ""), flags[i].name);
 +      had_prev = 1;
 +    }
 +
 +  }
 +
 +  return ret;
 +
 +}
 +
 +
 +/* Register new SSL signature. */
 +
 +void ssl_register_sig(u8 to_srv, u8 generic, s32 sig_class, u32 sig_name,
 +                      u8* sig_flavor, u32 label_id, u32* sys, u32 sys_cnt,
 +                      u8* val, u32 line_no) {
 +
 +  struct ssl_sig* ssig;
 +  struct ssl_sig_record* srec;
 +
 +  /* Client signatures only. */
 +  if (to_srv != 1) return;
 +
 +  /* Only "application" signatures supported, no "OS-identifying". */
 +  if (sig_class != -1)
 +    FATAL("OS-identifying SSL signatures not supported, use \"!\" instead "
 +          "of an OS class in line %u.", line_no);
 +
 +  ssig = DFL_ck_alloc(sizeof(struct ssl_sig));
 +
 +  signatures = DFL_ck_realloc(signatures, (signatures_cnt + 1) *
 +                              sizeof(struct ssl_sig_record));
 +
 +  srec = &signatures[signatures_cnt];
 +
 +
 +  int maj = strtol((char*)val, (char**)&val, 10);
 +  if (!val || *val != '.') FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +  int min = strtol((char*)val, (char**)&val, 10);
 +  if (!val || *val != ':') FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +  ssig->request_version = (maj << 8) | min;
 +
 +  ssig->cipher_suites = decode_hex_string((const u8**)&val, line_no);
 +  if (!val || *val != ':' || !ssig->cipher_suites)
 +    FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +  ssig->extensions = decode_hex_string((const u8**)&val, line_no);
 +  if (!val || *val != ':' || !ssig->extensions)
 +    FATAL("Malformed signature in line %u.", line_no);
 +  val ++;
 +
 +
 +  while (*val) {
 +
 +    int i;
 +    for (i = 0;  flags[i].name != NULL; i++) {
 +
 +      if (!strncmp((char*)val, flags[i].name, flags[i].name_len)) {
 +        ssig->flags |= flags[i].value;
 +        val += flags[i].name_len;
 +        goto flag_matched;
 +      }
 +
 +    }
 +
 +    FATAL("Unrecognized flag in line %u.", line_no);
 +
 +  flag_matched:
 +
 +    if (*val == ',') val++;
 +
 +  }
 +
 +  srec->class_id = sig_class;
 +  srec->name_id  = sig_name;
 +  srec->flavor   = sig_flavor;
 +  srec->label_id = label_id;
 +  srec->sys      = sys;
 +  srec->sys_cnt  = sys_cnt;
 +  srec->line_no  = line_no;
 +  srec->generic  = generic;
 +
 +  srec->sig      = ssig;
 +
 +  signatures_cnt++;
 +
 +}
 +
 +static void score_nat(u8 to_srv, struct packet_flow* f, struct ssl_sig* sig) {
 +
 +  struct ssl_sig_record* m = sig->matched;
 +  struct host_data* hd;
 +
 +  u8  score = 0;
 +  u16 reason = 0;
 +
 +  /* Client request only. */
 +  if (to_srv != 1) return;
 +
 +  hd = f->client;
 +
 +
 +  if (m && m->class_id == -1) {
 +
 +    /* Application signature: we might look at the OS-es mentioned by
 +       the signature, and make sure the OS from http and/or TCP
 +       matches. */
 +
 +    verify_tool_class(to_srv, f, m->sys, m->sys_cnt);
 +
 +  }
 +
 +  if (hd->ssl_remote_time && sig->remote_time &&
 +      (sig->flags & SSL_FLAG_RTIME) == 0) {
 +
 +    /* Time on the client should be increasing monotically */
 +
 +    s64 recv_diff    = ((s64)sig->recv_time) - hd->ssl_recv_time;
 +    s64 remote_diff  = ((s64)sig->remote_time) - hd->ssl_remote_time;
 +
 +    if (remote_diff < recv_diff - SSL_MAX_TIME_DIFF ||
 +        remote_diff > recv_diff + SSL_MAX_TIME_DIFF) {
 +
 +      DEBUG("[#] SSL gmt_unix_time skew too high (%lld in %lld sec).\n",
 +            remote_diff, recv_diff);
 +
 +      score += 4;
 +      reason |= NAT_APP_DATE;
 +
 +    }
 +
 +  }
 +
 +  add_nat_score(to_srv, f, reason, score);
 +
 +}
 +
 +
 +
 +/* Given an SSL client signature look it up and create an observation.  */
 +
 +static void fingerprint_ssl(u8 to_srv, struct packet_flow* f,
 +                            struct ssl_sig* sig) {
 +
 +  /* Client request only. */
 +  if (to_srv != 1) return;
 +
 +  ssl_find_match(sig);
 +
 +  struct ssl_sig_record* m = sig->matched;
 +
 +  start_observation("ssl request", 5, to_srv, f);
 +
 +  if (m) {
 +
 +    /* Found matching signature */
 +
 +    OBSERVF((m->class_id < 0) ? "app" : "os", "%s%s%s",
 +            fp_os_names[m->name_id], m->flavor ? " " : "",
 +            m->flavor ? m->flavor : (u8*)"");
 +
 +    add_observation_field("match_sig", dump_sig(sig->matched->sig, 0));
 +
 +  } else {
 +
 +    add_observation_field("app", NULL);
 +    add_observation_field("match_sig", NULL);
 +
 +  }
 +
 +  if ((sig->flags & (SSL_FLAG_RTIME | SSL_FLAG_STIME)) == 0) {
 +
 +    s64 drift = ((s64)sig->recv_time) - sig->remote_time;
 +    OBSERVF("drift", "%lld", drift);
 +
 +  } else {
 +
 +    add_observation_field("drift", NULL);
 +
 +  }
 +
 +  OBSERVF("remote_time", "%u", sig->remote_time);
 +
 +  add_observation_field("raw_sig", dump_sig(sig, 1));
 +
 +  score_nat(to_srv, f, sig);
 +
 +}
 +
 +
 +/* Examine request or response; returns 1 if more data needed and
 +   plausibly can be read. Note that the buffer is always NULL
 +   terminated. */
 +
 +u8 process_ssl(u8 to_srv, struct packet_flow* f) {
 +
 +  int success = 0;
 +  struct ssl_sig sig;
 +
 +
 +  /* Already decided this flow? */
 +
 +  if (f->in_ssl) return 0;
 +
 +
 +  /* Tracking requests only. */
 +
 +  if (!to_srv) return 0;
 +
 +
 +  u8 can_get_more = (f->req_len < MAX_FLOW_DATA);
 +
 +
 +  /* SSLv3 record is 5 bytes, message is 4 + 38; SSLv2 CLIENT-HELLO is
 +     11 bytes - we try to recognize protocol by looking at top 6
 +     bytes. */
 +
 +  if (f->req_len < 6) return can_get_more;
 +
 +  struct ssl2_hdr* hdr2 = (struct ssl2_hdr*)f->request;
 +  u16 msg_length = ntohs(hdr2->msg_length);
 +
 +  struct ssl3_record_hdr* hdr3 = (struct ssl3_record_hdr*)f->request;
 +  u16 fragment_len = ntohs(hdr3->length);
 +
 +
 +  /* Does it look like top 5 bytes of SSLv2? Most significant bit must
 +     be set, followed by 15 bits indicating record length, which must
 +     be at least 9. */
 +
 +  if ((msg_length & 0x8000) &&
 +      (msg_length & ~0x8000) >= sizeof(struct ssl2_hdr) - 2 &&
 +      hdr2->msg_type == 1 &&
 +      ((hdr2->ver_maj == 3 && hdr2->ver_min < 4) ||
 +       (hdr2->ver_min == 2 && hdr2->ver_maj == 0))) {
 +
 +    /* Clear top bit. */
 +    msg_length &= ~0x8000;
 +
 +    if (f->req_len < 2 + msg_length) return can_get_more;
 +
 +    memset(&sig, 0, sizeof(struct ssl_sig));
 +    sig.recv_time = f->client->last_seen;
 +    sig.flags |= SSL_FLAG_V2;
 +
 +    success = fingerprint_ssl_v2(&sig, f->request, msg_length + 2);
 +
 +  }
 +
 +
 +  /* Top 5 bytes of SSLv3/TLS header? Currently available TLS
 +     versions: 3.0 - 3.3. The rfc disallows fragment to have more than
 +     2^14 bytes. Also length less than 4 bytes doesn't make much
 +     sense. Additionally let's peek the meesage type. */
 +
 +  else if (hdr3->content_type == SSL3_REC_HANDSHAKE &&
 +           hdr3->ver_maj == 3 && hdr3->ver_min < 4 &&
 +           fragment_len > 3 && fragment_len < (1 << 14) &&
 +           f->request[5] == SSL3_MSG_CLIENT_HELLO) {
 +
 +    if (f->req_len < sizeof(struct ssl3_record_hdr) + fragment_len)
 +      return can_get_more;
 +
 +    memset(&sig, 0, sizeof(struct ssl_sig));
 +    sig.recv_time = f->client->last_seen;
 +    u16 record_version = (hdr3->ver_maj << 8) | hdr3->ver_min;
 +
 +    u8* fragment = f->request + sizeof(struct ssl3_record_hdr);
 +
 +    success = fingerprint_ssl_v3(&sig, fragment, fragment_len, record_version);
 +
 +  }
 +
 +  if (success != 1) {
 +
 +    DEBUG("[#] Does not look like SSLv2 nor SSLv3.\n");
 +
 +    f->in_ssl = -1;
 +    return 0;
 +
 +  }
 +
 +
 +  f->in_ssl = 1;
 +
 +  fingerprint_ssl(to_srv, f, &sig);
 +
 +  if (sig.remote_time && !(sig.flags & SSL_FLAG_RTIME)) {
 +    f->client->ssl_remote_time = sig.remote_time;
 +    f->client->ssl_recv_time   = sig.recv_time;
 +  }
 +
 +
 +  ck_free(sig.cipher_suites);
 +  ck_free(sig.extensions);
 +
 +  return 0;
 +
 +}
 diff --git a/fp_ssl.h b/fp_ssl.h
 new file mode 100644
 index 0000000..8eb1026
 --- /dev/null
 +++ b/fp_ssl.h
 @@ -0,0 +1,125 @@
 +/* -*-mode:c; tab-width: 8; indent-tabs-mode: nil; c-basic-offset: 2 -*- */
 +/*
 +  p0f - SSL fingerprinting
 +  -------------------------
 +
 +  Copyright (C) 2012 by Marek Majkowski <[email protected]>
 +
 +  Distributed under the terms and conditions of GNU LGPL.
 +
 + */
 +
 +#ifndef _HAVE_FP_SSL_H
 +#define _HAVE_FP_SSL_H
 +
 +#include "types.h"
 +
 +
 +/* Constants */
 +
 +#define MATCH_MAYBE 0x10000000  /* '?' - indicats a single optional match */
 +#define MATCH_ANY   0x20000000  /* '*' - match zero or more elements */
 +#define END_MARKER  0x40000000  /* internal marker */
 +
 +#define SSL_MAX_CIPHERS 128     /* max number of ciphers in a signature */
 +#define SSL_MAX_TIME_DIFF 10    /* remote clock scew limit, in seconds */
 +
 +
 +/* Flags */
 +
 +#define SSL_FLAG_V2    0x0001  /* SSLv2 handshake. */
 +#define SSL_FLAG_VER   0x0002  /* Record version different than ClientHello. */
 +#define SSL_FLAG_RTIME 0x0004  /* weird SSL time, (delta > 5 years), most likely random*/
 +#define SSL_FLAG_STIME 0x0008  /* small SSL time, (absolute value < 1 year)
 +                                  most likely time since reboot for old ff */
 +#define SSL_FLAG_COMPR 0x0010  /* Deflate compression support. */
 +#define SSL_FLAG_RAND  0x0020  /* Random data doesn't look randomly. */
 +#define SSL_FLAG_CHLEN 0x0040  /* SSLv2 challenge is not 16 bytes */
 +
 +
 +/* SSLv2 */
 +
 +struct ssl2_hdr {
 +
 +  u16 msg_length;
 +  u8 msg_type;
 +  u8 ver_maj;
 +  u8 ver_min;
 +
 +  u16 cipher_spec_length;
 +  u16 session_id_length;
 +  u16 challenge_length;
 +
 +} __attribute__((packed));
 +
 +
 +/* SSLv3 */
 +
 +#define SSL3_REC_HANDSHAKE 0x16    /* 22 */
 +#define SSL3_MSG_CLIENT_HELLO 0x01
 +
 +struct ssl3_record_hdr {
 +
 +  u8 content_type;
 +  u8 ver_maj;
 +  u8 ver_min;
 +  u16 length;
 +
 +} __attribute__((packed));
 +
 +
 +struct ssl3_message_hdr {
 +
 +  u8 message_type;
 +  u8 length[3];
 +
 +} __attribute__((packed));
 +
 +
 +
 +/* Internal data structures */
 +
 +struct ssl_sig_record;
 +
 +struct ssl_sig {
 +
 +  u16 request_version;   /* Requested SSL version (maj << 8) | min */
 +
 +  u32 remote_time;       /* ClientHello message gmt_unix_time field */
 +  u32 recv_time;         /* Actual receive time */
 +
 +  u32* cipher_suites;    /* List of SSL ciphers, END_MARKER terminated */
 +
 +  u32* extensions;       /* List of SSL extensions, END_MARKER terminated */
 +
 +  u32 flags;             /* SSL flags */
 +
 +  struct ssl_sig_record* matched; /* NULL = no match */
 +};
 +
 +struct ssl_sig_record {
 +
 +  s32 class_id;                         /* OS class ID (-1 = user)            */
 +  s32 name_id;                          /* OS name ID                         */
 +  u8* flavor;                           /* Human-readable flavor string       */
 +
 +  u32 label_id;                         /* Signature label ID                 */
 +
 +  u32* sys;                             /* OS class / name IDs for user apps  */
 +  u32  sys_cnt;                         /* Length of sys                      */
 +
 +  u32  line_no;                         /* Line number in p0f.fp              */
 +
 +  u8 generic;                           /* Generic signature?                 */
 +
 +  struct ssl_sig* sig;                  /* Actual signature data              */
 +
 +};
 +
 +void ssl_register_sig(u8 to_srv, u8 generic, s32 sig_class, u32 sig_name,
 +                      u8* sig_flavor, u32 label_id, u32* sys, u32 sys_cnt,
 +		      u8* val, u32 line_no);
 +
 +u8 process_ssl(u8 to_srv, struct packet_flow* f);
 +
 +#endif /* _HAVE_FP_SSL_H */
 diff --git a/p0f.fp b/p0f.fp
 index d02dc5f..acae4d0 100644
 --- a/p0f.fp
 +++ b/p0f.fp
 @@ -903,3 +903,311 @@ sys   = Linux
 sig   = *:Content-Type,X-Content-Type-Options=[nosniff],Date,Server=[sffe]:Connection,Accept-Ranges,Keep-Alive,Connection:
 sig   = *:Date,Content-Type,Server=[gws]:Connection,Accept-Ranges,Keep-Alive:
 sig   = *:Content-Type,X-Content-Type-Options=[nosniff],Server=[GSE]:Connection,Accept-Ranges,Keep-Alive:
 +
 +; =====================
 +; SSL client signatures
 +; =====================
 +
 +[ssl:request]
 +
 +;-----------------
 +; Windows specific
 +;-----------------
 +
 +; Windows NT 5.1, Windows NT 5.2 (XP)
 +label = s:!:any:MSIE or Safari on Windows XP
 +sys   = Windows
 +sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63:ff01:
 +; no MS10-049 applied
 +sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63,?ff::
 +sig   = 3.0:4,5,a,9,64,62,3,6,13,12,63,?ff::
 +
 +; with some SSL/TLS options tweaked
 +sig   = 3.0:4,5,a,10080,700c0,30080,9,60040,64,62,3,6,20080,40080,13,12,63,?ff::v2,chlen
 +sig   = 2.0:10080,700c0,30080,60040,20080,40080,ff::v2,chlen
 +
 +
 +; Windows NT 6.0 (Vista)
 +label = s:!:any:MSIE 5.5-6 or Chrome 1-4 or Safari on Windows Vista
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,a,b,ff01:
 +
 +label = s:!:any:MSIE 7-9 or Chrome 5 on Windows Vista
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,5,a,b,ff01:
 +
 +; look out for order of ff01 + time
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:stime
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:rtime
 +
 +
 +; Windows NT 6.1 (7)
 +label = s:!:MSIE:7-9 on Windows 7
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b:
 +; Weird IE9 on win 7: MSIE 9.0; Windows NT 6.1
 +sig   = 3.0:5,a,13,4,10080,700c0,?ff::v2,chlen
 +sig   = 3.0:5,a,13,4,?ff::
 +sig   = 3.0:5,a,13,4,?ff::v2,chlen
 +
 +label = s:!:Safari:on Windows 7
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b:
 +
 +
 +; Windows NT 6.2 ( 8)
 +; 23 usually means NT 6.2
 +label = s:!:MSIE:10 on Windows 8
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b,23:
 +
 +label = s:!:Safari:on Windows 8
 +sys   = Windows
 +sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b,23:
 +
 +
 +; ------
 +; Chrome
 +; ------
 +
 +; old chrome - TLS 1.0
 +label = s:!:Chrome:6-20
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:compr
 +
 +; newer chrome - TLS 1.1
 +label = s:!:Chrome:21
 +sys   = Windows,@unix
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:compr
 +
 +; newest chrome - no compr support
 +label = s:!:Chrome:22 or newer
 +sys   = Windows,@unix
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:ver
 +
 +
 +label = s:!:Chrome:degraded to SSL v3.0
 +sys   = Windows,@unix
 +sig   = 3.0:ff,88,87,39,38,84,35,45,44,66,33,32,96,41,4,5,2f,16,13,feff,a::
 +
 +; -------
 +; Firefox
 +; -------
 +
 +label = s:!:Firefox:1.X
 +sys   = Windows,@unix
 +sig   = 3.1:10080,30080,*,40080,39,38,35,*,64,62,3,6::v2,chlen
 +sig   = 3.1:39,38,35,*,64,62,3,6::stime
 +
 +label = s:!:Firefox:2.X
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,39,38,c00f,*,c00d,c003,feff,a:?0,a,b:
 +
 +label = s:!:Firefox:3.0-3.5
 +sys   = Windows,@unix
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,c005,84,35,c007,*,c003,feff,a:?0,a,b,23:
 +
 +label = s:!:Firefox:3.6.X
 +sys   = Windows,@unix
 +sig   = 3.1:ff,c00a,c014,88,87,38,c00f,c005,84,35,39,*,c00d,c003,feff,a:?0,a,b,23:
 +sig   = 3.0:88,87,38,84,35,39,45,44,33,32,96,41,4,5,2f,16,13,feff,a,ff::v2,chlen
 +
 +label = s:!:Firefox:4-12
 +sys   = Windows,@unix
 +sig   = 3.1:?ff,c00a,c014,88,87,39,38,c00f,c005,*,c003,feff,a:?0,?ff01,a,b,23:
 +
 +label = s:!:Firefox:11 (TOR)
 +sys   = Windows,@unix
 +; Lack of a single extension (SessionTicket TLS) is not a very strong signal.
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b:
 +
 +label = s:!:Firefox:13 or newer
 +sys   = Windows,@unix
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,45,44,33,32,*,c003,feff,a:?0,a,b,23,3374:
 +
 +; with TLS switched off
 +label = s:!:Firefox:3.6.X or newer
 +sys   = Windows,@unix
 +
 +; Catch all - suspects:
 +label = s:!:Firefox:
 +sys   = Windows,@unix
 +sig   = 3.0:?ff,88,87,39,38,84,35,45,44,33,32,96,41,4,5,2f,16,13,feff,a:?0,?23,?3374:
 +; no clue what 66 stands for
 +sig   = 3.0:?ff,88,87,39,38,84,35,45,44,66,33,32,96,41,5,4,2f,16,13,feff,a:?0,?23,?3374:
 +
 +; ------
 +; Safari
 +; ------
 +; Safari on old PowerPC box
 +label = s:!:Safari:4.X
 +sys   = Mac OS X
 +sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1::
 +sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1,10080,*,700c0::v2,chlen
 +
 +label = s:!:Safari:5.1.2
 +sys   = Mac OS X
 +sig   = 3.1:c00a,c009,c007,c008,c013,*,33,38,39,16,15,14,13,12,11:?0,a,b:
 +
 +label = s:!:Safari:5.1.3 or newer
 +sys   = Mac OS X
 +sig   = 3.1:c00a,c009,c007,c008,c013,*,33,38,39,16,13:?0,a,b:
 +
 +; confirmed on 5.0.6
 +label = s:!:Safari:5.x
 +sys   = Mac OS X
 +sig   = 3.1:2f,5,4,35,a,9,3,8,6,32,33,38,39,16,15,14,13,12,11:?0:
 +
 +label = s:!:Safari:6.x on Mac OS X 10.8.2+
 +sys   = Mac OS X
 +sig   = 3.1:ff,c00a,c009,c007,c008,c014,c013,*,c00d,2f,5,4,35,a,33,39,16:?0,a,b:
 +
 +
 +; -------
 +; Android
 +; -------
 +
 +; in http Android is treated as Linux, oh, well...
 +label = s:!:Android:2.3
 +sys   = Linux
 +sig   = 3.0:4,5,2f,33,32,a,16,13,9,15,12,3,8,14,11,?ff::
 +
 +label = s:!:Android:3.2
 +sys   = Linux
 +sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,15,12,9,14,11,8,6,3,ff:?0,b,a,23,3374:compr
 +
 +label = s:!:Android:4.X
 +sys   = Linux
 +sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,ff:?0,b,a,23,3374:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,*,c009,33,32,45,44,c00e,c004,2f,41,c011,c007,c00c,c002,5,4,ff:?0,b,a,23,3374:
 +
 +; -----------
 +; iPhone iPad
 +; -----------
 +
 +label = s:!:Safari:iOS 4.X
 +sys   = iOS
 +sig   = 3.1:c00a,c009,c007,*,33,39,16,15,14:?0,a,b:
 +
 +label = s:!:Safari:iOS 5.X
 +sys   = iOS
 +sig   = 3.3:ff,c024,c023,c00a,*,33,39,16:?0,a,b,d:
 +
 +label = s:!:Safari:iOS 6.X
 +sys   = iOS
 +sig   = 3.3:ff,c024,c023,c00a,*,33,39,16,c006,c010,c001,c00b,3b,2,1:?0,a,b,d:
 +
 +
 +; ------------
 +; Weird Mobile
 +; ------------
 +label = s:!:Opera Mini:11.X
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +sig   = 3.1:ff,39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +
 +label = s:!:HP-tablet:
 +sys   = Touchpad
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4:?0:
 +
 +; Confirmed on version 6.0
 +label = s:!:Blackberry:
 +sys   = Blackberry
 +sig   = 3.1:33,32,2f,39,38,35,5,4,16,13,a,15,12,9,3,14,11,8,?ff::
 +
 +; -----
 +; Opera
 +; -----
 +
 +label = s:!:Opera:10.x - 11.00
 +sys   = Windows,@unix
 +sig   = 3.2:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,5:ver
 +sig   = 3.3:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5,d:ver
 +
 +label = s:!:Opera:11.52 or newer
 +sys   = Windows,@unix
 +sig   = 3.1:6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +sig   = 3.1:ff,6b,6a,69,68,3d,39,38,37,36,35,67,40,3f,3e,3c,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 +
 +; On second connection Opera replies with the last used crypto in a first place I guess
 +label = s:!:Opera:
 +sys   = Windows,@unix
 +sig   = 3.1:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.1:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.2:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.2:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.3:*,6b,6a,69,68,3d,*,13,d,16,10,a:?0,?ff01,5:
 +sig   = 3.3:*,39,38,37,36,35,*,13,d,16,10,a:?0,?ff01,5:
 +
 +
 +; --------------
 +; Various things
 +; --------------
 +
 +label = g:!:gnutls:
 +sys   = @unix
 +sig   = 3.1:33,16,39,2f,a,35,5,4,32,13,38,66::compr
 +sig   = 3.2:2f,5,4,a,35,32,66,13,38,33,16,39,34,18,1b,3a,3::
 +sig   = 3.3:3c,2f,5,4,a,3d,35,40,32,66,13,6a,38,67,33,16,6b,39,6c,34,18,1b,6d,3a:ff01,d:
 +sig   = 3.1:*,2f,5,4,a,*,35,*,18,1b,3a,*:*:
 +
 +label = g:!:openssl:
 +sys   = @unix
 +
 +; This was seen on old Android 1.5-2
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3::
 +sig   = 3.0:39,38,35,16,13,a,33,32,2f,?7,5,4,15,12,9,14,11,8,6,3,?ff:?23:
 +sig   = 3.0:39,38,35,16,13,a,33,32,2f,?7,5,4,15,12,9,14,11,8,6,3,?ff:?23:compr
 +
 +; Observed:
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:compr
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:compr
 +
 +; darwin
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2,chlen
 +
 +sig   = 3.1:39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:b,a,23:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:?0,b,a:compr
 +
 +; wget on ubuntu
 +sig   = 3.2:c014,c00a,c022,c021,39,38,88,87,c00f,c005,35,84,*,14,11,8,6,3,ff:b,a,23,f:compr,ver
 +; curl on ubuntu
 +sig   = 3.2:c014,c00a,c022,c021,39,38,88,87,c00f,c005,35,84,*,14,11,8,6,3,ff:?0,b,a,f:compr,ver
 +
 +label = s:!:Epiphany:
 +sys   = Linux
 +; epiphany 2.x
 +sig   = 3.0:33,39,16,32,38,13,2f,35,a,5,4::
 +; epiphany 3.4.1
 +sig   = 3.0:33,39,16,32,38,13,2f,35,a,5,4,ff::
 +
 +
 +label = s:!:Google bot:
 +sys   = Windows,@unix
 +sig   = 3.1:c011,5,4,2f,a,35,33,32,16,13,39,38,ff:?0,b,a,23,f:
 +sig   = 3.1:4,5,a,13,16,2f,32,33,35,38,39,c011,ff:?0,b,a,23:
 +
 +label = s:!:Akregator bot:
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,7,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 +
 +label = s:!:BLP_bbot:
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,7,50080,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +
 +label = s:!:YandexBot:
 +sys   = Windows,@unix
 +sig   = 3.0:a,5,4::
 +
 +label = s:!:Facebook bot:
 +sys   = Windows,@unix
 +sig   = 3.1:c022,c021,39,38,88,87,35,84,c01c,c01b,16,13,a,c01f,c01e,33,32,9a,99,45,44,2f,96,41,5,4,15,12,9,14,11,8,6,3,ff:?0,f:compr
 +
 +label = s:!:Various bots:
 +sys   = Windows,@unix
 +sig   = 3.1:4,10080,5,2f,33,32,a,700c0,16,13,9,60040,15,12,3,20080,8,14,11,ff::v2
 diff --git a/process.c b/process.c
 index 3cee1da..66a0b21 100644
 --- a/process.c
 +++ b/process.c
 @@ -36,6 +36,7 @@
 #include "fp_tcp.h"
 #include "fp_mtu.h"
 #include "fp_http.h"
 +#include "fp_ssl.h"
 
 u64 packet_cnt;                         /* Total number of packets processed  */
 
 @@ -1368,6 +1369,7 @@ static void flow_dispatch(struct packet_data* pk) {
       if (!pk->pay_len) return;
 
       need_more |= process_http(to_srv, f);
 +      need_more |= process_ssl(to_srv, f);
 
       if (!need_more) {
 
 diff --git a/process.h b/process.h
 index 5433282..2addf27 100644
 --- a/process.h
 +++ b/process.h
 @@ -135,6 +135,9 @@ struct host_data {
 
   u16 http_resp_port;                   /* Port on which response seen        */
 
 +  u32 ssl_remote_time;                  /* Last client timestamp from SSL     */
 +  u32 ssl_recv_time;                    /* Time drift derived from SSL        */
 +
 };
 
 /* Reasons for NAT detection: */
 @@ -195,6 +198,8 @@ struct packet_flow {
 
   struct http_sig http_tmp;             /* Temporary signature                */
 
 +  s8  in_ssl;                           /* 0 = tbd, 1 = yes, -1 = no          */
 +
 };
 
 extern u64 packet_cnt;
 diff --git a/readfp.c b/readfp.c
 index 34a3cac..bafd466 100644
 --- a/readfp.c
 +++ b/readfp.c
 @@ -29,6 +29,7 @@
 #include "fp_tcp.h"
 #include "fp_mtu.h"
 #include "fp_http.h"
 +#include "fp_ssl.h"
 #include "readfp.h"
 
 static u32 sig_cnt;                     /* Total number of p0f.fp sigs        */
 @@ -270,6 +271,10 @@ static void config_parse_line(u8* line) {
 
       mod_type = CF_MOD_HTTP;
 
 +    } else if (!strcmp((char*)line, "ssl")) {
 +
 +      mod_type = CF_MOD_SSL;
 +
     } else {
 
       FATAL("Unrecognized fingerprinting module '%s' in line %u.", line, line_no);
 @@ -367,6 +372,11 @@ static void config_parse_line(u8* line) {
                           label_id, cur_sys, cur_sys_cnt, val, line_no);
         break;
 
 +      case CF_MOD_SSL:
 +        ssl_register_sig(mod_to_srv, generic, sig_class, sig_name, sig_flavor,
 +                         label_id, cur_sys, cur_sys_cnt, val, line_no);
 +        break;
 +
     }
 
     sig_cnt++;
 diff --git a/readfp.h b/readfp.h
 index 9998e00..6240462 100644
 --- a/readfp.h
 +++ b/readfp.h
 @@ -18,6 +18,7 @@
 #define CF_MOD_TCP           0x00       /* fp_tcp.c                           */
 #define CF_MOD_MTU           0x01       /* fp_mtu.c                           */
 #define CF_MOD_HTTP          0x02       /* fp_http.c                          */
 +#define CF_MOD_SSL           0x03       /* fp_ssl.c                           */
 
 /* Parser states: */
 
diff --git a/patch-05-17-to-11-01.diff b/patch-05-17-to-11-01.diff
 diff --git a/docs/README b/docs/README
 index f3f1ed7..943b9a2 100644
 --- a/docs/README
 +++ b/docs/README
 @@ -717,7 +717,7 @@ getting more than enough information just with a simple GET / HEAD fingerprint.
 
 P0f is capable of fingerprinting SSL / TLS requests. This can be used
 to verify if a browser using https is legitimate, and in some cases
 -can leak some details about client operating system.
 +can reveal some details about the client's operating system.
 
 SSL signatures have the following layout:
 
 @@ -731,32 +731,33 @@ sig = sslver:ciphers:extensions:sslflags
   ciphers    - comma-separated list of hex values representing SSL
                ciphers supported by the client.
 
 -               NEW SIGNATURES: Review the list and feel free to
 -               substitute parts that bring no information with the
 -               match all sign '*'.  For efficiency avoid using star in
 -               the beginning of the signature.
 +               NEW SIGNATURES: Review the list and substitute boring
 +               parts that don't bring newq information with the 'match
 +               all' sign '*'. For efficiency don't use star at the
 +               beginning of the signature.
 
 -               You may also use '?' to make particular value optional.
 +               You may also use '?' to make a single value optional.
 
 -  extensions - comma-separated list of hex values representing
 -               SSL extensions.
 +  extensions - comma-separated list of hex values representing SSL
 +               extensions.
 
                NEW SIGNATURES: Same as for ciphers.
 
 -  sslflags   - comma-separated list of SSL flags:
 +  sslflags   - comma-separated list of SSL flags. SSLv2 flags:
 
 -               v2    - client used an older SSLv2 handshake frame, instead
 -                       of more recent SSLv3 / TLSv1
 +               v2    - client used an SSLv2 handshake
 +               chlen - challenge data is not 32 bytes long
 
                Flags specific to SSLv3 handshake:
 
 -               ver   - requested SSL protocol was different on a protocol
 -                       (request) than on a record layer
 +               ver   - SSL protocol on a request layer was different
 +                       than on a record layer
                stime - gmt_unix_time field has unusually was small value,
                        most likely time since boot
                rtime - gmt_unix_time field has value that is very far off
                        local time, most likely it is random
                compr - client supports deflate compression
 +               rand  - supposedly random data doesn't look randomly
 
                NEW SIGNATURES: Copy literally.
 
 @@ -960,6 +961,7 @@ including:
   Jeff Weisberg
   Anthony Howe
   Tomoyuki Murakami
 +  Marek Majkowski
 
 If you wish to help, the most immediate way to do so is to simply gather new
 signatures, especially from less popular or older platforms (servers, networking
 diff --git a/docs/ssl-notes.txt b/docs/ssl-notes.txt
 index 1deeaba..bd7a86b 100644
 --- a/docs/ssl-notes.txt
 +++ b/docs/ssl-notes.txt
 @@ -2,99 +2,130 @@
 SSL fingerprinting
 ==================
 
 -Not many people realise that the unencrypted SSL / TLS handshake
 -(ClientHello message) reveals some fingerprintable details about the
 -client.
 +The first message of an SSL/TLS connection, ClientHello, contains some
 +data that could be used to identify the client. P0f SSL code creates a
 +fingerprint from this packet.
 
 -For a given application using SSL it is possible to fingerprint
 -underlying SSL library. In many cases it is even possible to pinpoint
 -specific application version.
 +For a given ClientHello message sometimes it is possible to identify
 +underlying SSL library. For applications which include a custom build
 +of an SSL library it may be even possible to pinpoint a specific
 +application version. For example some web browsers do it. Additionally
 +browsers selectively support newly introduced SSL features like SPDY,
 +ECC or 'renegotiation info' extension making it easier to distinguish
 +one from another.
 
 -This especially true in the browser world, where the SSL libraries are
 -most often bundled with a release. Additionally, on the web a lot of
 -innovation happens on SSL layer, like SPDY, ECC or 'renegotiation
 -info' extension.
 +SSL connections are usually directly forwarded by proxies without
 +mangling. In most cases SSL fingerprint comes straight from a genuine
 +application and is not affected by NAT or any specific network
 +configuration.
 
 -Additionally SSL traffic is unharmed by proxies or routers, we can
 -have high confidentiality that the SSL fingerprint does identify a
 -genuine application. This is useful for detecting NAT's.
 +Although initial frames sent from both SSL client and the server are
 +similar, only the one sent by the client (ClientHello) can be
 +passively fingerprinted. The packet from server (ServerHello) doesn't
 +contain enough information. If you wish to fingerprint an SSL server
 +an active fingerprinting tool is more suitable [1].
 
 -Although initial frames sent from both SSL client and server are
 -similar, only frame sent by the client 'ClientHello', can be passively
 -fingerprinted. The server frame 'ServerHello' doesn't have enough
 -information. If you wish to fingerprint SSL servers an active
 -fingerprinting tool may be more suitable.
 +Passively fingerprinting SSL is not a new idea, initial work was done
 +in mid 2009 by Ivan Ristic [2]. He was able to collect a few dozen
 +interesting signatures [3]. Unfortunately in his work he had ignored
 +the SSL extensions which add a lot of information. Especially the
 +ordering of extensions has high value, similarly to the TCP options
 +ordering for TCP/IP stack fingerprinting.
 
 +  [1] http://nmap.org/nsedoc/scripts/ssl-enum-ciphers.html
 +  [2] https://www.ssllabs.com/projects/client-fingerprinting/
 +  [3] http://blog.ivanristic.com/2009/07/examples-of-the-information-collected-from-ssl-handshakes.html
 
 -Fingerprinting SSL is not a new idea, initial work was done in mid
 -2009 by Ivan Ristic:
 +References to SSL/TLS specs:
 
 - * https://www.ssllabs.com/projects/client-fingerprinting/
 + * SSL 2.0: http://www.mozilla.org/projects/security/pki/nss/ssl/draft02.html
 + * SSL 3.0: http://tools.ietf.org/html/rfc6101
 + * TLS 1.0: http://tools.ietf.org/html/rfc2246
 + * TLS 1.1: http://tools.ietf.org/html/rfc4346
 + * TLS 1.2: http://tools.ietf.org/html/rfc5246
 
 -He was able to collect few dozen interesting signatures:
 
 - * http://blog.ivanristic.com/2009/07/examples-of-the-information-collected-from-ssl-handshakes.html
 +SSL fingerprint structure
 +=========================
 
 -Unfortunately on his works he seem to have totally ignored the SSL
 -extensions list, which add valuable information to a
 -fingerprint. Especially the ordering of extensions has high value,
 -similarly to the TCP options ordering for TCP/IP stack fingerprinting.
 +SSL fingerprint is generated from ClientHello SSL/TLS data. It's a
 +semicolon delimited string with layout as follows:
 
 +    ssl version : ciphers list : extensions list : flags
 
 -SSL handshake (ClientHello message) contains following fields:
 +For example a generated fingerprint might look like:
 +
 +    3.1:39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,ff:23:compr
 +
 +This reads:
 +
 +1) Client requests TLS 1.0 (value 3.1), using SSL 3.0+ record format
 +   (flag 'v2' is _not_ set), with 'request' layer SSL version equal to
 +   'record' layer version (flag 'ver' is _not_ set).
 +2) Client supports ciphers, in order:
 +    39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,ff
 +3) Client used only a single extension: 0x23 (SessionTicket TLS)
 +4) Client supports deflate compression (flag 'compr' is set).
 +5) Random blob looks legitimate (flag 'rand' is _not_ set).
 +6) Unix time reported by the client looks legitimate (flags 'stime'
 +   and 'rtime' are _not_ set).
 +
 +
 +ClientHello message structure
 +=============================
 +
 +SSL client handshake (ClientHello message) contains following fields:
 
  * SSL record version - browsers usually use version 3, Ivan Ristic
 -   research shows that web crawlers (including wget) still use 2.
 -   We set a flag:
 -     'v2'  - if request is SSL version 2
 -
 - * Requested SSL protocol version - most likely 3.1. Less likely values
 -   are 3.0 and 3.2. SSL crawlers trying invalid versions like 4.2 were
 -   seen in the wild. ('request_version')
 -   We set a flag:
 -     'ver' - if, for SSL version 3, the version on a 'request' layer
 -             is different than on 'record' layer. This behaviour was seen
 -             in Opera.
 -
 - * gmt_unix_time - a timestamp from the client. In the code we may
 -   set one of two flags derived from this value:
 -     'stime' - when the timestamp is unnaturally small (less than one year 
 -               since the epoch). This is behaviour was seen in old Firefox
 -               releases, when gmt_unix_time is set to seconds since boot.
 -     'rtime' - when the timestamp is far off the current value (delta
 -               is bigger than 5 years). Most often this means that gmt_unix_time
 -               field is set to a random value. This was seen in some
 -               SSL crawlers.
 +   showed that older clients and web crawlers (including wget) still
 +   use SSL 2 packet format. Flag 'v2' is set in that case.
 
 - * 28 random bytes - not very useful. In a debug build a warning will
 -   be printed if the values don't look very randomly.
 + * Requested SSL protocol version - most likely 3.1 (TLS 1.0). Less
 +   likely values are 3.0 (SSL 3.0) and 3.2 (TLS 1.1). SSL crawlers may
 +   try invalid versions like 4.2. We set a flag:
 
 - * session_id - a client may choose to resume previous SSL session. Ignored.
 +     'ver' - if, for SSLv3 / TLS handshake, the version on a 'request'
 +             layer is different than on 'record' layer. This behaviour
 +             was seen in Opera.
 
 - * cipher_suites - a list of supported encryption algorithms ('ciphers').
 -   For the meaning of the numbers refer to:
 -       http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-3
 + * Current unix timestamp from the client. In the code we may set one
 +   of two flags derived from this value:
 
 - * compression_methods - a list of supported compression methods. There
 -   is only one valid compression method available, so we set a flag:
 -       'compr' - if compression is enabled 
 +     'stime' - when the timestamp is too small (less than one year
 +               since the epoch). This is behaviour was seen in
 +               Firefox 2.X, when this value is set to the number of
 +               seconds since boot.
 
 - * extensions - a list of SSL extensions. Second, after cipher_suites,
 -   major source of data. For exact meaning refer to:
 -        http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xml
 +     'rtime' - when the timestamp is far off the current value (delta
 +               is greater than 5 years). Most often this means that
 +               the field is set to a random value. This was seen in
 +               some SSL crawlers.
 +
 + * 28 random bytes - not very useful. A flag is set:
 +
 +      'rand' - when the values don't look very randomly (0x00000000 or
 +               0xffffffff found in the supposedly random block)
 +
 + * session_id - ignored.
 
 + * cipher_suites - a list of supported encryption algorithms
 +   ('ciphers').  Iana maintains a list of valid values [4].
 
 -It's worth noting that SSL extensions contain not only a list of
 -supported extensions but also optional payload for every extension.
 -We ignore the payload as the experiments shown that it isn't an
 -interesting source of data - we can get the same entropy by just
 -looking at the ordering of SSL extensions.
 + * compression_methods - a list of supported compression
 +   methods. There is only one valid compression method available, we
 +   set 'compr' flag if compression is enabled
 
 + * extensions - a list of SSL extensions. Second, after
 +   'cipher_suites', major source of entropy. Iana maintains the list
 +   of valid extensions [5].
 +
 +It's worth noting that SSL extensions may contain a payload data for
 +every extension.  We ignore that - it doesn't seem to provide any more
 +entropy than by just looking at the order of SSL extensions.
 
 A special explanation should be given for '0' - 'server_name' SSL
 extension. This extension contains a host name of a remote host and is
 often described as Server Name Indication TLS extension from RFC 3546.
 -
 Unfortunately RFC forbids passing raw ip addresses in this
 option. That means this option must not be present if you enter a
 website using an ip address. For example, browser will valid extension
 @@ -106,3 +137,6 @@ with an optional sign '?'. This is quite optimistic and may not always
 be a good idea - it is possible the fingerprint when accessing
 'localhost' is completely different from '127.0.0.1'. Using '?' before
 this extension will only clutter a signature in such case.
 +
 +  [4] http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-3
 +  [5] http://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xml
 diff --git a/fp_ssl.c b/fp_ssl.c
 index c77dca0..6bffca6 100644
 --- a/fp_ssl.c
 +++ b/fp_ssl.c
 @@ -45,6 +45,8 @@ struct flag flags[] = {{"compr", 5, SSL_FLAG_COMPR},
                        {"ver",   3, SSL_FLAG_VER},
                        {"rtime", 5, SSL_FLAG_RTIME},
                        {"stime", 5, SSL_FLAG_STIME},
 +                       {"rand",  4, SSL_FLAG_RAND},
 +                       {"chlen", 5, SSL_FLAG_CHLEN},
                        {NULL, 0, 0}};
 
 
 @@ -238,7 +240,7 @@ static void ssl_find_match(struct ssl_sig* ts) {
 }
 
 
 -/* Unpack SSLv2 header to a signature. 
 +/* Unpack SSLv2 header to a signature.
    -1 on parsing error, 1 if signature was extracted. */
 
 static int fingerprint_ssl_v2(struct ssl_sig* sig, const u8* pay, u32 pay_len) {
 @@ -293,14 +295,45 @@ static int fingerprint_ssl_v2(struct ssl_sig* sig, const u8* pay, u32 pay_len) {
   u16 session_id_len = ntohs(hdr->session_id_length);
   u16 challenge_len = ntohs(hdr->challenge_length);
 
 +  /* Although SSLv2 states that challenge must be between 16 and 32
 +     bytes long, in practice no other values were recorded. 32 seems
 +     to be more popular than 16. */
 +
 +  if (challenge_len != 32) {
 +
 +    sig->flags |= SSL_FLAG_CHLEN;
 +
 +    if (challenge_len != 16) {
 +      DEBUG("[#] SSLv2 challenge_len %i req_ver=%04x\n", challenge_len,
 +            sig->request_version);
 +    }
 +
 +  }
 +
   if (pay + session_id_len + challenge_len > pay_end) {
 
 -    DEBUG("[#] SSLv2 frame truncated (but valid)\n");
 +    DEBUG("[#] SSLv2 frame truncated (but valid) req_ver=%04x\n",
 +          sig->request_version);
     goto truncated;
 
   }
 
 -  pay += session_id_len + challenge_len;
 +  pay += session_id_len;
 +
 +  u32 i;
 +  u32* challenge = (u32*)pay;
 +
 +  for (i = 0; i < challenge_len/4; i++) {
 +    if (challenge[i] == 0x00000000 || challenge[i] == 0xffffffff) {
 +
 +      sig->flags |= SSL_FLAG_RAND;
 +      break;
 +
 +    }
 +  }
 +
 +
 +  pay += challenge_len;
 
   if (pay != pay_end) {
 
 @@ -333,7 +366,7 @@ too_short:
  message.  -1 on parsing error, 1 if signature was extracted. */
 
 static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
 -                              u32 frag_len, u16 record_version, u32 local_time) {
 +                              u32 frag_len, u16 record_version) {
 
   int i;
   const u8* frag_end = fragment + frag_len;
 @@ -352,6 +385,9 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
      not for us. */
 
   if (pay_end > frag_end) {
 +    /* I've seen a packet which looked like fragmented, but upon
 +       inspection I noticed that 4th byte of data had been cleared
 +       (high bits of frame length). Weird, ain't? */
 
     DEBUG("[#] SSL Fragment coalescing not supported - %u bytes requested.\n",
           pay_end - frag_end);
 @@ -369,7 +405,7 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
        I guess we can assume that the first frame must be ClientHello.
     */
 
 -    DEBUG("[#] SSL First message type 0x%02x (%u bytes) not supported.\n",
 +    DEBUG("[#] SSL Message type 0x%02x (%u bytes) is not ClientHello.\n",
           msg->message_type, msg_len);
     return -1;
 
 @@ -393,7 +429,6 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
   sig->remote_time = ntohl(*((u32*)pay));
   pay += 4;
 
 -  sig->recv_time = local_time;
   s64 drift = ((s64)sig->recv_time) - sig->remote_time;
 
   if (sig->remote_time < 1*365*24*60*60) {
 @@ -406,20 +441,19 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
     /* More than 5 years difference - most likely random */
     sig->flags |= SSL_FLAG_RTIME;
 
 -    DEBUG("[#] SSL timer looks wrong: drift=%lld remote_time=%u.\n",
 +    DEBUG("[#] SSL timer looks random: drift=%lld remote_time=%u.\n",
           drift, sig->remote_time);
 
   }
 
   /* Random */
 -  u16* random = (u16*)pay;
 +  u32* random = (u32*)pay;
   pay += 28;
 
 -  for (i = 0; i < 14; i++) {
 -    if (random[i] == 0x0000 || random[i] == 0xffff) {
 +  for (i = 0; i < 7; i++) {
 +    if (random[i] == 0x00000000 || random[i] == 0xffffffff) {
 
 -      DEBUG("[#] SSL 0x%04x found in allegedly random blob at offset %i.\n",
 -            random[i], i);
 +      sig->flags |= SSL_FLAG_RAND;
       break;
 
     }
 @@ -473,6 +507,10 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
 
     if (pay[0] == 1) {
       sig->flags |= SSL_FLAG_COMPR;
 +    } else {
 +      if (pay[0] || pay + 1 != tmp_end)
 +        DEBUG("[#] SSL unknown compression method 0x%x at offset %i/%i.\n",
 +              pay[0], tmp_end - pay, compression_methods_len);
     }
 
     pay += 1;
 @@ -482,10 +520,15 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
 
   if (pay + 2 > pay_end) {
 
 -    /* Extensions are optional in SSLv3. This behaviour was considered
 -       as a flag, but it doesn't bring any entropy. In other words:
 -       noone who is able to send extensions sends an empty list.  An
 -       empty list of extensions is equal to SSLv2 or this branch. */
 +    /* SSL 3.0 and all versions of TLS allow the ClientHello message
 +       to be truncated after compression_methods and not specify
 +       extensions field at all. Although rarely, this does seem to
 +       occur in real world. Semantically it is no different to sending
 +       an empty list of extensions. In fact, no client was ever
 +       recorded to send an empty list of extensions if the extensions
 +       field was present. In other words - if the extensions field is
 +       empty you can assume the ClientHello packet was truncated after
 +       compression_methods. */
     goto truncated_ok;
 
   }
 @@ -547,7 +590,8 @@ static int fingerprint_ssl_v3(struct ssl_sig* sig, const u8* fragment,
   if (0) {
 truncated:
 
 -    DEBUG("[#] SSL packet truncated (but valid).\n");
 +    DEBUG("[#] SSL packet truncated (but valid) req_ver=%04x rec_ver=%04x\n",
 +          sig->request_version, record_version);
 
   }
 truncated_ok:
 @@ -752,7 +796,7 @@ static void score_nat(u8 to_srv, struct packet_flow* f, struct ssl_sig* sig) {
     if (remote_diff < recv_diff - SSL_MAX_TIME_DIFF ||
         remote_diff > recv_diff + SSL_MAX_TIME_DIFF) {
 
 -      DEBUG("[#] SSL gmt_unix_time distance too high (%lld in %lld sec).\n",
 +      DEBUG("[#] SSL gmt_unix_time skew too high (%lld in %lld sec).\n",
             remote_diff, recv_diff);
 
       score += 4;
 @@ -871,6 +915,7 @@ u8 process_ssl(u8 to_srv, struct packet_flow* f) {
     if (f->req_len < 2 + msg_length) return can_get_more;
 
     memset(&sig, 0, sizeof(struct ssl_sig));
 +    sig.recv_time = f->client->last_seen;
     sig.flags |= SSL_FLAG_V2;
 
     success = fingerprint_ssl_v2(&sig, f->request, msg_length + 2);
 @@ -892,13 +937,12 @@ u8 process_ssl(u8 to_srv, struct packet_flow* f) {
       return can_get_more;
 
     memset(&sig, 0, sizeof(struct ssl_sig));
 +    sig.recv_time = f->client->last_seen;
     u16 record_version = (hdr3->ver_maj << 8) | hdr3->ver_min;
 
     u8* fragment = f->request + sizeof(struct ssl3_record_hdr);
 
 -    success = fingerprint_ssl_v3(&sig, fragment, fragment_len,
 -                                 record_version,
 -                                 f->client->last_seen);
 +    success = fingerprint_ssl_v3(&sig, fragment, fragment_len, record_version);
 
   }
 
 diff --git a/fp_ssl.h b/fp_ssl.h
 index 4e0d57c..8eb1026 100644
 --- a/fp_ssl.h
 +++ b/fp_ssl.h
 @@ -33,6 +33,8 @@
 #define SSL_FLAG_STIME 0x0008  /* small SSL time, (absolute value < 1 year)
                                   most likely time since reboot for old ff */
 #define SSL_FLAG_COMPR 0x0010  /* Deflate compression support. */
 +#define SSL_FLAG_RAND  0x0020  /* Random data doesn't look randomly. */
 +#define SSL_FLAG_CHLEN 0x0040  /* SSLv2 challenge is not 16 bytes */
 
 
 /* SSLv2 */
 diff --git a/p0f.fp b/p0f.fp
 index b9627f5..acae4d0 100644
 --- a/p0f.fp
 +++ b/p0f.fp
 @@ -918,13 +918,13 @@ sig   = *:Content-Type,X-Content-Type-Options=[nosniff],Server=[GSE]:Connection,
 label = s:!:any:MSIE or Safari on Windows XP
 sys   = Windows
 sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63:ff01:
 -; no MS10-049 applied?
 -sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63::
 +; no MS10-049 applied
 +sig   = 3.1:4,5,a,9,64,62,3,6,13,12,63,?ff::
 +sig   = 3.0:4,5,a,9,64,62,3,6,13,12,63,?ff::
 
 ; with some SSL/TLS options tweaked
 -sig   = 3.0:4,5,a,9,64,62,3,6,13,12,63,ff::
 -sig   = 3.0:4,5,a,10080,700c0,30080,9,60040,64,62,3,6,20080,40080,13,12,63,ff::v2
 -sig   = 2.0:10080,700c0,30080,60040,20080,40080,ff::v2
 +sig   = 3.0:4,5,a,10080,700c0,30080,9,60040,64,62,3,6,20080,40080,13,12,63,?ff::v2,chlen
 +sig   = 2.0:10080,700c0,30080,60040,20080,40080,ff::v2,chlen
 
 
 ; Windows NT 6.0 (Vista)
 @@ -932,27 +932,37 @@ label = s:!:any:MSIE 5.5-6 or Chrome 1-4 or Safari on Windows Vista
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,a,b,ff01:
 
 -label = s:!:any:MSIE 7.0-9.0 or Chrome 5 on Windows Vista
 +label = s:!:any:MSIE 7-9 or Chrome 5 on Windows Vista
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:?0,5,a,b,ff01:
 
 +; look out for order of ff01 + time
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:stime
 +sig   = 3.1:2f,35,5,a,c009,c00a,c013,c014,32,38,13,4:ff01,?0,5,a,b:rtime
 +
 
 ; Windows NT 6.1 (7)
 label = s:!:MSIE:7-9 on Windows 7
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b:
 +; Weird IE9 on win 7: MSIE 9.0; Windows NT 6.1
 +sig   = 3.0:5,a,13,4,10080,700c0,?ff::v2,chlen
 +sig   = 3.0:5,a,13,4,?ff::
 +sig   = 3.0:5,a,13,4,?ff::v2,chlen
 
 label = s:!:Safari:on Windows 7
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b:
 
 +
 ; Windows NT 6.2 ( 8)
 ; 23 usually means NT 6.2
 label = s:!:MSIE:10 on Windows 8
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,5,a,b,23:
 
 -label = s:!:Safari:Safari on Windows 8
 +label = s:!:Safari:on Windows 8
 sys   = Windows
 sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b,23:
 
 @@ -961,14 +971,24 @@ sig   = 3.1:2f,35,5,a,c013,c014,c009,c00a,32,38,13,4:ff01,?0,a,b,23:
 ; Chrome
 ; ------
 
 -label = s:!:Chrome:6 or newer
 +; old chrome - TLS 1.0
 +label = s:!:Chrome:6-20
 sys   = Windows,@unix
 -sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23:compr
 -sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23,3374:compr
 -; 5 is on on windows
 -sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,c003,feff,a:?0,ff01,a,b,23,3374,5:compr
 +sig   = 3.1:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:compr
 
 -label = s:!:Chrome:degraded to SSLv3.0
 +; newer chrome - TLS 1.1
 +label = s:!:Chrome:21
 +sys   = Windows,@unix
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:compr
 +
 +; newest chrome - no compr support
 +label = s:!:Chrome:22 or newer
 +sys   = Windows,@unix
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:
 +sig   = 3.2:c00a,c014,88,87,39,38,c00f,*,45,44,66,33,32,*,c003,feff,a:?0,ff01,a,b,23,?3374,?5:ver
 +
 +
 +label = s:!:Chrome:degraded to SSL v3.0
 sys   = Windows,@unix
 sig   = 3.0:ff,88,87,39,38,84,35,45,44,66,33,32,96,41,4,5,2f,16,13,feff,a::
 
 @@ -978,7 +998,7 @@ sig   = 3.0:ff,88,87,39,38,84,35,45,44,66,33,32,96,41,4,5,2f,16,13,feff,a::
 
 label = s:!:Firefox:1.X
 sys   = Windows,@unix
 -sig   = 3.1:10080,30080,*,40080,39,38,35,*,64,62,3,6::v2
 +sig   = 3.1:10080,30080,*,40080,39,38,35,*,64,62,3,6::v2,chlen
 sig   = 3.1:39,38,35,*,64,62,3,6::stime
 
 label = s:!:Firefox:2.X
 @@ -992,27 +1012,31 @@ sig   = 3.1:c00a,c014,88,87,39,38,c00f,c005,84,35,c007,*,c003,feff,a:?0,a,b,23:
 label = s:!:Firefox:3.6.X
 sys   = Windows,@unix
 sig   = 3.1:ff,c00a,c014,88,87,38,c00f,c005,84,35,39,*,c00d,c003,feff,a:?0,a,b,23:
 +sig   = 3.0:88,87,38,84,35,39,45,44,33,32,96,41,4,5,2f,16,13,feff,a,ff::v2,chlen
 
 -label = s:!:Firefox:4-11
 +label = s:!:Firefox:4-12
 sys   = Windows,@unix
 -sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b,23:
 -; with SSLv2 disalbed
 -sig   = 3.1:c00a,c014,88,87,39,38,*,c003,feff,a:?0,ff01,a,b,23:
 +sig   = 3.1:?ff,c00a,c014,88,87,39,38,c00f,c005,*,c003,feff,a:?0,?ff01,a,b,23:
 
 label = s:!:Firefox:11 (TOR)
 sys   = Windows,@unix
 ; Lack of a single extension (SessionTicket TLS) is not a very strong signal.
 sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b:
 
 -label = s:!:Firefox:14 or newer
 +label = s:!:Firefox:13 or newer
 sys   = Windows,@unix
 -sig   = 3.1:ff,c00a,c014,88,87,39,38,*,c003,feff,a:?0,a,b,23,3374:
 +sig   = 3.1:ff,c00a,c014,88,87,39,38,*,45,44,33,32,*,c003,feff,a:?0,a,b,23,3374:
 
 ; with TLS switched off
 label = s:!:Firefox:3.6.X or newer
 sys   = Windows,@unix
 -sig   = 3.0:ff,88,87,39,38,84,35,45,44,33,32,96,41,4,5,2f,16,13,feff,a::
 
 +; Catch all - suspects:
 +label = s:!:Firefox:
 +sys   = Windows,@unix
 +sig   = 3.0:?ff,88,87,39,38,84,35,45,44,33,32,96,41,4,5,2f,16,13,feff,a:?0,?23,?3374:
 +; no clue what 66 stands for
 +sig   = 3.0:?ff,88,87,39,38,84,35,45,44,66,33,32,96,41,5,4,2f,16,13,feff,a:?0,?23,?3374:
 
 ; ------
 ; Safari
 @@ -1021,7 +1045,7 @@ sig   = 3.0:ff,88,87,39,38,84,35,45,44,33,32,96,41,4,5,2f,16,13,feff,a::
 label = s:!:Safari:4.X
 sys   = Mac OS X
 sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1::
 -sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1,10080,*,700c0::v2
 +sig   = 3.1:2f,5,4,35,a,ff83,*,17,19,1,10080,*,700c0::v2,chlen
 
 label = s:!:Safari:5.1.2
 sys   = Mac OS X
 @@ -1031,19 +1055,24 @@ label = s:!:Safari:5.1.3 or newer
 sys   = Mac OS X
 sig   = 3.1:c00a,c009,c007,c008,c013,*,33,38,39,16,13:?0,a,b:
 
 +; confirmed on 5.0.6
 +label = s:!:Safari:5.x
 +sys   = Mac OS X
 +sig   = 3.1:2f,5,4,35,a,9,3,8,6,32,33,38,39,16,15,14,13,12,11:?0:
 +
 +label = s:!:Safari:6.x on Mac OS X 10.8.2+
 +sys   = Mac OS X
 +sig   = 3.1:ff,c00a,c009,c007,c008,c014,c013,*,c00d,2f,5,4,35,a,33,39,16:?0,a,b:
 +
 
 ; -------
 ; Android
 ; -------
 
 ; in http Android is treated as Linux, oh, well...
 -label = s:!:Android:1.5-2.1
 -sys   = Linux
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3::
 -
 label = s:!:Android:2.3
 sys   = Linux
 -sig   = 3.1:4,5,2f,33,32,a,16,13,9,15,12,3,8,14,11,ff::
 +sig   = 3.0:4,5,2f,33,32,a,16,13,9,15,12,3,8,14,11,?ff::
 
 label = s:!:Android:3.2
 sys   = Linux
 @@ -1052,6 +1081,7 @@ sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,15,12,9,14,11,8,6,3,ff:
 label = s:!:Android:4.X
 sys   = Linux
 sig   = 3.1:c014,c00a,39,38,c00f,c005,35,*,c00c,c002,5,4,ff:?0,b,a,23,3374:compr
 +sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,*,c009,33,32,45,44,c00e,c004,2f,41,c011,c007,c00c,c002,5,4,ff:?0,b,a,23,3374:
 
 ; -----------
 ; iPhone iPad
 @@ -1065,6 +1095,10 @@ label = s:!:Safari:iOS 5.X
 sys   = iOS
 sig   = 3.3:ff,c024,c023,c00a,*,33,39,16:?0,a,b,d:
 
 +label = s:!:Safari:iOS 6.X
 +sys   = iOS
 +sig   = 3.3:ff,c024,c023,c00a,*,33,39,16,c006,c010,c001,c00b,3b,2,1:?0,a,b,d:
 +
 
 ; ------------
 ; Weird Mobile
 @@ -1074,10 +1108,14 @@ sys   = Windows,@unix
 sig   = 3.1:39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 sig   = 3.1:ff,39,38,37,36,35,33,32,31,30,2f,5,4,13,d,16,10,a:?0,ff01,5:
 
 -label = s:!:HP-tablet:unknown
 +label = s:!:HP-tablet:
 sys   = Touchpad
 sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4:?0:
 
 +; Confirmed on version 6.0
 +label = s:!:Blackberry:
 +sys   = Blackberry
 +sig   = 3.1:33,32,2f,39,38,35,5,4,16,13,a,15,12,9,3,14,11,8,?ff::
 
 ; -----
 ; Opera
 @@ -1115,23 +1153,61 @@ sig   = 3.2:2f,5,4,a,35,32,66,13,38,33,16,39,34,18,1b,3a,3::
 sig   = 3.3:3c,2f,5,4,a,3d,35,40,32,66,13,6a,38,67,33,16,6b,39,6c,34,18,1b,6d,3a:ff01,d:
 sig   = 3.1:*,2f,5,4,a,*,35,*,18,1b,3a,*:*:
 
 -
 label = g:!:openssl:
 sys   = @unix
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0:compr
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,23:
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0,23:compr
 -sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0:compr
 +
 +; This was seen on old Android 1.5-2
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3::
 +sig   = 3.0:39,38,35,16,13,a,33,32,2f,?7,5,4,15,12,9,14,11,8,6,3,?ff:?23:
 +sig   = 3.0:39,38,35,16,13,a,33,32,2f,?7,5,4,15,12,9,14,11,8,6,3,?ff:?23:compr
 +
 +; Observed:
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:compr
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,9a,99,96,5,4,15,12,9,14,11,8,6,3,ff:?0,?23:compr
 
 ; darwin
 -sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,9a,99,96,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2,chlen
 
 sig   = 3.1:39,38,88,87,35,84,16,13,a,33,32,9a,99,45,44,2f,96,41,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:b,a,23:compr
 sig   = 3.1:c014,c00a,39,38,88,87,c00f,c005,35,84,*,8,6,3,ff:?0,b,a:compr
 
 -label = s:!:Epiphany:2.X
 +; wget on ubuntu
 +sig   = 3.2:c014,c00a,c022,c021,39,38,88,87,c00f,c005,35,84,*,14,11,8,6,3,ff:b,a,23,f:compr,ver
 +; curl on ubuntu
 +sig   = 3.2:c014,c00a,c022,c021,39,38,88,87,c00f,c005,35,84,*,14,11,8,6,3,ff:?0,b,a,f:compr,ver
 +
 +label = s:!:Epiphany:
 sys   = Linux
 +; epiphany 2.x
 sig   = 3.0:33,39,16,32,38,13,2f,35,a,5,4::
 +; epiphany 3.4.1
 +sig   = 3.0:33,39,16,32,38,13,2f,35,a,5,4,ff::
 +
 +
 +label = s:!:Google bot:
 +sys   = Windows,@unix
 +sig   = 3.1:c011,5,4,2f,a,35,33,32,16,13,39,38,ff:?0,b,a,23,f:
 +sig   = 3.1:4,5,a,13,16,2f,32,33,35,38,39,c011,ff:?0,b,a,23:
 +
 +label = s:!:Akregator bot:
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,35,16,13,a,33,32,2f,7,5,4,15,12,9,14,11,8,6,3,ff:23:compr
 +
 +label = s:!:BLP_bbot:
 +sys   = Windows,@unix
 +sig   = 3.1:39,38,35,16,13,a,700c0,33,32,2f,7,50080,30080,5,4,10080,15,12,9,60040,14,11,8,6,40080,3,20080,ff::v2
 +
 +label = s:!:YandexBot:
 +sys   = Windows,@unix
 +sig   = 3.0:a,5,4::
 +
 +label = s:!:Facebook bot:
 +sys   = Windows,@unix
 +sig   = 3.1:c022,c021,39,38,88,87,35,84,c01c,c01b,16,13,a,c01f,c01e,33,32,9a,99,45,44,2f,96,41,5,4,15,12,9,14,11,8,6,3,ff:?0,f:compr
 +
 +label = s:!:Various bots:
 +sys   = Windows,@unix
 +sig   = 3.1:4,10080,5,2f,33,32,a,700c0,16,13,9,60040,15,12,3,20080,8,14,11,ff::v2