andresriancho

FROM ubuntu:14.04

# Update Ubuntu
RUN apt-get update
RUN apt-get -y upgrade

# Add oracle java 7 repository
RUN apt-get -y install software-properties-common
RUN add-apt-repository ppa:webupd8team/java
RUN apt-get -y update

andresriancho

Yesterday I completed the development of the REST API for w3af :) The documentation can be found here and the code is ready to use in the develop branch:

git clone https://github.com/andresriancho/w3af.git
cd w3af
git checkout develop
./w3af_api

Before merging it to the master branch I would love to hear your opinions, bug reports, etc. Thanks!

andresriancho

"""
    cswsh.py

    Copyright 2006 Andres Riancho

    This file is part of w3af, http://w3af.org/ .

    w3af is free software; you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation version 2 of the License.

andresriancho

TL;DR Use ruamel.ordereddict instead of Python's OrderedDict. Significant improvement in memory and CPU usage.

I was having memory issues with w3af so I started to experiment with different libraries for ordered dicts (since profiling was showing some strange things in that area). These are the results of some memory profiling tests I run:

• Lower memory usage: 11.574 MiB ruamel.ordereddict
• Higher memory usage: 69.742 MiB SQLMap's ordered dict; almost the same memory usage as the Python's collections.OrderedDict

When running the tests 100 times using timeit this is what I got on my workstation:

• Faster: ruamel.ordereddict 7.66 seconds

andresriancho

def debug_pickle(instance):
    """
    :return: Which attribute from this object can't be pickled?
    """
    attribute = None

    for k, v in instance.__dict__.iteritems():
        try:
            cPickle.dumps(v)
        except:

andresriancho

Description

In this talk we’ll discuss the challenge of profiling CPU, RAM and key performance metrics in non-trivial Python applications. There are many great tools for finding CPU and RAM bottlenecks such as memory_profiler [0], objgraph [1] and line_profiler [2], but they all seem to be focused on profiling small functions.

Real world Python applications have tens of thousand of lines of code, use threads, sub-processes and other difficult-to-profile technologies. In this situation, the previous libraries do not scale well.

The talk goal is to the attendees through the tools and techniques used to profile w3af, a complex Python application maintained by the speaker. The techniques [3] are reusable and allow to gain deep insight for:

• Top 10 functions with the most CPU usage
• Top 10 lines of code allocating the most memory

andresriancho

import time
import requests


while 1:
    try:
        time.sleep(0.1)
    except KeyboardInterrupt:
        break

andresriancho

>>> import hashlib
>>> secret = 'areallylongsecret'
>>> data = 'product_id=321&price=890.99'
>>> hashlib.md5(secret + data).hexdigest()
'99180b25a0c8a2b4e4981165a7223a8b'


$ hashpump 
Input Signature: 99180b25a0c8a2b4e4981165a7223a8b
Input Data: product_id=321&price=890.99

andresriancho

import socket
import ssl

sock = socket.create_connection(('dadario.com.br', 443))
sock = ssl.wrap_socket(sock, None, None, ssl_version=ssl.PROTOCOL_TLSv1)

andresriancho

    # Unfortunately, the main ipython launch script historically had no
    # "if __name__ == '__main__'" guard, so we work around that
    # by treating it like a __main__.py file
    # See https://github.com/ipython/ipython/issues/4698
    main_name = os.path.splitext(os.path.basename(main_path))[0]
    if main_name == 'ipython':
        return