chianingwang

$ cat elk-server-dpkg.yml
---
#
# Deploying ELK stack server using .deb files
#

- name: ELK server deployment
  hosts: elk-log-replay
  remote_user: ss
  become: yes

chianingwang

{u'action': u'create_container', u'headers': {}, u'container': u'24d3', u'response_dict': {'status': 201, 'headers': {u'date': u'Thu, 09 Mar 2017 05:41:49 GMT', u'content-length': u'0', u'content-type': u'text/html; charset=UTF-8', u'x-openstack-request-id': u'tx9edd1d7ba9f74509bea73-0058c0eb1d', u'x-trans-id': u'tx9edd1d7ba9f74509bea73-0058c0eb1d'}, 'reason': 'Created', 'response_dicts': [{'status': 201, 'headers': {u'date': u'Thu, 09 Mar 2017 05:41:49 GMT', u'content-length': u'0', u'content-type': u'text/html; charset=UTF-8', u'x-openstack-request-id': u'tx9edd1d7ba9f74509bea73-0058c0eb1d', u'x-trans-id': u'tx9edd1d7ba9f74509bea73-0058c0eb1d'}, 'reason': 'Created'}]}, u'success': True}
{u'status': u'uploaded', u'large_object': False, u'container': u'24d3', u'success': True, u'object': u'fc558eba5928785296b0c77e17e224d3', u'headers': {u'x-object-meta-mtime': u'1489038038.000000'}, u'attempts': 1, u'action': u'upload_object', u'path': '/Users/jwang/Documents/tmp/resource/slo/fc558eba5928785296b0c77e17e224d3'

chianingwang

Johnnys-MacBook-Pro:resource jwang$ python deduper.py ./libclsPyVariableChunk.dll ./slo | python -mjson.tool
[
    {
        "etag": "f27c7cf9be58f29fb4eb6170095f3b29",
        "path": "/3b29/f27c7cf9be58f29fb4eb6170095f3b29",
        "size_bytes": 371985
    },
    {
        "etag": "1cacd9462d54ccafd783122c1af1cb25",
        "path": "/cb25/1cacd9462d54ccafd783122c1af1cb25",

chianingwang

[global]
user = dedup
auth = https://cloud.swiftstack.com/auth/v1.0
key = dedup
insecure = False

# default is SLO
use_slo = False

# default segment_size is 10MB

chianingwang

On Windows many types from multiprocessing need to be picklable example

You can work around this by using an absolute import for OtherCustomClass:

from base_module import OtherCustomClass
I'm not exactly sure why, but it seems that when multiprocessing spawns a new process and imports your __main__, it's not able to handle the implicit relative import you're using with OtherCustomClass. If you explicitly import it from base_module, it works fine. My guess is that the spawned child process is not recognized as being part of the base_module package, so the implicit import fails, but that's just a guess.

Note that you shouldn't be using implicit relative imports anyway (they're altogether removed from Python 3), so switching to an absolute import isn't a bad thing.

Also of note, that doing an explicit relative import works on Python 3.4:

chianingwang

johnny@ubuntu-VirtualBox:~/Desktop$ java -jar JBox.jar q f
run: q, username:jb, password:jb, level:f
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|File Directory & Name                          |File GUID                       |File Content Hash               |Create Time         |Update Time         |Size(Byte)          |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
|/home/vagrant/Workspace/SyncFolder/ran_610K.0.0|dd091a049adf42deb86a406b3e6e60d5|a49ccd82312e779d86ef5f72c1097cb5|2017/02/21 01:06:56 |2017/02/21 01:06:56 |624640              |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
johnny@ubuntu-VirtualBox:~/D

chianingwang

C:\Users\Johnny\Documents\GitHub\swift-bulk-uploader>pyinstaller --onefile --windowed .\bulk_uploader\bulk_uploader.py
137 INFO: PyInstaller: 3.2.1
137 INFO: Python: 2.7.12
138 INFO: Platform: Windows-10-10.0.15031
144 INFO: wrote C:\Users\Johnny\Documents\GitHub\swift-bulk-uploader\bulk_uploader.spec
147 INFO: UPX is not available.
151 INFO: Extending PYTHONPATH with paths
['C:\\Users\\Johnny\\Documents\\GitHub\\swift-bulk-uploader',
 'C:\\Users\\Johnny\\Documents\\GitHub\\swift-bulk-uploader']
151 INFO: checking Analysis

chianingwang

[
    {
        "path": "/9f96/210f488c1594e003049809c0e9909f96",
        "size_bytes": 695383,
        "etag": "210f488c1594e003049809c0e9909f96"
    },
    {
        "path": "/b6cf/4697dc1a297b63f56433d9d797f9b6cf",
        "size_bytes": 1033323,
        "etag": "4697dc1a297b63f56433d9d797f9b6cf"

chianingwang

#import requests
from ctypes import *
import swiftclient
from swiftclient.service import SwiftService, SwiftUploadObject
#from swiftclient.client import put_object
import os
import glob
from sys import argv

_opts = {

chianingwang

from __future__ import print_function
import sys
import os
from ctypes import *
import requests
import subprocess
from subprocess import call
import re
import glob