Skip to content

Instantly share code, notes, and snippets.

@aodhan-domhnaill

aodhan-domhnaill/gist:495b56163df8d89bbeb5042be3bdc4b3

Created February 12, 2018 21:26
Show Gist options
  • Save aodhan-domhnaill/495b56163df8d89bbeb5042be3bdc4b3 to your computer and use it in GitHub Desktop.
Save aodhan-domhnaill/495b56163df8d89bbeb5042be3bdc4b3 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
gistfile1.txt
[21:02:50] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,776] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,776 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,776] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,776 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:50 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
[21:02:50] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,827] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,827 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,828] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,828 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:50 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
[21:02:50] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,867] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,867 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,868] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:50 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
2018-02-12 21:02:50,868 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[21:02:50] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,933] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,933 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,934] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,934 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:50 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
[21:02:50] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,957] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,957 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:50,958] ERROR in serving: [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:50,958 ERROR - model server - [21:02:50] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:50 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
[21:02:51] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:51,004] ERROR in serving: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:51,004 ERROR - model server - [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:51,005] ERROR in serving: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:51,005 ERROR - model server - [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:51 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
[21:02:51] /tmp/mxnet/dmlc-core/include/dmlc/./logging.h:308: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:51,085] ERROR in serving: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:51,085 ERROR - model server - [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
Traceback (most recent call last):
File "/opt/amazon/lib/python2.7/site-packages/container_support/serving.py", line 161, in _invoke
self.transformer.transform(content, input_content_type, requested_output_content_type)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 49, in transform
return self.transform_fn(self.model, input_data, content_type, accept)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 118, in transform_fn
p = predict_fn(block, i)
File "/opt/amazon/lib/python2.7/site-packages/mxnet_container/serve/transformer.py", line 150, in _default_predict_fn
return block(ndarray)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 290, in __call__
return self.forward(*args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 562, in forward
return self._call_cached_op(x, *args)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/gluon/block.py", line 394, in _call_cached_op
out = self._cached_op(*cargs)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/_ctypes/ndarray.py", line 146, in __call__
ctypes.byref(out_stypes)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/base.py", line 146, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
MXNetError: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
[2018-02-12 21:02:51,086] ERROR in serving: [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
2018-02-12 21:02:51,086 ERROR - model server - [21:02:51] src/core/pass.cc:43: Graph attr dependency shape is required by pass PlanMemory but is not available The attribute is provided by pass InferShape
Stack trace returned 10 entries:
[bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm11ApplyPassesENS_5GraphERKSt6vectorINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESaIS7_EE+0x1003) [0x7fc2616fe653]
[bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN4nnvm9ApplyPassENS_5GraphERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE+0xca) [0x7fc26109de0a]
[bt] (2) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10imperative10PlanMemoryEPN4nnvm5GraphEOSt6vectorIiSaIiEERKS4_IjSaIjEERKSt4pairIjjESF_+0x2a5) [0x7fc2610a5565]
[bt] (3) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp15GetForwardGraphEbRKSt6vectorIPNS_7NDArrayESaIS4_EE+0xab7) [0x7fc2610941e7]
[bt] (4) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(_ZN5mxnet10Imperative8CachedOp7ForwardERKSt6vectorIPNS_7NDArrayESaIS4_EES8_+0xa3) [0x7fc2610949d3]
[bt] (5) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOp+0xeba) [0x7fc26151bf8a]
[bt] (6) /usr/local/lib/python2.7/dist-packages/mxnet-0.12.1-py2.7.egg/mxnet/libmxnet.so(MXInvokeCachedOpEx+0x3ee) [0x7fc26151d8fe]
[bt] (7) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call_unix64+0x4c) [0x7fc26ed0ae40]
[bt] (8) /usr/lib/x86_64-linux-gnu/libffi.so.6(ffi_call+0x2eb) [0x7fc26ed0a8ab]
[bt] (9) /usr/lib/python2.7/lib-dynload/_ctypes.x86_64-linux-gnu.so(_ctypes_callproc+0x48f) [0x7fc26ef1a3df]
10.32.0.1 - - [12/Feb/2018:21:02:51 +0000] "POST /invocations HTTP/1.1" 500 0 "-" "AHC/2.0"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment