Skip to content

Instantly share code, notes, and snippets.

@davidberard98

davidberard98/nooptim.txt

Created November 7, 2022 23:19
Show Gist options
  • Save davidberard98/d7f49b110c1bb796208a9ad6e2cabb60 to your computer and use it in GitHub Desktop.
Save davidberard98/d7f49b110c1bb796208a9ad6e2cabb60 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
nooptim.txt
This file has been truncated, but you can view the full file.
WARNING:__main__:Sequence Length not defined for MobileBertForMaskedLM. Choosing 128 arbitrarily
[2022-11-07 20:06:13,575] torch._dynamo.testing: [WARNING] High loss value alert - 10.43. Can result in unstable gradients.
cuda train MobileBertForMaskedLM [2022-11-07 20:06:15,061] torch._dynamo.testing: [WARNING] High loss value alert - 10.43. Can result in unstable gradients.
[2022-11-07 20:06:16,693] torch._dynamo.testing: [WARNING] High loss value alert - 10.43. Can result in unstable gradients.
[2022-11-07 20:06:18,063] torch._dynamo.testing: [WARNING] High loss value alert - 10.43. Can result in unstable gradients.
[2022-11-07 20:06:19,501] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/contextlib.py
[2022-11-07 20:06:19,501] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/contextlib.py
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing run_n_iterations
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py:1019
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL range []
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST n [BuiltinVariable(range)]
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [BuiltinVariable(range), ConstantVariable(int)]
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBTRACT None [BuiltinVariable(range), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:19,506] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(range), ConstantVariable(int)]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [RangeVariable()]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 34 [ListIteratorVariable()]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST _ [ListIteratorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py:1020
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [ListIteratorVariable()]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR model_iter_fn [ListIteratorVariable(), UserDefinedObjectVariable(HuggingfaceRunner)]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mod [ListIteratorVariable(), UserDefinedObjectVariable(forward_and_backward_pass)]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs [ListIteratorVariable(), UserDefinedObjectVariable(forward_and_backward_pass), NNModuleVariable()]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST False [ListIteratorVariable(), UserDefinedObjectVariable(forward_and_backward_pass), NNModuleVariable(), ConstDictVariable()]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('collect_outputs',) [ListIteratorVariable(), UserDefinedObjectVariable(forward_and_backward_pass), NNModuleVariable(), ConstDictVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:19,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 3 [ListIteratorVariable(), UserDefinedObjectVariable(forward_and_backward_pass), NNModuleVariable(), ConstDictVariable(), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:19,509] torch._dynamo.convert_frame: [CODE] ORIGINAL BYTECODE run_n_iterations /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py line 1018
1019 0 LOAD_GLOBAL 0 (range)
2 LOAD_FAST 3 (n)
4 LOAD_CONST 1 (1)
6 BINARY_SUBTRACT
8 CALL_FUNCTION 1
10 GET_ITER
>> 12 FOR_ITER 20 (to 34)
14 STORE_FAST 4 (_)
1020 16 LOAD_FAST 0 (self)
18 LOAD_ATTR 1 (model_iter_fn)
20 LOAD_FAST 1 (mod)
22 LOAD_FAST 2 (inputs)
24 LOAD_CONST 2 (False)
26 LOAD_CONST 3 (('collect_outputs',))
28 CALL_FUNCTION_KW 3
30 POP_TOP
32 JUMP_ABSOLUTE 12
1021 >> 34 LOAD_FAST 0 (self)
36 LOAD_ATTR 1 (model_iter_fn)
38 LOAD_FAST 1 (mod)
40 LOAD_FAST 2 (inputs)
42 LOAD_CONST 4 (True)
44 LOAD_CONST 3 (('collect_outputs',))
46 CALL_FUNCTION_KW 3
48 RETURN_VALUE
[2022-11-07 20:06:19,510] torch._dynamo.convert_frame: [CODE] MODIFIED BYTECODE run_n_iterations /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py line 1018
1018 0 BUILD_TUPLE 0
2 GET_ITER
4 LOAD_FAST 0 (self)
6 LOAD_ATTR 1 (model_iter_fn)
8 LOAD_FAST 1 (mod)
10 LOAD_FAST 2 (inputs)
12 LOAD_CONST 2 (False)
14 LOAD_CONST 3 (('collect_outputs',))
1020 16 CALL_FUNCTION_KW 3
18 LOAD_GLOBAL 2 (__resume_at_30_0)
20 ROT_THREE
22 LOAD_FAST 0 (self)
24 LOAD_FAST 1 (mod)
26 LOAD_FAST 2 (inputs)
28 CALL_FUNCTION 5
30 RETURN_VALUE
[2022-11-07 20:06:19,687] torch._dynamo.convert_frame: [CODE] GUARDS:
-
local 'n' CONSTANT_MATCH
{
'guard_types': ['EQUALS_MATCH'],
'code': ['___check_type_id(n, 94677284981600)', 'n == 2'],
'obj_weakref': None
'guarded_class': <weakref at 0x7f1936276090; to 'type' at 0x561bc5f08b60 (int)>
}
-
local 'mod' NN_MODULE
{
'guard_types': ['ID_MATCH'],
'code': ['___check_obj_id(mod, 139732769588896)'],
'obj_weakref': <weakref at 0x7f1612016590; to 'MobileBertForMaskedLM' at 0x7f16121846a0>
'guarded_class': <weakref at 0x7f161b836090; to 'type' at 0x561bcdcaaad0 (MobileBertForMaskedLM)>
}
-
local 'self' TYPE_MATCH
{
'guard_types': ['TYPE_MATCH'],
'code': ['___check_type_id(self, 94677416862032)'],
'obj_weakref': <weakref at 0x7f16120166d0; to 'HuggingfaceRunner' at 0x7f161be94ee0>
'guarded_class': <weakref at 0x7f161bf23f90; to 'type' at 0x561bcdcce150 (HuggingfaceRunner)>
}
-
local 'inputs' DICT_KEYS
{
'guard_types': ['DICT_KEYS'],
'code': ['___check_type_id(inputs, 94677284979904)', "set(inputs.keys()) == {'labels', 'input_ids'}"],
'obj_weakref': None
'guarded_class': <weakref at 0x7f19362830e0; to 'type' at 0x561bc5f084c0 (dict)>
}
-
local "inputs['labels']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f16131e2450; to 'Tensor' at 0x7f1613a6a900>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
-
local 'self.model_iter_fn' TYPE_MATCH
{
'guard_types': ['TYPE_MATCH'],
'code': ['___check_type_id(self.model_iter_fn, 94677284851936)'],
'obj_weakref': None
'guarded_class': <weakref at 0x7f1936294ae0; to 'type' at 0x561bc5ee90e0 (method)>
}
-
local "inputs['input_ids']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f16131e2270; to 'Tensor' at 0x7f1613f97a40>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
-
global 'range' BUILTIN_MATCH
{
'guard_types': None,
'code': None,
'obj_weakref': None
'guarded_class': None
}
[2022-11-07 20:06:19,690] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing forward_and_backward_pass
[2022-11-07 20:06:19,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line benchmarks/dynamo/huggingface.py:429
[2022-11-07 20:06:19,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL clone_inputs []
[2022-11-07 20:06:19,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs [SkipFilesVariable()]
[2022-11-07 20:06:19,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [SkipFilesVariable(), ConstDictVariable()]
[2022-11-07 20:06:19,692] torch._dynamo.convert_frame: [CODE] ORIGINAL BYTECODE forward_and_backward_pass benchmarks/dynamo/huggingface.py line 428
429 0 LOAD_GLOBAL 0 (clone_inputs)
2 LOAD_FAST 2 (inputs)
4 CALL_FUNCTION 1
6 STORE_FAST 4 (cloned_inputs)
430 8 LOAD_FAST 0 (self)
10 LOAD_METHOD 1 (optimizer_zero_grad)
12 CALL_METHOD 0
14 POP_TOP
431 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (autocast)
20 CALL_METHOD 0
22 SETUP_WITH 26 (to 50)
24 POP_TOP
432 26 LOAD_FAST 1 (mod)
28 BUILD_TUPLE 0
30 LOAD_FAST 4 (cloned_inputs)
32 CALL_FUNCTION_EX 1
34 STORE_FAST 5 (pred)
433 36 LOAD_FAST 0 (self)
38 LOAD_METHOD 3 (compute_loss)
40 LOAD_FAST 5 (pred)
42 CALL_METHOD 1
44 STORE_FAST 6 (loss)
46 POP_BLOCK
48 BEGIN_FINALLY
>> 50 WITH_CLEANUP_START
52 WITH_CLEANUP_FINISH
54 END_FINALLY
434 56 LOAD_FAST 0 (self)
58 LOAD_ATTR 4 (grad_scaler)
60 LOAD_METHOD 5 (scale)
62 LOAD_FAST 6 (loss)
64 CALL_METHOD 1
66 LOAD_METHOD 6 (backward)
68 CALL_METHOD 0
70 POP_TOP
435 72 LOAD_FAST 0 (self)
74 LOAD_METHOD 7 (optimizer_step)
76 CALL_METHOD 0
78 POP_TOP
436 80 LOAD_FAST 3 (collect_outputs)
82 POP_JUMP_IF_FALSE 98
437 84 LOAD_GLOBAL 8 (collect_results)
86 LOAD_FAST 1 (mod)
88 LOAD_FAST 5 (pred)
90 LOAD_FAST 6 (loss)
92 LOAD_FAST 4 (cloned_inputs)
94 CALL_FUNCTION 4
96 RETURN_VALUE
438 >> 98 LOAD_CONST 0 (None)
100 RETURN_VALUE
[2022-11-07 20:06:19,692] torch._dynamo.convert_frame: [CODE] MODIFIED BYTECODE forward_and_backward_pass benchmarks/dynamo/huggingface.py line 428
428 0 LOAD_GLOBAL 0 (clone_inputs)
2 LOAD_FAST 2 (inputs)
429 4 CALL_FUNCTION 1
6 LOAD_GLOBAL 9 (__resume_at_6_1)
8 ROT_TWO
10 LOAD_FAST 0 (self)
12 LOAD_FAST 1 (mod)
14 LOAD_FAST 3 (collect_outputs)
16 CALL_FUNCTION 4
18 RETURN_VALUE
[2022-11-07 20:06:19,693] torch._dynamo.convert_frame: [CODE] GUARDS:
-
local 'inputs' DICT_KEYS
{
'guard_types': ['DICT_KEYS'],
'code': ['___check_type_id(inputs, 94677284979904)', "set(inputs.keys()) == {'labels', 'input_ids'}"],
'obj_weakref': None
'guarded_class': <weakref at 0x7f19362830e0; to 'type' at 0x561bc5f084c0 (dict)>
}
-
local "inputs['labels']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f16131e2450; to 'Tensor' at 0x7f1613a6a900>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
-
local "inputs['input_ids']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f16131e2270; to 'Tensor' at 0x7f1613f97a40>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
-
global 'clone_inputs' FUNCTION_MATCH
{
'guard_types': None,
'code': None,
'obj_weakref': None
'guarded_class': None
}
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping clone_inputs /scratch/dberard/dynamo38/pytorch/torch/_dynamo/utils.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping clone_input /scratch/dberard/dynamo38/pytorch/torch/_dynamo/utils.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /scratch/dberard/dynamo38/pytorch/torch/autograd/grad_mode.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping is_scripting /scratch/dberard/dynamo38/pytorch/torch/_jit_internal.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping __enter__ /scratch/dberard/dynamo38/pytorch/torch/autograd/grad_mode.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping __init__ /scratch/dberard/dynamo38/pytorch/torch/autograd/grad_mode.py
[2022-11-07 20:06:19,693] torch._dynamo.eval_frame: [DEBUG] skipping <genexpr> /scratch/dberard/dynamo38/pytorch/torch/_dynamo/utils.py
[2022-11-07 20:06:19,694] torch._dynamo.eval_frame: [DEBUG] skipping __exit__ /scratch/dberard/dynamo38/pytorch/torch/autograd/grad_mode.py
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing <graph break in forward_and_backward_pass>
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ___stack0 []
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 10 [ConstDictVariable()]
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST cloned_inputs [ConstDictVariable()]
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line benchmarks/dynamo/huggingface.py:430
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR optimizer_zero_grad [UserDefinedObjectVariable(HuggingfaceRunner)]
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(UserDefinedObjectVariable(HuggingfaceRunner), optimizer_zero_grad)]
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object optimizer_zero_grad at 0x7f16c48477c0, file "/scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py", line 1023>
1024 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (optimizer)
4 LOAD_CONST 0 (None)
6 COMPARE_OP 9 (is not)
8 POP_JUMP_IF_FALSE 22
1025 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 0 (optimizer)
14 LOAD_METHOD 1 (zero_grad)
16 LOAD_CONST 1 (True)
18 CALL_METHOD 1
20 POP_TOP
>> 22 LOAD_CONST 0 (None)
24 RETURN_VALUE
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py:1024
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR optimizer [UserDefinedObjectVariable(HuggingfaceRunner)]
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 22 [ConstantVariable(bool)]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None []
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object optimizer_zero_grad at 0x7f16c48477c0, file "/scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py", line 1023>
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_TOP None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line benchmarks/dynamo/huggingface.py:431
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR autocast [UserDefinedObjectVariable(HuggingfaceRunner)]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [UserDefinedClassVariable()]
[2022-11-07 20:06:19,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE SETUP_WITH 54 [UserDefinedObjectVariable(NullContext)]
[2022-11-07 20:06:19,699] torch._dynamo.convert_frame: [CODE] ORIGINAL BYTECODE <graph break in forward_and_backward_pass> benchmarks/dynamo/huggingface.py line 429
429 0 LOAD_FAST 0 (___stack0)
2 JUMP_ABSOLUTE 10
4 LOAD_GLOBAL 0 (clone_inputs)
6 LOAD_FAST 4 (inputs)
8 CALL_FUNCTION 1
>> 10 STORE_FAST 5 (cloned_inputs)
430 12 LOAD_FAST 1 (self)
14 LOAD_ATTR 1 (optimizer_zero_grad)
16 CALL_FUNCTION 0
18 POP_TOP
431 20 LOAD_FAST 1 (self)
22 LOAD_ATTR 2 (autocast)
24 CALL_FUNCTION 0
26 SETUP_WITH 26 (to 54)
28 POP_TOP
432 30 LOAD_FAST 2 (mod)
32 BUILD_TUPLE 0
34 LOAD_FAST 5 (cloned_inputs)
36 CALL_FUNCTION_EX 1
38 STORE_FAST 6 (pred)
433 40 LOAD_FAST 1 (self)
42 LOAD_ATTR 3 (compute_loss)
44 LOAD_FAST 6 (pred)
46 CALL_FUNCTION 1
48 STORE_FAST 7 (loss)
50 POP_BLOCK
52 BEGIN_FINALLY
>> 54 WITH_CLEANUP_START
56 WITH_CLEANUP_FINISH
58 END_FINALLY
434 60 LOAD_FAST 1 (self)
62 LOAD_ATTR 4 (grad_scaler)
64 LOAD_ATTR 5 (scale)
66 LOAD_FAST 7 (loss)
68 CALL_FUNCTION 1
70 LOAD_ATTR 6 (backward)
72 CALL_FUNCTION 0
74 POP_TOP
435 76 LOAD_FAST 1 (self)
78 LOAD_ATTR 7 (optimizer_step)
80 CALL_FUNCTION 0
82 POP_TOP
436 84 LOAD_FAST 3 (collect_outputs)
86 POP_JUMP_IF_FALSE 102
437 88 LOAD_GLOBAL 8 (collect_results)
90 LOAD_FAST 2 (mod)
92 LOAD_FAST 6 (pred)
94 LOAD_FAST 7 (loss)
96 LOAD_FAST 5 (cloned_inputs)
98 CALL_FUNCTION 4
100 RETURN_VALUE
438 >> 102 LOAD_CONST 0 (None)
104 RETURN_VALUE
[2022-11-07 20:06:19,699] torch._dynamo.convert_frame: [CODE] MODIFIED BYTECODE <graph break in forward_and_backward_pass> benchmarks/dynamo/huggingface.py line 429
429 0 LOAD_FAST 0 (___stack0)
2 STORE_FAST 5 (cloned_inputs)
431 4 LOAD_FAST 1 (self)
6 LOAD_ATTR 2 (autocast)
8 CALL_FUNCTION 0
10 SETUP_WITH 26 (to 38)
12 POP_TOP
432 14 LOAD_FAST 2 (mod)
16 BUILD_TUPLE 0
18 LOAD_FAST 5 (cloned_inputs)
20 CALL_FUNCTION_EX 1
22 STORE_FAST 6 (pred)
433 24 LOAD_FAST 1 (self)
26 LOAD_ATTR 3 (compute_loss)
28 LOAD_FAST 6 (pred)
30 CALL_FUNCTION 1
32 STORE_FAST 7 (loss)
34 POP_BLOCK
36 BEGIN_FINALLY
>> 38 WITH_CLEANUP_START
40 WITH_CLEANUP_FINISH
42 END_FINALLY
434 44 LOAD_FAST 1 (self)
46 LOAD_ATTR 4 (grad_scaler)
48 LOAD_ATTR 5 (scale)
50 LOAD_FAST 7 (loss)
52 CALL_FUNCTION 1
54 LOAD_ATTR 6 (backward)
56 CALL_FUNCTION 0
58 POP_TOP
435 60 LOAD_FAST 1 (self)
62 LOAD_ATTR 7 (optimizer_step)
64 CALL_FUNCTION 0
66 POP_TOP
436 68 LOAD_FAST 3 (collect_outputs)
70 POP_JUMP_IF_FALSE 86
437 72 LOAD_GLOBAL 8 (collect_results)
74 LOAD_FAST 2 (mod)
76 LOAD_FAST 6 (pred)
78 LOAD_FAST 7 (loss)
80 LOAD_FAST 5 (cloned_inputs)
82 CALL_FUNCTION 4
84 RETURN_VALUE
438 >> 86 LOAD_CONST 0 (None)
88 RETURN_VALUE
[2022-11-07 20:06:19,700] torch._dynamo.convert_frame: [CODE] GUARDS:
-
local 'self' TYPE_MATCH
{
'guard_types': ['TYPE_MATCH'],
'code': ['___check_type_id(self, 94677416862032)'],
'obj_weakref': <weakref at 0x7f16120166d0; to 'HuggingfaceRunner' at 0x7f161be94ee0>
'guarded_class': <weakref at 0x7f161bf23f90; to 'type' at 0x561bcdcce150 (HuggingfaceRunner)>
}
-
local '___stack0' DICT_KEYS
{
'guard_types': ['DICT_KEYS'],
'code': ['___check_type_id(___stack0, 94677284979904)', "set(___stack0.keys()) == {'labels', 'input_ids'}"],
'obj_weakref': None
'guarded_class': <weakref at 0x7f19362830e0; to 'type' at 0x561bc5f084c0 (dict)>
}
-
local 'self.optimizer' CONSTANT_MATCH
{
'guard_types': ['ID_MATCH'],
'code': ['___check_obj_id(self.optimizer, 94677284969920)'],
'obj_weakref': None
'guarded_class': None
}
-
local "___stack0['labels']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f16120251d0; to 'Tensor' at 0x7f1612016b30>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
-
local "___stack0['input_ids']" TENSOR_MATCH
{
'guard_types': ['TENSOR_MATCH'],
'code': None,
'obj_weakref': <weakref at 0x7f161201def0; to 'Tensor' at 0x7f16120ce720>
'guarded_class': <weakref at 0x7f176a5b7400; to 'torch._C._TensorMeta' at 0x561bcadaaa80 (Tensor)>
}
[2022-11-07 20:06:19,700] torch._dynamo.convert_frame: [DEBUG] skipping because no torch.* __enter__ /scratch/dberard/dynamo38/pytorch/benchmarks/dynamo/common.py 185
[2022-11-07 20:06:19,700] torch._dynamo.eval_frame: [DEBUG] skipping _call_impl /scratch/dberard/dynamo38/pytorch/torch/nn/modules/module.py
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [INFO] Step 1: torchdynamo start tracing forward
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1087
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST return_dict []
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 12 [ConstantVariable(bool)]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [NNModuleVariable()]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_return_dict [HFPretrainedConfigVariable()]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST return_dict [ConstantVariable(bool)]
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1089
[2022-11-07 20:06:19,706] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR mobilebert [NNModuleVariable()]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1090
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids [NNModuleVariable()]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1091
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1092
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_ids [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1093
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST position_ids [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1094
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1095
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1096
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1097
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1098
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST return_dict [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:1089
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('attention_mask', 'token_type_ids', 'position_ids', 'head_mask', 'inputs_embeds', 'output_attentions', 'output_hidden_states', 'return_dict') [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:19,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 9 [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:19,708] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b821f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 843>
862 0 LOAD_FAST 8 (output_attentions)
2 LOAD_CONST 0 (None)
4 COMPARE_OP 9 (is not)
6 POP_JUMP_IF_FALSE 12
8 LOAD_FAST 8 (output_attentions)
10 JUMP_FORWARD 6 (to 18)
>> 12 LOAD_FAST 0 (self)
14 LOAD_ATTR 0 (config)
16 LOAD_ATTR 1 (output_attentions)
>> 18 STORE_FAST 8 (output_attentions)
864 20 LOAD_FAST 7 (output_hidden_states)
22 LOAD_CONST 0 (None)
24 COMPARE_OP 9 (is not)
26 POP_JUMP_IF_FALSE 32
28 LOAD_FAST 7 (output_hidden_states)
30 JUMP_FORWARD 6 (to 38)
>> 32 LOAD_FAST 0 (self)
34 LOAD_ATTR 0 (config)
36 LOAD_ATTR 2 (output_hidden_states)
863 >> 38 STORE_FAST 7 (output_hidden_states)
866 40 LOAD_FAST 9 (return_dict)
42 LOAD_CONST 0 (None)
44 COMPARE_OP 9 (is not)
46 POP_JUMP_IF_FALSE 52
48 LOAD_FAST 9 (return_dict)
50 JUMP_FORWARD 6 (to 58)
>> 52 LOAD_FAST 0 (self)
54 LOAD_ATTR 0 (config)
56 LOAD_ATTR 3 (use_return_dict)
>> 58 STORE_FAST 9 (return_dict)
868 60 LOAD_FAST 1 (input_ids)
62 LOAD_CONST 0 (None)
64 COMPARE_OP 9 (is not)
66 POP_JUMP_IF_FALSE 86
68 LOAD_FAST 6 (inputs_embeds)
70 LOAD_CONST 0 (None)
72 COMPARE_OP 9 (is not)
74 POP_JUMP_IF_FALSE 86
869 76 LOAD_GLOBAL 4 (ValueError)
78 LOAD_CONST 1 ('You cannot specify both input_ids and inputs_embeds at the same time')
80 CALL_FUNCTION 1
82 RAISE_VARARGS 1
84 JUMP_FORWARD 52 (to 138)
870 >> 86 LOAD_FAST 1 (input_ids)
88 LOAD_CONST 0 (None)
90 COMPARE_OP 9 (is not)
92 POP_JUMP_IF_FALSE 104
871 94 LOAD_FAST 1 (input_ids)
96 LOAD_METHOD 5 (size)
98 CALL_METHOD 0
100 STORE_FAST 10 (input_shape)
102 JUMP_FORWARD 34 (to 138)
872 >> 104 LOAD_FAST 6 (inputs_embeds)
106 LOAD_CONST 0 (None)
108 COMPARE_OP 9 (is not)
110 POP_JUMP_IF_FALSE 130
873 112 LOAD_FAST 6 (inputs_embeds)
114 LOAD_METHOD 5 (size)
116 CALL_METHOD 0
118 LOAD_CONST 0 (None)
120 LOAD_CONST 2 (-1)
122 BUILD_SLICE 2
124 BINARY_SUBSCR
126 STORE_FAST 10 (input_shape)
128 JUMP_FORWARD 8 (to 138)
875 >> 130 LOAD_GLOBAL 4 (ValueError)
132 LOAD_CONST 3 ('You have to specify either input_ids or inputs_embeds')
134 CALL_FUNCTION 1
136 RAISE_VARARGS 1
877 >> 138 LOAD_FAST 1 (input_ids)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 152
146 LOAD_FAST 1 (input_ids)
148 LOAD_ATTR 6 (device)
150 JUMP_FORWARD 4 (to 156)
>> 152 LOAD_FAST 6 (inputs_embeds)
154 LOAD_ATTR 6 (device)
>> 156 STORE_FAST 11 (device)
879 158 LOAD_FAST 2 (attention_mask)
160 LOAD_CONST 0 (None)
162 COMPARE_OP 8 (is)
164 POP_JUMP_IF_FALSE 180
880 166 LOAD_GLOBAL 7 (torch)
168 LOAD_ATTR 8 (ones)
170 LOAD_FAST 10 (input_shape)
172 LOAD_FAST 11 (device)
174 LOAD_CONST 4 (('device',))
176 CALL_FUNCTION_KW 2
178 STORE_FAST 2 (attention_mask)
881 >> 180 LOAD_FAST 3 (token_type_ids)
182 LOAD_CONST 0 (None)
184 COMPARE_OP 8 (is)
186 POP_JUMP_IF_FALSE 206
882 188 LOAD_GLOBAL 7 (torch)
190 LOAD_ATTR 9 (zeros)
192 LOAD_FAST 10 (input_shape)
194 LOAD_GLOBAL 7 (torch)
196 LOAD_ATTR 10 (long)
198 LOAD_FAST 11 (device)
200 LOAD_CONST 5 (('dtype', 'device'))
202 CALL_FUNCTION_KW 3
204 STORE_FAST 3 (token_type_ids)
886 >> 206 LOAD_FAST 0 (self)
208 LOAD_METHOD 11 (get_extended_attention_mask)
210 LOAD_FAST 2 (attention_mask)
212 LOAD_FAST 10 (input_shape)
214 CALL_METHOD 2
216 STORE_FAST 12 (extended_attention_mask)
893 218 LOAD_FAST 0 (self)
220 LOAD_METHOD 12 (get_head_mask)
222 LOAD_FAST 5 (head_mask)
224 LOAD_FAST 0 (self)
226 LOAD_ATTR 0 (config)
228 LOAD_ATTR 13 (num_hidden_layers)
230 CALL_METHOD 2
232 STORE_FAST 5 (head_mask)
895 234 LOAD_FAST 0 (self)
236 LOAD_ATTR 14 (embeddings)
896 238 LOAD_FAST 1 (input_ids)
240 LOAD_FAST 4 (position_ids)
242 LOAD_FAST 3 (token_type_ids)
244 LOAD_FAST 6 (inputs_embeds)
895 246 LOAD_CONST 6 (('input_ids', 'position_ids', 'token_type_ids', 'inputs_embeds'))
248 CALL_FUNCTION_KW 4
250 STORE_FAST 13 (embedding_output)
898 252 LOAD_FAST 0 (self)
254 LOAD_ATTR 15 (encoder)
899 256 LOAD_FAST 13 (embedding_output)
900 258 LOAD_FAST 12 (extended_attention_mask)
901 260 LOAD_FAST 5 (head_mask)
902 262 LOAD_FAST 8 (output_attentions)
903 264 LOAD_FAST 7 (output_hidden_states)
904 266 LOAD_FAST 9 (return_dict)
898 268 LOAD_CONST 7 (('attention_mask', 'head_mask', 'output_attentions', 'output_hidden_states', 'return_dict'))
270 CALL_FUNCTION_KW 6
272 STORE_FAST 14 (encoder_outputs)
906 274 LOAD_FAST 14 (encoder_outputs)
276 LOAD_CONST 8 (0)
278 BINARY_SUBSCR
280 STORE_FAST 15 (sequence_output)
907 282 LOAD_FAST 0 (self)
284 LOAD_ATTR 16 (pooler)
286 LOAD_CONST 0 (None)
288 COMPARE_OP 9 (is not)
290 EXTENDED_ARG 1
292 POP_JUMP_IF_FALSE 304
294 LOAD_FAST 0 (self)
296 LOAD_METHOD 16 (pooler)
298 LOAD_FAST 15 (sequence_output)
300 CALL_METHOD 1
302 JUMP_FORWARD 2 (to 306)
>> 304 LOAD_CONST 0 (None)
>> 306 STORE_FAST 16 (pooled_output)
909 308 LOAD_FAST 9 (return_dict)
310 EXTENDED_ARG 1
312 POP_JUMP_IF_TRUE 334
910 314 LOAD_FAST 15 (sequence_output)
316 LOAD_FAST 16 (pooled_output)
318 BUILD_TUPLE 2
320 LOAD_FAST 14 (encoder_outputs)
322 LOAD_CONST 9 (1)
324 LOAD_CONST 0 (None)
326 BUILD_SLICE 2
328 BINARY_SUBSCR
330 BINARY_ADD
332 RETURN_VALUE
912 >> 334 LOAD_GLOBAL 17 (BaseModelOutputWithPooling)
913 336 LOAD_FAST 15 (sequence_output)
914 338 LOAD_FAST 16 (pooled_output)
915 340 LOAD_FAST 14 (encoder_outputs)
342 LOAD_ATTR 18 (hidden_states)
916 344 LOAD_FAST 14 (encoder_outputs)
346 LOAD_ATTR 19 (attentions)
912 348 LOAD_CONST 10 (('last_hidden_state', 'pooler_output', 'hidden_states', 'attentions'))
350 CALL_FUNCTION_KW 4
352 RETURN_VALUE
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:862
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 12 [ConstantVariable(bool)]
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [NNModuleVariable()]
[2022-11-07 20:06:19,709] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output_attentions [HFPretrainedConfigVariable()]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST output_attentions [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:864
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states []
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 32 [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [NNModuleVariable()]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output_hidden_states [HFPretrainedConfigVariable()]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:863
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST output_hidden_states [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:866
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST return_dict []
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(bool), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 52 [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST return_dict []
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 58 [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST return_dict [ConstantVariable(bool)]
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:868
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:19,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 86 [ConstantVariable(bool)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 86 [ConstantVariable(bool)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:870
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 104 [ConstantVariable(bool)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:871
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST input_shape [SizeVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 138 []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:877
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 152 [ConstantVariable(bool)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR device [TensorVariable()]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 156 [TorchVariable(cuda:0)]
[2022-11-07 20:06:19,711] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST device [TorchVariable(cuda:0)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:879
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 180 [ConstantVariable(bool)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:880
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ones [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_shape [TorchVariable(<built-in method ones of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST device [TorchVariable(<built-in method ones of type object at 0x7f19049e30c0>), SizeVariable()]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('device',) [TorchVariable(<built-in method ones of type object at 0x7f19049e30c0>), SizeVariable(), TorchVariable(cuda:0)]
[2022-11-07 20:06:19,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<built-in method ones of type object at 0x7f19049e30c0>), SizeVariable(), TorchVariable(cuda:0), ConstantVariable(tuple)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:881
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_ids []
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 206 [ConstantVariable(bool)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:882
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR zeros [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_shape [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:19,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>), SizeVariable()]
[2022-11-07 20:06:19,716] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR long [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>), SizeVariable(), TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:19,716] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST device [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>), SizeVariable(), ConstantVariable(dtype)]
[2022-11-07 20:06:19,716] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dtype', 'device') [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>), SizeVariable(), ConstantVariable(dtype), TorchVariable(cuda:0)]
[2022-11-07 20:06:19,716] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 3 [TorchVariable(<built-in method zeros of type object at 0x7f19049e30c0>), SizeVariable(), ConstantVariable(dtype), TorchVariable(cuda:0), ConstantVariable(tuple)]
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST token_type_ids [TensorVariable()]
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:886
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR get_extended_attention_mask [NNModuleVariable()]
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [UserMethodVariable(<function ModuleUtilsMixin.get_extended_attention_mask at 0x7f161c552af0>, NNModuleVariable())]
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_shape [UserMethodVariable(<function ModuleUtilsMixin.get_extended_attention_mask at 0x7f161c552af0>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:19,718] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [UserMethodVariable(<function ModuleUtilsMixin.get_extended_attention_mask at 0x7f161c552af0>, NNModuleVariable()), TensorVariable(), SizeVariable()]
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object get_extended_attention_mask at 0x7f161cf6f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 794>
809 0 LOAD_FAST 1 (attention_mask)
2 LOAD_METHOD 0 (dim)
4 CALL_METHOD 0
6 LOAD_CONST 1 (2)
8 COMPARE_OP 2 (==)
10 POP_JUMP_IF_FALSE 20
12 LOAD_FAST 0 (self)
14 LOAD_ATTR 1 (config)
16 LOAD_ATTR 2 (is_decoder)
18 POP_JUMP_IF_TRUE 40
811 >> 20 LOAD_FAST 3 (device)
22 LOAD_CONST 2 (None)
24 COMPARE_OP 9 (is not)
26 POP_JUMP_IF_FALSE 40
812 28 LOAD_GLOBAL 3 (warnings)
30 LOAD_METHOD 4 (warn)
813 32 LOAD_CONST 3 ('The `device` argument is deprecated and will be removed in v5 of Transformers.')
34 LOAD_GLOBAL 5 (FutureWarning)
812 36 CALL_METHOD 2
38 POP_TOP
817 >> 40 LOAD_FAST 1 (attention_mask)
42 LOAD_METHOD 0 (dim)
44 CALL_METHOD 0
46 LOAD_CONST 4 (3)
48 COMPARE_OP 2 (==)
50 POP_JUMP_IF_FALSE 82
818 52 LOAD_FAST 1 (attention_mask)
54 LOAD_CONST 2 (None)
56 LOAD_CONST 2 (None)
58 BUILD_SLICE 2
60 LOAD_CONST 2 (None)
62 LOAD_CONST 2 (None)
64 LOAD_CONST 2 (None)
66 BUILD_SLICE 2
68 LOAD_CONST 2 (None)
70 LOAD_CONST 2 (None)
72 BUILD_SLICE 2
74 BUILD_TUPLE 4
76 BINARY_SUBSCR
78 STORE_FAST 4 (extended_attention_mask)
80 JUMP_FORWARD 86 (to 168)
819 >> 82 LOAD_FAST 1 (attention_mask)
84 LOAD_METHOD 0 (dim)
86 CALL_METHOD 0
88 LOAD_CONST 1 (2)
90 COMPARE_OP 2 (==)
92 POP_JUMP_IF_FALSE 144
823 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 1 (config)
98 LOAD_ATTR 2 (is_decoder)
100 POP_JUMP_IF_FALSE 118
824 102 LOAD_GLOBAL 6 (ModuleUtilsMixin)
104 LOAD_METHOD 7 (create_extended_attention_mask_for_decoder)
825 106 LOAD_FAST 2 (input_shape)
108 LOAD_FAST 1 (attention_mask)
110 LOAD_FAST 3 (device)
824 112 CALL_METHOD 3
114 STORE_FAST 4 (extended_attention_mask)
116 JUMP_ABSOLUTE 168
828 >> 118 LOAD_FAST 1 (attention_mask)
120 LOAD_CONST 2 (None)
122 LOAD_CONST 2 (None)
124 BUILD_SLICE 2
126 LOAD_CONST 2 (None)
128 LOAD_CONST 2 (None)
130 LOAD_CONST 2 (None)
132 LOAD_CONST 2 (None)
134 BUILD_SLICE 2
136 BUILD_TUPLE 4
138 BINARY_SUBSCR
140 STORE_FAST 4 (extended_attention_mask)
142 JUMP_FORWARD 24 (to 168)
830 >> 144 LOAD_GLOBAL 8 (ValueError)
831 146 LOAD_CONST 5 ('Wrong shape for input_ids (shape ')
148 LOAD_FAST 2 (input_shape)
150 FORMAT_VALUE 0
152 LOAD_CONST 6 (') or attention_mask (shape ')
154 LOAD_FAST 1 (attention_mask)
156 LOAD_ATTR 9 (shape)
158 FORMAT_VALUE 0
160 LOAD_CONST 7 (')')
162 BUILD_STRING 5
830 164 CALL_FUNCTION 1
166 RAISE_VARARGS 1
839 >> 168 LOAD_FAST 4 (extended_attention_mask)
170 LOAD_ATTR 10 (to)
172 LOAD_FAST 0 (self)
174 LOAD_ATTR 11 (dtype)
176 LOAD_CONST 8 (('dtype',))
178 CALL_FUNCTION_KW 1
180 STORE_FAST 4 (extended_attention_mask)
840 182 LOAD_CONST 9 (1.0)
184 LOAD_FAST 4 (extended_attention_mask)
186 BINARY_SUBTRACT
188 LOAD_CONST 10 (-10000.0)
190 BINARY_MULTIPLY
192 STORE_FAST 4 (extended_attention_mask)
841 194 LOAD_FAST 4 (extended_attention_mask)
196 RETURN_VALUE
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:809
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dim [TensorVariable()]
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), dim)]
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [ConstantVariable(int)]
[2022-11-07 20:06:19,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP == [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 20 [ConstantVariable(bool)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [NNModuleVariable()]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR is_decoder [HFPretrainedConfigVariable()]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 40 [ConstantVariable(bool)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:811
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST device []
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 40 [ConstantVariable(bool)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:817
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dim [TensorVariable()]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), dim)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [ConstantVariable(int)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP == [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:19,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 82 [ConstantVariable(bool)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:819
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dim [TensorVariable()]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), dim)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [ConstantVariable(int)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP == [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 144 [ConstantVariable(bool)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:823
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [NNModuleVariable()]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR is_decoder [HFPretrainedConfigVariable()]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 118 [ConstantVariable(bool)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:828
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), SliceVariable()]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), SliceVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType), SliceVariable()]
[2022-11-07 20:06:19,722] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TensorVariable(), TupleVariable()]
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST extended_attention_mask [TensorVariable()]
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 168 []
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:839
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST extended_attention_mask []
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR to [TensorVariable()]
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [GetAttrVariable(TensorVariable(), to)]
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dtype [GetAttrVariable(TensorVariable(), to), NNModuleVariable()]
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object dtype at 0x7f161cf6f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 727>
732 0 LOAD_GLOBAL 0 (get_parameter_dtype)
2 LOAD_FAST 0 (self)
4 CALL_FUNCTION 1
6 RETURN_VALUE
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:732
[2022-11-07 20:06:19,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL get_parameter_dtype []
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [UserFunctionVariable()]
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserFunctionVariable(), NNModuleVariable()]
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object get_parameter_dtype at 0x7f161cf43ea0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 153>
157 0 LOAD_CONST 1 (None)
2 STORE_FAST 1 (last_dtype)
158 4 LOAD_FAST 0 (parameter)
6 LOAD_METHOD 0 (parameters)
8 CALL_METHOD 0
10 GET_ITER
>> 12 FOR_ITER 28 (to 42)
14 STORE_FAST 2 (t)
159 16 LOAD_FAST 2 (t)
18 LOAD_ATTR 1 (dtype)
20 STORE_FAST 1 (last_dtype)
160 22 LOAD_FAST 2 (t)
24 LOAD_METHOD 2 (is_floating_point)
26 CALL_METHOD 0
28 POP_JUMP_IF_FALSE 12
161 30 LOAD_FAST 2 (t)
32 LOAD_ATTR 1 (dtype)
34 ROT_TWO
36 POP_TOP
38 RETURN_VALUE
40 JUMP_ABSOLUTE 12
163 >> 42 LOAD_FAST 1 (last_dtype)
44 LOAD_CONST 1 (None)
46 COMPARE_OP 9 (is not)
48 POP_JUMP_IF_FALSE 54
165 50 LOAD_FAST 1 (last_dtype)
52 RETURN_VALUE
169 >> 54 LOAD_GLOBAL 3 (nn)
56 LOAD_ATTR 4 (Module)
58 LOAD_GLOBAL 5 (List)
60 LOAD_GLOBAL 6 (Tuple)
62 LOAD_GLOBAL 7 (str)
64 LOAD_GLOBAL 8 (Tensor)
66 BUILD_TUPLE 2
68 BINARY_SUBSCR
70 BINARY_SUBSCR
72 LOAD_CONST 2 (('module', 'return'))
74 BUILD_CONST_KEY_MAP 2
76 LOAD_CONST 3 (<code object find_tensor_attributes at 0x7f161cf43df0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 169>)
78 LOAD_CONST 4 ('get_parameter_dtype.<locals>.find_tensor_attributes')
80 MAKE_FUNCTION 4 (annotations)
82 STORE_FAST 3 (find_tensor_attributes)
173 84 LOAD_FAST 0 (parameter)
86 LOAD_ATTR 9 (_named_members)
88 LOAD_FAST 3 (find_tensor_attributes)
90 LOAD_CONST 5 (('get_members_fn',))
92 CALL_FUNCTION_KW 1
94 STORE_FAST 4 (gen)
174 96 LOAD_CONST 1 (None)
98 STORE_FAST 5 (last_tuple)
175 100 LOAD_FAST 4 (gen)
102 GET_ITER
>> 104 FOR_ITER 34 (to 140)
106 STORE_FAST 6 (tuple)
176 108 LOAD_FAST 6 (tuple)
110 STORE_FAST 5 (last_tuple)
177 112 LOAD_FAST 6 (tuple)
114 LOAD_CONST 6 (1)
116 BINARY_SUBSCR
118 LOAD_METHOD 2 (is_floating_point)
120 CALL_METHOD 0
122 POP_JUMP_IF_FALSE 104
178 124 LOAD_FAST 6 (tuple)
126 LOAD_CONST 6 (1)
128 BINARY_SUBSCR
130 LOAD_ATTR 1 (dtype)
132 ROT_TWO
134 POP_TOP
136 RETURN_VALUE
138 JUMP_ABSOLUTE 104
181 >> 140 LOAD_FAST 5 (last_tuple)
142 LOAD_CONST 6 (1)
144 BINARY_SUBSCR
146 LOAD_ATTR 1 (dtype)
148 RETURN_VALUE
150 LOAD_CONST 1 (None)
152 RETURN_VALUE
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:157
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None []
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST last_dtype [ConstantVariable(NoneType)]
[2022-11-07 20:06:19,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:158
[2022-11-07 20:06:19,730] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST parameter []
[2022-11-07 20:06:19,730] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR parameters [NNModuleVariable()]
[2022-11-07 20:06:19,730] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [UserMethodVariable(<function Module.parameters at 0x7f16c7890670>, NNModuleVariable())]
[2022-11-07 20:06:22,684] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [ListIteratorVariable()]
[2022-11-07 20:06:22,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 42 [ListIteratorVariable()]
[2022-11-07 20:06:22,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST t [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:22,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:159
[2022-11-07 20:06:22,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST t [ListIteratorVariable()]
[2022-11-07 20:06:22,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dtype [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST last_dtype [ListIteratorVariable(), TorchVariable(torch.float32)]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:160
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST t [ListIteratorVariable()]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR is_floating_point [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [ListIteratorVariable(), GetAttrVariable(TensorVariable(), is_floating_point)]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 12 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:161
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST t [ListIteratorVariable()]
[2022-11-07 20:06:22,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dtype [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:22,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE ROT_TWO None [ListIteratorVariable(), TorchVariable(torch.float32)]
[2022-11-07 20:06:22,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_TOP None [TorchVariable(torch.float32), ListIteratorVariable()]
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TorchVariable(torch.float32)]
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object get_parameter_dtype at 0x7f161cf43ea0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 153>
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TorchVariable(torch.float32)]
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object dtype at 0x7f161cf6f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 727>
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dtype',) [GetAttrVariable(TensorVariable(), to), TorchVariable(torch.float32)]
[2022-11-07 20:06:22,697] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 1 [GetAttrVariable(TensorVariable(), to), TorchVariable(torch.float32), ConstantVariable(tuple)]
[2022-11-07 20:06:22,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST extended_attention_mask [TensorVariable()]
[2022-11-07 20:06:22,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:840
[2022-11-07 20:06:22,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1.0 []
[2022-11-07 20:06:22,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST extended_attention_mask [ConstantVariable(float)]
[2022-11-07 20:06:22,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBTRACT None [ConstantVariable(float), TensorVariable()]
[2022-11-07 20:06:22,700] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -10000.0 [TensorVariable()]
[2022-11-07 20:06:22,700] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:22,701] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST extended_attention_mask [TensorVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:841
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST extended_attention_mask []
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object get_extended_attention_mask at 0x7f161cf6f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 794>
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST extended_attention_mask [TensorVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:893
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR get_head_mask [NNModuleVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [UserMethodVariable(<function ModuleUtilsMixin.get_head_mask at 0x7f161c552b80>, NNModuleVariable())]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [UserMethodVariable(<function ModuleUtilsMixin.get_head_mask at 0x7f161c552b80>, NNModuleVariable()), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR config [UserMethodVariable(<function ModuleUtilsMixin.get_head_mask at 0x7f161c552b80>, NNModuleVariable()), ConstantVariable(NoneType), NNModuleVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_hidden_layers [UserMethodVariable(<function ModuleUtilsMixin.get_head_mask at 0x7f161c552b80>, NNModuleVariable()), ConstantVariable(NoneType), HFPretrainedConfigVariable()]
[2022-11-07 20:06:22,702] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [UserMethodVariable(<function ModuleUtilsMixin.get_head_mask at 0x7f161c552b80>, NNModuleVariable()), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,703] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object get_head_mask at 0x7f161cf6f870, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 843>
861 0 LOAD_FAST 1 (head_mask)
2 LOAD_CONST 1 (None)
4 COMPARE_OP 9 (is not)
6 POP_JUMP_IF_FALSE 40
862 8 LOAD_FAST 0 (self)
10 LOAD_METHOD 0 (_convert_head_mask_to_5d)
12 LOAD_FAST 1 (head_mask)
14 LOAD_FAST 2 (num_hidden_layers)
16 CALL_METHOD 2
18 STORE_FAST 1 (head_mask)
863 20 LOAD_FAST 3 (is_attention_chunked)
22 LOAD_CONST 2 (True)
24 COMPARE_OP 8 (is)
26 POP_JUMP_IF_FALSE 50
864 28 LOAD_FAST 1 (head_mask)
30 LOAD_METHOD 1 (unsqueeze)
32 LOAD_CONST 3 (-1)
34 CALL_METHOD 1
36 STORE_FAST 1 (head_mask)
38 JUMP_FORWARD 10 (to 50)
866 >> 40 LOAD_CONST 1 (None)
42 BUILD_LIST 1
44 LOAD_FAST 2 (num_hidden_layers)
46 BINARY_MULTIPLY
48 STORE_FAST 1 (head_mask)
868 >> 50 LOAD_FAST 1 (head_mask)
52 RETURN_VALUE
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:861
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 40 [ConstantVariable(bool)]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:866
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None []
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_LIST 1 [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST num_hidden_layers [ListVariable()]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST head_mask [ListVariable()]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py:868
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [ListVariable()]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object get_head_mask at 0x7f161cf6f870, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/modeling_utils.py", line 843>
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST head_mask [ListVariable()]
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:895
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,704] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:896
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids [NNModuleVariable()]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST position_ids [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_ids [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), TensorVariable()]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:895
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('input_ids', 'position_ids', 'token_type_ids', 'inputs_embeds') [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,705] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 4 [NNModuleVariable(), TensorVariable(), ConstantVariable(NoneType), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(tuple)]
[2022-11-07 20:06:22,706] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4d40, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 197>
204 0 LOAD_FAST 1 (input_ids)
2 LOAD_CONST 0 (None)
4 COMPARE_OP 9 (is not)
6 POP_JUMP_IF_FALSE 18
205 8 LOAD_FAST 1 (input_ids)
10 LOAD_METHOD 0 (size)
12 CALL_METHOD 0
14 STORE_FAST 5 (input_shape)
16 JUMP_FORWARD 16 (to 34)
207 >> 18 LOAD_FAST 4 (inputs_embeds)
20 LOAD_METHOD 0 (size)
22 CALL_METHOD 0
24 LOAD_CONST 0 (None)
26 LOAD_CONST 1 (-1)
28 BUILD_SLICE 2
30 BINARY_SUBSCR
32 STORE_FAST 5 (input_shape)
209 >> 34 LOAD_FAST 5 (input_shape)
36 LOAD_CONST 2 (1)
38 BINARY_SUBSCR
40 STORE_FAST 6 (seq_length)
211 42 LOAD_FAST 3 (position_ids)
44 LOAD_CONST 0 (None)
46 COMPARE_OP 8 (is)
48 POP_JUMP_IF_FALSE 72
212 50 LOAD_FAST 0 (self)
52 LOAD_ATTR 1 (position_ids)
54 LOAD_CONST 0 (None)
56 LOAD_CONST 0 (None)
58 BUILD_SLICE 2
60 LOAD_CONST 0 (None)
62 LOAD_FAST 6 (seq_length)
64 BUILD_SLICE 2
66 BUILD_TUPLE 2
68 BINARY_SUBSCR
70 STORE_FAST 3 (position_ids)
214 >> 72 LOAD_FAST 2 (token_type_ids)
74 LOAD_CONST 0 (None)
76 COMPARE_OP 8 (is)
78 POP_JUMP_IF_FALSE 102
215 80 LOAD_GLOBAL 2 (torch)
82 LOAD_ATTR 3 (zeros)
84 LOAD_FAST 5 (input_shape)
86 LOAD_GLOBAL 2 (torch)
88 LOAD_ATTR 4 (long)
90 LOAD_FAST 0 (self)
92 LOAD_ATTR 1 (position_ids)
94 LOAD_ATTR 5 (device)
96 LOAD_CONST 3 (('dtype', 'device'))
98 CALL_FUNCTION_KW 3
100 STORE_FAST 2 (token_type_ids)
216 >> 102 LOAD_FAST 4 (inputs_embeds)
104 LOAD_CONST 0 (None)
106 COMPARE_OP 8 (is)
108 POP_JUMP_IF_FALSE 120
217 110 LOAD_FAST 0 (self)
112 LOAD_METHOD 6 (word_embeddings)
114 LOAD_FAST 1 (input_ids)
116 CALL_METHOD 1
118 STORE_FAST 4 (inputs_embeds)
219 >> 120 LOAD_FAST 0 (self)
122 LOAD_ATTR 7 (trigram_input)
124 POP_JUMP_IF_FALSE 230
227 126 LOAD_GLOBAL 2 (torch)
128 LOAD_ATTR 8 (cat)
229 130 LOAD_GLOBAL 9 (nn)
132 LOAD_ATTR 10 (functional)
134 LOAD_ATTR 11 (pad)
136 LOAD_FAST 4 (inputs_embeds)
138 LOAD_CONST 0 (None)
140 LOAD_CONST 0 (None)
142 BUILD_SLICE 2
144 LOAD_CONST 2 (1)
146 LOAD_CONST 0 (None)
148 BUILD_SLICE 2
150 BUILD_TUPLE 2
152 BINARY_SUBSCR
154 LOAD_CONST 4 (0)
156 LOAD_CONST 4 (0)
158 LOAD_CONST 4 (0)
160 LOAD_CONST 2 (1)
162 LOAD_CONST 4 (0)
164 LOAD_CONST 4 (0)
166 BUILD_LIST 6
168 LOAD_CONST 5 (0.0)
170 LOAD_CONST 6 (('value',))
172 CALL_FUNCTION_KW 3
230 174 LOAD_FAST 4 (inputs_embeds)
231 176 LOAD_GLOBAL 9 (nn)
178 LOAD_ATTR 10 (functional)
180 LOAD_ATTR 11 (pad)
182 LOAD_FAST 4 (inputs_embeds)
184 LOAD_CONST 0 (None)
186 LOAD_CONST 0 (None)
188 BUILD_SLICE 2
190 LOAD_CONST 0 (None)
192 LOAD_CONST 1 (-1)
194 BUILD_SLICE 2
196 BUILD_TUPLE 2
198 BINARY_SUBSCR
200 LOAD_CONST 4 (0)
202 LOAD_CONST 4 (0)
204 LOAD_CONST 2 (1)
206 LOAD_CONST 4 (0)
208 LOAD_CONST 4 (0)
210 LOAD_CONST 4 (0)
212 BUILD_LIST 6
214 LOAD_CONST 5 (0.0)
216 LOAD_CONST 6 (('value',))
218 CALL_FUNCTION_KW 3
228 220 BUILD_LIST 3
233 222 LOAD_CONST 7 (2)
227 224 LOAD_CONST 8 (('dim',))
226 CALL_FUNCTION_KW 2
228 STORE_FAST 4 (inputs_embeds)
235 >> 230 LOAD_FAST 0 (self)
232 LOAD_ATTR 7 (trigram_input)
234 POP_JUMP_IF_TRUE 250
236 LOAD_FAST 0 (self)
238 LOAD_ATTR 12 (embedding_size)
240 LOAD_FAST 0 (self)
242 LOAD_ATTR 13 (hidden_size)
244 COMPARE_OP 3 (!=)
246 EXTENDED_ARG 1
248 POP_JUMP_IF_FALSE 260
236 >> 250 LOAD_FAST 0 (self)
252 LOAD_METHOD 14 (embedding_transformation)
254 LOAD_FAST 4 (inputs_embeds)
256 CALL_METHOD 1
258 STORE_FAST 4 (inputs_embeds)
240 >> 260 LOAD_FAST 0 (self)
262 LOAD_METHOD 15 (position_embeddings)
264 LOAD_FAST 3 (position_ids)
266 CALL_METHOD 1
268 STORE_FAST 7 (position_embeddings)
241 270 LOAD_FAST 0 (self)
272 LOAD_METHOD 16 (token_type_embeddings)
274 LOAD_FAST 2 (token_type_ids)
276 CALL_METHOD 1
278 STORE_FAST 8 (token_type_embeddings)
242 280 LOAD_FAST 4 (inputs_embeds)
282 LOAD_FAST 7 (position_embeddings)
284 BINARY_ADD
286 LOAD_FAST 8 (token_type_embeddings)
288 BINARY_ADD
290 STORE_FAST 9 (embeddings)
243 292 LOAD_FAST 0 (self)
294 LOAD_METHOD 17 (LayerNorm)
296 LOAD_FAST 9 (embeddings)
298 CALL_METHOD 1
300 STORE_FAST 9 (embeddings)
244 302 LOAD_FAST 0 (self)
304 LOAD_METHOD 18 (dropout)
306 LOAD_FAST 9 (embeddings)
308 CALL_METHOD 1
310 STORE_FAST 9 (embeddings)
245 312 LOAD_FAST 9 (embeddings)
314 RETURN_VALUE
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:204
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 18 [ConstantVariable(bool)]
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:205
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids []
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:22,707] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST input_shape [SizeVariable()]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 34 []
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:209
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_shape []
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [SizeVariable()]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST seq_length [ConstantVariable(int)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:211
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST position_ids []
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 72 [ConstantVariable(bool)]
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:212
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,708] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR position_ids [NNModuleVariable()]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable(), SliceVariable()]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST seq_length [TensorVariable(), SliceVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TensorVariable(), SliceVariable(), SliceVariable()]
[2022-11-07 20:06:22,710] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TensorVariable(), TupleVariable()]
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST position_ids [TensorVariable()]
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:214
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_ids []
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 102 [ConstantVariable(bool)]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:216
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds []
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 120 [ConstantVariable(bool)]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:217
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR word_embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_ids [NNModuleVariable()]
[2022-11-07 20:06:22,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST inputs_embeds [TensorVariable()]
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:219
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR trigram_input [NNModuleVariable()]
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 230 [ConstantVariable(bool)]
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:227
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR cat [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:229
[2022-11-07 20:06:22,720] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR pad [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable()]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable()]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), SliceVariable()]
[2022-11-07 20:06:22,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), TupleVariable()]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable()]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_LIST 6 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0.0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable()]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('value',) [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable(), ConstantVariable(float)]
[2022-11-07 20:06:22,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 3 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable(), ConstantVariable(float), ConstantVariable(tuple)]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:230
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:231
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR pad [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>)]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable()]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable()]
[2022-11-07 20:06:22,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), SliceVariable(), SliceVariable()]
[2022-11-07 20:06:22,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), TupleVariable()]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable()]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_LIST 6 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0.0 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable()]
[2022-11-07 20:06:22,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('value',) [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable(), ConstantVariable(float)]
[2022-11-07 20:06:22,730] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 3 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TorchVariable(<built-in function pad>), TensorVariable(), ListVariable(), ConstantVariable(float), ConstantVariable(tuple)]
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:228
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_LIST 3 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:233
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), ListVariable()]
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:227
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<built-in method cat of type object at 0x7f19049e30c0>), ListVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:22,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST inputs_embeds [TensorVariable()]
[2022-11-07 20:06:22,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:235
[2022-11-07 20:06:22,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR trigram_input [NNModuleVariable()]
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 250 [ConstantVariable(bool)]
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:236
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR embedding_transformation [NNModuleVariable()]
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds [NNModuleVariable()]
[2022-11-07 20:06:22,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST inputs_embeds [TensorVariable()]
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:240
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR position_embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST position_ids [NNModuleVariable()]
[2022-11-07 20:06:22,746] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST position_embeddings [TensorVariable()]
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:241
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR token_type_embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_ids [NNModuleVariable()]
[2022-11-07 20:06:22,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,758] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST token_type_embeddings [TensorVariable()]
[2022-11-07 20:06:22,758] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:242
[2022-11-07 20:06:22,758] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST inputs_embeds []
[2022-11-07 20:06:22,758] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST position_embeddings [TensorVariable()]
[2022-11-07 20:06:22,758] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,759] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST token_type_embeddings [TensorVariable()]
[2022-11-07 20:06:22,759] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,760] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST embeddings [TensorVariable()]
[2022-11-07 20:06:22,760] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:243
[2022-11-07 20:06:22,761] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,761] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:22,761] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,761] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,762] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:22,762] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:22,762] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:22,762] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,762] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,765] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,769] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,770] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST embeddings [TensorVariable()]
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:244
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST embeddings [NNModuleVariable()]
[2022-11-07 20:06:22,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST embeddings [TensorVariable()]
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:245
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST embeddings []
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4d40, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 197>
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST embedding_output [TensorVariable()]
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:898
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,772] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR encoder [NNModuleVariable()]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:899
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST embedding_output [NNModuleVariable()]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:900
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST extended_attention_mask [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:901
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:902
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:903
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:904
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST return_dict [NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(bool), ConstantVariable(bool)]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:898
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('attention_mask', 'head_mask', 'output_attentions', 'output_hidden_states', 'return_dict') [NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(bool), ConstantVariable(bool), ConstantVariable(bool)]
[2022-11-07 20:06:22,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(bool), ConstantVariable(bool), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:22,774] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fea0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 566>
575 0 LOAD_FAST 5 (output_hidden_states)
2 POP_JUMP_IF_FALSE 8
4 LOAD_CONST 1 (())
6 JUMP_FORWARD 2 (to 10)
>> 8 LOAD_CONST 0 (None)
>> 10 STORE_FAST 7 (all_hidden_states)
576 12 LOAD_FAST 4 (output_attentions)
14 POP_JUMP_IF_FALSE 20
16 LOAD_CONST 1 (())
18 JUMP_FORWARD 2 (to 22)
>> 20 LOAD_CONST 0 (None)
>> 22 STORE_FAST 8 (all_attentions)
577 24 LOAD_GLOBAL 0 (enumerate)
26 LOAD_FAST 0 (self)
28 LOAD_ATTR 1 (layer)
30 CALL_FUNCTION 1
32 GET_ITER
>> 34 FOR_ITER 66 (to 102)
36 UNPACK_SEQUENCE 2
38 STORE_FAST 9 (i)
40 STORE_FAST 10 (layer_module)
578 42 LOAD_FAST 5 (output_hidden_states)
44 POP_JUMP_IF_FALSE 56
579 46 LOAD_FAST 7 (all_hidden_states)
48 LOAD_FAST 1 (hidden_states)
50 BUILD_TUPLE 1
52 BINARY_ADD
54 STORE_FAST 7 (all_hidden_states)
581 >> 56 LOAD_FAST 10 (layer_module)
582 58 LOAD_FAST 1 (hidden_states)
583 60 LOAD_FAST 2 (attention_mask)
584 62 LOAD_FAST 3 (head_mask)
64 LOAD_FAST 9 (i)
66 BINARY_SUBSCR
585 68 LOAD_FAST 4 (output_attentions)
581 70 CALL_FUNCTION 4
72 STORE_FAST 11 (layer_outputs)
587 74 LOAD_FAST 11 (layer_outputs)
76 LOAD_CONST 2 (0)
78 BINARY_SUBSCR
80 STORE_FAST 1 (hidden_states)
589 82 LOAD_FAST 4 (output_attentions)
84 POP_JUMP_IF_FALSE 34
590 86 LOAD_FAST 8 (all_attentions)
88 LOAD_FAST 11 (layer_outputs)
90 LOAD_CONST 3 (1)
92 BINARY_SUBSCR
94 BUILD_TUPLE 1
96 BINARY_ADD
98 STORE_FAST 8 (all_attentions)
100 JUMP_ABSOLUTE 34
593 >> 102 LOAD_FAST 5 (output_hidden_states)
104 POP_JUMP_IF_FALSE 116
594 106 LOAD_FAST 7 (all_hidden_states)
108 LOAD_FAST 1 (hidden_states)
110 BUILD_TUPLE 1
112 BINARY_ADD
114 STORE_FAST 7 (all_hidden_states)
596 >> 116 LOAD_FAST 6 (return_dict)
118 POP_JUMP_IF_TRUE 144
597 120 LOAD_GLOBAL 2 (tuple)
122 LOAD_CONST 4 (<code object <genexpr> at 0x7f161b81fdf0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 597>)
124 LOAD_CONST 5 ('MobileBertEncoder.forward.<locals>.<genexpr>')
126 MAKE_FUNCTION 0
128 LOAD_FAST 1 (hidden_states)
130 LOAD_FAST 7 (all_hidden_states)
132 LOAD_FAST 8 (all_attentions)
134 BUILD_TUPLE 3
136 GET_ITER
138 CALL_FUNCTION 1
140 CALL_FUNCTION 1
142 RETURN_VALUE
598 >> 144 LOAD_GLOBAL 3 (BaseModelOutput)
599 146 LOAD_FAST 1 (hidden_states)
148 LOAD_FAST 7 (all_hidden_states)
150 LOAD_FAST 8 (all_attentions)
598 152 LOAD_CONST 6 (('last_hidden_state', 'hidden_states', 'attentions'))
154 CALL_FUNCTION_KW 3
156 RETURN_VALUE
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:575
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states []
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 8 [ConstantVariable(bool)]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None []
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST all_hidden_states [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:576
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 20 [ConstantVariable(bool)]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None []
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST all_attentions [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:577
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR layer [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:22,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:22,778] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:22,778] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 102 [ListIteratorVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:578
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [ListIteratorVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_module [ListIteratorVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:582
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:583
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:584
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST i [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:22,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:585
[2022-11-07 20:06:22,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:22,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:22,781] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
519 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (use_bottleneck)
4 POP_JUMP_IF_FALSE 26
520 6 LOAD_FAST 0 (self)
8 LOAD_METHOD 1 (bottleneck)
10 LOAD_FAST 1 (hidden_states)
12 CALL_METHOD 1
14 UNPACK_SEQUENCE 4
16 STORE_FAST 5 (query_tensor)
18 STORE_FAST 6 (key_tensor)
20 STORE_FAST 7 (value_tensor)
22 STORE_FAST 8 (layer_input)
24 JUMP_FORWARD 18 (to 44)
522 >> 26 LOAD_FAST 1 (hidden_states)
28 BUILD_LIST 1
30 LOAD_CONST 1 (4)
32 BINARY_MULTIPLY
34 UNPACK_SEQUENCE 4
36 STORE_FAST 5 (query_tensor)
38 STORE_FAST 6 (key_tensor)
40 STORE_FAST 7 (value_tensor)
42 STORE_FAST 8 (layer_input)
524 >> 44 LOAD_FAST 0 (self)
46 LOAD_ATTR 2 (attention)
525 48 LOAD_FAST 5 (query_tensor)
526 50 LOAD_FAST 6 (key_tensor)
527 52 LOAD_FAST 7 (value_tensor)
528 54 LOAD_FAST 8 (layer_input)
529 56 LOAD_FAST 2 (attention_mask)
530 58 LOAD_FAST 3 (head_mask)
531 60 LOAD_FAST 4 (output_attentions)
524 62 LOAD_CONST 2 (('output_attentions',))
64 CALL_FUNCTION_KW 7
66 STORE_FAST 9 (self_attention_outputs)
533 68 LOAD_FAST 9 (self_attention_outputs)
70 LOAD_CONST 3 (0)
72 BINARY_SUBSCR
74 STORE_FAST 10 (attention_output)
534 76 LOAD_FAST 10 (attention_output)
78 BUILD_TUPLE 1
80 STORE_FAST 11 (s)
535 82 LOAD_FAST 9 (self_attention_outputs)
84 LOAD_CONST 4 (1)
86 LOAD_CONST 0 (None)
88 BUILD_SLICE 2
90 BINARY_SUBSCR
92 STORE_FAST 12 (outputs)
537 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 3 (num_feedforward_networks)
98 LOAD_CONST 4 (1)
100 COMPARE_OP 3 (!=)
102 POP_JUMP_IF_FALSE 142
538 104 LOAD_GLOBAL 4 (enumerate)
106 LOAD_FAST 0 (self)
108 LOAD_ATTR 5 (ffn)
110 CALL_FUNCTION 1
112 GET_ITER
>> 114 FOR_ITER 26 (to 142)
116 UNPACK_SEQUENCE 2
118 STORE_FAST 13 (i)
120 STORE_FAST 14 (ffn_module)
539 122 LOAD_FAST 14 (ffn_module)
124 LOAD_FAST 10 (attention_output)
126 CALL_FUNCTION 1
128 STORE_FAST 10 (attention_output)
540 130 LOAD_FAST 11 (s)
132 LOAD_FAST 10 (attention_output)
134 BUILD_TUPLE 1
136 INPLACE_ADD
138 STORE_FAST 11 (s)
140 JUMP_ABSOLUTE 114
542 >> 142 LOAD_FAST 0 (self)
144 LOAD_METHOD 6 (intermediate)
146 LOAD_FAST 10 (attention_output)
148 CALL_METHOD 1
150 STORE_FAST 15 (intermediate_output)
543 152 LOAD_FAST 0 (self)
154 LOAD_METHOD 7 (output)
156 LOAD_FAST 15 (intermediate_output)
158 LOAD_FAST 10 (attention_output)
160 LOAD_FAST 1 (hidden_states)
162 CALL_METHOD 3
164 STORE_FAST 16 (layer_output)
545 166 LOAD_FAST 16 (layer_output)
168 BUILD_TUPLE 1
546 170 LOAD_FAST 12 (outputs)
545 172 BINARY_ADD
548 174 LOAD_GLOBAL 8 (torch)
176 LOAD_METHOD 9 (tensor)
178 LOAD_CONST 5 (1000)
180 CALL_METHOD 1
549 182 LOAD_FAST 5 (query_tensor)
550 184 LOAD_FAST 6 (key_tensor)
551 186 LOAD_FAST 7 (value_tensor)
552 188 LOAD_FAST 8 (layer_input)
553 190 LOAD_FAST 10 (attention_output)
554 192 LOAD_FAST 15 (intermediate_output)
547 194 BUILD_TUPLE 7
545 196 BINARY_ADD
556 198 LOAD_FAST 11 (s)
545 200 BINARY_ADD
544 202 STORE_FAST 12 (outputs)
558 204 LOAD_FAST 12 (outputs)
206 RETURN_VALUE
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:519
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:520
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,782] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,783] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
464 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (input)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (bottlenecked_hidden_states)
465 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck_attention)
14 POP_JUMP_IF_FALSE 26
466 16 LOAD_FAST 2 (bottlenecked_hidden_states)
18 BUILD_TUPLE 1
20 LOAD_CONST 1 (4)
22 BINARY_MULTIPLY
24 RETURN_VALUE
467 >> 26 LOAD_FAST 0 (self)
28 LOAD_ATTR 2 (key_query_shared_bottleneck)
30 POP_JUMP_IF_FALSE 54
468 32 LOAD_FAST 0 (self)
34 LOAD_METHOD 3 (attention)
36 LOAD_FAST 1 (hidden_states)
38 CALL_METHOD 1
40 STORE_FAST 3 (shared_attention_input)
469 42 LOAD_FAST 3 (shared_attention_input)
44 LOAD_FAST 3 (shared_attention_input)
46 LOAD_FAST 1 (hidden_states)
48 LOAD_FAST 2 (bottlenecked_hidden_states)
50 BUILD_TUPLE 4
52 RETURN_VALUE
471 >> 54 LOAD_FAST 1 (hidden_states)
56 LOAD_FAST 1 (hidden_states)
58 LOAD_FAST 1 (hidden_states)
60 LOAD_FAST 2 (bottlenecked_hidden_states)
62 BUILD_TUPLE 4
64 RETURN_VALUE
66 LOAD_CONST 0 (None)
68 RETURN_VALUE
[2022-11-07 20:06:22,783] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:464
[2022-11-07 20:06:22,784] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,784] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR input [NNModuleVariable()]
[2022-11-07 20:06:22,784] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,784] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,785] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:22,785] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:22,785] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,785] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:22,786] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,786] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:22,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,798] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:22,798] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:22,798] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:22,798] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,798] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,801] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,803] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,803] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,805] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST bottlenecked_hidden_states [TensorVariable()]
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:465
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,807] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck_attention [NNModuleVariable()]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:467
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key_query_shared_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 54 [ConstantVariable(bool)]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:468
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,808] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,809] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:22,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:22,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:22,810] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,810] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:22,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,822] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:22,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:22,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:22,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,825] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,827] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,827] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,829] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:469
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input []
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST bottlenecked_hidden_states [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:22,831] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 4 [TupleVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_tensor [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_tensor [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_tensor [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 44 []
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:525
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:526
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:527
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:528
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:529
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:530
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:531
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,832] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:22,833] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('output_attentions',) [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:22,833] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 7 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:22,833] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
358 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (self)
359 4 LOAD_FAST 1 (query_tensor)
360 6 LOAD_FAST 2 (key_tensor)
361 8 LOAD_FAST 3 (value_tensor)
362 10 LOAD_FAST 5 (attention_mask)
363 12 LOAD_FAST 6 (head_mask)
364 14 LOAD_FAST 7 (output_attentions)
358 16 CALL_METHOD 6
18 STORE_FAST 8 (self_outputs)
368 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 1 (output)
24 LOAD_FAST 8 (self_outputs)
26 LOAD_CONST 1 (0)
28 BINARY_SUBSCR
30 LOAD_FAST 4 (layer_input)
32 CALL_METHOD 2
34 STORE_FAST 9 (attention_output)
369 36 LOAD_FAST 9 (attention_output)
38 BUILD_TUPLE 1
40 LOAD_FAST 8 (self_outputs)
42 LOAD_CONST 2 (1)
44 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
48 BINARY_SUBSCR
50 BINARY_ADD
52 STORE_FAST 10 (outputs)
370 54 LOAD_FAST 10 (outputs)
56 RETURN_VALUE
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR self [NNModuleVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:359
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:360
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:361
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:362
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:363
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:364
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:22,834] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:22,835] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
276 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (query)
4 LOAD_FAST 1 (query_tensor)
6 CALL_METHOD 1
8 STORE_FAST 7 (mixed_query_layer)
277 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (key)
14 LOAD_FAST 2 (key_tensor)
16 CALL_METHOD 1
18 STORE_FAST 8 (mixed_key_layer)
278 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (value)
24 LOAD_FAST 3 (value_tensor)
26 CALL_METHOD 1
28 STORE_FAST 9 (mixed_value_layer)
280 30 LOAD_FAST 0 (self)
32 LOAD_METHOD 3 (transpose_for_scores)
34 LOAD_FAST 7 (mixed_query_layer)
36 CALL_METHOD 1
38 STORE_FAST 10 (query_layer)
281 40 LOAD_FAST 0 (self)
42 LOAD_METHOD 3 (transpose_for_scores)
44 LOAD_FAST 8 (mixed_key_layer)
46 CALL_METHOD 1
48 STORE_FAST 11 (key_layer)
282 50 LOAD_FAST 0 (self)
52 LOAD_METHOD 3 (transpose_for_scores)
54 LOAD_FAST 9 (mixed_value_layer)
56 CALL_METHOD 1
58 STORE_FAST 12 (value_layer)
285 60 LOAD_GLOBAL 4 (torch)
62 LOAD_METHOD 5 (matmul)
64 LOAD_FAST 10 (query_layer)
66 LOAD_FAST 11 (key_layer)
68 LOAD_METHOD 6 (transpose)
70 LOAD_CONST 1 (-1)
72 LOAD_CONST 2 (-2)
74 CALL_METHOD 2
76 CALL_METHOD 2
78 STORE_FAST 13 (attention_scores)
286 80 LOAD_FAST 13 (attention_scores)
82 LOAD_GLOBAL 7 (math)
84 LOAD_METHOD 8 (sqrt)
86 LOAD_FAST 0 (self)
88 LOAD_ATTR 9 (attention_head_size)
90 CALL_METHOD 1
92 BINARY_TRUE_DIVIDE
94 STORE_FAST 13 (attention_scores)
287 96 LOAD_FAST 4 (attention_mask)
98 LOAD_CONST 0 (None)
100 COMPARE_OP 9 (is not)
102 POP_JUMP_IF_FALSE 112
289 104 LOAD_FAST 13 (attention_scores)
106 LOAD_FAST 4 (attention_mask)
108 BINARY_ADD
110 STORE_FAST 13 (attention_scores)
291 >> 112 LOAD_GLOBAL 10 (nn)
114 LOAD_ATTR 11 (functional)
116 LOAD_ATTR 12 (softmax)
118 LOAD_FAST 13 (attention_scores)
120 LOAD_CONST 1 (-1)
122 LOAD_CONST 3 (('dim',))
124 CALL_FUNCTION_KW 2
126 STORE_FAST 14 (attention_probs)
294 128 LOAD_FAST 0 (self)
130 LOAD_METHOD 13 (dropout)
132 LOAD_FAST 14 (attention_probs)
134 CALL_METHOD 1
136 STORE_FAST 14 (attention_probs)
296 138 LOAD_FAST 5 (head_mask)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 154
297 146 LOAD_FAST 14 (attention_probs)
148 LOAD_FAST 5 (head_mask)
150 BINARY_MULTIPLY
152 STORE_FAST 14 (attention_probs)
298 >> 154 LOAD_GLOBAL 4 (torch)
156 LOAD_METHOD 5 (matmul)
158 LOAD_FAST 14 (attention_probs)
160 LOAD_FAST 12 (value_layer)
162 CALL_METHOD 2
164 STORE_FAST 15 (context_layer)
299 166 LOAD_FAST 15 (context_layer)
168 LOAD_METHOD 14 (permute)
170 LOAD_CONST 4 (0)
172 LOAD_CONST 5 (2)
174 LOAD_CONST 6 (1)
176 LOAD_CONST 7 (3)
178 CALL_METHOD 4
180 LOAD_METHOD 15 (contiguous)
182 CALL_METHOD 0
184 STORE_FAST 15 (context_layer)
300 186 LOAD_FAST 15 (context_layer)
188 LOAD_METHOD 16 (size)
190 CALL_METHOD 0
192 LOAD_CONST 0 (None)
194 LOAD_CONST 2 (-2)
196 BUILD_SLICE 2
198 BINARY_SUBSCR
200 LOAD_FAST 0 (self)
202 LOAD_ATTR 17 (all_head_size)
204 BUILD_TUPLE 1
206 BINARY_ADD
208 STORE_FAST 16 (new_context_layer_shape)
301 210 LOAD_FAST 15 (context_layer)
212 LOAD_METHOD 18 (view)
214 LOAD_FAST 16 (new_context_layer_shape)
216 CALL_METHOD 1
218 STORE_FAST 15 (context_layer)
302 220 LOAD_FAST 6 (output_attentions)
222 POP_JUMP_IF_FALSE 232
224 LOAD_FAST 15 (context_layer)
226 LOAD_FAST 14 (attention_probs)
228 BUILD_TUPLE 2
230 JUMP_FORWARD 4 (to 236)
>> 232 LOAD_FAST 15 (context_layer)
234 BUILD_TUPLE 1
>> 236 STORE_FAST 17 (outputs)
303 238 LOAD_FAST 17 (outputs)
240 RETURN_VALUE
[2022-11-07 20:06:22,836] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:276
[2022-11-07 20:06:22,836] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,836] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR query [NNModuleVariable()]
[2022-11-07 20:06:22,836] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:22,836] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_query_layer [TensorVariable()]
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:277
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key [NNModuleVariable()]
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable()]
[2022-11-07 20:06:22,848] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,859] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_key_layer [TensorVariable()]
[2022-11-07 20:06:22,859] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:278
[2022-11-07 20:06:22,859] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,859] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR value [NNModuleVariable()]
[2022-11-07 20:06:22,860] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable()]
[2022-11-07 20:06:22,860] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_value_layer [TensorVariable()]
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:280
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_query_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:22,871] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:22,872] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:22,872] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:22,872] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,872] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:22,872] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:22,873] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:22,873] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,873] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,873] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:22,874] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,876] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_layer [TensorVariable()]
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:281
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,878] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:22,879] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_key_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:22,879] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:22,879] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:22,879] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,880] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:22,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:22,882] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,882] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:22,882] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:22,882] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_layer [TensorVariable()]
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:282
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_value_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:22,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:22,887] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:22,887] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:22,887] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,887] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:22,887] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:22,888] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:22,889] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,891] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_layer [TensorVariable()]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:285
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose)]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int)]
[2022-11-07 20:06:22,893] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,895] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:286
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL math [TensorVariable()]
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR sqrt [TensorVariable(), TorchVariable(<module 'math' from '/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>)]
[2022-11-07 20:06:22,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable(), TorchVariable(<built-in function sqrt>)]
[2022-11-07 20:06:22,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [TensorVariable(), TorchVariable(<built-in function sqrt>), NNModuleVariable()]
[2022-11-07 20:06:22,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TensorVariable(), TorchVariable(<built-in function sqrt>), ConstantVariable(int)]
[2022-11-07 20:06:22,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_TRUE_DIVIDE None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:287
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 112 [ConstantVariable(bool)]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:289
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:22,903] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,904] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:22,904] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:291
[2022-11-07 20:06:22,904] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn []
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR softmax [TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores [TorchVariable(<function softmax at 0x7f16c76f6a60>)]
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable()]
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:294
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [NNModuleVariable()]
[2022-11-07 20:06:22,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:296
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 154 [ConstantVariable(bool)]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:298
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:22,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:299
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,918] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,920] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR contiguous [TensorVariable()]
[2022-11-07 20:06:22,920] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), contiguous)]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:300
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:22,922] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR all_head_size [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_context_layer_shape [TupleVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:301
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:22,923] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_context_layer_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:22,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:302
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 232 [ConstantVariable(bool)]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:303
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
[2022-11-07 20:06:22,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:368
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [NNModuleVariable()]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [NNModuleVariable(), TupleVariable()]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [NNModuleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,927] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
316 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
317 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 26
318 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 3 (layer_outputs)
22 CALL_METHOD 1
24 STORE_FAST 3 (layer_outputs)
319 >> 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 3 (layer_outputs)
32 LOAD_FAST 2 (residual_tensor)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 3 (layer_outputs)
320 40 LOAD_FAST 3 (layer_outputs)
42 RETURN_VALUE
[2022-11-07 20:06:22,927] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:316
[2022-11-07 20:06:22,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:22,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:317
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:319
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,939] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:22,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:22,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,942] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:22,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:22,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:22,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,945] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,947] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,947] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,950] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:320
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:369
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:22,951] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), TupleVariable(), SliceVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:370
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:22,952] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_attention_outputs [TupleVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:533
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TupleVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:534
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [TupleVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:535
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable()]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:22,953] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), SliceVariable()]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:537
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_feedforward_networks [NNModuleVariable()]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [ConstantVariable(int)]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP != [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 142 [ConstantVariable(bool)]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:538
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:22,954] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ffn [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:22,955] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:22,956] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:22,956] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,957] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,958] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,959] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:22,959] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:22,959] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,959] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:22,960] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,960] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,971] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:22,971] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:22,971] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,971] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:22,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:22,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:22,976] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,976] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,976] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:22,977] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:22,977] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,977] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:22,977] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:22,977] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:22,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:22,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:22,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:22,989] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:22,989] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,989] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,990] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:22,991] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:22,991] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:22,991] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:22,991] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,991] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,994] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:22,995] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:22,995] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:22,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,004] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,004] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,004] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,004] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,005] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,005] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,016] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,019] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,020] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,020] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,020] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,021] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,033] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,034] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,035] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,035] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,035] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,035] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,035] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,042] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,044] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,047] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,048] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,063] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,064] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,064] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,064] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,065] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,076] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,076] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,076] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,076] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,077] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,077] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,077] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,078] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,079] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,083] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,084] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,084] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:542
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable()]
[2022-11-07 20:06:23,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,102] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,102] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,102] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,102] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,103] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,103] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,105] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:543
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,106] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 3 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,107] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
416 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (intermediate_states)
6 CALL_METHOD 1
8 STORE_FAST 4 (layer_output)
417 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 42
418 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 4 (layer_output)
22 CALL_METHOD 1
24 STORE_FAST 4 (layer_output)
419 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 4 (layer_output)
32 LOAD_FAST 2 (residual_tensor_1)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 4 (layer_output)
40 JUMP_FORWARD 26 (to 68)
421 >> 42 LOAD_FAST 0 (self)
44 LOAD_METHOD 3 (LayerNorm)
46 LOAD_FAST 4 (layer_output)
48 LOAD_FAST 2 (residual_tensor_1)
50 BINARY_ADD
52 CALL_METHOD 1
54 STORE_FAST 4 (layer_output)
422 56 LOAD_FAST 0 (self)
58 LOAD_METHOD 4 (bottleneck)
60 LOAD_FAST 4 (layer_output)
62 LOAD_FAST 3 (residual_tensor_2)
64 CALL_METHOD 2
66 STORE_FAST 4 (layer_output)
423 >> 68 LOAD_FAST 4 (layer_output)
70 RETURN_VALUE
[2022-11-07 20:06:23,107] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:416
[2022-11-07 20:06:23,107] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,107] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,108] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_states [NNModuleVariable()]
[2022-11-07 20:06:23,108] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:417
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 42 [ConstantVariable(bool)]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:421
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,119] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,121] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,122] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,127] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,127] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:422
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_2 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,132] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,132] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
396 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
397 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (dropout)
14 LOAD_FAST 3 (layer_outputs)
16 CALL_METHOD 1
18 STORE_FAST 3 (layer_outputs)
398 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (LayerNorm)
24 LOAD_FAST 3 (layer_outputs)
26 LOAD_FAST 2 (residual_tensor)
28 BINARY_ADD
30 CALL_METHOD 1
32 STORE_FAST 3 (layer_outputs)
399 34 LOAD_FAST 3 (layer_outputs)
36 RETURN_VALUE
[2022-11-07 20:06:23,133] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:396
[2022-11-07 20:06:23,133] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,133] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,133] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,133] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:397
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:398
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,146] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,148] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,151] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,153] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,153] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,155] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:399
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:423
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
[2022-11-07 20:06:23,157] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:546
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:548
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TupleVariable()]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR tensor [TupleVariable(), TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1000 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,158] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>), ConstantVariable(int)]
[2022-11-07 20:06:23,159] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:549
[2022-11-07 20:06:23,159] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:550
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [TupleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:551
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:552
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:553
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:554
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:547
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 7 [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:556
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [TupleVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:544
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:558
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,160] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:587
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [ListIteratorVariable()]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:589
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable()]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 34 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 102 [ListIteratorVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:578
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [ListIteratorVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_module [ListIteratorVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:582
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:583
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:584
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST i [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:23,162] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:585
[2022-11-07 20:06:23,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,164] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
519 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (use_bottleneck)
4 POP_JUMP_IF_FALSE 26
520 6 LOAD_FAST 0 (self)
8 LOAD_METHOD 1 (bottleneck)
10 LOAD_FAST 1 (hidden_states)
12 CALL_METHOD 1
14 UNPACK_SEQUENCE 4
16 STORE_FAST 5 (query_tensor)
18 STORE_FAST 6 (key_tensor)
20 STORE_FAST 7 (value_tensor)
22 STORE_FAST 8 (layer_input)
24 JUMP_FORWARD 18 (to 44)
522 >> 26 LOAD_FAST 1 (hidden_states)
28 BUILD_LIST 1
30 LOAD_CONST 1 (4)
32 BINARY_MULTIPLY
34 UNPACK_SEQUENCE 4
36 STORE_FAST 5 (query_tensor)
38 STORE_FAST 6 (key_tensor)
40 STORE_FAST 7 (value_tensor)
42 STORE_FAST 8 (layer_input)
524 >> 44 LOAD_FAST 0 (self)
46 LOAD_ATTR 2 (attention)
525 48 LOAD_FAST 5 (query_tensor)
526 50 LOAD_FAST 6 (key_tensor)
527 52 LOAD_FAST 7 (value_tensor)
528 54 LOAD_FAST 8 (layer_input)
529 56 LOAD_FAST 2 (attention_mask)
530 58 LOAD_FAST 3 (head_mask)
531 60 LOAD_FAST 4 (output_attentions)
524 62 LOAD_CONST 2 (('output_attentions',))
64 CALL_FUNCTION_KW 7
66 STORE_FAST 9 (self_attention_outputs)
533 68 LOAD_FAST 9 (self_attention_outputs)
70 LOAD_CONST 3 (0)
72 BINARY_SUBSCR
74 STORE_FAST 10 (attention_output)
534 76 LOAD_FAST 10 (attention_output)
78 BUILD_TUPLE 1
80 STORE_FAST 11 (s)
535 82 LOAD_FAST 9 (self_attention_outputs)
84 LOAD_CONST 4 (1)
86 LOAD_CONST 0 (None)
88 BUILD_SLICE 2
90 BINARY_SUBSCR
92 STORE_FAST 12 (outputs)
537 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 3 (num_feedforward_networks)
98 LOAD_CONST 4 (1)
100 COMPARE_OP 3 (!=)
102 POP_JUMP_IF_FALSE 142
538 104 LOAD_GLOBAL 4 (enumerate)
106 LOAD_FAST 0 (self)
108 LOAD_ATTR 5 (ffn)
110 CALL_FUNCTION 1
112 GET_ITER
>> 114 FOR_ITER 26 (to 142)
116 UNPACK_SEQUENCE 2
118 STORE_FAST 13 (i)
120 STORE_FAST 14 (ffn_module)
539 122 LOAD_FAST 14 (ffn_module)
124 LOAD_FAST 10 (attention_output)
126 CALL_FUNCTION 1
128 STORE_FAST 10 (attention_output)
540 130 LOAD_FAST 11 (s)
132 LOAD_FAST 10 (attention_output)
134 BUILD_TUPLE 1
136 INPLACE_ADD
138 STORE_FAST 11 (s)
140 JUMP_ABSOLUTE 114
542 >> 142 LOAD_FAST 0 (self)
144 LOAD_METHOD 6 (intermediate)
146 LOAD_FAST 10 (attention_output)
148 CALL_METHOD 1
150 STORE_FAST 15 (intermediate_output)
543 152 LOAD_FAST 0 (self)
154 LOAD_METHOD 7 (output)
156 LOAD_FAST 15 (intermediate_output)
158 LOAD_FAST 10 (attention_output)
160 LOAD_FAST 1 (hidden_states)
162 CALL_METHOD 3
164 STORE_FAST 16 (layer_output)
545 166 LOAD_FAST 16 (layer_output)
168 BUILD_TUPLE 1
546 170 LOAD_FAST 12 (outputs)
545 172 BINARY_ADD
548 174 LOAD_GLOBAL 8 (torch)
176 LOAD_METHOD 9 (tensor)
178 LOAD_CONST 5 (1000)
180 CALL_METHOD 1
549 182 LOAD_FAST 5 (query_tensor)
550 184 LOAD_FAST 6 (key_tensor)
551 186 LOAD_FAST 7 (value_tensor)
552 188 LOAD_FAST 8 (layer_input)
553 190 LOAD_FAST 10 (attention_output)
554 192 LOAD_FAST 15 (intermediate_output)
547 194 BUILD_TUPLE 7
545 196 BINARY_ADD
556 198 LOAD_FAST 11 (s)
545 200 BINARY_ADD
544 202 STORE_FAST 12 (outputs)
558 204 LOAD_FAST 12 (outputs)
206 RETURN_VALUE
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:519
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:520
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,165] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,166] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
464 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (input)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (bottlenecked_hidden_states)
465 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck_attention)
14 POP_JUMP_IF_FALSE 26
466 16 LOAD_FAST 2 (bottlenecked_hidden_states)
18 BUILD_TUPLE 1
20 LOAD_CONST 1 (4)
22 BINARY_MULTIPLY
24 RETURN_VALUE
467 >> 26 LOAD_FAST 0 (self)
28 LOAD_ATTR 2 (key_query_shared_bottleneck)
30 POP_JUMP_IF_FALSE 54
468 32 LOAD_FAST 0 (self)
34 LOAD_METHOD 3 (attention)
36 LOAD_FAST 1 (hidden_states)
38 CALL_METHOD 1
40 STORE_FAST 3 (shared_attention_input)
469 42 LOAD_FAST 3 (shared_attention_input)
44 LOAD_FAST 3 (shared_attention_input)
46 LOAD_FAST 1 (hidden_states)
48 LOAD_FAST 2 (bottlenecked_hidden_states)
50 BUILD_TUPLE 4
52 RETURN_VALUE
471 >> 54 LOAD_FAST 1 (hidden_states)
56 LOAD_FAST 1 (hidden_states)
58 LOAD_FAST 1 (hidden_states)
60 LOAD_FAST 2 (bottlenecked_hidden_states)
62 BUILD_TUPLE 4
64 RETURN_VALUE
66 LOAD_CONST 0 (None)
68 RETURN_VALUE
[2022-11-07 20:06:23,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:464
[2022-11-07 20:06:23,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR input [NNModuleVariable()]
[2022-11-07 20:06:23,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,167] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,180] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,180] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,180] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,180] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,181] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,181] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,182] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,182] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,182] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,182] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,182] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,185] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,186] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,186] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,190] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,190] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST bottlenecked_hidden_states [TensorVariable()]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:465
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck_attention [NNModuleVariable()]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:467
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key_query_shared_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 54 [ConstantVariable(bool)]
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:468
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,191] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,204] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,205] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,205] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,205] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,206] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,213] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:469
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input []
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST bottlenecked_hidden_states [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 4 [TupleVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_tensor [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_tensor [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_tensor [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 44 []
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:525
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:526
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,215] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:527
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:528
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:529
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:530
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:531
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('output_attentions',) [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,216] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 7 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
358 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (self)
359 4 LOAD_FAST 1 (query_tensor)
360 6 LOAD_FAST 2 (key_tensor)
361 8 LOAD_FAST 3 (value_tensor)
362 10 LOAD_FAST 5 (attention_mask)
363 12 LOAD_FAST 6 (head_mask)
364 14 LOAD_FAST 7 (output_attentions)
358 16 CALL_METHOD 6
18 STORE_FAST 8 (self_outputs)
368 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 1 (output)
24 LOAD_FAST 8 (self_outputs)
26 LOAD_CONST 1 (0)
28 BINARY_SUBSCR
30 LOAD_FAST 4 (layer_input)
32 CALL_METHOD 2
34 STORE_FAST 9 (attention_output)
369 36 LOAD_FAST 9 (attention_output)
38 BUILD_TUPLE 1
40 LOAD_FAST 8 (self_outputs)
42 LOAD_CONST 2 (1)
44 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
48 BINARY_SUBSCR
50 BINARY_ADD
52 STORE_FAST 10 (outputs)
370 54 LOAD_FAST 10 (outputs)
56 RETURN_VALUE
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR self [NNModuleVariable()]
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:359
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:360
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:361
[2022-11-07 20:06:23,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:362
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:363
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:364
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:23,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,219] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
276 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (query)
4 LOAD_FAST 1 (query_tensor)
6 CALL_METHOD 1
8 STORE_FAST 7 (mixed_query_layer)
277 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (key)
14 LOAD_FAST 2 (key_tensor)
16 CALL_METHOD 1
18 STORE_FAST 8 (mixed_key_layer)
278 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (value)
24 LOAD_FAST 3 (value_tensor)
26 CALL_METHOD 1
28 STORE_FAST 9 (mixed_value_layer)
280 30 LOAD_FAST 0 (self)
32 LOAD_METHOD 3 (transpose_for_scores)
34 LOAD_FAST 7 (mixed_query_layer)
36 CALL_METHOD 1
38 STORE_FAST 10 (query_layer)
281 40 LOAD_FAST 0 (self)
42 LOAD_METHOD 3 (transpose_for_scores)
44 LOAD_FAST 8 (mixed_key_layer)
46 CALL_METHOD 1
48 STORE_FAST 11 (key_layer)
282 50 LOAD_FAST 0 (self)
52 LOAD_METHOD 3 (transpose_for_scores)
54 LOAD_FAST 9 (mixed_value_layer)
56 CALL_METHOD 1
58 STORE_FAST 12 (value_layer)
285 60 LOAD_GLOBAL 4 (torch)
62 LOAD_METHOD 5 (matmul)
64 LOAD_FAST 10 (query_layer)
66 LOAD_FAST 11 (key_layer)
68 LOAD_METHOD 6 (transpose)
70 LOAD_CONST 1 (-1)
72 LOAD_CONST 2 (-2)
74 CALL_METHOD 2
76 CALL_METHOD 2
78 STORE_FAST 13 (attention_scores)
286 80 LOAD_FAST 13 (attention_scores)
82 LOAD_GLOBAL 7 (math)
84 LOAD_METHOD 8 (sqrt)
86 LOAD_FAST 0 (self)
88 LOAD_ATTR 9 (attention_head_size)
90 CALL_METHOD 1
92 BINARY_TRUE_DIVIDE
94 STORE_FAST 13 (attention_scores)
287 96 LOAD_FAST 4 (attention_mask)
98 LOAD_CONST 0 (None)
100 COMPARE_OP 9 (is not)
102 POP_JUMP_IF_FALSE 112
289 104 LOAD_FAST 13 (attention_scores)
106 LOAD_FAST 4 (attention_mask)
108 BINARY_ADD
110 STORE_FAST 13 (attention_scores)
291 >> 112 LOAD_GLOBAL 10 (nn)
114 LOAD_ATTR 11 (functional)
116 LOAD_ATTR 12 (softmax)
118 LOAD_FAST 13 (attention_scores)
120 LOAD_CONST 1 (-1)
122 LOAD_CONST 3 (('dim',))
124 CALL_FUNCTION_KW 2
126 STORE_FAST 14 (attention_probs)
294 128 LOAD_FAST 0 (self)
130 LOAD_METHOD 13 (dropout)
132 LOAD_FAST 14 (attention_probs)
134 CALL_METHOD 1
136 STORE_FAST 14 (attention_probs)
296 138 LOAD_FAST 5 (head_mask)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 154
297 146 LOAD_FAST 14 (attention_probs)
148 LOAD_FAST 5 (head_mask)
150 BINARY_MULTIPLY
152 STORE_FAST 14 (attention_probs)
298 >> 154 LOAD_GLOBAL 4 (torch)
156 LOAD_METHOD 5 (matmul)
158 LOAD_FAST 14 (attention_probs)
160 LOAD_FAST 12 (value_layer)
162 CALL_METHOD 2
164 STORE_FAST 15 (context_layer)
299 166 LOAD_FAST 15 (context_layer)
168 LOAD_METHOD 14 (permute)
170 LOAD_CONST 4 (0)
172 LOAD_CONST 5 (2)
174 LOAD_CONST 6 (1)
176 LOAD_CONST 7 (3)
178 CALL_METHOD 4
180 LOAD_METHOD 15 (contiguous)
182 CALL_METHOD 0
184 STORE_FAST 15 (context_layer)
300 186 LOAD_FAST 15 (context_layer)
188 LOAD_METHOD 16 (size)
190 CALL_METHOD 0
192 LOAD_CONST 0 (None)
194 LOAD_CONST 2 (-2)
196 BUILD_SLICE 2
198 BINARY_SUBSCR
200 LOAD_FAST 0 (self)
202 LOAD_ATTR 17 (all_head_size)
204 BUILD_TUPLE 1
206 BINARY_ADD
208 STORE_FAST 16 (new_context_layer_shape)
301 210 LOAD_FAST 15 (context_layer)
212 LOAD_METHOD 18 (view)
214 LOAD_FAST 16 (new_context_layer_shape)
216 CALL_METHOD 1
218 STORE_FAST 15 (context_layer)
302 220 LOAD_FAST 6 (output_attentions)
222 POP_JUMP_IF_FALSE 232
224 LOAD_FAST 15 (context_layer)
226 LOAD_FAST 14 (attention_probs)
228 BUILD_TUPLE 2
230 JUMP_FORWARD 4 (to 236)
>> 232 LOAD_FAST 15 (context_layer)
234 BUILD_TUPLE 1
>> 236 STORE_FAST 17 (outputs)
303 238 LOAD_FAST 17 (outputs)
240 RETURN_VALUE
[2022-11-07 20:06:23,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:276
[2022-11-07 20:06:23,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR query [NNModuleVariable()]
[2022-11-07 20:06:23,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,231] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_query_layer [TensorVariable()]
[2022-11-07 20:06:23,231] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:277
[2022-11-07 20:06:23,231] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,231] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key [NNModuleVariable()]
[2022-11-07 20:06:23,232] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,232] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_key_layer [TensorVariable()]
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:278
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR value [NNModuleVariable()]
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,243] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_value_layer [TensorVariable()]
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:280
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_query_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,255] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,256] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,256] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,256] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,256] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,256] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,257] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,258] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,258] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,261] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,261] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_layer [TensorVariable()]
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:281
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_key_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,262] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_layer [TensorVariable()]
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:282
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_value_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,269] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,270] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,270] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,270] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,270] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,270] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,271] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,271] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,271] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,271] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,272] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,273] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,273] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,274] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,275] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,275] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,275] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_layer [TensorVariable()]
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:285
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:23,276] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose)]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int)]
[2022-11-07 20:06:23,277] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,278] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:286
[2022-11-07 20:06:23,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:23,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL math [TensorVariable()]
[2022-11-07 20:06:23,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR sqrt [TensorVariable(), TorchVariable(<module 'math' from '/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>)]
[2022-11-07 20:06:23,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable(), TorchVariable(<built-in function sqrt>)]
[2022-11-07 20:06:23,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [TensorVariable(), TorchVariable(<built-in function sqrt>), NNModuleVariable()]
[2022-11-07 20:06:23,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TensorVariable(), TorchVariable(<built-in function sqrt>), ConstantVariable(int)]
[2022-11-07 20:06:23,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_TRUE_DIVIDE None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:287
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 112 [ConstantVariable(bool)]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:289
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:23,286] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,287] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,287] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:291
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn []
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR softmax [TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores [TorchVariable(<function softmax at 0x7f16c76f6a60>)]
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable()]
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,288] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:23,293] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:23,294] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:294
[2022-11-07 20:06:23,294] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,294] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:23,294] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [NNModuleVariable()]
[2022-11-07 20:06:23,294] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:296
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 154 [ConstantVariable(bool)]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:298
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:23,295] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,300] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,300] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:299
[2022-11-07 20:06:23,300] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,301] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,302] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR contiguous [TensorVariable()]
[2022-11-07 20:06:23,303] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), contiguous)]
[2022-11-07 20:06:23,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:300
[2022-11-07 20:06:23,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,305] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,305] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,305] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,305] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR all_head_size [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_context_layer_shape [TupleVariable()]
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:301
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,306] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,307] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_context_layer_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,307] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:302
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 232 [ConstantVariable(bool)]
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:303
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:368
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [NNModuleVariable(), TupleVariable()]
[2022-11-07 20:06:23,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [NNModuleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
316 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
317 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 26
318 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 3 (layer_outputs)
22 CALL_METHOD 1
24 STORE_FAST 3 (layer_outputs)
319 >> 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 3 (layer_outputs)
32 LOAD_FAST 2 (residual_tensor)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 3 (layer_outputs)
320 40 LOAD_FAST 3 (layer_outputs)
42 RETURN_VALUE
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:316
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,322] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,322] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:317
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:319
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,325] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,325] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,326] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,326] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,326] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,326] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,328] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,330] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,330] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,333] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,334] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,334] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,334] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:320
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:369
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), TupleVariable(), SliceVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:370
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_attention_outputs [TupleVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:533
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TupleVariable()]
[2022-11-07 20:06:23,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:534
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [TupleVariable()]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:535
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable()]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), SliceVariable()]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:537
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_feedforward_networks [NNModuleVariable()]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [ConstantVariable(int)]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP != [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 142 [ConstantVariable(bool)]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:538
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ffn [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:23,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:23,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:23,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,344] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,355] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,355] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,355] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,355] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,356] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,356] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,358] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,358] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,359] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,360] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,360] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,360] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,360] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,373] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,374] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,375] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,375] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,375] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,375] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,377] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,379] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,379] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,382] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,384] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,386] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,388] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,400] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,403] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,404] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,404] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,404] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,405] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,416] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,416] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,416] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,416] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,417] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,417] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,417] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,418] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,419] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,419] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,419] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,419] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,419] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,423] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,424] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,424] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,427] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,428] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,428] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,429] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,430] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,430] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,430] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,431] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,432] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,433] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,433] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,434] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,434] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,434] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,434] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,448] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,449] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,450] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,462] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,463] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,464] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,464] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,464] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,464] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,464] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,469] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,469] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,474] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:542
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable()]
[2022-11-07 20:06:23,475] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,476] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,476] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,476] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,476] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,477] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,477] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,488] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:543
[2022-11-07 20:06:23,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,492] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,492] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,492] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,492] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,492] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 3 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,493] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
416 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (intermediate_states)
6 CALL_METHOD 1
8 STORE_FAST 4 (layer_output)
417 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 42
418 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 4 (layer_output)
22 CALL_METHOD 1
24 STORE_FAST 4 (layer_output)
419 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 4 (layer_output)
32 LOAD_FAST 2 (residual_tensor_1)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 4 (layer_output)
40 JUMP_FORWARD 26 (to 68)
421 >> 42 LOAD_FAST 0 (self)
44 LOAD_METHOD 3 (LayerNorm)
46 LOAD_FAST 4 (layer_output)
48 LOAD_FAST 2 (residual_tensor_1)
50 BINARY_ADD
52 CALL_METHOD 1
54 STORE_FAST 4 (layer_output)
422 56 LOAD_FAST 0 (self)
58 LOAD_METHOD 4 (bottleneck)
60 LOAD_FAST 4 (layer_output)
62 LOAD_FAST 3 (residual_tensor_2)
64 CALL_METHOD 2
66 STORE_FAST 4 (layer_output)
423 >> 68 LOAD_FAST 4 (layer_output)
70 RETURN_VALUE
[2022-11-07 20:06:23,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:416
[2022-11-07 20:06:23,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_states [NNModuleVariable()]
[2022-11-07 20:06:23,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:417
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 42 [ConstantVariable(bool)]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:421
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,505] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,507] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,508] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,511] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,513] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,513] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,515] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:422
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,518] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,518] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_2 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,518] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,518] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
396 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
397 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (dropout)
14 LOAD_FAST 3 (layer_outputs)
16 CALL_METHOD 1
18 STORE_FAST 3 (layer_outputs)
398 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (LayerNorm)
24 LOAD_FAST 3 (layer_outputs)
26 LOAD_FAST 2 (residual_tensor)
28 BINARY_ADD
30 CALL_METHOD 1
32 STORE_FAST 3 (layer_outputs)
399 34 LOAD_FAST 3 (layer_outputs)
36 RETURN_VALUE
[2022-11-07 20:06:23,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:396
[2022-11-07 20:06:23,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:397
[2022-11-07 20:06:23,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:23,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:398
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,532] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,534] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,535] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,535] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,535] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,535] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,535] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,538] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,539] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,539] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,542] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,543] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,543] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:399
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:423
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:546
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:548
[2022-11-07 20:06:23,545] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TupleVariable()]
[2022-11-07 20:06:23,545] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR tensor [TupleVariable(), TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,545] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1000 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,545] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>), ConstantVariable(int)]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:549
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:550
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [TupleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:551
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:552
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:553
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:554
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,546] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:547
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 7 [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:556
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:544
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:558
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:587
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [ListIteratorVariable()]
[2022-11-07 20:06:23,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:589
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable()]
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 34 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,548] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 102 [ListIteratorVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:578
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [ListIteratorVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_module [ListIteratorVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:582
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:583
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:584
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST i [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:23,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,550] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:585
[2022-11-07 20:06:23,550] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,550] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,550] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,551] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
519 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (use_bottleneck)
4 POP_JUMP_IF_FALSE 26
520 6 LOAD_FAST 0 (self)
8 LOAD_METHOD 1 (bottleneck)
10 LOAD_FAST 1 (hidden_states)
12 CALL_METHOD 1
14 UNPACK_SEQUENCE 4
16 STORE_FAST 5 (query_tensor)
18 STORE_FAST 6 (key_tensor)
20 STORE_FAST 7 (value_tensor)
22 STORE_FAST 8 (layer_input)
24 JUMP_FORWARD 18 (to 44)
522 >> 26 LOAD_FAST 1 (hidden_states)
28 BUILD_LIST 1
30 LOAD_CONST 1 (4)
32 BINARY_MULTIPLY
34 UNPACK_SEQUENCE 4
36 STORE_FAST 5 (query_tensor)
38 STORE_FAST 6 (key_tensor)
40 STORE_FAST 7 (value_tensor)
42 STORE_FAST 8 (layer_input)
524 >> 44 LOAD_FAST 0 (self)
46 LOAD_ATTR 2 (attention)
525 48 LOAD_FAST 5 (query_tensor)
526 50 LOAD_FAST 6 (key_tensor)
527 52 LOAD_FAST 7 (value_tensor)
528 54 LOAD_FAST 8 (layer_input)
529 56 LOAD_FAST 2 (attention_mask)
530 58 LOAD_FAST 3 (head_mask)
531 60 LOAD_FAST 4 (output_attentions)
524 62 LOAD_CONST 2 (('output_attentions',))
64 CALL_FUNCTION_KW 7
66 STORE_FAST 9 (self_attention_outputs)
533 68 LOAD_FAST 9 (self_attention_outputs)
70 LOAD_CONST 3 (0)
72 BINARY_SUBSCR
74 STORE_FAST 10 (attention_output)
534 76 LOAD_FAST 10 (attention_output)
78 BUILD_TUPLE 1
80 STORE_FAST 11 (s)
535 82 LOAD_FAST 9 (self_attention_outputs)
84 LOAD_CONST 4 (1)
86 LOAD_CONST 0 (None)
88 BUILD_SLICE 2
90 BINARY_SUBSCR
92 STORE_FAST 12 (outputs)
537 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 3 (num_feedforward_networks)
98 LOAD_CONST 4 (1)
100 COMPARE_OP 3 (!=)
102 POP_JUMP_IF_FALSE 142
538 104 LOAD_GLOBAL 4 (enumerate)
106 LOAD_FAST 0 (self)
108 LOAD_ATTR 5 (ffn)
110 CALL_FUNCTION 1
112 GET_ITER
>> 114 FOR_ITER 26 (to 142)
116 UNPACK_SEQUENCE 2
118 STORE_FAST 13 (i)
120 STORE_FAST 14 (ffn_module)
539 122 LOAD_FAST 14 (ffn_module)
124 LOAD_FAST 10 (attention_output)
126 CALL_FUNCTION 1
128 STORE_FAST 10 (attention_output)
540 130 LOAD_FAST 11 (s)
132 LOAD_FAST 10 (attention_output)
134 BUILD_TUPLE 1
136 INPLACE_ADD
138 STORE_FAST 11 (s)
140 JUMP_ABSOLUTE 114
542 >> 142 LOAD_FAST 0 (self)
144 LOAD_METHOD 6 (intermediate)
146 LOAD_FAST 10 (attention_output)
148 CALL_METHOD 1
150 STORE_FAST 15 (intermediate_output)
543 152 LOAD_FAST 0 (self)
154 LOAD_METHOD 7 (output)
156 LOAD_FAST 15 (intermediate_output)
158 LOAD_FAST 10 (attention_output)
160 LOAD_FAST 1 (hidden_states)
162 CALL_METHOD 3
164 STORE_FAST 16 (layer_output)
545 166 LOAD_FAST 16 (layer_output)
168 BUILD_TUPLE 1
546 170 LOAD_FAST 12 (outputs)
545 172 BINARY_ADD
548 174 LOAD_GLOBAL 8 (torch)
176 LOAD_METHOD 9 (tensor)
178 LOAD_CONST 5 (1000)
180 CALL_METHOD 1
549 182 LOAD_FAST 5 (query_tensor)
550 184 LOAD_FAST 6 (key_tensor)
551 186 LOAD_FAST 7 (value_tensor)
552 188 LOAD_FAST 8 (layer_input)
553 190 LOAD_FAST 10 (attention_output)
554 192 LOAD_FAST 15 (intermediate_output)
547 194 BUILD_TUPLE 7
545 196 BINARY_ADD
556 198 LOAD_FAST 11 (s)
545 200 BINARY_ADD
544 202 STORE_FAST 12 (outputs)
558 204 LOAD_FAST 12 (outputs)
206 RETURN_VALUE
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:519
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:520
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,552] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,553] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
464 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (input)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (bottlenecked_hidden_states)
465 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck_attention)
14 POP_JUMP_IF_FALSE 26
466 16 LOAD_FAST 2 (bottlenecked_hidden_states)
18 BUILD_TUPLE 1
20 LOAD_CONST 1 (4)
22 BINARY_MULTIPLY
24 RETURN_VALUE
467 >> 26 LOAD_FAST 0 (self)
28 LOAD_ATTR 2 (key_query_shared_bottleneck)
30 POP_JUMP_IF_FALSE 54
468 32 LOAD_FAST 0 (self)
34 LOAD_METHOD 3 (attention)
36 LOAD_FAST 1 (hidden_states)
38 CALL_METHOD 1
40 STORE_FAST 3 (shared_attention_input)
469 42 LOAD_FAST 3 (shared_attention_input)
44 LOAD_FAST 3 (shared_attention_input)
46 LOAD_FAST 1 (hidden_states)
48 LOAD_FAST 2 (bottlenecked_hidden_states)
50 BUILD_TUPLE 4
52 RETURN_VALUE
471 >> 54 LOAD_FAST 1 (hidden_states)
56 LOAD_FAST 1 (hidden_states)
58 LOAD_FAST 1 (hidden_states)
60 LOAD_FAST 2 (bottlenecked_hidden_states)
62 BUILD_TUPLE 4
64 RETURN_VALUE
66 LOAD_CONST 0 (None)
68 RETURN_VALUE
[2022-11-07 20:06:23,554] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:464
[2022-11-07 20:06:23,554] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,554] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR input [NNModuleVariable()]
[2022-11-07 20:06:23,554] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,554] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,555] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,555] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,555] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,555] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,555] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,556] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,569] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,569] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,569] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,569] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,569] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST bottlenecked_hidden_states [TensorVariable()]
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:465
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck_attention [NNModuleVariable()]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:467
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key_query_shared_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 54 [ConstantVariable(bool)]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:468
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,579] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,580] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,580] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,580] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,580] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,581] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,581] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,593] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,596] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,598] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,598] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,601] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,602] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:469
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input []
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST bottlenecked_hidden_states [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 4 [TupleVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_tensor [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_tensor [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_tensor [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 44 []
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,603] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:525
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:526
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:527
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:528
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:529
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:530
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:531
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('output_attentions',) [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,604] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 7 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:23,605] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
358 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (self)
359 4 LOAD_FAST 1 (query_tensor)
360 6 LOAD_FAST 2 (key_tensor)
361 8 LOAD_FAST 3 (value_tensor)
362 10 LOAD_FAST 5 (attention_mask)
363 12 LOAD_FAST 6 (head_mask)
364 14 LOAD_FAST 7 (output_attentions)
358 16 CALL_METHOD 6
18 STORE_FAST 8 (self_outputs)
368 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 1 (output)
24 LOAD_FAST 8 (self_outputs)
26 LOAD_CONST 1 (0)
28 BINARY_SUBSCR
30 LOAD_FAST 4 (layer_input)
32 CALL_METHOD 2
34 STORE_FAST 9 (attention_output)
369 36 LOAD_FAST 9 (attention_output)
38 BUILD_TUPLE 1
40 LOAD_FAST 8 (self_outputs)
42 LOAD_CONST 2 (1)
44 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
48 BINARY_SUBSCR
50 BINARY_ADD
52 STORE_FAST 10 (outputs)
370 54 LOAD_FAST 10 (outputs)
56 RETURN_VALUE
[2022-11-07 20:06:23,605] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:23,605] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,605] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR self [NNModuleVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:359
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:360
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:361
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:362
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:363
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:364
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:23,606] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,607] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
276 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (query)
4 LOAD_FAST 1 (query_tensor)
6 CALL_METHOD 1
8 STORE_FAST 7 (mixed_query_layer)
277 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (key)
14 LOAD_FAST 2 (key_tensor)
16 CALL_METHOD 1
18 STORE_FAST 8 (mixed_key_layer)
278 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (value)
24 LOAD_FAST 3 (value_tensor)
26 CALL_METHOD 1
28 STORE_FAST 9 (mixed_value_layer)
280 30 LOAD_FAST 0 (self)
32 LOAD_METHOD 3 (transpose_for_scores)
34 LOAD_FAST 7 (mixed_query_layer)
36 CALL_METHOD 1
38 STORE_FAST 10 (query_layer)
281 40 LOAD_FAST 0 (self)
42 LOAD_METHOD 3 (transpose_for_scores)
44 LOAD_FAST 8 (mixed_key_layer)
46 CALL_METHOD 1
48 STORE_FAST 11 (key_layer)
282 50 LOAD_FAST 0 (self)
52 LOAD_METHOD 3 (transpose_for_scores)
54 LOAD_FAST 9 (mixed_value_layer)
56 CALL_METHOD 1
58 STORE_FAST 12 (value_layer)
285 60 LOAD_GLOBAL 4 (torch)
62 LOAD_METHOD 5 (matmul)
64 LOAD_FAST 10 (query_layer)
66 LOAD_FAST 11 (key_layer)
68 LOAD_METHOD 6 (transpose)
70 LOAD_CONST 1 (-1)
72 LOAD_CONST 2 (-2)
74 CALL_METHOD 2
76 CALL_METHOD 2
78 STORE_FAST 13 (attention_scores)
286 80 LOAD_FAST 13 (attention_scores)
82 LOAD_GLOBAL 7 (math)
84 LOAD_METHOD 8 (sqrt)
86 LOAD_FAST 0 (self)
88 LOAD_ATTR 9 (attention_head_size)
90 CALL_METHOD 1
92 BINARY_TRUE_DIVIDE
94 STORE_FAST 13 (attention_scores)
287 96 LOAD_FAST 4 (attention_mask)
98 LOAD_CONST 0 (None)
100 COMPARE_OP 9 (is not)
102 POP_JUMP_IF_FALSE 112
289 104 LOAD_FAST 13 (attention_scores)
106 LOAD_FAST 4 (attention_mask)
108 BINARY_ADD
110 STORE_FAST 13 (attention_scores)
291 >> 112 LOAD_GLOBAL 10 (nn)
114 LOAD_ATTR 11 (functional)
116 LOAD_ATTR 12 (softmax)
118 LOAD_FAST 13 (attention_scores)
120 LOAD_CONST 1 (-1)
122 LOAD_CONST 3 (('dim',))
124 CALL_FUNCTION_KW 2
126 STORE_FAST 14 (attention_probs)
294 128 LOAD_FAST 0 (self)
130 LOAD_METHOD 13 (dropout)
132 LOAD_FAST 14 (attention_probs)
134 CALL_METHOD 1
136 STORE_FAST 14 (attention_probs)
296 138 LOAD_FAST 5 (head_mask)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 154
297 146 LOAD_FAST 14 (attention_probs)
148 LOAD_FAST 5 (head_mask)
150 BINARY_MULTIPLY
152 STORE_FAST 14 (attention_probs)
298 >> 154 LOAD_GLOBAL 4 (torch)
156 LOAD_METHOD 5 (matmul)
158 LOAD_FAST 14 (attention_probs)
160 LOAD_FAST 12 (value_layer)
162 CALL_METHOD 2
164 STORE_FAST 15 (context_layer)
299 166 LOAD_FAST 15 (context_layer)
168 LOAD_METHOD 14 (permute)
170 LOAD_CONST 4 (0)
172 LOAD_CONST 5 (2)
174 LOAD_CONST 6 (1)
176 LOAD_CONST 7 (3)
178 CALL_METHOD 4
180 LOAD_METHOD 15 (contiguous)
182 CALL_METHOD 0
184 STORE_FAST 15 (context_layer)
300 186 LOAD_FAST 15 (context_layer)
188 LOAD_METHOD 16 (size)
190 CALL_METHOD 0
192 LOAD_CONST 0 (None)
194 LOAD_CONST 2 (-2)
196 BUILD_SLICE 2
198 BINARY_SUBSCR
200 LOAD_FAST 0 (self)
202 LOAD_ATTR 17 (all_head_size)
204 BUILD_TUPLE 1
206 BINARY_ADD
208 STORE_FAST 16 (new_context_layer_shape)
301 210 LOAD_FAST 15 (context_layer)
212 LOAD_METHOD 18 (view)
214 LOAD_FAST 16 (new_context_layer_shape)
216 CALL_METHOD 1
218 STORE_FAST 15 (context_layer)
302 220 LOAD_FAST 6 (output_attentions)
222 POP_JUMP_IF_FALSE 232
224 LOAD_FAST 15 (context_layer)
226 LOAD_FAST 14 (attention_probs)
228 BUILD_TUPLE 2
230 JUMP_FORWARD 4 (to 236)
>> 232 LOAD_FAST 15 (context_layer)
234 BUILD_TUPLE 1
>> 236 STORE_FAST 17 (outputs)
303 238 LOAD_FAST 17 (outputs)
240 RETURN_VALUE
[2022-11-07 20:06:23,608] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:276
[2022-11-07 20:06:23,608] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,608] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR query [NNModuleVariable()]
[2022-11-07 20:06:23,609] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,609] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_query_layer [TensorVariable()]
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:277
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key [NNModuleVariable()]
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,631] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_key_layer [TensorVariable()]
[2022-11-07 20:06:23,631] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:278
[2022-11-07 20:06:23,631] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,631] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR value [NNModuleVariable()]
[2022-11-07 20:06:23,632] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,632] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_value_layer [TensorVariable()]
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:280
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_query_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,643] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,644] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,644] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,644] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,644] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,644] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,645] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,645] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,645] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,645] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,646] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,647] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,648] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,649] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_layer [TensorVariable()]
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:281
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,650] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,651] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_key_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,651] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,651] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,652] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,653] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,653] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,653] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,653] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,653] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,654] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,656] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_layer [TensorVariable()]
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:282
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,658] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:23,659] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_value_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:23,659] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:23,659] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,660] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,661] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,661] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,661] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,661] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,662] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,664] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_layer [TensorVariable()]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:285
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose)]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int)]
[2022-11-07 20:06:23,666] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,668] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:286
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL math [TensorVariable()]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR sqrt [TensorVariable(), TorchVariable(<module 'math' from '/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>)]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable(), TorchVariable(<built-in function sqrt>)]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [TensorVariable(), TorchVariable(<built-in function sqrt>), NNModuleVariable()]
[2022-11-07 20:06:23,673] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TensorVariable(), TorchVariable(<built-in function sqrt>), ConstantVariable(int)]
[2022-11-07 20:06:23,674] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_TRUE_DIVIDE None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:287
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,675] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 112 [ConstantVariable(bool)]
[2022-11-07 20:06:23,676] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:289
[2022-11-07 20:06:23,676] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:23,676] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:23,676] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:291
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn []
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR softmax [TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores [TorchVariable(<function softmax at 0x7f16c76f6a60>)]
[2022-11-07 20:06:23,677] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable()]
[2022-11-07 20:06:23,678] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,678] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:294
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [NNModuleVariable()]
[2022-11-07 20:06:23,683] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:296
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 154 [ConstantVariable(bool)]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:298
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:23,685] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:299
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,690] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,692] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR contiguous [TensorVariable()]
[2022-11-07 20:06:23,692] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), contiguous)]
[2022-11-07 20:06:23,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:300
[2022-11-07 20:06:23,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:23,694] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:23,695] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR all_head_size [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_context_layer_shape [TupleVariable()]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:301
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_context_layer_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:23,696] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:302
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 232 [ConstantVariable(bool)]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:303
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:368
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,698] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,699] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,699] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [NNModuleVariable(), TupleVariable()]
[2022-11-07 20:06:23,699] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [NNModuleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,699] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,699] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,700] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
316 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
317 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 26
318 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 3 (layer_outputs)
22 CALL_METHOD 1
24 STORE_FAST 3 (layer_outputs)
319 >> 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 3 (layer_outputs)
32 LOAD_FAST 2 (residual_tensor)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 3 (layer_outputs)
320 40 LOAD_FAST 3 (layer_outputs)
42 RETURN_VALUE
[2022-11-07 20:06:23,700] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:316
[2022-11-07 20:06:23,700] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,700] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,701] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,701] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:317
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:319
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,712] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,713] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,714] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,715] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,715] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,719] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,721] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,724] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,725] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,725] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:320
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:369
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,726] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), TupleVariable(), SliceVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:370
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_attention_outputs [TupleVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:533
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TupleVariable()]
[2022-11-07 20:06:23,727] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:534
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [TupleVariable()]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:535
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable()]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,728] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), SliceVariable()]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:537
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_feedforward_networks [NNModuleVariable()]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [ConstantVariable(int)]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP != [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 142 [ConstantVariable(bool)]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:538
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:23,729] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ffn [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:23,730] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:23,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:23,731] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,732] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,733] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,733] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,734] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,734] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,735] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,735] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,735] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,735] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,735] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,747] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,750] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,751] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,751] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,751] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,752] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,763] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,763] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,763] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,763] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,764] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,764] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,764] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,765] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,766] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,766] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,769] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,770] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,771] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,773] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,775] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,776] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,777] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,778] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,778] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,778] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,779] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,780] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,791] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,791] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,791] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,791] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,792] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,792] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,795] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,796] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,796] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,796] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,797] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,809] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,810] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,811] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,811] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,812] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,812] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,812] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,814] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,816] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,816] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,819] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,820] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,820] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,821] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,822] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,823] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,824] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,825] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,826] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,826] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,826] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,826] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,826] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,837] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,840] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,840] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,840] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,840] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,840] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,841] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,842] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:23,842] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:23,842] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,842] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,842] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,843] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,854] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,856] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,857] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,857] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,857] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,857] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,857] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,860] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,862] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,862] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,865] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,866] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,866] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,867] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:542
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable()]
[2022-11-07 20:06:23,868] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,869] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:23,869] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:23,869] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,869] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,869] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,870] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,881] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:543
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,884] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:23,885] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:23,885] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,885] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,885] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 3 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,886] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
416 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (intermediate_states)
6 CALL_METHOD 1
8 STORE_FAST 4 (layer_output)
417 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 42
418 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 4 (layer_output)
22 CALL_METHOD 1
24 STORE_FAST 4 (layer_output)
419 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 4 (layer_output)
32 LOAD_FAST 2 (residual_tensor_1)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 4 (layer_output)
40 JUMP_FORWARD 26 (to 68)
421 >> 42 LOAD_FAST 0 (self)
44 LOAD_METHOD 3 (LayerNorm)
46 LOAD_FAST 4 (layer_output)
48 LOAD_FAST 2 (residual_tensor_1)
50 BINARY_ADD
52 CALL_METHOD 1
54 STORE_FAST 4 (layer_output)
422 56 LOAD_FAST 0 (self)
58 LOAD_METHOD 4 (bottleneck)
60 LOAD_FAST 4 (layer_output)
62 LOAD_FAST 3 (residual_tensor_2)
64 CALL_METHOD 2
66 STORE_FAST 4 (layer_output)
423 >> 68 LOAD_FAST 4 (layer_output)
70 RETURN_VALUE
[2022-11-07 20:06:23,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:416
[2022-11-07 20:06:23,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,886] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_states [NNModuleVariable()]
[2022-11-07 20:06:23,887] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:417
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 42 [ConstantVariable(bool)]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:421
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,898] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,899] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,900] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,901] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,901] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,904] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,905] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,909] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,910] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,910] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:422
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_2 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,911] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,912] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
396 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
397 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (dropout)
14 LOAD_FAST 3 (layer_outputs)
16 CALL_METHOD 1
18 STORE_FAST 3 (layer_outputs)
398 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (LayerNorm)
24 LOAD_FAST 3 (layer_outputs)
26 LOAD_FAST 2 (residual_tensor)
28 BINARY_ADD
30 CALL_METHOD 1
32 STORE_FAST 3 (layer_outputs)
399 34 LOAD_FAST 3 (layer_outputs)
36 RETURN_VALUE
[2022-11-07 20:06:23,912] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:396
[2022-11-07 20:06:23,912] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,912] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,913] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:397
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,924] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:398
[2022-11-07 20:06:23,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,925] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:23,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,926] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,927] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,928] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,928] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,931] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,933] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,933] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,935] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:399
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,937] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:423
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:546
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:548
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TupleVariable()]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR tensor [TupleVariable(), TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1000 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:23,938] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>), ConstantVariable(int)]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:549
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [TupleVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:550
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [TupleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:551
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:552
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:553
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:554
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:547
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 7 [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,940] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:556
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:544
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:558
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:587
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [ListIteratorVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,941] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:23,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:589
[2022-11-07 20:06:23,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable()]
[2022-11-07 20:06:23,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 34 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,942] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 102 [ListIteratorVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:578
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [ListIteratorVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_module [ListIteratorVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:582
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:583
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:584
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST i [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:23,943] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:23,944] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:585
[2022-11-07 20:06:23,944] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,944] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:23,944] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,945] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
519 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (use_bottleneck)
4 POP_JUMP_IF_FALSE 26
520 6 LOAD_FAST 0 (self)
8 LOAD_METHOD 1 (bottleneck)
10 LOAD_FAST 1 (hidden_states)
12 CALL_METHOD 1
14 UNPACK_SEQUENCE 4
16 STORE_FAST 5 (query_tensor)
18 STORE_FAST 6 (key_tensor)
20 STORE_FAST 7 (value_tensor)
22 STORE_FAST 8 (layer_input)
24 JUMP_FORWARD 18 (to 44)
522 >> 26 LOAD_FAST 1 (hidden_states)
28 BUILD_LIST 1
30 LOAD_CONST 1 (4)
32 BINARY_MULTIPLY
34 UNPACK_SEQUENCE 4
36 STORE_FAST 5 (query_tensor)
38 STORE_FAST 6 (key_tensor)
40 STORE_FAST 7 (value_tensor)
42 STORE_FAST 8 (layer_input)
524 >> 44 LOAD_FAST 0 (self)
46 LOAD_ATTR 2 (attention)
525 48 LOAD_FAST 5 (query_tensor)
526 50 LOAD_FAST 6 (key_tensor)
527 52 LOAD_FAST 7 (value_tensor)
528 54 LOAD_FAST 8 (layer_input)
529 56 LOAD_FAST 2 (attention_mask)
530 58 LOAD_FAST 3 (head_mask)
531 60 LOAD_FAST 4 (output_attentions)
524 62 LOAD_CONST 2 (('output_attentions',))
64 CALL_FUNCTION_KW 7
66 STORE_FAST 9 (self_attention_outputs)
533 68 LOAD_FAST 9 (self_attention_outputs)
70 LOAD_CONST 3 (0)
72 BINARY_SUBSCR
74 STORE_FAST 10 (attention_output)
534 76 LOAD_FAST 10 (attention_output)
78 BUILD_TUPLE 1
80 STORE_FAST 11 (s)
535 82 LOAD_FAST 9 (self_attention_outputs)
84 LOAD_CONST 4 (1)
86 LOAD_CONST 0 (None)
88 BUILD_SLICE 2
90 BINARY_SUBSCR
92 STORE_FAST 12 (outputs)
537 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 3 (num_feedforward_networks)
98 LOAD_CONST 4 (1)
100 COMPARE_OP 3 (!=)
102 POP_JUMP_IF_FALSE 142
538 104 LOAD_GLOBAL 4 (enumerate)
106 LOAD_FAST 0 (self)
108 LOAD_ATTR 5 (ffn)
110 CALL_FUNCTION 1
112 GET_ITER
>> 114 FOR_ITER 26 (to 142)
116 UNPACK_SEQUENCE 2
118 STORE_FAST 13 (i)
120 STORE_FAST 14 (ffn_module)
539 122 LOAD_FAST 14 (ffn_module)
124 LOAD_FAST 10 (attention_output)
126 CALL_FUNCTION 1
128 STORE_FAST 10 (attention_output)
540 130 LOAD_FAST 11 (s)
132 LOAD_FAST 10 (attention_output)
134 BUILD_TUPLE 1
136 INPLACE_ADD
138 STORE_FAST 11 (s)
140 JUMP_ABSOLUTE 114
542 >> 142 LOAD_FAST 0 (self)
144 LOAD_METHOD 6 (intermediate)
146 LOAD_FAST 10 (attention_output)
148 CALL_METHOD 1
150 STORE_FAST 15 (intermediate_output)
543 152 LOAD_FAST 0 (self)
154 LOAD_METHOD 7 (output)
156 LOAD_FAST 15 (intermediate_output)
158 LOAD_FAST 10 (attention_output)
160 LOAD_FAST 1 (hidden_states)
162 CALL_METHOD 3
164 STORE_FAST 16 (layer_output)
545 166 LOAD_FAST 16 (layer_output)
168 BUILD_TUPLE 1
546 170 LOAD_FAST 12 (outputs)
545 172 BINARY_ADD
548 174 LOAD_GLOBAL 8 (torch)
176 LOAD_METHOD 9 (tensor)
178 LOAD_CONST 5 (1000)
180 CALL_METHOD 1
549 182 LOAD_FAST 5 (query_tensor)
550 184 LOAD_FAST 6 (key_tensor)
551 186 LOAD_FAST 7 (value_tensor)
552 188 LOAD_FAST 8 (layer_input)
553 190 LOAD_FAST 10 (attention_output)
554 192 LOAD_FAST 15 (intermediate_output)
547 194 BUILD_TUPLE 7
545 196 BINARY_ADD
556 198 LOAD_FAST 11 (s)
545 200 BINARY_ADD
544 202 STORE_FAST 12 (outputs)
558 204 LOAD_FAST 12 (outputs)
206 RETURN_VALUE
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:519
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:520
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,946] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,947] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
464 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (input)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (bottlenecked_hidden_states)
465 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck_attention)
14 POP_JUMP_IF_FALSE 26
466 16 LOAD_FAST 2 (bottlenecked_hidden_states)
18 BUILD_TUPLE 1
20 LOAD_CONST 1 (4)
22 BINARY_MULTIPLY
24 RETURN_VALUE
467 >> 26 LOAD_FAST 0 (self)
28 LOAD_ATTR 2 (key_query_shared_bottleneck)
30 POP_JUMP_IF_FALSE 54
468 32 LOAD_FAST 0 (self)
34 LOAD_METHOD 3 (attention)
36 LOAD_FAST 1 (hidden_states)
38 CALL_METHOD 1
40 STORE_FAST 3 (shared_attention_input)
469 42 LOAD_FAST 3 (shared_attention_input)
44 LOAD_FAST 3 (shared_attention_input)
46 LOAD_FAST 1 (hidden_states)
48 LOAD_FAST 2 (bottlenecked_hidden_states)
50 BUILD_TUPLE 4
52 RETURN_VALUE
471 >> 54 LOAD_FAST 1 (hidden_states)
56 LOAD_FAST 1 (hidden_states)
58 LOAD_FAST 1 (hidden_states)
60 LOAD_FAST 2 (bottlenecked_hidden_states)
62 BUILD_TUPLE 4
64 RETURN_VALUE
66 LOAD_CONST 0 (None)
68 RETURN_VALUE
[2022-11-07 20:06:23,947] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:464
[2022-11-07 20:06:23,947] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,947] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR input [NNModuleVariable()]
[2022-11-07 20:06:23,948] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,948] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,949] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,961] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,961] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,961] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,961] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,962] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,962] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,963] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,963] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,963] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,963] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,963] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,966] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,967] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,967] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,970] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST bottlenecked_hidden_states [TensorVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:465
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck_attention [NNModuleVariable()]
[2022-11-07 20:06:23,972] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:467
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key_query_shared_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 54 [ConstantVariable(bool)]
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:468
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,973] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,974] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:23,974] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:23,974] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,974] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:23,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:23,975] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,986] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,986] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:23,986] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,986] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:23,987] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:23,987] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,987] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:23,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:23,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:23,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,988] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,991] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,992] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:23,992] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:23,995] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,996] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:469
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input []
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST bottlenecked_hidden_states [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:23,997] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 4 [TupleVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_tensor [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_tensor [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_tensor [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 44 []
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:525
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:526
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:527
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:528
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:529
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:530
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:531
[2022-11-07 20:06:23,998] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:23,999] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:23,999] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('output_attentions',) [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:23,999] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 7 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
358 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (self)
359 4 LOAD_FAST 1 (query_tensor)
360 6 LOAD_FAST 2 (key_tensor)
361 8 LOAD_FAST 3 (value_tensor)
362 10 LOAD_FAST 5 (attention_mask)
363 12 LOAD_FAST 6 (head_mask)
364 14 LOAD_FAST 7 (output_attentions)
358 16 CALL_METHOD 6
18 STORE_FAST 8 (self_outputs)
368 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 1 (output)
24 LOAD_FAST 8 (self_outputs)
26 LOAD_CONST 1 (0)
28 BINARY_SUBSCR
30 LOAD_FAST 4 (layer_input)
32 CALL_METHOD 2
34 STORE_FAST 9 (attention_output)
369 36 LOAD_FAST 9 (attention_output)
38 BUILD_TUPLE 1
40 LOAD_FAST 8 (self_outputs)
42 LOAD_CONST 2 (1)
44 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
48 BINARY_SUBSCR
50 BINARY_ADD
52 STORE_FAST 10 (outputs)
370 54 LOAD_FAST 10 (outputs)
56 RETURN_VALUE
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR self [NNModuleVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:359
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:360
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:361
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:362
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:363
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,000] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:364
[2022-11-07 20:06:24,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:24,001] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:24,002] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
276 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (query)
4 LOAD_FAST 1 (query_tensor)
6 CALL_METHOD 1
8 STORE_FAST 7 (mixed_query_layer)
277 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (key)
14 LOAD_FAST 2 (key_tensor)
16 CALL_METHOD 1
18 STORE_FAST 8 (mixed_key_layer)
278 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (value)
24 LOAD_FAST 3 (value_tensor)
26 CALL_METHOD 1
28 STORE_FAST 9 (mixed_value_layer)
280 30 LOAD_FAST 0 (self)
32 LOAD_METHOD 3 (transpose_for_scores)
34 LOAD_FAST 7 (mixed_query_layer)
36 CALL_METHOD 1
38 STORE_FAST 10 (query_layer)
281 40 LOAD_FAST 0 (self)
42 LOAD_METHOD 3 (transpose_for_scores)
44 LOAD_FAST 8 (mixed_key_layer)
46 CALL_METHOD 1
48 STORE_FAST 11 (key_layer)
282 50 LOAD_FAST 0 (self)
52 LOAD_METHOD 3 (transpose_for_scores)
54 LOAD_FAST 9 (mixed_value_layer)
56 CALL_METHOD 1
58 STORE_FAST 12 (value_layer)
285 60 LOAD_GLOBAL 4 (torch)
62 LOAD_METHOD 5 (matmul)
64 LOAD_FAST 10 (query_layer)
66 LOAD_FAST 11 (key_layer)
68 LOAD_METHOD 6 (transpose)
70 LOAD_CONST 1 (-1)
72 LOAD_CONST 2 (-2)
74 CALL_METHOD 2
76 CALL_METHOD 2
78 STORE_FAST 13 (attention_scores)
286 80 LOAD_FAST 13 (attention_scores)
82 LOAD_GLOBAL 7 (math)
84 LOAD_METHOD 8 (sqrt)
86 LOAD_FAST 0 (self)
88 LOAD_ATTR 9 (attention_head_size)
90 CALL_METHOD 1
92 BINARY_TRUE_DIVIDE
94 STORE_FAST 13 (attention_scores)
287 96 LOAD_FAST 4 (attention_mask)
98 LOAD_CONST 0 (None)
100 COMPARE_OP 9 (is not)
102 POP_JUMP_IF_FALSE 112
289 104 LOAD_FAST 13 (attention_scores)
106 LOAD_FAST 4 (attention_mask)
108 BINARY_ADD
110 STORE_FAST 13 (attention_scores)
291 >> 112 LOAD_GLOBAL 10 (nn)
114 LOAD_ATTR 11 (functional)
116 LOAD_ATTR 12 (softmax)
118 LOAD_FAST 13 (attention_scores)
120 LOAD_CONST 1 (-1)
122 LOAD_CONST 3 (('dim',))
124 CALL_FUNCTION_KW 2
126 STORE_FAST 14 (attention_probs)
294 128 LOAD_FAST 0 (self)
130 LOAD_METHOD 13 (dropout)
132 LOAD_FAST 14 (attention_probs)
134 CALL_METHOD 1
136 STORE_FAST 14 (attention_probs)
296 138 LOAD_FAST 5 (head_mask)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 154
297 146 LOAD_FAST 14 (attention_probs)
148 LOAD_FAST 5 (head_mask)
150 BINARY_MULTIPLY
152 STORE_FAST 14 (attention_probs)
298 >> 154 LOAD_GLOBAL 4 (torch)
156 LOAD_METHOD 5 (matmul)
158 LOAD_FAST 14 (attention_probs)
160 LOAD_FAST 12 (value_layer)
162 CALL_METHOD 2
164 STORE_FAST 15 (context_layer)
299 166 LOAD_FAST 15 (context_layer)
168 LOAD_METHOD 14 (permute)
170 LOAD_CONST 4 (0)
172 LOAD_CONST 5 (2)
174 LOAD_CONST 6 (1)
176 LOAD_CONST 7 (3)
178 CALL_METHOD 4
180 LOAD_METHOD 15 (contiguous)
182 CALL_METHOD 0
184 STORE_FAST 15 (context_layer)
300 186 LOAD_FAST 15 (context_layer)
188 LOAD_METHOD 16 (size)
190 CALL_METHOD 0
192 LOAD_CONST 0 (None)
194 LOAD_CONST 2 (-2)
196 BUILD_SLICE 2
198 BINARY_SUBSCR
200 LOAD_FAST 0 (self)
202 LOAD_ATTR 17 (all_head_size)
204 BUILD_TUPLE 1
206 BINARY_ADD
208 STORE_FAST 16 (new_context_layer_shape)
301 210 LOAD_FAST 15 (context_layer)
212 LOAD_METHOD 18 (view)
214 LOAD_FAST 16 (new_context_layer_shape)
216 CALL_METHOD 1
218 STORE_FAST 15 (context_layer)
302 220 LOAD_FAST 6 (output_attentions)
222 POP_JUMP_IF_FALSE 232
224 LOAD_FAST 15 (context_layer)
226 LOAD_FAST 14 (attention_probs)
228 BUILD_TUPLE 2
230 JUMP_FORWARD 4 (to 236)
>> 232 LOAD_FAST 15 (context_layer)
234 BUILD_TUPLE 1
>> 236 STORE_FAST 17 (outputs)
303 238 LOAD_FAST 17 (outputs)
240 RETURN_VALUE
[2022-11-07 20:06:24,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:276
[2022-11-07 20:06:24,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,002] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR query [NNModuleVariable()]
[2022-11-07 20:06:24,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,003] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_query_layer [TensorVariable()]
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:277
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key [NNModuleVariable()]
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,015] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,026] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_key_layer [TensorVariable()]
[2022-11-07 20:06:24,026] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:278
[2022-11-07 20:06:24,026] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,026] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR value [NNModuleVariable()]
[2022-11-07 20:06:24,027] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,027] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_value_layer [TensorVariable()]
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:280
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_query_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,038] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,039] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,039] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,039] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,039] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,039] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,040] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,041] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,042] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,042] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,042] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,043] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,043] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,043] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,043] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,044] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,045] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,045] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_layer [TensorVariable()]
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:281
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_key_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,046] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,047] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,047] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,048] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,049] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,052] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_layer [TensorVariable()]
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:282
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_value_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,054] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,055] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,055] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,055] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,055] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,056] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,056] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,056] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,056] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,056] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,057] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,058] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,059] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,059] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,059] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,060] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,061] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_layer [TensorVariable()]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:285
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose)]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int)]
[2022-11-07 20:06:24,062] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,064] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:286
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL math [TensorVariable()]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR sqrt [TensorVariable(), TorchVariable(<module 'math' from '/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>)]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable(), TorchVariable(<built-in function sqrt>)]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [TensorVariable(), TorchVariable(<built-in function sqrt>), NNModuleVariable()]
[2022-11-07 20:06:24,069] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TensorVariable(), TorchVariable(<built-in function sqrt>), ConstantVariable(int)]
[2022-11-07 20:06:24,070] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_TRUE_DIVIDE None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:287
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 112 [ConstantVariable(bool)]
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:289
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:24,071] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:24,072] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:291
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn []
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR softmax [TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores [TorchVariable(<function softmax at 0x7f16c76f6a60>)]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable()]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,073] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:294
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [NNModuleVariable()]
[2022-11-07 20:06:24,079] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:296
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 154 [ConstantVariable(bool)]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:298
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:24,081] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,086] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,086] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:299
[2022-11-07 20:06:24,086] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,086] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,087] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,088] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR contiguous [TensorVariable()]
[2022-11-07 20:06:24,089] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), contiguous)]
[2022-11-07 20:06:24,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:300
[2022-11-07 20:06:24,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,090] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,091] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR all_head_size [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_context_layer_shape [TupleVariable()]
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:301
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,092] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,093] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_context_layer_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,093] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:302
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 232 [ConstantVariable(bool)]
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:303
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:24,094] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:368
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [NNModuleVariable(), TupleVariable()]
[2022-11-07 20:06:24,095] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [NNModuleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,096] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,096] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
316 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
317 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 26
318 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 3 (layer_outputs)
22 CALL_METHOD 1
24 STORE_FAST 3 (layer_outputs)
319 >> 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 3 (layer_outputs)
32 LOAD_FAST 2 (residual_tensor)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 3 (layer_outputs)
320 40 LOAD_FAST 3 (layer_outputs)
42 RETURN_VALUE
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:316
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,097] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:317
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:319
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,109] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,111] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,112] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,112] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,112] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,112] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,112] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,116] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,117] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,117] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,120] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:320
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:369
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,122] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), TupleVariable(), SliceVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:370
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,123] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_attention_outputs [TupleVariable()]
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:533
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TupleVariable()]
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:24,124] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:534
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [TupleVariable()]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:535
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable()]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), SliceVariable()]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:537
[2022-11-07 20:06:24,125] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_feedforward_networks [NNModuleVariable()]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [ConstantVariable(int)]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP != [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 142 [ConstantVariable(bool)]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:538
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ffn [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:24,126] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:24,127] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:24,127] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,129] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,130] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,131] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,131] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,132] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,132] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,144] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:24,147] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,148] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,149] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,161] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,163] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,163] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,164] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,166] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,168] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,171] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,172] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,172] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,173] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,174] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:24,174] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,175] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,176] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,177] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,178] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,189] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,192] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,192] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,193] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,194] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:24,194] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:24,194] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,194] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,195] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,195] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,207] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,208] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,209] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,210] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,212] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,214] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,217] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,218] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,218] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,219] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:24,220] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,221] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,222] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,223] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,224] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,224] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,224] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,224] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,224] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,235] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,235] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,235] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,235] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,236] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,236] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,239] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,240] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:24,241] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:24,241] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,241] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,241] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,241] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,252] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,254] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,255] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,255] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,259] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,260] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,260] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,263] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,264] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,264] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:24,265] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:542
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable()]
[2022-11-07 20:06:24,266] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,267] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,267] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,268] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,268] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,279] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,282] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:543
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,283] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 3 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,284] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
416 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (intermediate_states)
6 CALL_METHOD 1
8 STORE_FAST 4 (layer_output)
417 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 42
418 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 4 (layer_output)
22 CALL_METHOD 1
24 STORE_FAST 4 (layer_output)
419 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 4 (layer_output)
32 LOAD_FAST 2 (residual_tensor_1)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 4 (layer_output)
40 JUMP_FORWARD 26 (to 68)
421 >> 42 LOAD_FAST 0 (self)
44 LOAD_METHOD 3 (LayerNorm)
46 LOAD_FAST 4 (layer_output)
48 LOAD_FAST 2 (residual_tensor_1)
50 BINARY_ADD
52 CALL_METHOD 1
54 STORE_FAST 4 (layer_output)
422 56 LOAD_FAST 0 (self)
58 LOAD_METHOD 4 (bottleneck)
60 LOAD_FAST 4 (layer_output)
62 LOAD_FAST 3 (residual_tensor_2)
64 CALL_METHOD 2
66 STORE_FAST 4 (layer_output)
423 >> 68 LOAD_FAST 4 (layer_output)
70 RETURN_VALUE
[2022-11-07 20:06:24,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:416
[2022-11-07 20:06:24,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,284] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,285] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_states [NNModuleVariable()]
[2022-11-07 20:06:24,285] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:417
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 42 [ConstantVariable(bool)]
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:421
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,296] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,297] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:24,297] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,297] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,298] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,299] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,299] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,299] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,299] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,299] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,302] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,304] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,308] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,309] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,309] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:422
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output [NNModuleVariable()]
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor_2 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,310] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,311] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
396 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
397 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (dropout)
14 LOAD_FAST 3 (layer_outputs)
16 CALL_METHOD 1
18 STORE_FAST 3 (layer_outputs)
398 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (LayerNorm)
24 LOAD_FAST 3 (layer_outputs)
26 LOAD_FAST 2 (residual_tensor)
28 BINARY_ADD
30 CALL_METHOD 1
32 STORE_FAST 3 (layer_outputs)
399 34 LOAD_FAST 3 (layer_outputs)
36 RETURN_VALUE
[2022-11-07 20:06:24,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:396
[2022-11-07 20:06:24,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,311] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,312] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,312] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:397
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,323] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,324] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,324] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:398
[2022-11-07 20:06:24,324] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,324] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,325] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,325] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,325] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,326] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,327] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,327] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,327] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,327] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,327] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,330] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,332] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,332] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,335] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:399
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,336] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4b30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 395>
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:423
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4f50, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 413>
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_output [TensorVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_output []
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:546
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:548
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch [TupleVariable()]
[2022-11-07 20:06:24,337] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR tensor [TupleVariable(), TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:24,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1000 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:24,338] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TupleVariable(), TorchVariable(<built-in method tensor of type object at 0x7f19049e30c0>), ConstantVariable(int)]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:549
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:550
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [TupleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:551
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:552
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:553
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:554
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:547
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 7 [TupleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:24,339] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:556
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:545
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:544
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:558
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:587
[2022-11-07 20:06:24,340] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [ListIteratorVariable()]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:589
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable()]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 34 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:24,341] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 102 [ListIteratorVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:578
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_hidden_states [ListIteratorVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 56 [ListIteratorVariable(), ConstantVariable(bool)]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_module [ListIteratorVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:582
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:583
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:584
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST i [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable()]
[2022-11-07 20:06:24,342] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ListVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,343] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:585
[2022-11-07 20:06:24,343] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,343] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:581
[2022-11-07 20:06:24,343] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [ListIteratorVariable(), NNModuleVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:24,344] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81fb30, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 512>
519 0 LOAD_FAST 0 (self)
2 LOAD_ATTR 0 (use_bottleneck)
4 POP_JUMP_IF_FALSE 26
520 6 LOAD_FAST 0 (self)
8 LOAD_METHOD 1 (bottleneck)
10 LOAD_FAST 1 (hidden_states)
12 CALL_METHOD 1
14 UNPACK_SEQUENCE 4
16 STORE_FAST 5 (query_tensor)
18 STORE_FAST 6 (key_tensor)
20 STORE_FAST 7 (value_tensor)
22 STORE_FAST 8 (layer_input)
24 JUMP_FORWARD 18 (to 44)
522 >> 26 LOAD_FAST 1 (hidden_states)
28 BUILD_LIST 1
30 LOAD_CONST 1 (4)
32 BINARY_MULTIPLY
34 UNPACK_SEQUENCE 4
36 STORE_FAST 5 (query_tensor)
38 STORE_FAST 6 (key_tensor)
40 STORE_FAST 7 (value_tensor)
42 STORE_FAST 8 (layer_input)
524 >> 44 LOAD_FAST 0 (self)
46 LOAD_ATTR 2 (attention)
525 48 LOAD_FAST 5 (query_tensor)
526 50 LOAD_FAST 6 (key_tensor)
527 52 LOAD_FAST 7 (value_tensor)
528 54 LOAD_FAST 8 (layer_input)
529 56 LOAD_FAST 2 (attention_mask)
530 58 LOAD_FAST 3 (head_mask)
531 60 LOAD_FAST 4 (output_attentions)
524 62 LOAD_CONST 2 (('output_attentions',))
64 CALL_FUNCTION_KW 7
66 STORE_FAST 9 (self_attention_outputs)
533 68 LOAD_FAST 9 (self_attention_outputs)
70 LOAD_CONST 3 (0)
72 BINARY_SUBSCR
74 STORE_FAST 10 (attention_output)
534 76 LOAD_FAST 10 (attention_output)
78 BUILD_TUPLE 1
80 STORE_FAST 11 (s)
535 82 LOAD_FAST 9 (self_attention_outputs)
84 LOAD_CONST 4 (1)
86 LOAD_CONST 0 (None)
88 BUILD_SLICE 2
90 BINARY_SUBSCR
92 STORE_FAST 12 (outputs)
537 94 LOAD_FAST 0 (self)
96 LOAD_ATTR 3 (num_feedforward_networks)
98 LOAD_CONST 4 (1)
100 COMPARE_OP 3 (!=)
102 POP_JUMP_IF_FALSE 142
538 104 LOAD_GLOBAL 4 (enumerate)
106 LOAD_FAST 0 (self)
108 LOAD_ATTR 5 (ffn)
110 CALL_FUNCTION 1
112 GET_ITER
>> 114 FOR_ITER 26 (to 142)
116 UNPACK_SEQUENCE 2
118 STORE_FAST 13 (i)
120 STORE_FAST 14 (ffn_module)
539 122 LOAD_FAST 14 (ffn_module)
124 LOAD_FAST 10 (attention_output)
126 CALL_FUNCTION 1
128 STORE_FAST 10 (attention_output)
540 130 LOAD_FAST 11 (s)
132 LOAD_FAST 10 (attention_output)
134 BUILD_TUPLE 1
136 INPLACE_ADD
138 STORE_FAST 11 (s)
140 JUMP_ABSOLUTE 114
542 >> 142 LOAD_FAST 0 (self)
144 LOAD_METHOD 6 (intermediate)
146 LOAD_FAST 10 (attention_output)
148 CALL_METHOD 1
150 STORE_FAST 15 (intermediate_output)
543 152 LOAD_FAST 0 (self)
154 LOAD_METHOD 7 (output)
156 LOAD_FAST 15 (intermediate_output)
158 LOAD_FAST 10 (attention_output)
160 LOAD_FAST 1 (hidden_states)
162 CALL_METHOD 3
164 STORE_FAST 16 (layer_output)
545 166 LOAD_FAST 16 (layer_output)
168 BUILD_TUPLE 1
546 170 LOAD_FAST 12 (outputs)
545 172 BINARY_ADD
548 174 LOAD_GLOBAL 8 (torch)
176 LOAD_METHOD 9 (tensor)
178 LOAD_CONST 5 (1000)
180 CALL_METHOD 1
549 182 LOAD_FAST 5 (query_tensor)
550 184 LOAD_FAST 6 (key_tensor)
551 186 LOAD_FAST 7 (value_tensor)
552 188 LOAD_FAST 8 (layer_input)
553 190 LOAD_FAST 10 (attention_output)
554 192 LOAD_FAST 15 (intermediate_output)
547 194 BUILD_TUPLE 7
545 196 BINARY_ADD
556 198 LOAD_FAST 11 (s)
545 200 BINARY_ADD
544 202 STORE_FAST 12 (outputs)
558 204 LOAD_FAST 12 (outputs)
206 RETURN_VALUE
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:519
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:520
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,345] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,346] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
464 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (input)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (bottlenecked_hidden_states)
465 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck_attention)
14 POP_JUMP_IF_FALSE 26
466 16 LOAD_FAST 2 (bottlenecked_hidden_states)
18 BUILD_TUPLE 1
20 LOAD_CONST 1 (4)
22 BINARY_MULTIPLY
24 RETURN_VALUE
467 >> 26 LOAD_FAST 0 (self)
28 LOAD_ATTR 2 (key_query_shared_bottleneck)
30 POP_JUMP_IF_FALSE 54
468 32 LOAD_FAST 0 (self)
34 LOAD_METHOD 3 (attention)
36 LOAD_FAST 1 (hidden_states)
38 CALL_METHOD 1
40 STORE_FAST 3 (shared_attention_input)
469 42 LOAD_FAST 3 (shared_attention_input)
44 LOAD_FAST 3 (shared_attention_input)
46 LOAD_FAST 1 (hidden_states)
48 LOAD_FAST 2 (bottlenecked_hidden_states)
50 BUILD_TUPLE 4
52 RETURN_VALUE
471 >> 54 LOAD_FAST 1 (hidden_states)
56 LOAD_FAST 1 (hidden_states)
58 LOAD_FAST 1 (hidden_states)
60 LOAD_FAST 2 (bottlenecked_hidden_states)
62 BUILD_TUPLE 4
64 RETURN_VALUE
66 LOAD_CONST 0 (None)
68 RETURN_VALUE
[2022-11-07 20:06:24,347] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:464
[2022-11-07 20:06:24,347] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,347] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR input [NNModuleVariable()]
[2022-11-07 20:06:24,347] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,347] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,348] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:24,348] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:24,348] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,348] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,349] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,349] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:24,361] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,362] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,362] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,362] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,362] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,362] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,365] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,367] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,367] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,369] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST bottlenecked_hidden_states [TensorVariable()]
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:465
[2022-11-07 20:06:24,371] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck_attention [NNModuleVariable()]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:467
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key_query_shared_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 54 [ConstantVariable(bool)]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:468
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,372] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,373] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
433 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (layer_input)
434 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 2 (layer_input)
16 CALL_METHOD 1
18 STORE_FAST 2 (layer_input)
435 20 LOAD_FAST 2 (layer_input)
22 RETURN_VALUE
[2022-11-07 20:06:24,374] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:433
[2022-11-07 20:06:24,374] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,374] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,374] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,374] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:434
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable()]
[2022-11-07 20:06:24,385] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,386] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,387] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,389] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,391] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,391] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,394] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,395] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,395] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,395] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:24,395] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:435
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input []
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f190, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 432>
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:469
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input []
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST shared_attention_input [TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST bottlenecked_hidden_states [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 4 [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f3a0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 447>
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 4 [TupleVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_tensor [TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_tensor [TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_tensor [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,396] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_input [TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_FORWARD 44 []
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention [NNModuleVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:525
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:526
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:527
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:528
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:529
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:530
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:531
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:524
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('output_attentions',) [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:24,397] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 7 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool), ConstantVariable(tuple)]
[2022-11-07 20:06:24,398] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
358 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (self)
359 4 LOAD_FAST 1 (query_tensor)
360 6 LOAD_FAST 2 (key_tensor)
361 8 LOAD_FAST 3 (value_tensor)
362 10 LOAD_FAST 5 (attention_mask)
363 12 LOAD_FAST 6 (head_mask)
364 14 LOAD_FAST 7 (output_attentions)
358 16 CALL_METHOD 6
18 STORE_FAST 8 (self_outputs)
368 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 1 (output)
24 LOAD_FAST 8 (self_outputs)
26 LOAD_CONST 1 (0)
28 BINARY_SUBSCR
30 LOAD_FAST 4 (layer_input)
32 CALL_METHOD 2
34 STORE_FAST 9 (attention_output)
369 36 LOAD_FAST 9 (attention_output)
38 BUILD_TUPLE 1
40 LOAD_FAST 8 (self_outputs)
42 LOAD_CONST 2 (1)
44 LOAD_CONST 0 (None)
46 BUILD_SLICE 2
48 BINARY_SUBSCR
50 BINARY_ADD
52 STORE_FAST 10 (outputs)
370 54 LOAD_FAST 10 (outputs)
56 RETURN_VALUE
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR self [NNModuleVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:359
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:360
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:361
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:362
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:363
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:364
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:358
[2022-11-07 20:06:24,399] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 6 [NNModuleVariable(), TensorVariable(), TensorVariable(), TensorVariable(), TensorVariable(), ConstantVariable(NoneType), ConstantVariable(bool)]
[2022-11-07 20:06:24,401] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
276 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (query)
4 LOAD_FAST 1 (query_tensor)
6 CALL_METHOD 1
8 STORE_FAST 7 (mixed_query_layer)
277 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (key)
14 LOAD_FAST 2 (key_tensor)
16 CALL_METHOD 1
18 STORE_FAST 8 (mixed_key_layer)
278 20 LOAD_FAST 0 (self)
22 LOAD_METHOD 2 (value)
24 LOAD_FAST 3 (value_tensor)
26 CALL_METHOD 1
28 STORE_FAST 9 (mixed_value_layer)
280 30 LOAD_FAST 0 (self)
32 LOAD_METHOD 3 (transpose_for_scores)
34 LOAD_FAST 7 (mixed_query_layer)
36 CALL_METHOD 1
38 STORE_FAST 10 (query_layer)
281 40 LOAD_FAST 0 (self)
42 LOAD_METHOD 3 (transpose_for_scores)
44 LOAD_FAST 8 (mixed_key_layer)
46 CALL_METHOD 1
48 STORE_FAST 11 (key_layer)
282 50 LOAD_FAST 0 (self)
52 LOAD_METHOD 3 (transpose_for_scores)
54 LOAD_FAST 9 (mixed_value_layer)
56 CALL_METHOD 1
58 STORE_FAST 12 (value_layer)
285 60 LOAD_GLOBAL 4 (torch)
62 LOAD_METHOD 5 (matmul)
64 LOAD_FAST 10 (query_layer)
66 LOAD_FAST 11 (key_layer)
68 LOAD_METHOD 6 (transpose)
70 LOAD_CONST 1 (-1)
72 LOAD_CONST 2 (-2)
74 CALL_METHOD 2
76 CALL_METHOD 2
78 STORE_FAST 13 (attention_scores)
286 80 LOAD_FAST 13 (attention_scores)
82 LOAD_GLOBAL 7 (math)
84 LOAD_METHOD 8 (sqrt)
86 LOAD_FAST 0 (self)
88 LOAD_ATTR 9 (attention_head_size)
90 CALL_METHOD 1
92 BINARY_TRUE_DIVIDE
94 STORE_FAST 13 (attention_scores)
287 96 LOAD_FAST 4 (attention_mask)
98 LOAD_CONST 0 (None)
100 COMPARE_OP 9 (is not)
102 POP_JUMP_IF_FALSE 112
289 104 LOAD_FAST 13 (attention_scores)
106 LOAD_FAST 4 (attention_mask)
108 BINARY_ADD
110 STORE_FAST 13 (attention_scores)
291 >> 112 LOAD_GLOBAL 10 (nn)
114 LOAD_ATTR 11 (functional)
116 LOAD_ATTR 12 (softmax)
118 LOAD_FAST 13 (attention_scores)
120 LOAD_CONST 1 (-1)
122 LOAD_CONST 3 (('dim',))
124 CALL_FUNCTION_KW 2
126 STORE_FAST 14 (attention_probs)
294 128 LOAD_FAST 0 (self)
130 LOAD_METHOD 13 (dropout)
132 LOAD_FAST 14 (attention_probs)
134 CALL_METHOD 1
136 STORE_FAST 14 (attention_probs)
296 138 LOAD_FAST 5 (head_mask)
140 LOAD_CONST 0 (None)
142 COMPARE_OP 9 (is not)
144 POP_JUMP_IF_FALSE 154
297 146 LOAD_FAST 14 (attention_probs)
148 LOAD_FAST 5 (head_mask)
150 BINARY_MULTIPLY
152 STORE_FAST 14 (attention_probs)
298 >> 154 LOAD_GLOBAL 4 (torch)
156 LOAD_METHOD 5 (matmul)
158 LOAD_FAST 14 (attention_probs)
160 LOAD_FAST 12 (value_layer)
162 CALL_METHOD 2
164 STORE_FAST 15 (context_layer)
299 166 LOAD_FAST 15 (context_layer)
168 LOAD_METHOD 14 (permute)
170 LOAD_CONST 4 (0)
172 LOAD_CONST 5 (2)
174 LOAD_CONST 6 (1)
176 LOAD_CONST 7 (3)
178 CALL_METHOD 4
180 LOAD_METHOD 15 (contiguous)
182 CALL_METHOD 0
184 STORE_FAST 15 (context_layer)
300 186 LOAD_FAST 15 (context_layer)
188 LOAD_METHOD 16 (size)
190 CALL_METHOD 0
192 LOAD_CONST 0 (None)
194 LOAD_CONST 2 (-2)
196 BUILD_SLICE 2
198 BINARY_SUBSCR
200 LOAD_FAST 0 (self)
202 LOAD_ATTR 17 (all_head_size)
204 BUILD_TUPLE 1
206 BINARY_ADD
208 STORE_FAST 16 (new_context_layer_shape)
301 210 LOAD_FAST 15 (context_layer)
212 LOAD_METHOD 18 (view)
214 LOAD_FAST 16 (new_context_layer_shape)
216 CALL_METHOD 1
218 STORE_FAST 15 (context_layer)
302 220 LOAD_FAST 6 (output_attentions)
222 POP_JUMP_IF_FALSE 232
224 LOAD_FAST 15 (context_layer)
226 LOAD_FAST 14 (attention_probs)
228 BUILD_TUPLE 2
230 JUMP_FORWARD 4 (to 236)
>> 232 LOAD_FAST 15 (context_layer)
234 BUILD_TUPLE 1
>> 236 STORE_FAST 17 (outputs)
303 238 LOAD_FAST 17 (outputs)
240 RETURN_VALUE
[2022-11-07 20:06:24,401] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:276
[2022-11-07 20:06:24,401] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,401] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR query [NNModuleVariable()]
[2022-11-07 20:06:24,402] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,402] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,413] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_query_layer [TensorVariable()]
[2022-11-07 20:06:24,413] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:277
[2022-11-07 20:06:24,413] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,413] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR key [NNModuleVariable()]
[2022-11-07 20:06:24,414] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,414] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_key_layer [TensorVariable()]
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:278
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR value [NNModuleVariable()]
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_tensor [NNModuleVariable()]
[2022-11-07 20:06:24,425] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,436] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST mixed_value_layer [TensorVariable()]
[2022-11-07 20:06:24,436] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:280
[2022-11-07 20:06:24,436] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,436] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,437] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_query_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,437] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,438] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,438] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,438] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,438] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,438] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,439] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,440] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,442] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST query_layer [TensorVariable()]
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:281
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,444] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_key_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,445] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,446] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,446] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,446] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,446] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,446] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,447] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,448] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,451] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST key_layer [TensorVariable()]
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:282
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose_for_scores [NNModuleVariable()]
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST mixed_value_layer [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable())]
[2022-11-07 20:06:24,453] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [UserMethodVariable(<function MobileBertSelfAttention.transpose_for_scores at 0x7f161bea5e50>, NNModuleVariable()), TensorVariable()]
[2022-11-07 20:06:24,454] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
263 0 LOAD_FAST 1 (x)
2 LOAD_METHOD 0 (size)
4 CALL_METHOD 0
6 LOAD_CONST 0 (None)
8 LOAD_CONST 1 (-1)
10 BUILD_SLICE 2
12 BINARY_SUBSCR
14 LOAD_FAST 0 (self)
16 LOAD_ATTR 1 (num_attention_heads)
18 LOAD_FAST 0 (self)
20 LOAD_ATTR 2 (attention_head_size)
22 BUILD_TUPLE 2
24 BINARY_ADD
26 STORE_FAST 2 (new_x_shape)
264 28 LOAD_FAST 1 (x)
30 LOAD_METHOD 3 (view)
32 LOAD_FAST 2 (new_x_shape)
34 CALL_METHOD 1
36 STORE_FAST 1 (x)
265 38 LOAD_FAST 1 (x)
40 LOAD_METHOD 4 (permute)
42 LOAD_CONST 2 (0)
44 LOAD_CONST 3 (2)
46 LOAD_CONST 4 (1)
48 LOAD_CONST 5 (3)
50 CALL_METHOD 4
52 RETURN_VALUE
[2022-11-07 20:06:24,454] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:263
[2022-11-07 20:06:24,454] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,454] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,455] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,455] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,455] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,455] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,455] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_attention_heads [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [SizeVariable(), ConstantVariable(int), NNModuleVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 2 [SizeVariable(), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_x_shape [TupleVariable()]
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:264
[2022-11-07 20:06:24,456] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,457] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,457] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_x_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,457] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,458] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST x [TensorVariable()]
[2022-11-07 20:06:24,458] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:265
[2022-11-07 20:06:24,458] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST x []
[2022-11-07 20:06:24,458] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,459] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,459] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,459] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,459] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,459] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,460] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,460] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object transpose_for_scores at 0x7f161bea47c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 262>
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST value_layer [TensorVariable()]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:285
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST query_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST key_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR transpose [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose)]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int)]
[2022-11-07 20:06:24,461] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), GetAttrVariable(TensorVariable(), transpose), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,463] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:286
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL math [TensorVariable()]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR sqrt [TensorVariable(), TorchVariable(<module 'math' from '/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'>)]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable(), TorchVariable(<built-in function sqrt>)]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR attention_head_size [TensorVariable(), TorchVariable(<built-in function sqrt>), NNModuleVariable()]
[2022-11-07 20:06:24,468] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [TensorVariable(), TorchVariable(<built-in function sqrt>), ConstantVariable(int)]
[2022-11-07 20:06:24,469] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_TRUE_DIVIDE None [TensorVariable(), ConstantVariable(float)]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:287
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask []
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TensorVariable()]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [TensorVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 112 [ConstantVariable(bool)]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:289
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores []
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_mask [TensorVariable()]
[2022-11-07 20:06:24,470] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_scores [TensorVariable()]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:291
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL nn []
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR functional [TorchVariable(<module 'torch.nn' from '/scratch/dberard/dynamo38/pytorch/torch/nn/__init__.py'>)]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR softmax [TorchVariable(<module 'torch.nn.functional' from '/scratch/dberard/dynamo38/pytorch/torch/nn/functional.py'>)]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_scores [TorchVariable(<function softmax at 0x7f16c76f6a60>)]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -1 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable()]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST ('dim',) [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,472] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION_KW 2 [TorchVariable(<function softmax at 0x7f16c76f6a60>), TensorVariable(), ConstantVariable(int), ConstantVariable(tuple)]
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:294
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dropout [NNModuleVariable()]
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [NNModuleVariable()]
[2022-11-07 20:06:24,478] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,479] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_probs [TensorVariable()]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:296
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST head_mask []
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [ConstantVariable(NoneType)]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP is not [ConstantVariable(NoneType), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 154 [ConstantVariable(bool)]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:298
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL torch []
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR matmul [TorchVariable(<module 'torch' from '/scratch/dberard/dynamo38/pytorch/torch/__init__.py'>)]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_probs [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>)]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST value_layer [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable()]
[2022-11-07 20:06:24,480] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [TorchVariable(<built-in method matmul of type object at 0x7f19049e30c0>), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:299
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR permute [TensorVariable()]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [GetAttrVariable(TensorVariable(), permute)]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 2 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int)]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 3 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,485] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 4 [GetAttrVariable(TensorVariable(), permute), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,487] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR contiguous [TensorVariable()]
[2022-11-07 20:06:24,487] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), contiguous)]
[2022-11-07 20:06:24,489] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,489] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:300
[2022-11-07 20:06:24,489] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,489] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR size [TensorVariable()]
[2022-11-07 20:06:24,489] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 0 [GetAttrVariable(TensorVariable(), size)]
[2022-11-07 20:06:24,490] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [SizeVariable()]
[2022-11-07 20:06:24,490] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST -2 [SizeVariable(), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,490] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [SizeVariable(), ConstantVariable(NoneType), ConstantVariable(int)]
[2022-11-07 20:06:24,490] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [SizeVariable(), SliceVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [SizeVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR all_head_size [SizeVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [SizeVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [SizeVariable(), TupleVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST new_context_layer_shape [TupleVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:301
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR view [TensorVariable()]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST new_context_layer_shape [GetAttrVariable(TensorVariable(), view)]
[2022-11-07 20:06:24,491] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [GetAttrVariable(TensorVariable(), view), TupleVariable()]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST context_layer [TensorVariable()]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:302
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST output_attentions []
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 232 [ConstantVariable(bool)]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST context_layer []
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:303
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,493] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4660, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 267>
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:368
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [NNModuleVariable(), TupleVariable()]
[2022-11-07 20:06:24,494] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [NNModuleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,495] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_input [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,495] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
316 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
317 10 LOAD_FAST 0 (self)
12 LOAD_ATTR 1 (use_bottleneck)
14 POP_JUMP_IF_TRUE 26
318 16 LOAD_FAST 0 (self)
18 LOAD_METHOD 2 (dropout)
20 LOAD_FAST 3 (layer_outputs)
22 CALL_METHOD 1
24 STORE_FAST 3 (layer_outputs)
319 >> 26 LOAD_FAST 0 (self)
28 LOAD_METHOD 3 (LayerNorm)
30 LOAD_FAST 3 (layer_outputs)
32 LOAD_FAST 2 (residual_tensor)
34 BINARY_ADD
36 CALL_METHOD 1
38 STORE_FAST 3 (layer_outputs)
320 40 LOAD_FAST 3 (layer_outputs)
42 RETURN_VALUE
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:316
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,496] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:317
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR use_bottleneck [NNModuleVariable()]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_TRUE 26 [ConstantVariable(bool)]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:319
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,508] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,509] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,510] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,511] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,511] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,511] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,511] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,511] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,515] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,517] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,519] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:320
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4240, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 315>
[2022-11-07 20:06:24,521] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:369
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_outputs [TupleVariable()]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), TupleVariable(), SliceVariable()]
[2022-11-07 20:06:24,522] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:370
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST outputs []
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TupleVariable()]
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea42f0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 348>
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST self_attention_outputs [TupleVariable()]
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:533
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 0 [TupleVariable()]
[2022-11-07 20:06:24,523] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [TensorVariable()]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:534
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output []
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [TensorVariable()]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [TupleVariable()]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:535
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self_attention_outputs []
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [TupleVariable()]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST None [TupleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_SLICE 2 [TupleVariable(), ConstantVariable(int), ConstantVariable(NoneType)]
[2022-11-07 20:06:24,524] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_SUBSCR None [TupleVariable(), SliceVariable()]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST outputs [TupleVariable()]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:537
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR num_feedforward_networks [NNModuleVariable()]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_CONST 1 [ConstantVariable(int)]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE COMPARE_OP != [ConstantVariable(int), ConstantVariable(int)]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE POP_JUMP_IF_FALSE 142 [ConstantVariable(bool)]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:538
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_GLOBAL enumerate []
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [BuiltinVariable(enumerate)]
[2022-11-07 20:06:24,525] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR ffn [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:24,526] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [BuiltinVariable(enumerate), NNModuleVariable()]
[2022-11-07 20:06:24,526] torch._dynamo.symbolic_convert: [DEBUG] TRACE GET_ITER None [TupleVariable()]
[2022-11-07 20:06:24,527] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,528] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,529] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,529] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,529] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,529] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,530] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,531] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,543] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,543] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,543] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,543] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,544] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,547] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,548] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:24,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:24,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,549] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,560] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,560] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:24,560] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,560] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,561] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,561] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,561] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,562] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,563] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,563] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,563] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,563] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,563] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,566] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,567] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,568] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,570] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,572] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:24,573] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,574] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,575] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,575] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,575] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,576] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,577] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
383 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 1 (hidden_states)
384 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (intermediate_act_fn)
14 LOAD_FAST 1 (hidden_states)
16 CALL_METHOD 1
18 STORE_FAST 1 (hidden_states)
385 20 LOAD_FAST 1 (hidden_states)
22 RETURN_VALUE
[2022-11-07 20:06:24,577] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:383
[2022-11-07 20:06:24,577] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,577] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,578] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:384
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate_act_fn [NNModuleVariable()]
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,589] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST hidden_states [TensorVariable()]
[2022-11-07 20:06:24,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:385
[2022-11-07 20:06:24,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states []
[2022-11-07 20:06:24,592] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,592] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161bea4710, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 382>
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST intermediate_output [TensorVariable()]
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:494
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR output [NNModuleVariable()]
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST intermediate_output [NNModuleVariable()]
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,593] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 2 [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,594] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
481 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (dense)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 3 (layer_outputs)
482 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (LayerNorm)
14 LOAD_FAST 3 (layer_outputs)
16 LOAD_FAST 2 (residual_tensor)
18 BINARY_ADD
20 CALL_METHOD 1
22 STORE_FAST 3 (layer_outputs)
483 24 LOAD_FAST 3 (layer_outputs)
26 RETURN_VALUE
[2022-11-07 20:06:24,594] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:481
[2022-11-07 20:06:24,594] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,594] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR dense [NNModuleVariable()]
[2022-11-07 20:06:24,595] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST hidden_states [NNModuleVariable()]
[2022-11-07 20:06:24,595] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:482
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR LayerNorm [NNModuleVariable()]
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs [NNModuleVariable()]
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST residual_tensor [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,607] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [NNModuleVariable(), TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,609] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,610] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
168 0 LOAD_FAST 1 (input_tensor)
2 LOAD_FAST 0 (self)
4 LOAD_ATTR 0 (weight)
6 BINARY_MULTIPLY
8 LOAD_FAST 0 (self)
10 LOAD_ATTR 1 (bias)
12 BINARY_ADD
14 RETURN_VALUE
[2022-11-07 20:06:24,610] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:168
[2022-11-07 20:06:24,610] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST input_tensor []
[2022-11-07 20:06:24,610] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,610] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR weight [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,613] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_MULTIPLY None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,614] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self [TensorVariable()]
[2022-11-07 20:06:24,614] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR bias [TensorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,617] torch._dynamo.symbolic_convert: [DEBUG] TRACE BINARY_ADD None [TensorVariable(), TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161beab7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 167>
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:483
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f5b0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 480>
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST layer_outputs [TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:495
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST layer_outputs []
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] TRACE RETURN_VALUE None [TensorVariable()]
[2022-11-07 20:06:24,619] torch._dynamo.symbolic_convert: [DEBUG] DONE INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST attention_output [ListIteratorVariable(), TensorVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:540
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST s [ListIteratorVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE BUILD_TUPLE 1 [ListIteratorVariable(), TupleVariable(), TensorVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE INPLACE_ADD None [ListIteratorVariable(), TupleVariable(), TupleVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST s [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE JUMP_ABSOLUTE 114 [ListIteratorVariable()]
[2022-11-07 20:06:24,620] torch._dynamo.symbolic_convert: [DEBUG] TRACE FOR_ITER 142 [ListIteratorVariable()]
[2022-11-07 20:06:24,621] torch._dynamo.symbolic_convert: [DEBUG] TRACE UNPACK_SEQUENCE 2 [ListIteratorVariable(), TupleVariable()]
[2022-11-07 20:06:24,621] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST i [ListIteratorVariable(), NNModuleVariable(), ConstantVariable(int)]
[2022-11-07 20:06:24,621] torch._dynamo.symbolic_convert: [DEBUG] TRACE STORE_FAST ffn_module [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,621] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:539
[2022-11-07 20:06:24,622] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST ffn_module [ListIteratorVariable()]
[2022-11-07 20:06:24,622] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST attention_output [ListIteratorVariable(), NNModuleVariable()]
[2022-11-07 20:06:24,622] torch._dynamo.symbolic_convert: [DEBUG] TRACE CALL_FUNCTION 1 [ListIteratorVariable(), NNModuleVariable(), TensorVariable()]
[2022-11-07 20:06:24,623] torch._dynamo.symbolic_convert: [DEBUG] INLINING <code object forward at 0x7f161b81f7c0, file "/data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py", line 492>
493 0 LOAD_FAST 0 (self)
2 LOAD_METHOD 0 (intermediate)
4 LOAD_FAST 1 (hidden_states)
6 CALL_METHOD 1
8 STORE_FAST 2 (intermediate_output)
494 10 LOAD_FAST 0 (self)
12 LOAD_METHOD 1 (output)
14 LOAD_FAST 2 (intermediate_output)
16 LOAD_FAST 1 (hidden_states)
18 CALL_METHOD 2
20 STORE_FAST 3 (layer_outputs)
495 22 LOAD_FAST 3 (layer_outputs)
24 RETURN_VALUE
[2022-11-07 20:06:24,623] torch._dynamo.symbolic_convert: [DEBUG] TRACE starts_line /data/home/dberard/miniconda/envs/dynamo38/lib/python3.8/site-packages/transformers/models/mobilebert/modeling_mobilebert.py:493
[2022-11-07 20:06:24,623] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_FAST self []
[2022-11-07 20:06:24,623] torch._dynamo.symbolic_convert: [DEBUG] TRACE LOAD_ATTR intermediate [NNModuleVariable()]
[2022-11-07 20:06:24,623] tor
Raw
optim.txt
View raw

(Sorry about that, but we can’t show files that are this big right now.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment