Skip to content

Instantly share code, notes, and snippets.

@davidberard98
Created March 7, 2023 19:19
Show Gist options
  • Save davidberard98/7e5e9443a1912fd19337d0a38952e042 to your computer and use it in GitHub Desktop.
Save davidberard98/7e5e9443a1912fd19337d0a38952e042 to your computer and use it in GitHub Desktop.
WARNING:root:Super_SloMo failed to load
Eager model failed to run
Traceback (most recent call last):
File "/scratch/dberard/bisectdynamo/pytorch/benchmarks/dynamo/common.py", line 1115, in validate_model
self.model_iter_fn(model, example_inputs)
File "benchmarks/dynamo/torchbench.py", line 370, in forward_and_backward_pass
self.grad_scaler.scale(loss).backward()
File "/scratch/dberard/bisectdynamo/pytorch/torch/_tensor.py", line 487, in backward
torch.autograd.backward(
File "/scratch/dberard/bisectdynamo/pytorch/torch/autograd/__init__.py", line 204, in backward
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
RuntimeError: grid_sampler_2d_backward_cuda does not have a deterministic implementation, but you set 'torch.use_deterministic_algorithms(True)'. You can turn off determinism just for this operation, or you can use the 'warn_only=True' option, if that's acceptable for your application. You can also file an issue at https://github.com/pytorch/pytorch/issues to help us prioritize adding deterministic support for this operation.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/scratch/dberard/bisectdynamo/pytorch/benchmarks/dynamo/common.py", line 2224, in run
) = runner.load_model(device, model_name, batch_size=batch_size)
File "benchmarks/dynamo/torchbench.py", line 310, in load_model
self.validate_model(model, example_inputs)
File "/scratch/dberard/bisectdynamo/pytorch/benchmarks/dynamo/common.py", line 1117, in validate_model
raise NotImplementedError("Eager model failed to run") from e
NotImplementedError: Eager model failed to run
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment