xformers attention bias limitations

gistfile1.txt

No operator found for `memory_efficient_attention_forward` with inputs:
     query       : shape=(32, 4096, 1, 40) (torch.float16)
     key         : shape=(32, 77, 1, 40) (torch.float16)
     value       : shape=(32, 77, 1, 40) (torch.float16)
     attn_bias   : <class 'torch.Tensor'>
     p           : 0.0
`cutlassF` is not supported because:
    attn_bias.shape[-1] % 8 != 0
`flshattF` is not supported because:
    attn_bias type is <class 'torch.Tensor'>
`tritonflashattF` is not supported because:
    attn_bias type is <class 'torch.Tensor'>
    requires A100 GPU
`smallkF` is not supported because:
    dtype=torch.float16 (supported: {torch.float32})
    max(query.shape[-1] != value.shape[-1]) > 32
    bias with non-zero stride not supported
    unsupported embed per head: 40