Edit /etc/pam.d/sudo
add after header:
auth sufficient pam_tid.so
File should look like this in the end:
# sudo: auth account password session
auth sufficient pam_tid.so
auth sufficient pam_smartcard.so
auth required pam_opendirectory.so
account required pam_permit.so
password required pam_deny.so
session required pam_permit.so