Skip to content

Instantly share code, notes, and snippets.

@jrzaurin
Last active June 13, 2021 09:12
Show Gist options
  • Save jrzaurin/9efc4c3bd5abbd8bc408cedc12e510b6 to your computer and use it in GitHub Desktop.
Save jrzaurin/9efc4c3bd5abbd8bc408cedc12e510b6 to your computer and use it in GitHub Desktop.
Results for the Adult Census using Tabnet
n_steps step_dim attn_dim ghost_bn virtual_batch_size momentum gamma dropout embed_dropout lr batch_size weight_decay lambda_sparse optimizer lr_scheduler base_lr max_lr div_factor final_div_factor n_cycles val_loss_or_metric
5 32 32 False 128 0.98 1.5 0.1 0.0 0.03 128 0.0 0.0001 AdamW ReduceLROnPlateau 0.001 0.01 25 10000.0 5.0 0.2916
5 64 64 False 128 0.98 1.5 0.2 0.0 0.03 128 0.0 0.0001 Adam ReduceLROnPlateau 0.001 0.01 25 10000.0 5.0 0.2938
5 32 32 False 128 0.98 1.5 0.1 0.0 0.03 128 0.0 0.0001 Adam ReduceLROnPlateau 0.001 0.01 25 10000.0 5.0 0.2939
5 64 64 False 128 0.98 1.5 0.2 0.0 0.03 128 0.0 0.0001 AdamW ReduceLROnPlateau 0.001 0.01 25 10000.0 5.0 0.2945
5 64 64 False 128 0.98 1.5 0.2 0.0 0.05 128 0.0 0.0001 RAdam ReduceLROnPlateau 0.001 0.01 25 10000.0 5.0 0.2962
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment