filter the countries where the VAT number field will show up for

wc-eu-vat-countries.php

add_filter( 'woocommerce_eu_vat_number_country_codes', 'woo_custom_eu_vat_number_country_codes' );
function woo_custom_eu_vat_number_country_codes( $vat_countries ) {
// only show field for users in BE
return array( 'BE' );
}

For anyone that is manipulating/removing the countries through an unset or such, pass the returning array through array_values first. I discovered that the values embedded on the page also picked up the array keys which was causing my VAT field from stop appearing completely.

@rynaldos can you update the script?

If you need to remove a couple of countries use array_diff passing an array of the country codes you wish to remove as the second param, no need to loop and unset values.
As @jakecausier mentioned pass the modified array through array_values to reindex the values.

add_filter( 'woocommerce_eu_vat_number_country_codes', 'woo_custom_eu_vat_number_country_codes' );
function woo_custom_eu_vat_number_country_codes( $vat_countries ) {
    $display_vat = array_diff($vat_countries, ['SE', 'GB']); // remove countries in second array
    return array_values($display_vat); // reindex array
}