Last active
January 23, 2019 16:56
-
-
Save adam-phillipps/ea8c24d19bfb62c390d778b7f9697ea7 to your computer and use it in GitHub Desktop.
a path-bug in a demo in the json_normalize function
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
args = ( | |
[["State", "Name"]], | |
[ | |
"InstanceId", "VpcId", "PublicIpAddress", "PrivateIpAddress", | |
"PublicDnsName", "PrivateDnsName", "ImageId", "SubnetId", | |
"KeyName", "InstanceType", "LaunchTime", | |
["State", "Name"] | |
] | |
) | |
clean = lambda data: pd.io.json.json_normalize(data, *args, errors='ignore') | |
ec2 = boto3.client('ec2') | |
i, r = "Instances", "Reservations" | |
instances = [reserv[i][-1] for reserv in ec2.describe_instances()[r] if i in reserv] | |
data = [clean(inst) for inst in instances] | |
results_table = pd.concat(data) | |
unique_records_table = results_table.drop_duplicates("InstanceId") | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
unique_records_table | |
0 InstanceId VpcId PublicIpAddress ... KeyName InstanceType LaunchTime State.Name | |
0 r i-08071c704a45c09af vpc-c4e93fa1 NaN ... key-name-name-of-it t2.medium 2019-01-23 12:01:21+00:00 running | |
0 r i-c5707433 vpc-c4e93fa1 12.23.345.567 ... key-name-name-of-it t2.small 2015-06-17 01:29:14+00:00 running | |
0 r i-05a817254f1f0bf3a vpc-c4e93fa1 NaN ... key-name-name-of-it t2.medium 2018-09-04 15:38:35+00:00 running | |
0 r i-262f46e3 vpc-c4e93fa1 12.23.345.567 ... key-name-name-of-it t2.small 2015-08-26 17:57:54+00:00 running | |
0 r i-04ae135070a1ae6c3 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.small 2019-01-23 12:01:21+00:00 running | |
0 r i-0a3ba15d3ba74903b vpc-c4e93fa1 NaN ... key-name-name-of-it t2.medium 2018-03-12 18:15:34+00:00 running | |
0 s i-83356d46 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.small 2015-09-11 17:30:31+00:00 stopped | |
0 r i-070045e7be71e5268 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.medium 2018-07-17 17:03:11+00:00 running | |
0 r i-0e0ce3b7ad6429af9 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.micro 2018-08-31 16:21:39+00:00 running | |
0 r i-0ae87df89b65c2943 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.small 2018-02-28 21:01:21+00:00 running | |
0 r i-0c8321aee91358c40 vpc-c4e93fa1 NaN ... key-name-name-of-it t2.medium 2018-11-11 02:03:50+00:00 running | |
... |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment