Skip to content

Instantly share code, notes, and snippets.

View sjtalkar's full-sized avatar
💭
Immersion training in Python and Power BI for DataScience

Simi Talkar sjtalkar

💭
Immersion training in Python and Power BI for DataScience
  • Mondelez Intl.
  • Bellevue, WA
View GitHub Profile
@sjtalkar
sjtalkar / gist:65ce7a2f6df2c820acf1a6e485b9af1b
Last active April 28, 2023 17:04
Dynamic Programming Cost Matrix
def dp(dist_mat):
"""
Find minimum-cost path through matrix `dist_mat` using dynamic programming.
The cost of a path is defined as the sum of the matrix entries on that
path. See the following for details of the algorithm:
- http://en.wikipedia.org/wiki/Dynamic_time_warping
- https://www.ee.columbia.edu/~dpwe/resources/matlab/dtw/dp.m
@sjtalkar
sjtalkar / gist:b10e0ff3139b65bfcbd9e4738a94a4b3
Last active November 25, 2022 01:05
Convolution layer output size calculation
#Source Aurelion Geron https://colab.research.google.com/github/ageron/handson-ml3/blob/main/14_deep_computer_vision_with_cnns.ipynb#scrollTo=jisXP9jfpKz2
import numpy as np
def conv_output_size(input_size, kernel_size, strides=1, padding="valid"):
if padding=="valid":
z = input_size - kernel_size + strides
output_size = z // strides
num_ignored = z % strides
return output_size, num_ignored
else:
@sjtalkar
sjtalkar / gist:50bcefe0086c6736ce74a8b070c65e10
Created October 22, 2022 16:49
Get tweet location example
#Credit : https://brittarude.github.io/blog/2021/08/01/Location-and-geo-information-in-twitter
new_search = "EdTech -filter:retweets"
tweets = tweepy.Cursor(api.search_tweets,
q=new_search,
place_country="DE",
lang="de",
since='2020-07-14').items(100)
users_locs = [[tweet.text, tweet.user.location, tweet.place] for tweet in tweets]
let
Source = Odbc.DataSource("dsn=CData SugarCRM Sys", [HierarchicalNavigation=true]),
CData_Database = Source{[Name="CData",Kind="Database"]}[Data],
SugarCRM_Schema = CData_Database{[Name="SugarCRM",Kind="Schema"]}[Data],
Accounts_Table = SugarCRM_Schema{[Name="Accounts",Kind="Table"]}[Data],
table_to_work_with = Table.SelectColumns(Accounts_Table,{"Id", "Tag"}),
tags_table = Table.TransformColumnTypes(table_to_work_with,{{"Tag", type text}}),
split_by_colon = Table.SplitColumn(tags_table, "Tag", Splitter.SplitTextByDelimiter(":", QuoteStyle.Csv), {"Tag.1", "Tag.2", "Tag.3", "Tag.4", "Tag.5", "Tag.6", "Tag.7", "Tag.8", "Tag.9", "Tag.10", "Tag.11", "Tag.12", "Tag.13", "Tag.14", "Tag.15", "Tag.16", "Tag.17"}),
tag_cols = List.Range (Table.ColumnNames(split_by_colon) , 2),
@sjtalkar
sjtalkar / gist:753a9e60d07e9d6ed1dbc103d9fbd721
Created July 22, 2022 17:46
Set all column types dynamically and select multiple items from list
= Table.TransformColumnTypes (reset_header, List.Transform (transform_cols_list, each {_, type text}))
= List.Select(transform_cols_list, each Text.StartsWith (_, "Federal"))
@sjtalkar
sjtalkar / gist:87f235428f4b089de0a327548a0149cb
Last active July 21, 2022 14:23
Reading the current workbook filename Excel and Powe Query
Create two cells with names FilePath and FileName in Excel
FilePath cell has formula : G:\My Drive\BBSI\Round 2 PDFs\
FileName Cell has formula : =MID(CELL("filename",A2),FIND("[",CELL("filename",A2))+1,FIND("]", CELL("filename",A2))-FIND("[",CELL("filename",A2))-1)
Create a function name function_get_current_file in Power Query:
function_get_current_file
()=>
let
current_dir = Excel.CurrentWorkbook(){[Name="FilePath"]}[Content]{0}[Column1],
@sjtalkar
sjtalkar / gist:4175921b5d43251fb694c13bfe1994ea
Created July 15, 2022 16:48
Remove null columns in Power Query M Language with iteration
get_curr_cols = Table.ColumnNames (set_section_headers),
get_null_cols = List.Generate(
() => [counter = 0, x = get_curr_cols, result="False"] ,
each [counter] < List.Count([x]),
each [counter=[counter]+1 ,
x= get_curr_cols,
result= if List.NonNullCount(Table.Column(set_section_headers, [x]{counter})) <= 0 then "True" else "False"
],
each [result] ),
@sjtalkar
sjtalkar / gist:6d436a6f15e833936dbf7c1878ce7b73
Created June 27, 2022 22:34
Accessing a Table in a column in a query
= group_by_employee{[Column1 = "Lastname, FirstName" ]}[merge_employee]
//Here group_by_employee is the last step of a query and so it is a table containing a column of employee names. The next column mer_employee
contains a Table of all data for that employee
@sjtalkar
sjtalkar / gist:fdf38dbb198701997d630390ea9653b6
Created June 23, 2022 05:06
Use List.Generate to loop through columns in a table
let
Source = Pdf.Tables(File.Contents("G:\My Drive\BBSI\Paychex 2022-01-28 Payroll_Journal_Wed_Jan_26_14_02_00_EST_2022.pdf"), [Implementation="1.3"]),
this_page = Source{[Id="Page015"]}[Data],
//There is consistently two header and two footer rows that need to be removed
remove_botttom_three = Table.RemoveLastN(this_page,3),
remove_wccode = Table.ReplaceValue( remove_botttom_three, each [Column1], each if Text.StartsWith([Column1],"(") then null else [Column1] , Replacer.ReplaceValue,{"Column1"} ),
remove_title_and_company_name = Table.Skip(remove_wccode,2),
@sjtalkar
sjtalkar / gist:b76740fd152b962314747fb4ef7c4478
Created May 5, 2022 16:28
Reorder cols dynamically in M Query
curr_col_list = Table.ColumnNames(fill_section_down),
last_col = List.LastN(curr_col_list, 1),
first_cols = List.FirstN(curr_col_list, List.Count(curr_col_list) - 1),
new_col_order = List.Combine(last_col, first_cols),
reordered_table = Table.ReorderColumns(remove_first_col, new_col_order),