Igor Cotruta Hugoberry
Hugoberry
/ CRC32.m
Last active
January 30, 2024 22:33
CRC32 implementation in Power Query M mostly inpired by the article about CRC32 computation http://www.sunshine2k.de/articles/coding/crc/understanding_crc.html and Bit Hacks https://graphics.stanford.edu/~seander/bithacks.htm
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (StringInput)=> | |
| let | |
| /*function for reversing the bits in a 8-bit number*/ | |
| reverse = (x) => | |
| Number.Mod(Number.BitwiseAnd((x* 0x0202020202),0x010884422010),1023), | |
| /*function for reversing the bits in a 32-bit number*/ | |
| reverse32 = (x) => | |
| let | |
| b0 = Number.BitwiseAnd(x,0xff), |
Hugoberry
/ ReversingBits.ps1
Last active
January 24, 2017 16:37
An example of reversing bits in a 32-bit number
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| function reverse(){ | |
| param([Int32]$b) | |
| $b = (($b -band 0xf0) -shr 4) -bor (($b -band 0x0f) -shl 4) | |
| $b = (($b -band 0xcc) -shr 2) -bor (($b -band 0x33) -shl 2) | |
| $b = (($b -band 0xaa) -shr 1) -bor (($b -band 0x55) -shl 1) | |
| $b | |
| } | |
| # on a 64bit machine this is an alternative to reverse() function | |
| function reverse64(){ | |
| param([Int32]$x) |
Hugoberry
/ Fisher-Yates-shuffle.m
Last active
February 1, 2017 18:11
The Fisher–Yates shuffle is an algorithm for generating a random permutation of a finite set—in plain terms, the algorithm shuffles the set. The algorithm effectively puts all the elements into a hat; it continually determines the next element by randomly drawing an element from the hat until no elements remain. The algorithm produces an unbiase…
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (n) => | |
| let | |
| swapAnyOrder = (_,from,to)=> if from = to then _ else | |
| (if from < to then swap(_,from,to) else swap(_,to,from)), | |
| swap = (_,from,to)=> List.Range(_,0,from) | |
| &{_{to}} | |
| &List.Range(_,from+1,to-from-1) | |
| &{_{from}} | |
| &List.Range(_,to+1) | |
| in |
Hugoberry
/ MatrixMultiplication.m
Last active
January 29, 2018 14:59
Matrix Multiplication in Power Query M. Test example = MatrixMultiplication(#table({"A","B"},{{1,2},{3,4}}), #table({"A","B"},{{1,2},{3,4}}))
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (A,B) => | |
| let | |
| //vector multiplication | |
| dotProduct = (v1,v2) => | |
| List.Accumulate( | |
| List.Zip({v1,v2}), | |
| 0, | |
| (agg,_)=> | |
| agg+(_{1}*_{0})) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (ZIPFile) => | |
| let | |
| Header = BinaryFormat.Record([ Signature = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32,ByteOrder.LittleEndian), | |
| Version = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian), | |
| Flags = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian), | |
| Compression = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian), | |
| ModTime = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian), | |
| ModDate = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16,ByteOrder.LittleEndian), | |
| CRC32 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32,ByteOrder.LittleEndian), | |
| CompressedSize = BinaryFormat.ByteOrde |
Hugoberry
/ ZIPdissect.m
Last active
February 17, 2018 19:13
Dissecting ZIP file structure in Power Query M following explanation from https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (ZIPFile) => | |
| let | |
| //shorthand | |
| UInt32 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger32, | |
| ByteOrder.LittleEndian), | |
| UInt16 = BinaryFormat.ByteOrder(BinaryFormat.UnsignedInteger16, | |
| ByteOrder.LittleEndian), | |
| //Local File Header | |
| Header = BinaryFormat.Record([ | |
| Version = UInt16, |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| = List.Generate( | |
| ()=> [ | |
| can_expand=true, | |
| level =1, | |
| input =json, | |
| output =[level.0=Record.FieldNames(json){0}] | |
| ], | |
| each [can_expand], | |
| each [ | |
| can_expand =Record.FieldValues([input]){0} is record, |
Hugoberry
/ ExpandableColumns.m
Last active
February 23, 2017 17:55
Get a list of expandable columns from a table input in Power Query
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| expandableColumns = (_) => List.Accumulate( | |
| Table.ColumnNames(_), | |
| {}, | |
| (s,c)=>s&(if Type.Is(Value.Type(Record.Field(_{0},c)), type record) | |
| or Type.Is(Value.Type(Record.Field(_{0},c)), type list) | |
| then {c} | |
| else {} | |
| ) | |
| ), | |
| columnHasConsistentType = (T,Cname) => |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (json) => | |
| let | |
| //List the expandable columns | |
| expandableColumns = (_) => List.Accumulate( | |
| Table.ColumnNames(_), | |
| {}, | |
| (s,c)=>s&(if Type.Is(Value.Type(Record.Field(_{0},c)), type record) | |
| or Type.Is(Value.Type(Record.Field(_{0},c)), type list) | |
| then {c} | |
| else {}) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| reverse = (x) => | |
| Number.Mod(Number.BitwiseAnd((x* 0x0202020202),0x010884422010),1023), |
OlderNewer