Skip to content

Instantly share code, notes, and snippets.

@rifkiamil
rifkiamil / Looker_Studio_Community_Connector_for_Speckle.md
Created May 11, 2025 19:06
Building a Looker Studio Community Connector for Speckle API

Let's look at the Speckle APIs and 3 methods getting data into Looker Studio.

  1. https://speckle.guide/dev/server-rest-api.html
  2. https://developers.google.com/looker-studio/connector
  3. https://developers.google.com/apps-script/reference/url-fetch/url-fetch-app

Speckle Service APIs: GraphQL vs. REST

Speckle provides both GraphQL and REST APIs to interact with your data on a Speckle Server. They serve similar purposes (accessing streams, commits, objects, etc.) but differ in how you structure your requests and receive data.

@rifkiamil
rifkiamil / Trino_JSON_Processing_gemini.md
Created May 5, 2025 15:13
Trino's Efficiency in Processing JSON Files in Data Lakes: A Technical Deep Dive

Trino's Efficiency in Processing JSON Files in Data Lakes: A Technical Deep Dive

Introduction

Trino, the distributed SQL query engine formerly known as PrestoSQL, is engineered for high-performance, interactive analytics across a multitude of heterogeneous data sources.1 Its architecture is particularly well-suited for querying large datasets residing in data lakes, whether deployed on-premises using HDFS or in the cloud on object storage systems like Amazon S3, Google Cloud Storage, or Azure Blob Storage.2 A key capability enabling this is Trino's schema-on-read approach, allowing users to query data in various formats directly where it resides, without requiring upfront transformation and loading into a proprietary storage system.3

JSON (JavaScript Object Notation) has become ubiquitous in modern data ecosystems, frequently used for API payloads, application logs, configuration files, and semi-structured data exchange.7 However, querying JSON efficiently at scale presents significant challen

@rifkiamil
rifkiamil / Trino_JSON_Processing_chatgpt.md
Created May 5, 2025 15:07
Trino JSON Processing in Data Lakes: Engine Internals, Comparisons, and SIMD Aspects

Trino JSON Processing in Data Lakes: Engine Internals, Comparisons, and SIMD Aspects

Engine Internals and JSON Processing Enhancements

Trino’s JSON architecture: Trino (formerly PrestoSQL) is a distributed MPP query engine where workers scan data in parallel and pipeline results in memory. JSON in a data lake (e.g. files on S3 or HDFS) is typically handled via the Hive connector, which treats JSON files as line-oriented text. Each JSON object (or array) is expected to be a record – often one JSON per line (NDJSON). Trino splits large JSON files into segments for parallel reading, aligning splits on record boundaries (usually newline delimited) so that no JSON object is cut in half between workers. This ensures each split contains whole JSON records for valid parsing. Internally, Trino uses a LinePageSource to read text files and find record boundaries (e.g. newline positions) so that each worker thread reads a chunk of the file and emits complete JSON rows. For extremely large JSON objects th

@rifkiamil
rifkiamil / Synapse_SQLDW_Table_Creation_from_Query.sql
Created April 11, 2025 16:39
Template for Azure Synapse Dedicated Pool Table Creation from Query with CLUSTERED COLUMNSTORE INDEX ORDER
-- Template for Azure Synapse Dedicated Pool Table Creation from Query with CLUSTERED COLUMNSTORE INDEX ORDER
-- Schema: testing_rif_k
-- Goal: Make easy to find the commands that only work on Azure Synapse Dedicated Pool ( SQL DW )
CREATE TABLE testing_rif_k.banking_data
WITH
(
DISTRIBUTION = HASH(AccountId), -- Recommended for even data distribution based on a key
CLUSTERED COLUMNSTORE INDEX ORDER (TransactionDate DESC) -- Good for analytical queries on date ranges
-- DISTRIBUTION = ROUND_ROBIN, -- Useful for small tables or when no clear distribution key exists
@rifkiamil
rifkiamil / Google-Play-Console-and-Firebase-SHA-1 fingerprints-andriod.md
Last active March 29, 2025 01:03
Google Sign-In with Google Play Console and Firebase - configuration problem related to SHA-1 fingerprints

Rrocess to ensure that both your debug and release builds (including those distributed via Google Play) are correctly recognized by Firebase for Google Sign-In. Option Reading Might want to read SHA1 key hell regarding Gmail authentication and sign-in.


Overview

  • Debug Builds:
    Your development builds are signed with the debug keystore (typically located in ~/.android/debug.keystore). The SHA-1 fingerprint from this keystore is already added to Firebase, which is why sign-in works in the emulator.

  • Release Builds & Google Play App Signing:

@rifkiamil
rifkiamil / football-withjoins.DBML
Last active October 27, 2021 22:43
Football Data Database Markup Language - with joins
Project project_name {
database_type: 'BigQuery'
Note: 'Football Data by https://github.com/dcaribou'
}
Table appearances {
player_id INTEGER
game_id INTEGER
appearance_id STRING [pk]
league_id STRING
@rifkiamil
rifkiamil / football-nojoins.DBML
Last active October 27, 2021 22:05
Football Data Database Markup Language - no joins
Project project_name {
database_type: 'BigQuery'
Note: 'Football data'
}
Table appearances {
player_id INTEGER
game_id INTEGER
appearance_id STRING [pk]
league_id STRING
@rifkiamil
rifkiamil / sql-eth-03.sql
Created February 23, 2021 07:11
Find Ethereum - Internal transactions - Matchs Advanced view on Etherscan.io
## Find Ethereum - Internal transactions - Matchs Advanced view on Etherscan.io
SELECT *
FROM `bigquery-public-data.crypto_ethereum.traces`
WHERE DATE(block_timestamp) = "2021-02-12"
and trace_address IS NOT NULL
and block_number = 11838934
ORDER BY transaction_index
@rifkiamil
rifkiamil / sql-eth-02.sql
Created February 23, 2021 06:54
Find Ethereum - Internal transactions with values - Matchs default view on Etherscan.io
## Find Ethereum - Internal transactions with values - Matchs default view on Etherscan.io
SELECT *
FROM `bigquery-public-data.crypto_ethereum.traces`
WHERE DATE(block_timestamp) = "2021-02-12"
and trace_address IS NOT NULL
and block_number = 11838934
and value > 0
ORDER BY transaction_index
@rifkiamil
rifkiamil / sql-eth-01.sql
Created February 19, 2021 12:36
Top 20 balances on Ethereum with correct decimal place
## Below is the query that will give you top 20 balances on Ethereum
SELECT Address as EthAddress,
CAST(eth_balance as NUMERIC)/1000000000000000000 as EthBalance
FROM `bigquery-public-data.crypto_ethereum.balances`
ORDER BY Eth_balance DESC
LIMIT 20