Skip to content

Instantly share code, notes, and snippets.

@rferraton

rferraton/FastTransfer PG2PG

Last active March 17, 2026 08:11
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save rferraton/8db4d697c6b1bc18c6e62713b9972767 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save rferraton/8db4d697c6b1bc18c6e62713b9972767 to your computer and use it in GitHub Desktop.
Download ZIP
FastTransfer log for a parallel transfer from pg15 to pg16 using ctid method and degre -2 (half cpu => 32/2=16 here)
Raw
FastTransfer PG2PG
(base) PS D:\FastTransfer\latest> .\FastTransfer.exe `
>> --sourceconnectiontype pgcopy `
>> --sourceserver "localhost:5432" `
>> --sourcedatabase "tpch" `
>> --sourceuser "FastUser" `
>> --sourcepassword "FastPassword" `
>> --sourceschema "tpch_10" `
>> --sourcetable "orders" `
>> --targetconnectiontype pgcopy `
>> --targetserver "localhost:15432" `
>> --targetdatabase "tpch" `
>> --targetuser "FastUser" `
>> --targetpassword "FastPassword" `
>> --targetschema "tpch_10" `
>> --targettable "orders" `
>> --loadmode Truncate `
>> --mapmethod Name `
>> --method Ctid `
>> --degree -2
╔════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║ ║
║ ███████ █████ ███████ ████████ ████████ ██████ █████ ███ ██ ███████ ███████ ███████ ██████ ║
║ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ████ ██ ██ ██ ██ ██ ██ ║
║ █████ ███████ ███████ ██ ██ ██████ ███████ ██ ██ ██ ███████ █████ █████ ██████ ║
║ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ██ ║
║ ██ ██ ██ ███████ ██ ██ ██ ██ ██ ██ ██ ████ ███████ ██ ███████ ██ ██ ║
║ ║
║ High-Performance Transfer and Import Data Tool ║
║ ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║ Version: 0.16.1.0 License: Commercial ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║ ║
║ DB Data Sources: 12 File Datasources: 5 Targets: 8 ║
║ ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║ Data Sources ║
║ - SQL Server - PostgreSQL - MySQL - Oracle ║
║ - Netezza - ClickHouse - SAP HANA - Teradata ║
║ - ODBC - OleDB - DuckDB - and more... ║
║ ║
║ Input File Formats (using DuckDBStream connection type) ║
║ - CSV - TSV - JSON - BSON ║
║ - Parquet - Excel (XLSX) ║
║ ║
║ Targets ║
║ - SQL Server - PostgreSQL - MySQL - Oracle ║
║ - Netezza - ClickHouse - SAP HANA - Teradata ║
║ - DuckDB ║
║ ║
╚════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝
2026-03-17T00:21:04.391 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- The "FastTransfer_Settings.json" file does not exist. Using default settings. Console Only with loglevel=Information
2026-03-17T00:21:04.403 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- FastTransfer - running in trial mode – trial mode will end on 2026‑03‑25 - normal licensed mode will then start (8 day(s) left).
2026-03-17T00:21:04.404 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Starting
2026-03-17T00:21:04.406 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- FastTransfer Version : "0.16.1.0" Architecture : "X64" - Framework : ".NET 8.0.24"
2026-03-17T00:21:04.407 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- OS : "Microsoft Windows 10.0.26200"
2026-03-17T00:21:04.408 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Process ID : 37764
2026-03-17T00:21:04.409 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Generated Run ID : "b99f5336-0e06-45a8-9135-6f0319c971cc"
2026-03-17T00:21:04.409 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Connection Type : "pgcopy"
2026-03-17T00:21:04.410 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source SqlInstance : "localhost:5432"
2026-03-17T00:21:04.411 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Database : "tpch"
2026-03-17T00:21:04.412 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Trusted Connection : False
2026-03-17T00:21:04.413 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source User : "FastUser"
2026-03-17T00:21:04.413 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Schema : "tpch_10"
2026-03-17T00:21:04.414 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Table : "orders"
2026-03-17T00:21:04.415 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Type : "pgcopy"
2026-03-17T00:21:04.416 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Server : "localhost:15432"
2026-03-17T00:21:04.416 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Database : "tpch"
2026-03-17T00:21:04.417 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Schema : "tpch_10"
2026-03-17T00:21:04.418 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Table : "orders"
2026-03-17T00:21:04.419 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Trusted Connection : False
2026-03-17T00:21:04.420 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target User : "FastUser"
2026-03-17T00:21:04.421 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Columns Map Method : "Name"
2026-03-17T00:21:04.422 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Degree : -2
2026-03-17T00:21:04.423 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Distribute Method : "Ctid"
2026-03-17T00:21:04.423 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Bulkcopy Batch Size : 1048576
2026-03-17T00:21:04.424 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Load Mode : "Truncate"
2026-03-17T00:21:04.425 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Use Work Tables : False
2026-03-17T00:21:04.429 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Encoding used : "Unicode (UTF-8) - 65001 - utf-8"
2026-03-17T00:21:04.432 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Process Priority set to High
2026-03-17T00:21:04.593 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Connection String : "Host=localhost;Port=5432;Database=tpch;Trust Server Certificate=True;Application Name=FastTransfer;Timeout=15;Command Timeout=10800;Username=FastUser;Password=xxxxx"
2026-03-17T00:21:04.596 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Connection String : "Host=localhost;Port=15432;Database=tpch;Trust Server Certificate=True;Application Name=FastTransfer;Timeout=15;Command Timeout=10800;Username=FastUser;Password=xxxxx"
2026-03-17T00:21:04.596 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Source Database Version : "PostgreSQL 15.1, compiled by Visual C++ build 1914, 64-bit"
2026-03-17T00:21:04.598 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Target Database Version : "PostgreSQL 16.1, compiled by Visual C++ build 1937, 64-bit"
2026-03-17T00:21:04.682 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Degree of parallelism was computed to 16 (=> 32\2)
2026-03-17T00:21:04.695 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Snapshot ID = "00000006-000000C7-1" Created in 9 ms
2026-03-17T00:21:05.023 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed CTID Max Retrieval in 0 ms
2026-03-17T00:21:05.025 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Max CTID for table "\"tpch_10\""."\"orders\"" @snapshotid="00000006-000000C7-1" = "(275681,42)"
2026-03-17T00:21:05.038 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.205 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.207 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.215 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.215 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.225 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.226 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.228 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.244 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.247 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.257 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.257 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.261 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.261 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.263 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:05.264 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Start Loading Data using distribution method Ctid
2026-03-17T00:21:09.560 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 3 for ctid between 34460 and 51690 : 140113130 Bytes in 4525ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:09.823 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 10 for ctid between 155070 and 172300 : 140121348 Bytes in 4789ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.009 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 6 for ctid between 86150 and 103380 : 140120613 Bytes in 4974ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.021 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 4 for ctid between 51690 and 68920 : 140111882 Bytes in 4987ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.042 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 15 for ctid between 241220 and 258450 : 140114384 Bytes in 5008ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.081 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 16 for ctid between 258450 and 2147483647 : 140133290 Bytes in 5047ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.128 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 14 for ctid between 223990 and 241220 : 140123391 Bytes in 5093ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.143 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 1 for ctid between 0 and 17230 : 140142663 Bytes in 5109ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.162 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 13 for ctid between 206760 and 223990 : 140086424 Bytes in 5128ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.169 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 11 for ctid between 172300 and 189530 : 140100830 Bytes in 5135ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.174 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 8 for ctid between 120610 and 137840 : 140087224 Bytes in 5139ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.246 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 5 for ctid between 68920 and 86150 : 140112939 Bytes in 5211ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.249 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 12 for ctid between 189530 and 206760 : 140090533 Bytes in 5215ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.255 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 7 for ctid between 103380 and 120610 : 140103694 Bytes in 5220ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.338 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 9 for ctid between 137840 and 155070 : 140105282 Bytes in 5303ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.445 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load Query 2 for ctid between 17230 and 34460 : 140123798 Bytes in 5411ms with snapshot "00000006-000000C7-1"
2026-03-17T00:21:10.450 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Total size : 2241791425 Bytes
2026-03-17T00:21:10.451 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Throughput : 388653767 Bytes/s
2026-03-17T00:21:10.452 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Transfert time : Elapsed=5770 ms
2026-03-17T00:21:10.453 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Total time : Elapsed=6148 ms
2026-03-17T00:21:10.454 +01:00 -|- FastTransfer -|- b99f5336-0e06-45a8-9135-6f0319c971cc -|- INFORMATION -|- tpch.tpch_10.orders -|- Completed Load
@rferraton

rferraton commented Mar 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Author
  • orders SF10 = 15Mrows x 9 columns
CREATE UNLOGGED TABLE tpch_10.orders (
	o_orderkey serial4 NOT NULL,
	o_custkey int8 NOT NULL,
	o_orderstatus bpchar(1) NULL,
	o_totalprice numeric NULL,
	o_orderdate date NULL,
	o_orderpriority bpchar(15) NULL,
	o_clerk bpchar(15) NULL,
	o_shippriority int4 NULL,
	o_comment varchar(79) NULL,
	CONSTRAINT orders_pkey PRIMARY KEY (o_orderkey)
);
  • Nota :
    • for a logged heap table : timing is 11s with the same machine
    • for a unlogged heap table : 3.3s with the same machine

@rferraton

rferraton commented Mar 16, 2026

Copy link
Copy Markdown
Author

Machine Configuration :

  • MSI Vector gp68 HX 13v
  • Processor 13th Gen Intel(R) Core(TM) i9-13950HX (2.20 GHz)
  • Memory RAM 64.0 Go (63.7 Go utilisable)
  • Windows 11

@rferraton

rferraton commented Mar 16, 2026

Copy link
Copy Markdown
Author

Targeting a Heap Unlogged : 3.6s

CREATE UNLOGGED TABLE tpch_10.orders_heap (
	o_orderkey serial4 NOT NULL,
	o_custkey int8 NOT NULL,
	o_orderstatus bpchar(1) NULL,
	o_totalprice numeric NULL,
	o_orderdate date NULL,
	o_orderpriority bpchar(15) NULL,
	o_clerk bpchar(15) NULL,
	o_shippriority int4 NULL,
	o_comment varchar(79) NULL
)
WITH (
	parallel_workers=6
);
(base) PS D:\FastTransfer\latest> .\FastTransfer.exe `
>>   --sourceconnectiontype pgcopy `
>>   --sourceserver "localhost:5432" `
>>   --sourcedatabase "tpch" `
>>   --sourceuser "FastUser" `
>>   --sourcepassword "FastPassword" `
>>   --sourceschema "tpch_10" `
>>   --sourcetable "orders" `
>>   --targetconnectiontype pgcopy `
>>   --targetserver "localhost:15432" `
>>   --targetdatabase "tpch" `
>>   --targetuser "FastUser" `
>>   --targetpassword "FastPassword" `
>>   --targetschema "tpch_10" `
>>   --targettable "orders_heap" `
>>   --loadmode Truncate `
>>   --mapmethod Name `
>>   --method Ctid `
>>   --degree -2

╔════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║                                                                                                                ║
║      ███████  █████  ███████ ████████  ████████ ██████   █████  ███    ██ ███████ ███████ ███████ ██████       ║
║      ██      ██   ██ ██         ██        ██    ██   ██ ██   ██ ████   ██ ██      ██      ██      ██   ██      ║
║      █████   ███████ ███████    ██        ██    ██████  ███████ ██ ██  ██ ███████ █████   █████   ██████       ║
║      ██      ██   ██      ██    ██        ██    ██   ██ ██   ██ ██  ██ ██      ██ ██      ██      ██   ██      ║
║      ██      ██   ██ ███████    ██        ██    ██   ██ ██   ██ ██   ████ ███████ ██      ███████ ██   ██      ║
║                                                                                                                ║
║                               High-Performance Transfer and Import Data Tool                                   ║
║                                                                                                                ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║  Version: 0.16.1.0             License: Commercial                                                             ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║                                                                                                                ║
║           DB Data Sources: 12               File Datasources: 5               Targets:  8                      ║
║                                                                                                                ║
╠════════════════════════════════════════════════════════════════════════════════════════════════════════════════╣
║  Data Sources                                                                                                  ║
║          - SQL Server           - PostgreSQL           - MySQL                - Oracle                         ║
║          - Netezza              - ClickHouse           - SAP HANA             - Teradata                       ║
║          - ODBC                 - OleDB                - DuckDB               - and more...                    ║
║                                                                                                                ║
║  Input File Formats (using DuckDBStream connection type)                                                       ║
║          - CSV                  - TSV                  - JSON                 - BSON                           ║
║          - Parquet              - Excel (XLSX)                                                                 ║
║                                                                                                                ║
║  Targets                                                                                                       ║
║          - SQL Server           - PostgreSQL           - MySQL                - Oracle                         ║
║          - Netezza              - ClickHouse           - SAP HANA             - Teradata                       ║
║          - DuckDB                                                                                              ║
║                                                                                                                ║
╚════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝

2026-03-17T00:37:46.549 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- The "FastTransfer_Settings.json" file does not exist. Using default settings. Console Only with loglevel=Information
2026-03-17T00:37:46.561 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- FastTransfer - running in trial mode – trial mode will end on 2026‑03‑25 - normal licensed mode will then start (8 day(s) left).
2026-03-17T00:37:46.563 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Starting
2026-03-17T00:37:46.564 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- FastTransfer Version : "0.16.1.0" Architecture : "X64" - Framework : ".NET 8.0.24"
2026-03-17T00:37:46.565 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- OS : "Microsoft Windows 10.0.26200"
2026-03-17T00:37:46.566 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Process ID : 11572
2026-03-17T00:37:46.567 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Generated Run ID : "ad43f4ea-b868-4f94-ae32-c369a5fdfa22"
2026-03-17T00:37:46.567 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Connection Type : "pgcopy"
2026-03-17T00:37:46.568 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source SqlInstance : "localhost:5432"
2026-03-17T00:37:46.569 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Database : "tpch"
2026-03-17T00:37:46.571 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Trusted Connection : False
2026-03-17T00:37:46.572 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source User : "FastUser"
2026-03-17T00:37:46.573 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Schema : "tpch_10"
2026-03-17T00:37:46.573 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Table : "orders"
2026-03-17T00:37:46.574 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Type : "pgcopy"
2026-03-17T00:37:46.575 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Server : "localhost:15432"
2026-03-17T00:37:46.576 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Database : "tpch"
2026-03-17T00:37:46.577 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Schema : "tpch_10"
2026-03-17T00:37:46.577 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Table : "orders_heap"
2026-03-17T00:37:46.578 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Trusted Connection : False
2026-03-17T00:37:46.579 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target User : "FastUser"
2026-03-17T00:37:46.580 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Columns Map Method : "Name"
2026-03-17T00:37:46.581 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Degree : -2
2026-03-17T00:37:46.581 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Distribute Method : "Ctid"
2026-03-17T00:37:46.582 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Bulkcopy Batch Size : 1048576
2026-03-17T00:37:46.583 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Load Mode : "Truncate"
2026-03-17T00:37:46.584 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Use Work Tables : False
2026-03-17T00:37:46.588 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Encoding used : "Unicode (UTF-8) - 65001 - utf-8"
2026-03-17T00:37:46.591 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Process Priority set to High
2026-03-17T00:37:46.757 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Connection String : "Host=localhost;Port=5432;Database=tpch;Trust Server Certificate=True;Application Name=FastTransfer;Timeout=15;Command Timeout=10800;Username=FastUser;Password=xxxxx"
2026-03-17T00:37:46.760 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Connection String : "Host=localhost;Port=15432;Database=tpch;Trust Server Certificate=True;Application Name=FastTransfer;Timeout=15;Command Timeout=10800;Username=FastUser;Password=xxxxx"
2026-03-17T00:37:46.761 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Source Database Version : "PostgreSQL 15.1, compiled by Visual C++ build 1914, 64-bit"
2026-03-17T00:37:46.761 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Target Database Version : "PostgreSQL 16.1, compiled by Visual C++ build 1937, 64-bit"
2026-03-17T00:37:46.801 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Degree of parallelism was computed to 16 (=> 32\2)
2026-03-17T00:37:46.810 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Snapshot ID = "00000006-000000F6-1" Created in 4 ms
2026-03-17T00:37:47.149 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed CTID Max Retrieval in 0 ms
2026-03-17T00:37:47.151 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Max CTID for table "\"tpch_10\""."\"orders\"" @snapshotid="00000006-000000F6-1" = "(275681,42)"
2026-03-17T00:37:47.161 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.308 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.325 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.347 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.355 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.355 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.367 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.373 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.373 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.378 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.385 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.386 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.387 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.388 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.389 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:47.390 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Start Loading Data using distribution method Ctid
2026-03-17T00:37:49.288 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 13 for ctid between 206760 and 223990 : 140086424 Bytes in 2128ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.667 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 2 for ctid between 17230 and 34460 : 140123798 Bytes in 2506ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.686 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 11 for ctid between 172300 and 189530 : 140100830 Bytes in 2526ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.698 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 4 for ctid between 51690 and 68920 : 140111882 Bytes in 2538ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.716 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 9 for ctid between 137840 and 155070 : 140105282 Bytes in 2556ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.729 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 12 for ctid between 189530 and 206760 : 140090533 Bytes in 2569ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.741 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 14 for ctid between 223990 and 241220 : 140123391 Bytes in 2581ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.748 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 15 for ctid between 241220 and 258450 : 140114384 Bytes in 2588ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.766 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 10 for ctid between 155070 and 172300 : 140121348 Bytes in 2606ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.784 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 3 for ctid between 34460 and 51690 : 140113130 Bytes in 2624ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.785 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 6 for ctid between 86150 and 103380 : 140120613 Bytes in 2624ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.793 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 5 for ctid between 68920 and 86150 : 140112939 Bytes in 2633ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.795 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 7 for ctid between 103380 and 120610 : 140103694 Bytes in 2635ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.802 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 1 for ctid between 0 and 17230 : 140142663 Bytes in 2641ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.803 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 16 for ctid between 258450 and 2147483647 : 140133290 Bytes in 2643ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.813 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load Query 8 for ctid between 120610 and 137840 : 140087224 Bytes in 2653ms with snapshot "00000006-000000F6-1"
2026-03-17T00:37:49.818 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Total size : 2241791425 Bytes
2026-03-17T00:37:49.819 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Throughput : 743167346 Bytes/s
2026-03-17T00:37:49.821 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Transfert time : Elapsed=3019 ms
2026-03-17T00:37:49.822 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Total time : Elapsed=3361 ms
2026-03-17T00:37:49.823 +01:00 -|- FastTransfer -|- ad43f4ea-b868-4f94-ae32-c369a5fdfa22 -|- INFORMATION -|- tpch.tpch_10.orders_heap -|- Completed Load
(base) PS D:\FastTransfer\latest>

@rferraton

rferraton commented Mar 16, 2026

Copy link
Copy Markdown
Author

Data available as pgdump here : https://aetpshared.s3.eu-west-1.amazonaws.com/data/tpch/sf10/pgdump/orders/dump-orders-tpch-202603170051.zip

@rferraton

rferraton commented Mar 17, 2026

Copy link
Copy Markdown
Author

Compare with fdw (pull from fdw) : 2min 43s

image

@rferraton

rferraton commented Mar 17, 2026

Copy link
Copy Markdown
Author 
create schema foreign_tpch;
	
	
	CREATE FOREIGN TABLE IF NOT EXISTS foreign_tpch.orders(
    o_orderkey integer OPTIONS (column_name 'o_orderkey') NOT NULL,
    o_custkey bigint OPTIONS (column_name 'o_custkey') NOT NULL,
    o_orderstatus character(1) OPTIONS (column_name 'o_orderstatus') COLLATE pg_catalog."default",
    o_totalprice numeric OPTIONS (column_name 'o_totalprice'),
    o_orderdate date OPTIONS (column_name 'o_orderdate'),
    o_orderpriority character(15) OPTIONS (column_name 'o_orderpriority') COLLATE pg_catalog."default",
    o_clerk character(15) OPTIONS (column_name 'o_clerk') COLLATE pg_catalog."default",
    o_shippriority integer OPTIONS (column_name 'o_shippriority'),
    o_comment character varying(79) OPTIONS (column_name 'o_comment') COLLATE pg_catalog."default"
)
    SERVER tpch_src
    OPTIONS (schema_name 'tpch_10', table_name 'orders', async_capable 'true', batch_size '10000');

	ALTER FOREIGN TABLE  foreign_tpch.orders
	OPTIONS (fetch_size '10000');

ALTER FOREIGN TABLE foreign_tpch.orders
    OWNER TO postgres;

CREATE ROLE "PG_FDW_ROLE";

GRANT "PG_FDW_ROLE" TO "postgres" WITH ADMIN OPTION;

CREATE USER MAPPING FOR "PG_FDW_ROLE"
SERVER tpch_src
OPTIONS (user 'FastUser', password 'FastPassword');

CREATE USER MAPPING FOR "postgres"
SERVER tpch_src
OPTIONS (user 'FastUser', password 'FastPassword');

SELECT count(*) from foreign_tpch.orders;

INSERT into tpch10.orders
SELECT * from foreign_tpch.orders;

@rferraton

rferraton commented Mar 17, 2026

Copy link
Copy Markdown
Author

more detail on FastTransfer here https://fasttransfer.arpe.io

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment