From this tweet
Q: The fastest way to read a text file line by line
47Gb file -> 5hrs
Console.Write
each loop -> 45 mins to read the file (.NET 6)
1.File.ReadLines()
2.Write and read messages from a Dataflow block
3.Memory-mapped files
4.ref: HaveIBeenPwned Helper
5. StreamReader寫法
6. 同4.解法,有用到 System.IO.Pipelines
7. unsafe 解法
8. System.Threading.Channels 解法
9. bufferedstream
https://devblogs.microsoft.com/dotnet/system-io-pipelines-high-performance-io-in-net/
https://timiskhakov.github.io/posts/exploring-spans-and-pipelines
https://www.meziantou.net/split-a-string-into-lines-without-allocation.htm
From this tweet
Another optimization problem
https://github.com/jeremybytes/digit-display-csharp-comparison (check branches and PRs)
https://github.com/ReubenBond/digit-display-csharp-comparison/tree/reubenbond
How do I generate large files in the short time?