Skip to content

Instantly share code, notes, and snippets.

@Weiyuan-Lane
Created February 15, 2021 18:41
Show Gist options
  • Save Weiyuan-Lane/2e1a8a29624849a9b9aa8f3f55705a8f to your computer and use it in GitHub Desktop.
Save Weiyuan-Lane/2e1a8a29624849a9b9aa8f3f55705a8f to your computer and use it in GitHub Desktop.
Comparison of file writing strategies
Write all lines to file at once, mode "w"
1.651B memsize ( 0.000 retained)
75.008k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Write all lines to file at once, mode "a"
1.651B memsize ( 40.000 retained)
75.008k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Write all lines to file at once, mode "w+"
1.651B memsize ( 40.000 retained)
75.008k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Write all lines to file at once, mode "a+"
1.651B memsize ( 40.000 retained)
75.008k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file, mode "w"
2.009M memsize ( 0.000 retained)
50.007k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file, mode "a"
2.009M memsize ( 0.000 retained)
50.007k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file, mode "w+"
2.009M memsize ( 0.000 retained)
50.007k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file, mode "a+"
2.009M memsize ( 0.000 retained)
50.007k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file (manual), mode "w"
2.009M memsize ( 0.000 retained)
50.007k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file (manual), mode "a"
2.009M memsize ( 40.000 retained)
50.007k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file (manual), mode "w+"
2.009M memsize ( 40.000 retained)
50.007k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file (manual), mode "a+"
2.009M memsize ( 40.000 retained)
50.007k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks, mode "w"
2.870M memsize ( 0.000 retained)
50.703k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks, mode "a"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks, mode "w+"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks, mode "a+"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks (manual), mode "w"
2.870M memsize ( 0.000 retained)
50.703k objects ( 0.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks (manual), mode "a"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks (manual), mode "w+"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Iterate and write per line to file chunks (manual), mode "a+"
2.870M memsize ( 40.000 retained)
50.703k objects ( 1.000 retained)
50.000 strings ( 0.000 retained)
Comparison:
Iterate and write per line to file (manual), mode "w+": 2008696 allocated
Iterate and write per line to file (manual), mode "a+": 2008696 allocated - same
Iterate and write per line to file (manual), mode "a": 2008696 allocated - same
Iterate and write per line to file (manual), mode "w": 2008696 allocated - same
Iterate and write per line to file, mode "w": 2008696 allocated - same
Iterate and write per line to file, mode "a": 2008696 allocated - same
Iterate and write per line to file, mode "w+": 2008696 allocated - same
Iterate and write per line to file, mode "a+": 2008696 allocated - same
Iterate and write per line to file chunks (manual), mode "w": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks (manual), mode "a": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks, mode "a+": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks (manual), mode "w+": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks, mode "w": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks, mode "a": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks, mode "w+": 2869720 allocated - 1.43x more
Iterate and write per line to file chunks (manual), mode "a+": 2869720 allocated - 1.43x more
Write all lines to file at once, mode "a+": 1650858087 allocated - 821.86x more
Write all lines to file at once, mode "w+": 1650858087 allocated - 821.86x more
Write all lines to file at once, mode "a": 1650858087 allocated - 821.86x more
Write all lines to file at once, mode "w": 1650858087 allocated - 821.86x more
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment