Skip to content

Instantly share code, notes, and snippets.

@sooop
Created July 20, 2018 05:09
Show Gist options
  • Save sooop/0280d620cad8d91d3bf533785f08eb65 to your computer and use it in GitHub Desktop.
Save sooop/0280d620cad8d91d3bf533785f08eb65 to your computer and use it in GitHub Desktop.
Read a file line by line in reversed order (lazily) in python3
def reversed_lines(filepath, sep='\n'):
token = sep.encode()
buffer = bytearray()
chunk_size = 1024
c = 0
with open(filepath, 'rb') as f:
total_bytes = f.seek(0,2)
# 버퍼에 한 번을 읽어들인다.
while True:
c += 1
pos = total_bytes - (chunk_size * c)
# 시작위치가 0보다 작은 경우는 문제 없는데
# chunk_size * -1 보다 작으면 더 읽을 수 없다.
if pos <= -chunk_size:
break
f.seek(pos if pos > 0 else 0, 0)
buffer[:0] = f.read(chunk_size if pos >= 0 else chunk_size + pos)
# 버퍼에서 구분자까지를 뒤에서부터 찾는다.
while True:
i = buffer.rfind(token)
if i < 0:
break
result = buffer[i+len(token):].decode()
buffer[i:] = []
yield(result)
if buffer:
yield(buffer.decode())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment