Hugh Phoenix-Hulme HughPH
HughPH
/ jserv_hf_fast.py
Last active
September 9, 2021 21:23
— forked from abodacs/jserv_hf_fast.py
Run HuggingFace converted GPT-J-6B checkpoint using FastAPI and Ngrok on local GPU (3090 or Titan)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # So you want to run GPT-J-6B using HuggingFace+FastAPI on a local rig (3090 or TITAN) ... tricky. | |
| # special help from the Kolob Colab server https://colab.research.google.com/drive/1VFh5DOkCJjWIrQ6eB82lxGKKPgXmsO5D?usp=sharing#scrollTo=iCHgJvfL4alW | |
| # Conversion to HF format (12.6GB tar image) found at https://drive.google.com/u/0/uc?id=1NXP75l1Xa5s9K18yf3qLoZcR6p4Wced1&export=download | |
| # Uses GDOWN to get the image | |
| # You will need 26 GB of space, 12+GB for the tar and 12+GB expanded (you can nuke the tar after expansion) | |
| # HPPH: Not sure where you'll find this file, the links I found didn't work and the GDOWN was returning unauthorised errors. Maybe I'll make it a torrent. | |
| # HPPH: I also dumped the kobold endpoint. And added one for getting token counts so you can prune your prompt if necessary. | |
| # HPPH: And finally... Now the prompt goes in the POST body, which simplifies matters significantly. | |
| # Near Simplest Language model API, with room to expand! |
HughPH
/ gist:1315b6d10218aa810910f015749b601c
Created
August 30, 2018 18:29
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Callstack Function(desc) Line Bytes File Process Address | |
| --------------------------- ---- ----- ---- ------- ------- | |
| _IO_fwrite() ... + 0x3a3ba8e0 ? | |
| Register Info | |
| --------------------------- | |
| EDI: 0x00007F47440024E0 ESI: 0x0000000044000080 EAX: 0x0000000000000000 | |
| EBX: 0x0000000000000000 ECX: 0x00007F47440024C0 EDX: 0x0000000000000001 | |
| EIP: 0x00007F483A3BA8E0 EBP: 0x0000000044000080 SegCs: 0x0000000000000033 | |
| EFlags: 0x0000000000010246 ESP: 0x00007F465289F4A0 SegSs: 0x000000000000002B |
HughPH
/ gist:392a3c7d119a6c78a60899689154bb81
Created
August 27, 2018 21:37
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2604.421:0008:0031:trace:seh:NtRaiseException code=40010006 flags=0 addr=0x7b44abbc ip=7b44abbc tid=0031 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException info[0]=0000000000000016 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException info[1]=0000000001280f30 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException rax=0000000001280590 rbx=00007fffffea4000 rcx=0000000001280590 rdx=00000000012805b0 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException rsi=0000000001280690 rdi=00000000012805c0 rbp=0000000001280640 rsp=0000000001280570 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException r8=0000000000000002 r9=0000000001280680 r10=00000000ffffffe8 r11=0000000000000000 | |
| 2604.421:0008:0031:trace:seh:NtRaiseException r12=0000000000000000 r13=0000000000000001 r14=00007f3df201b3d0 r15=0000000000000001 | |
| 2604.421:0008:0031:trace:seh:dwarf_virtual_unwind function 7b44abbc base 0x7b44ab50 cie 0x7b6351d8 len 14 id 0 version 1 aug 'zR' code_align 1 data_align -8 retaddr %rip | |
| 2604.421:0008:0031:trace:seh:execute_cfa_instructions 7b44ab50: DW_CFA_def_cfa %r |
HughPH
/ gist:94701afc757803c72bb6c9cb03c26d1a
Created
August 26, 2018 20:12
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Computer Information: | |
| Manufacturer: Unknown | |
| Model: Unknown | |
| Form Factor: Laptop | |
| No Touch Input Detected | |
| Processor Information: | |
| CPU Vendor: GenuineIntel | |
| CPU Brand: Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz | |
| CPU Family: 0x6 |