Use latest prebuilt toolchain: https://goo.gl/QpX5zM (the one at the top)
New build step has been added, after vita-elf-create you need to run: vita-make-fself homebrew.velf eboot.bin
You need to run vita-make-fself on the .velf, not .elf The resulting eboot.bin can then be put into a .vpk archive, read below
Homebrew installer uses a .vpk format which is just a ZIP file, start from this template: https://github.com/xyzz/Vita_Doom/releases/download/1.0/vitadoom.vpk (try installing it from the shell)
- replace eboot.bin with your eboot.bin
- you can also add template.xml and stuff like icon/background/etc that will be displayed in livearea, check out how it's done in molecular shell (launch it and go to app0)
- make sure to run pngquant on all your png images https://pngquant.org/
- to make param.sfo, use
vita-mksfoex -s TITLE_ID=XXXX00001 "homebrew name" output/param.sfo