Tibor Teebor-Choka

On-premise LLM model deployments

LLM models used as an external resource offer multiple disadvantages:

Privacy - guarantees regarding confidential imformation usage by the models and intellectual property (IP) safety cannot be guaranteed
Flexibility - specialization and fine-tuning of more general models offers efficiency gains in terms of resources used and outputs achieved
Stability - reducing vendor dependency allows internal system stability through guaranteed availability

An approach, whereby LLM models and deployed, fine-tuned and utilized locally (on-premise) allows:

(using relationship - "uses")

A - B or A -> B (A is associated with B)

Tricks:

compilers do a lot less in Debug, use it in development
- minimze optimizations
remove unreachable or dead code
- cppcheck, Coverity, Understand, CppDepend, OCLint, gcov...
unused sources, libraries
commonly used sources => static libraries
use granular includes

What to consider before starting a project:

Interesting language features:

+ before argument-less lambda forces cast to a pointer to function: https://stackoverflow.com/a/18889029
meta: decltype(..., void()) and void_t are the same
operator overloading: https://stackoverflow.com/q/4421706

	#!/usr/bin/env python3

	from json import dumps
	import os

	from httplib2 import Http


	def main():
	url = os.environ["GOOGLE_CHAT_URL"]

	CREATE ROLE test_ro_access;
	GRANT CONNECT ON DATABASE test TO test_ro_access;
	GRANT USAGE ON SCHEMA public TO test_ro_access;
	GRANT SELECT ON ALL TABLES IN SCHEMA public TO test_ro_access;

	CREATE USER test_read_user WITH PASSWORD 'passw0rd';
	GRANT test_ro_access TO test_read_user;

	/**
	* Sentinels: http://stackoverflow.com/a/5384593
	*/

	template<typename T>
	decltype(auto) find_without_sentinel(const T& c)
	{
	for (iterator i=c.begin(); i!=c.end(); ++i) // first branch here
	{
	if (*i == x) // second branch here