prasanthj

Enable native memory tracking in JVM by specifying the following flag

-XX:NativeMemoryTracking=detail

Know the <PID> of the java process

jps

To print ps based RSS

ps -p <PID> -o pcpu,rss,size,vsize

To print native memory tracking summary

yurishkuro

• What do Etcd, Consul, and Zookeeper do?
  • Service Registration:
    • Host, port number, and sometimes authentication credentials, protocols, versions numbers, and/or environment details.
  • Service Discovery:
    • Ability for client application to query the central registry to learn of service location.
  • Consistent and durable general-purpose K/V store across distributed system.
    • Some solutions support this better than others.
    • Based on Paxos or some derivative (i.e. Raft) algorithm to quickly converge to a consistent state.
• Centralized locking can be based on this K/V store.

akostadinov

@Grapes([
  @Grab(group='org.gperfutils', module='gbench', version='[0.4,)'),
  @Grab(group='org.apache.commons', module='commons-lang3', version='[3.7,)'),
  @Grab(group='joda-time', module='joda-time', version='[2.9.9,)')
])

/**
 * Java 8 DateTimeFormatter
 */
import java.util.Date;

Artefact2

Which GGUF is right for me? (Opinionated)

Good question! I am collecting human data on how quantization affects outputs. See here for more information: ggml-org/llama.cpp#5962

In the meantime, use the largest that fully fits in your GPU. If you can comfortably fit Q4_K_S, try using a model with more parameters.

llama.cpp feature matrix

See the wiki upstream: https://github.com/ggerganov/llama.cpp/wiki/Feature-matrix