filmo

q_learning walkthrough. GATech OMSCS. (School policy prohibits our publishing code for honor policy)

filmo

Used a 2-layer Fully Connected network with H1=100, H2=60 and ReLU

He Initialization of weights
Adam Optimizer. Initial learning rate = 0.001 Learning rate reduced using gamma of 0.50 every 350 episodes

Gamma = 0.99
Eps   = 1.00
Eps Decay = 0.98
Eps Decay every new episode. (not each step)

filmo

import numpy as np
np.set_printoptions(precision=4,suppress=True,linewidth=180)

def clamp(n, min, max):
    if n < min:
        return min
    if n > max:
        return max
    return n

filmo

On line 35 shuffle is imported from sklearn

from sklearn.utils import shuffle

and then later called inside the data generator with:

shuffle(keys)

However, shuffle from sklearn does ~not~ shuffle in-place. It returns a shuffled list, leaving the input parameter untouched.

filmo

# llm is a valid OpenAI|Ollama object
# Settings.embed_model is a valid "nomic-embed-text" embedding model producing 768-d vectors

# This code works and 'default_vector_store.json' contains the embeddings 
documents = SimpleDirectoryReader(
	"data/DIR_WORD_DOCs",
	filename_as_id=False,
	file_extractor={'.docx':DocxReader()}
).load_data(num_workers=num_workers)

filmo

# set of actions for which a system would require additional confirmation from the user
ACTIONS_REQUIRING_CONFIRMATION = [
    "saving a file to disk",
    "executing a file",
    "stopping a computer process",
]


# This function takes the original user input and attempts to determine if any actions are being requested
# that require human confirmation. This is more flexible the searching for a string literal.