Skip to content

Instantly share code, notes, and snippets.

View Cdaprod's full-sized avatar
🫠
Learn something new everyday!

David Cdaprod

🫠
Learn something new everyday!
View GitHub Profile
from langchain.chains.openai_functions import create_structured_output_runnable
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.pydantic_v1 import BaseModel, Field
class Insight(BaseModel):
insight: str = Field(description="""insight""")
chat_model = ChatOpenAI(model_name="gpt-4-1106-preview")
@mberman84
mberman84 / gist:ea207e7d9e5f8c5f6a3252883ef16df3
Created November 29, 2023 15:31
AutoGen + Ollama Instructions
1. # create new .py file with code found below
2. # install ollama
3. # install model you want “ollama run mistral”
4. conda create -n autogen python=3.11
5. conda activate autogen
6. which python
7. python -m pip install pyautogen
7. ollama run mistral
8. ollama run codellama
9. # open new terminal
from langchain.chat_models import ChatOpenAI
from langchain.prompts import ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
import requests
from bs4 import BeautifulSoup
from langchain.schema.runnable import RunnablePassthrough, RunnableLambda
from langchain.utilities import DuckDuckGoSearchAPIWrapper
import json
RESULTS_PER_QUESTION = 3
@Cdaprod
Cdaprod / storage_transformer.md
Last active October 20, 2023 01:07
This Python script is designed to perform several operations related to text and data processing using Minio for object storage and the Transformers library for text tokenization. The script is organized into various functions, each responsible for a specific task.

Modified Python Script for Text and Data Processing with Minio and Transformers

This Python script is designed to perform several operations related to text and data processing using Minio for object storage and the Transformers library for text tokenization. The script is organized into various functions, each responsible for a specific task.

Functions:

  1. init_logger(log_level: str): Initializes the logging mechanism with a given log level.

  2. check_env(): Checks for the existence of required environment variables for Minio client setup.

@Cdaprod
Cdaprod / write_functions_to_files.py
Last active October 1, 2023 03:08
Prototype - write_functions_to_files() for populating code as a small object `code`. Attempting a RAG system as a personal code base by parsing personal Python documentation. This will be utilized in an automated CD platform for deploying code base using `functions_code` embeddings
import os
import ast
from pydantic import BaseModel, Field
from typing import List
# Pydantic model for function code blocks
class FunctionCodeBlock(BaseModel):
code: str = Field(..., description="The Python code block")
class Config:
@Cdaprod
Cdaprod / yt-babyagi.ipynb
Created August 18, 2023 19:51
yt-babyagi.ipynb
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Cdaprod
Cdaprod / recon_data_to_aws_dynamodb.ipynb
Last active August 1, 2023 17:57
This Jupyter Notebook provides an interactive tutorial on how to ingest reconnaissance data into AWS DynamoDB. Tailored for cybersecurity enthusiasts, researchers, and professionals, it offers a systematic guide to seamlessly parse recon tool outputs and store them efficiently on AWS. Whether you're new to data storage on the cloud or a seasoned…
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
System: Respond to the human as helpfully and accurately as possible. You have access to the following tools:
click_element: Click on an element with the given CSS selector, args: {{'selector': {{'title': 'Selector', 'description': 'CSS selector for the element to click', 'type': 'string'}}}}
navigate_browser: Navigate a browser to the specified URL, args: {{'url': {{'title': 'Url', 'description': 'url to navigate to', 'type': 'string'}}}}
previous_webpage: Navigate back to the previous page in the browser history, args: {{}}
extract_text: Extract all the text on the current webpage, args: {{}}
extract_hyperlinks: Extract all hyperlinks on the current webpage, args: {{'absolute_urls': {{'title': 'Absolute Urls', 'description': 'Return absolute URLs instead of relative URLs', 'default': False, 'type': 'boolean'}}}}
get_elements: Retrieve elements in the current web page matching the given CSS selector, args: {{'selector': {{'title': 'Selector', 'description': "CSS selector, such as '*', 'div', 'p', 'a', #id,
# STEP 1: Load
# Load documents using LangChain's DocumentLoaders
# This is from https://langchain.readthedocs.io/en/latest/modules/document_loaders/examples/csv.html
from langchain.document_loaders.csv_loader import CSVLoader
loader = CSVLoader(file_path='./example_data/mlb_teams_2012.csv')
data = loader.load()