- Introduction
- Architecture Overview
- Setting Up the Environment 3.1. Azure Setup 3.2. Snowflake Setup 3.3. DBT Setup 3.4. Python Environment Setup
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # Check for required arguments | |
| if [ $# -lt 2 ]; then | |
| echo "Usage: $0 <directory> <delimiter> [--include=<pattern>] [--exclude=<pattern>]" | |
| exit 1 | |
| fi | |
| # Assign the first argument as the directory | |
| DIRECTORY=$1 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| from fastapi import FastAPI, HTTPException | |
| from pydantic import BaseModel | |
| from typing import List, Optional, Dict | |
| import faiss | |
| import numpy as np | |
| import os | |
| app = FastAPI() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import asyncio | |
| import aiohttp | |
| import os | |
| from dotenv import load_dotenv | |
| # Load environment variables | |
| load_dotenv() | |
| API_KEY = os.getenv("ANTHROPIC_API_KEY") | |
| API_URL = "https://api.anthropic.com/v1/messages" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| import subprocess | |
| from pydub import AudioSegment | |
| from groq import Groq | |
| import argparse | |
| # Initialize the Groq client | |
| client = Groq() | |
| def convert_video_to_audio(video_path, audio_path): |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import json | |
| import logging | |
| import time | |
| from typing import Dict, Any | |
| import requests | |
| from bs4 import BeautifulSoup | |
| from openai import OpenAI | |
| from requests.exceptions import RequestException | |
| from tenacity import retry, stop_after_attempt, wait_random_exponential |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import os | |
| import base64 | |
| import json | |
| import logging | |
| from datetime import datetime, timedelta | |
| from typing import List | |
| from google.oauth2.credentials import Credentials | |
| from google_auth_oauthlib.flow import InstalledAppFlow | |
| from googleapiclient.discovery import Resource, build | |
| from googleapiclient.errors import HttpError |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import json | |
| import logging | |
| import csv | |
| from typing import List, Dict, Any, Optional | |
| from pydantic import BaseModel, create_model, Field, validator, field_validator | |
| from openai import OpenAI | |
| import time | |
| # Set up logging | |
| logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s') |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #!/bin/bash | |
| # Update and upgrade Homebrew | |
| echo "Updating Homebrew..." | |
| brew update | |
| brew upgrade | |
| # Install nvm (Node Version Manager) | |
| echo "Installing nvm..." |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| # settings we’re about to change | |
| osascript -e 'tell application "System Preferences" to quit' | |
| # Ask for the administrator password upfront | |
| sudo -v | |
| # Keep-alive: update existing `sudo` time stamp until `.macos` has finished | |
| while true; do | |
| sudo -n true | |
| sleep 60 | |
| kill -0 "$$" || exit |