Skip to content

Instantly share code, notes, and snippets.

View johnidm's full-sized avatar

Johni Douglas Marangon johnidm

View GitHub Profile
@johnidm
johnidm / readme.md
Created January 29, 2025 13:49
PHP - Parsing PDF to Txt

How to parse a PDF file to txt.

Install the dependence: composer require smalot/pdfparser.

Run the following code:

<?php
require __DIR__ . '/vendor/autoload.php';
@johnidm
johnidm / readme.md
Last active January 17, 2025 19:15
Surya and docTR: Powerful OCR Toolkits for Document Processing

Surya and docTR: Powerful OCR Toolkits for Document Processing

Surya is a Python-based document OCR toolkit designed for flexibility and ease of use in processing and extracting text from scanned documents. Developed as a lightweight and customizable solution, it allows developers to work with OCR tasks seamlessly, making it a great choice for building tailored document processing workflows. Surya supports multiple OCR engines and focuses on accessibility for diverse use cases. GitHub

from PIL import Image
from surya.ocr import run_ocr
from surya.model.detection.model import load_model as load_det_model, load_processor as load_det_processor
from surya.model.recognition.model import load_model as load_rec_model
from surya.model.recognition.processor import load_processor as load_rec_processor
@johnidm
johnidm / readme.md
Last active December 11, 2024 13:15
Example How to Use Structured Outputs - OpenAI

This code demonstrates a structured approach to querying OpenAI models and handling the response, making it easier to integrate advanced use cases like math tutoring into applications.

Defining Data Models:

  • Step: Represents an individual step in the solution process. Each step contains:
    • explanation: A description of what is happening in this step.
    • output: The result or equation at this stage.
  • MathResponse: Represents the full structured response. It contains:
    • steps: A list of Step objects, representing the solution's breakdown.
    • final_answer: The final solution to the problem.
@johnidm
johnidm / readme.md
Created November 2, 2024 16:46
Explore OpenAI Assistant in Python - While True to Run a Conversation

A variant of the ongoing conversation employs the while true strategy, utilizing an infinite loop to sustain the dialogue.

def run_conversation(message: str, thread_id, assistant_id):
    message = client.beta.threads.messages.create(
        thread_id=thread_id,
        role="user",
        content=message,
    )
@johnidm
johnidm / readme.md
Created October 5, 2024 18:29
OpenAI - Calculating Token Counts and Estimating Costs

Utilize this code to quickly estimate processing costs for your dataset with OpenAI.

dataset.csv

text
Music is a universal language that connects people across cultures.
Listening to music can improve your mood and reduce stress.
Classical music has a rich history and deeply influences modern genres.
Jazz is known for its improvisational style and complex harmonies.
@johnidm
johnidm / readme.md
Last active October 4, 2024 14:21
Basic Real-Time Applications with WebSockets and FastAPI

Install dependencies - pip install fastapi 'uvicorn[standard]'

Run the example - uvicorn main:app --host 0.0.0.0 --port 8000

Access multiple pages from http://localhost:8000/ and send messages

Source code files

main.py

@johnidm
johnidm / readme.md
Created July 31, 2024 17:24
Securing Application with API Key Authentication
from fastapi.security import APIKeyHeader
from fastapi import Security, HTTPException, status
from fastapi import APIRouter, FastAPI, Depends
import sqlite3


print(sqlite3.sqlite_version)
@johnidm
johnidm / readme.md
Last active December 27, 2024 13:35
Simple Background Job Management with FastAPI (with status)

This code snippet is a simple FastAPI application that manages a background job with status.

Install the dependecies

pip install fastapi uvicorn

Example Usage

@johnidm
johnidm / dataset.csv
Created May 26, 2024 12:57
Dataset - Full United State Addresses
We can make this file beautiful and searchable if this error is corrected: Unclosed quoted field in line 2.
document
"It was a beautiful summer day, the sun was shining bright and the birds were chirping happily. I was on my way to visit my friend, Tom, who recently moved to a new place. He had been telling me about it for weeks and I was eager to see it in person. He had described it as a charming little house with a big backyard, nestled in the heart of a vibrant and bustling neighborhood.
He had sent me the address a few days ago, ""123 Elm Street, Springfield, Illinois, 62704"". I had punched it into my GPS and off I went, cruising down the highway with my favorite music playing on the radio. I was excited about the prospect of spending the day with Tom, catching up on old times and exploring his new neighborhood.
As I drove, I couldn't help but marvel at the beauty of the surrounding landscape. The lush green fields, the clear blue sky, it was a sight to behold. I could see why Tom had chosen to move here. It was peaceful, serene, and yet full of life and energy. I was sure he was going to love it here.
Aft
@johnidm
johnidm / README.md
Last active January 25, 2025 15:10
SQLite - Use two Database in-memory - Example in SQLAlchemy

Using two SQLite database connection in-memory.

from sqlalchemy import create_engine, MetaData
from sqlalchemy import text
from sqlalchemy.orm import DeclarativeBase

from sqlalchemy import select
from sqlalchemy import insert
from sqlalchemy.orm import Mapped, mapped_column