This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
data = ["Giants hit 3 HRs to down Dodgers", | |
"Giants 5 Dodgers 4 final", | |
"Dodgers drop Game 2 against the Giants, 5-4", | |
"Blue Jays beat Red Sox final score 2-1", | |
"Red Sox lost to the Blue Jays, 2-1", | |
"Blue Jays at Red Sox is over. Score: 2-1", | |
"Phillies win over the Braves, 5-0", | |
"Phillies 5 Braves 0 final", | |
"Final: Braves lose to the Phillies in the series opener, 5-0", | |
"Lightning goaltender pulled, lose to Flyers 4-1", |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
embeddings.save("index") | |
embeddings = Embeddings() | |
embeddings.load("index") | |
uid = embeddings.search("climate change", 1)[0][0] | |
print(data[uid]) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from tldrstory.source.source import Source | |
class Neuspo(Source): | |
""" | |
Articles have the following schema: | |
uid - unique id | |
source - source name | |
date - article date | |
title - article title | |
url - reference url for data |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# GitHub Actions build workflow | |
name: build | |
on: ["push", "pull_request"] | |
jobs: | |
build: | |
runs-on: ubuntu-latest | |
steps: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# GitHub Actions build workflow | |
name: build | |
on: ["push", "pull_request"] | |
jobs: | |
build: | |
runs-on: ${{ matrix.os }} | |
strategy: | |
matrix: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from txtai.pipeline import Summary | |
# Create summary model | |
summary = Summary() | |
text = ("Search is the base of many applications. Once data starts to pile up, users want to be able to find it. It’s the foundation " | |
"of the internet and an ever-growing challenge that is never solved or done. The field of Natural Language Processing (NLP) is " | |
"rapidly evolving with a number of new developments. Large-scale general language models are an exciting new capability " | |
"allowing us to add amazing functionality quickly with limited compute and people. Innovation continues with new models " | |
"and advancements coming in at what seems a weekly basis. This article introduces txtai, an AI-powered search engine " |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from txtai.pipeline import Textractor | |
# Create textractor model | |
textractor = Textractor() | |
textractor("txtai/article.pdf") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from txtai.pipeline import Transcription | |
# Create transcription model | |
transcribe = Transcription("facebook/wav2vec2-large-960h") | |
transcribe("Make_huge_profits.wav") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from txtai.pipeline import Translation | |
# Create translation model | |
translate = Translation() | |
translate("This is a test translation into Spanish", "es") |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from txtai.embeddings import Embeddings | |
embeddings = Embeddings({"method": "sentence-transformers", "path": "clip-ViT-B-32"}) | |
embeddings.index(images(directory)) | |
embeddings.search(query, 1) |