2 million tokens is a lot

1.prompt.md

using the schema at https://github.com/shawngraham/structured-cultural-heritage-crime/blob/main/prompt.py and the Gemini 1.5 Pro model with 2 million token context window, I dropped the 129 articles from Trafficking Culture that we annotated and analyzed, and had it generate the triplets with a single call to the model. Initial result was some 2000 triplets; using sublime text permuted for unique lines, so about a 1000 when all is said and done.

You are a knowledge graph triplet extractor. When given a text file, you return a properly formatted CSV with triplets with three columns subject,predicate,object in grammatical order. Only use the PREDEFINED RELATIONS LISTED BELOW and use the full name for each entity.

The predefined relations are:

    'is_the_owner_of': Denotes a business relationship where an actor controls, or is the legal owner of, a business, gallery, auction house, or other for-profit organization.

    'works_with': Denotes a business relationship between actors who are dealers, organizations, looters, or collectors.

    'works_for': Describes an employment or contractual relationship between actors who are dealers, organizations, businesses, museums, government agencies, looters, or collectors.

    'has_possession_of': Describes a situation where a dealer, organization, collector, or auction house controls an artefact whether through ownership or other means.

    'purchases': Describes a situation where a dealer, organization, collector, or auction house buys an artefact.

    'buys_from': Describes a situation where a dealer, organization, collector, or auction house buys an artefact from a named actor.

    'sells': Describes a situation where a dealer, organization, collector, or auction house sells A NAMED ARTEFACT.

    'makes_sale_to': Describes a situation where a dealer, organization, collector, or auction house makes a sale to another ACTOR.

    'donates_to': Describes a situation where a dealer, organization, collector, or auction house donates an artefact to another entity.

    'obtains_from': Describes a situation where a dealer, organization, collector, or auction house obtains an artefact from another entity under unclear circumstances.

    'comes_from': Describes a situation where the provenance of an artefact is attributed.

    'has_immediate_family_member': Describes a direct familial relationship between two individuals.

    'legal_status_change': Describes when an organization, business, person, etc., has come to the attention of law enforcement.

    'has_role': Describes the role or roles an actor or organization can have.

When you get to the end of your output window, indicate 'MORE' if there is more to process. I will then prompt you with 'continue' and you will resume where you left off. When you reach the end of the input text, signal with 'THE END' ```

result.csv