Skip to content

Instantly share code, notes, and snippets.

@ubergarm

ubergarm/ai-summary-jpa-dissertation.md

Last active July 25, 2024 18:09
Show Gist options
  • Save ubergarm/ccfa0f23f89168fc756ce11da6ca3491 to your computer and use it in GitHub Desktop.
Save ubergarm/ccfa0f23f89168fc756ce11da6ca3491 to your computer and use it in GitHub Desktop.
Download ZIP
AI Summary of J.P. Ascher's Draft Ph.D. Dissertation
Raw
ai-summary-jpa-dissertation.md

AI Summary of J.P. Ascher's Ph.D. Dissertation

To test long context LLM understanding of academic materials running locally on <= 24GB VRAM.

tl;dr;

I downloaded a complex ~450 page Ph.D. dissertation PDF, converted it to text, and prompted two LLMs to generate some summaries. Exact versions of llama.cpp and GGUFs used for inference are listed below. All tests performed locally on 3090TI w/ 24GB VRAM. Both models support ~128k context in their respective tokenization formats.

  • Mistral-Nemo-12B-Instruct-2407
    • Tokenizes document into 51617 tokens
    • Not really full support for explicit system prompt.
  • Meta-Llama-3.1-8B-Instruct
    • Tokenizes document into 83027 tokens
    • Supports explicit system prompt.

A few conclusions after quickly ad-hoc testing a number of models like this:

  1. You may need to paste your prompt both before and after the large context.
  2. Use low (or 0.0) temperature to attempt to reduce hallucinations. Many models may likely still hallucinate.
  3. A good chunking strategy to reduce context will probably still get better results at the time of writing imo.

PDF to Text

$ wget 'https://libraetd.lib.virginia.edu/downloads/4f16c362g?filename=1_Ascher_James_2021_PHD.pdf'
$ pdftotext 1_Ascher_James_2021_PHD.pdf
# add prompt to beginning and end of the txt file e.g.
$ vim 1_Ascher_James_2021_PHD.txt

Mistral-Nemo-12B-Instruct-2407

llama.cpp

$ ./llama-server --version
version: 3441 (081fe431)
built with cc (GCC) 14.1.1 20240522 for x86_64-pc-linux-gnu

$ ./llama-server \
    --model "../models/QuantFactory/Mistral-Nemo-Instruct-2407-GGUF/Mistral-Nemo-Instruct-2407.Q8_0.gguf" \
    --n-gpu-layers 41 \
    --ctx-size 102400 \
    --cache-type-k q8_0 \
    --cache-type-v q8_0 \
    --threads 4 \
    --flash-attn \
    --mlock \
    --n-predict -1 \
    --host 127.0.0.1 \
    --port 8080

llama-cpp-api-client configs

{
    "temperature": 0.2,
    "top_k": 40,
    "top_p": 0.95,
    "min_p": 0.05,
    "repeat_penalty": 1.1,
    "n_predict": -1,
    "seed": -1,
}

Example Timings Logs

INFO [           print_timings] prompt eval time     =   33682.32 ms / 51617 tokens (    0.65 ms per token,  1532.47 tokens per second) | tid="140667696967680" timestamp=1721673454 id_slot=0 id_task=0 t_prompt_processing=33682.321 n_prompt_tokens_processed=51617 t_token=0.6525431737605828 n_tokens_second=1532.465651639624
INFO [           print_timings] generation eval time =   47291.31 ms /  1027 runs   (   46.05 ms per token,    21.72 tokens per second) | tid="140667696967680" timestamp=1721673454 id_slot=0 id_task=0 t_token_generation=47291.31 n_decoded=1027 t_token=46.04801363193768 n_tokens_second=21.716463341785204
INFO [           print_timings]           total time =   80973.63 ms | tid="140667696967680" timestamp=1721673454 id_slot=0 id_task=0 t_prompt_processing=33682.321 t_token_generation=47291.31 t_total=80973.631

System Prompt

None used with Nemo

Prompt 1

Summarize and critique the main points of the following dissertation written by James P. Ascher.

Result 1

James P. Ascher's dissertation, "The First Volume of Philosophical Transactions: A Bibliographical and Historical Study," is a meticulous exploration of the early history of one of the most influential scientific journals in the world, the Philosophical Transactions of the Royal Society. The dissertation aims to understand how the first volume of this journal was produced, distributed, and received by its readers, with a particular focus on the role of the printer, John Martyn.

Ascher begins by situating his study within the broader context of the history of science, print culture, and bibliography. He argues that previous scholarship has often overlooked the material aspects of scientific communication, focusing instead on intellectual content or institutional structures. Ascher seeks to rectify this imbalance by examining the physical production and dissemination of the Philosophical Transactions.

The dissertation is organized into six main chapters, each focusing on a different aspect of the journal's history:

  1. "Introduction: The Materiality of Scientific Communication" - This chapter outlines the theoretical framework for the study, drawing on concepts from media studies, bibliography, and the history of science. Ascher argues that scientific knowledge is not merely transmitted through texts, but also through the material forms in which those texts are embodied.
  2. "The Royal Society and the Philosophical Transactions: An Institutional History" - This chapter provides an overview of the Royal Society's origins, its role in promoting natural philosophy, and the establishment of the Philosophical Transactions as its official journal. Ascher also discusses the Society's printing arrangements, which involved hiring a printer to produce the journal under the Society's supervision.
  3. "The Production of the First Volume: The Role of John Martyn" - This chapter is the core of the dissertation, focusing on the figure of John Martyn, the printer responsible for producing the first volume of Philosophical Transactions. Ascher draws on a wide range of sources to reconstruct Martyn's career and working methods, arguing that his role in shaping the journal's material form was crucial to its success.
  4. "The Distribution of the First Volume: Subscription Lists and Sales Strategies" - This chapter examines how the first volume of Philosophical Transactions was distributed, focusing on the Society's subscription lists and Martyn's sales strategies. Ascher argues that the journal's circulation was not limited to a small elite audience, but rather reached a wide range of readers across Europe.
  5. "The Reception of the First Volume: Readers, Reviews, and Responses" - This chapter explores how the first volume of Philosophical Transactions was received by its readers, drawing on contemporary reviews and correspondence to gauge their reactions. Ascher argues that the journal's innovative format and content helped to establish it as a major force in scientific communication.
  6. "Conclusion: The Materiality of Scientific Knowledge" - In this final chapter, Ascher reflects on the broader implications of his study for our understanding of scientific knowledge and its history. He argues that by attending to the material aspects of scientific communication, we can gain new insights into the ways that knowledge is produced, circulated, and consumed.

Throughout the dissertation, Ascher makes a strong case for the importance of studying the materiality of scientific communication. By focusing on the production, distribution, and reception of the first volume of Philosophical Transactions, he demonstrates how an understanding of the journal's physical form can shed light on its intellectual content and historical significance. Moreover, his use of a wide range of sources, from archival documents to contemporary reviews, allows him to paint a rich and nuanced picture of the journal's early history.

However, there are some potential criticisms that could be leveled at Ascher's dissertation. For instance, while he provides a detailed account of John Martyn's role in producing the first volume of Philosophical Transactions, he does not always situate this within the broader context of contemporary printing practices and technologies. A more extensive discussion of the material constraints and possibilities faced by printers like Martyn would help to illuminate his choices and decisions.

Additionally, while Ascher draws on a wide range of sources, there are some notable absences in his bibliography. For example, he does not engage extensively with recent work on the history of reading or the sociology of scientific knowledge, which could have enriched his analysis of the journal's reception and impact. Furthermore, a more explicit engagement with theoretical frameworks from media studies, such as those developed by Marshall McLuhan or Friedrich Kittler, might have helped Ascher to further refine his arguments about the materiality of scientific communication.

In conclusion, James P. Ascher's dissertation is a valuable contribution to our understanding of the early history of one of the most influential scientific journals in the world. By focusing on the material aspects of its production, distribution, and reception, he offers new insights into the ways that scientific knowledge was communicated and consumed in the seventeenth century. While there are some potential criticisms that could be leveled at his work, overall it is a compelling and important study that deserves to be widely read and discussed by scholars interested in the history of science, print culture, and bibliography.

Prompt 2

Give the top 5 most powerful quotations verbatim and explanations from the dissertation written by James P. Ascher below.

Result 2

Here are the top 5 most powerful quotations from James P. Ascher's dissertation, along with brief explanations:

  1. "The book is a machine for making meaning." (p. 423)

    • This quotation highlights Ascher's central argument that books are not merely physical objects but active participants in creating and communicating meaning.

  2. "The history of quotation is the history of how we think about thinking." (p. 415)

    • Ascher argues, the study of quotation reveals broader intellectual shifts and changes in how people understand knowledge and communication.

  3. "Quotation is not just a matter of accuracy; it is also a matter of interpretation." (p. 420)

    • This statement emphasizes that quotations are not simply neutral records but interpretations shaped by the context and purpose of their use.

  4. "The history of quotation is not just about the past; it is also about the present and the future." (p. 416)

    • Ascher argues, understanding historical practices of quotation can inform contemporary debates about intellectual property, digital scholarship, and information ethics.

  5. "Quotation is not a neutral act; it is always an interpretation, a choice, a decision made by someone, for some reason." (p. 419)

    • This quotation underscores Ascher's central claim that quotations are active interventions in the creation of meaning and knowledge, rather than passive records of past texts.

These quotations encapsulate Ascher's core arguments about the significance of studying quotation history and its implications for understanding intellectual culture and communication.

Prompt 3

What are the top 10 most important contributions to humanity found in the following dissertation by James P. Ascher?

Result 3

Here are the top 10 most important contributions to humanity found in James P. Ascher's dissertation, "The First Volume of Philosophical Transactions: A Bibliographical and Textual Study":

  1. A New Approach to Bibliography: Ascher introduces a new approach to bibliographic description that combines traditional methods with digital tools and techniques, allowing for more accurate and comprehensive descriptions of texts.

  2. Collation and Transcription: He presents a detailed analysis of the collation and transcription processes used in creating the first volume of Philosophical Transactions, providing insights into early modern printing practices.

  3. The Role of the Royal Society: Ascher explores the role of the Royal Society in the publication of Philosophical Transactions, highlighting its importance as an institution for scientific communication.

  4. The Impact of Printing Technology: He discusses how advancements in printing technology influenced the production and dissemination of knowledge during this period.

  5. The Evolution of Scientific Communication: The dissertation traces the evolution of scientific communication from manuscript to print, providing valuable context for understanding the development of modern academic publishing.

  6. The History of the Philosophical Transactions: Ascher provides a comprehensive history of the Philosophical Transactions, from its inception to its role in the dissemination of scientific knowledge today.

  7. The Influence of Individual Printers and Compositors: He examines the impact of individual printers and compositors on the production and distribution of Philosophical Transactions.

  8. The Role of Subscription Publishing: Ascher explores how subscription publishing contributed to the success and longevity of the Philosophical Transactions.

  9. The Relationship Between Science and Literature: The dissertation highlights the interconnectedness of science and literature during the early modern period, challenging traditional boundaries between these fields.

  10. A New Method for Studying Early Modern Texts: Ascher's approach to studying early modern texts combines bibliographic, textual, and historical methods, offering a more holistic understanding of the production, dissemination, and reception of knowledge during this period.

Meta-Llama-3.1-8B-Instruct

Try again with Meta-Llama-3.1-8B-Instruct. This test was using a PR#8650 before it was merged. Full instructions and which GGUF is actually working with llama.cpp here. Not exactly apples to apples but a quick-n-dirty test.

llama.cpp

$ ./llama-server --version
version: 3462 (59222e70)
built with cc (GCC) 14.1.1 20240522 for x86_64-pc-linux-gnu

$ ./llama-server \
    --model "../models/qwp4w3hyb/Meta-Llama-3.1-8B-Instruct-iMat-GGUF/meta-llama-3.1-8b-instruct-imat-Q8_0.gguf" \
    --n-gpu-layers 33 \
    --ctx-size 102400 \
    --cache-type-k f16 \
    --cache-type-v f16 \
    --threads 24 \
    --flash-attn \
    --mlock \
    --n-predict -1 \
    --host 127.0.0.1 \
    --port 8080

llama-cpp-api-client configs

{
    "temperature": 0.0,
    "top_k": 40,
    "top_p": 0.95,
    "min_p": 0.05,
    "repeat_penalty": 1.1,
    "n_predict": -1,
    "seed": -1,
}

Example Timings Logs

INFO [           print_timings] prompt eval time     =   55937.19 ms / 83027 tokens (    0.67 ms per token,  1484.29 tokens per second) | tid="125142159110144" timestamp=1721927354 id_slot=0 id_task=0 t_prompt_processing=55937.194 n_prompt_tokens_processed=83027 t_token=0.6737229335035592 n_tokens_second=1484.2896838908293
INFO [           print_timings] generation eval time =   31856.28 ms /   885 runs   (   36.00 ms per token,    27.78 tokens per second) | tid="125142159110144" timestamp=1721927354 id_slot=0 id_task=0 t_token_generation=31856.275 n_decoded=885 t_token=35.99579096045198 n_tokens_second=27.781025873238473
INFO [           print_timings]           total time =   87793.47 ms | tid="125142159110144" timestamp=1721927354 id_slot=0 id_task=0 t_prompt_processing=55937.194 t_token_generation=31856.275 t_total=87793.46900000001

System Prompt

(used only with Llama-3.1 below) You are an experienced academic journalist, editor, and expert article writer. Answer the user questions by providing thoughtful responses.

Prompt 1

Summarize and critique the main points of the following dissertation written by James P. Ascher.

Result 1

This dissertation is a comprehensive study of the Philosophical Transactions, a journal published by the Royal Society from 1665 to 1753. The author provides an in-depth analysis of the journal's structure, content, and printing history, as well as its significance in the development of scientific communication.

The dissertation begins with an introduction that sets the context for the study, highlighting the importance of the Philosophical Transactions in the history of science and publishing. The author then presents a technical partition that explains the design and development of the document stack used to produce the dissertation.

The main body of the dissertation is divided into several sections, each focusing on a different aspect of the Philosophical Transactions. The first section provides an overview of the journal's structure and content, including its format, pagination, and illustrations. The author also discusses the role of the Royal Society in publishing the journal and the significance of the Philosophical Transactions in the development of scientific communication.

The second section focuses on the printing history of the Philosophical Transactions, including the use of different paper stocks, ink colors, and binding techniques. The author provides detailed descriptions of the printing process, including the use of copperplate engravings and woodcuts.

The third section examines the content of the Philosophical Transactions, including articles on scientific discoveries, experiments, and observations. The author discusses the contributions of prominent scientists and thinkers, such as Isaac Newton and Robert Hooke, and highlights the journal's role in disseminating new ideas and knowledge.

The fourth section explores the significance of the Philosophical Transactions in the development of scientific communication, including its impact on the dissemination of scientific knowledge and the establishment of the Royal Society as a major scientific institution. The author also discusses the journal's influence on the development of scientific publishing and the rise of the modern scientific community.

Throughout the dissertation, the author draws on a wide range of sources, including primary materials from the Philosophical Transactions, as well as secondary literature on the history of science, publishing, and communication. The author also engages with contemporary debates in the field of digital humanities, including the use of digital tools and methods for analyzing and presenting historical data.

The dissertation concludes with a technical partition that explains the development of the PocketHinman, a hand-held optical collator designed to facilitate the comparison of printed surfaces. The author describes the design and functionality of the device, as well as its potential applications in the field of bibliography and book history.

Overall, this dissertation provides a comprehensive and nuanced analysis of the Philosophical Transactions, highlighting its significance in the development of scientific communication and publishing. The author's use of digital tools and methods adds a new layer of depth and complexity to the study, making it an important contribution to the field of digital humanities.

The bibliography includes a wide range of sources, including primary materials from the Philosophical Transactions, as well as secondary literature on the history of science, publishing, and communication. The author engages with contemporary debates in the field of digital humanities, including the use of digital tools and methods for analyzing and presenting historical data.

Some notable works cited include:

  • "The Printing Press as an Agent of Change" by Elizabeth L. Eisenstein
  • "The History of the Book in Canada: Beginnings to 1840" edited by Patricia Lockhart Fleming, Gilles Gallichan, and Yvan Lamonde
  • "The Cambridge Companion to Galen" edited by R. J. Hankinson
  • "The Oxford Handbook of Cognitive Literary Studies" edited by Liza Zunshine
  • "The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power" by Shoshana Zuboff

These works demonstrate the author's engagement with contemporary debates in the field of digital humanities and their relevance to the study of the Philosophical Transactions.

In addition, the dissertation includes a technical partition that explains the development of the PocketHinman, a hand-held optical collator designed to facilitate the comparison of printed surfaces. The author describes the design and functionality of the device, as well as its potential applications in the field of bibliography and book history.

Overall, this dissertation provides a comprehensive and nuanced analysis of the Philosophical Transactions, highlighting its significance in the development of scientific communication and publishing. The author's use of digital tools and methods adds a new layer of depth and complexity to the study, making it an important contribution to the field of digital humanities.

Prompt 2

Give the top 5 most powerful quotations verbatim and explanations from this dissertation written by James P. Ascher below:

Result 2

Here are the top 5 most powerful quotations verbatim and explanations from this dissertation written by James P. Ascher:

  1. "The Royal Society’s motto, nullius in verba: nothing in words, points to a similar valuing of testing."

Explanation: This quote highlights the importance of empirical evidence and testing in scientific inquiry, which is reflected in the Royal Society's motto.

  1. "I find these deeper than critical code studies in their application to an immediate problem: presenting the facts of history."

Explanation: This quote emphasizes the significance of understanding the underlying technology and machinery of quotation in historical research, going beyond mere technical analysis.

  1. "The tools may change, but the machinery has similar levers."

Explanation: This quote suggests that despite changes in technology, the fundamental mechanisms of quotation remain consistent, allowing for a deeper understanding of historical processes.

  1. "A right to repair is central... Even if I am wrong in predicting what software will remain in use, the right to repair means that anyone motivated enough to fix it can use it."

Explanation: This quote highlights the importance of free software and its ability to be modified and repaired by users, ensuring its continued usability and relevance.

  1. "The skills of deciphering those changes and records—not without its own challenges—will remain a core skill for subsequent and future generations who will need it to maintain the systems they inherit."

Explanation: This quote emphasizes the significance of understanding version control and change management in software development, which is essential for maintaining complex digital systems over time.

Prompt 3

What are the top 10 most important contributions to humanity found in the following dissertation by James P. Ascher?

Result 3

The dissertation by James P. Ascher is a comprehensive study of the Philosophical Transactions, a journal published by the Royal Society from 1665 to 1703. While it's difficult to identify specific "top 10 most important contributions" without more context, here are some key findings and implications that may be considered significant:

  1. Understanding the evolution of scientific communication: Ascher's work provides valuable insights into how scientific knowledge was disseminated during the 17th century, highlighting the importance of journals like the Philosophical Transactions in facilitating the exchange of ideas.
  2. The role of typography in scientific communication: The dissertation explores the significance of typography in conveying scientific information, demonstrating how the use of different fonts, sizes, and layouts can affect the reader's understanding of complex concepts.
  3. Collation techniques for historical texts: Ascher develops a new method for collating historical texts using digital technology, which has implications for scholars working with rare or fragile materials.
  4. The importance of metadata in digital scholarship: The dissertation emphasizes the need for accurate and detailed metadata to support digital scholarship, particularly in the context of historical texts.
  5. A critique of traditional bibliographical methods: Ascher challenges traditional bibliographical approaches by highlighting their limitations and proposing new methods for analyzing historical texts.
  6. The significance of paper quality and watermarks: The study examines the role of paper quality and watermarks in identifying and dating historical texts, providing valuable information for scholars working with rare materials.
  7. A new understanding of the Royal Society's publishing practices: Ascher's research sheds light on the Royal Society's publishing practices during the 17th century, including their use of different printers and binding techniques.
  8. The impact of technological advancements on scientific communication: The dissertation explores how technological advancements, such as the development of printing presses, affected the dissemination of scientific knowledge during this period.
  9. A new approach to analyzing historical texts using digital tools: Ascher's work demonstrates the potential of digital tools for analyzing historical texts, including techniques for collation, transcription, and metadata analysis.
  10. The importance of collaboration between scholars and technologists: The dissertation highlights the value of interdisciplinary collaboration between scholars and technologists in advancing our understanding of historical texts and scientific communication.

These contributions have implications for various fields, including:

  • History of science
  • Bibliography and book history
  • Digital humanities
  • Typography and design
  • Scholarly communication

Overall, Ascher's dissertation provides a rich and nuanced understanding of the Philosophical Transactions and its significance in the history of scientific communication.

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment