ChatGPT passes the 2022 APCSA free response section

README.md

ChatGPT passes the 2022 AP Computer Science A free response section

For fun, I had ChatGPT take the free response section of the 2022 AP Computer Science A exam. (The exam also has a multiple-choice section, but the College Board doesn't publish this.) It scored 32/36.

Methodology

• For each question, I pasted in the full text of the question and took the response given.
• I tried each question once and took the response given: no cherry-picking. For readability, I've added indentation in some cases, and included method signatures where they were provided in the question and ChatGPT only provided a body. I've added question numbers; any other comments are ChatGPT's.
• Many questions have examples containing tables or diagrams; because those don't translate well to plain text, I excluded those tables/diagrams and any text that referenced them.
• I excluded the initial instructions at the top of the exam booklet, the "Class information for this question" boxes, and instructions about where/how to write the response in the free response booklet.
• I clicked "reset thread" between each of the four numbered questions.
• For questions with a part (a) and (b), I pasted in the full question up to the end of part (a), got the response for part (a), then pasted in (in the same thread) just part (b) and got the answer to that part.
  • In one case (question 1), ChatGPT provided an implementation for both parts after just getting asked for part (a), so I just took that implementation; as such, it didn't get to see the details on page 7 of the exam.

Scoring and notes

ChatGPT scored 32/36 according to my best interpretation of the College Board's scoring guidelines. It missed the following points:

• Question 1, point 1: fails to call getPoints or goalReached on a level object (it tries to access a levels array which doesn't exist)
• Quesiton 1, point 4: initializes to 3 instead of tripling
• Question 1, point 6: initializes a new Game each time instead of calling methods on this (this works, and actually seems a little more sensible, but the text of the scoring guidelines seem to imply I should dock points here)
• Question 4, point 4: off-by-one error in generating random numbers (it uses rand.nextInt(MAX - 1) + 1; nextInt's parameter is exclusive, so the highest value we could get is MAX - 1, but the question states that MAX should be inclusive)

Several questions use parts of Java that aren't part of the AP subset; fair enough, I didn't tell ChatGPT about the subset. In any case, I don't think there's a rule against that as long as the solution works.

The implementation of question 3b (collectComments()) is needlessly convoluted, but it looks like it'd work fine.

solutions.java

// 1a
public int getScore()
{
    int score = 0;

    // Check if this is a bonus game
    if (isBonus()) {
        // If it is, triple the final score
        score = 3;
    }

    // Iterate through each level
    for (int i = 1; i <= 3; i++) {
        // Check if the goal for the current level has been reached
        if (level[i].goalReached()) {
            // If it has, add the points for that level to the score
            score += level[i].getPoints();
        }
    }

    // Return the final score
    return score;
}

// 1b
public int playManyTimes(int num)
{
    // Keep track of the highest score seen so far
    int highestScore = 0;

    // Iterate for the number of games specified in the num parameter
    for (int i = 0; i < num; i++) {
        // Create a new game
        Game game = new Game();

        // Simulate the play of the game
        game.play();

        // Compute the score for the game
        int score = game.getScore();

        // If the score for this game is higher than the highest score seen so far,
        // update the highest score
        if (score > highestScore) {
            highestScore = score;
        }
    }

    // Return the highest score seen
    return highestScore;
}

// 2
public class Textbook extends Book {
  private int edition;

  /**
   * Creates a new Textbook with given title, price, and edition number.
   *
   * @param bookTitle the title of the textbook
   * @param bookPrice the price of the textbook
   * @param bookEdition the edition number of the textbook
   */
  public Textbook(String bookTitle, double bookPrice, int bookEdition) {
    super(bookTitle, bookPrice);
    this.edition = bookEdition;
  }

  /**
   * Returns the edition number of the textbook.
   *
   * @return the edition number of the textbook
   */
  public int getEdition() {
    return edition;
  }

  /**
   * Returns the title, price, and edition number of the Textbook as a string.
   *
   * @return the title, price, and edition number of the Textbook as a string
   */
  @Override
  public String getBookInfo() {
    return super.getBookInfo() + "-" + edition;
  }

  /**
   * Returns true if this Textbook is a valid substitute for the Textbook referenced by the parameter
   * of the canSubstituteFor method. Returns false otherwise.
   *
   * @param other the Textbook to compare to this Textbook
   * @return true if this Textbook is a valid substitute for the Textbook referenced by the parameter
   *     of the canSubstituteFor method, false otherwise
   */
  public boolean canSubstituteFor(Textbook other) {
    return this.getTitle().equals(other.getTitle()) && this.getEdition() >= other.getEdition();
  }
}

// 3a

public double getAverageRating()
{
  double totalRating = 0;
  int numReviews = allReviews.length;

  for (Review review : allReviews)
  {
      totalRating += review.getRating();
  }

  return totalRating / numReviews;
}

// 3b

public ArrayList<String> collectComments()
{
  ArrayList<String> formattedComments = new ArrayList<>();

  for (int i = 0; i < allReviews.length; i++)
  {
      String comment = allReviews[i].getComment();
      if (comment.contains("!"))
      {
          formattedComments.add(i + "-" + comment);

          if (!(comment.endsWith(".") || comment.endsWith("!")))
          {
              formattedComments.set(formattedComments.size() - 1, formattedComments.get(formattedComments.size() - 1) + ".");
          }
      }
  }

  return formattedComments;
}

// 4a

public void repopulate()
{
  Random rand = new Random();
  for (int i = 0; i < grid.length; i++) {
    for (int j = 0; j < grid[i].length; j++) {
      int val = rand.nextInt(MAX - 1) + 1;
      while (val % 10 != 0 || val % 100 == 0) {
        val = rand.nextInt(MAX - 1) + 1;
      }
      grid[i][j] = val;
    }
  }
}

// 4b

public int countIncreasingCols()
{
  int count = 0;
  for (int i = 0; i < grid[0].length; i++) {
    boolean isIncreasing = true;
    for (int j = 1; j < grid.length; j++) {
      if (grid[j][i] < grid[j - 1][i]) {
        isIncreasing = false;
        break;
      }
    }
    if (isIncreasing) {
      count++;
    }
  }
  return count;
}

The 1a solution has several issues:

• Uses the fictitious levels array instead of the levelOne, levelTwo, and levelThree variables.
• Off-by-one error in the for loop over levels
• Doesn't take into account the requirement that, e.g., level two points are only earned if level one's goal was reached
• As you noted, doesn't triple the score correctly

Doesn't take into account the requirement that, e.g., level two points are only earned if level one's goal was reached

Oh, good point - it might also miss question 1 point 2:

Determines if points are earned based on goalReached return values
Responses can still earn the point even if they

• calculate the score total incorrectly
• call goalReached incorrectly
• fail to distinguish all cases correctly

Responses will not earn the point if they

• fail to use a nested if statement or equivalent

I think the lack of a break means this isn't "equivalent" to a nested if statement, so it loses this point.

But why its IQ is only 83. https://pbs.twimg.com/media/Fi7DNj9WICAf_Tu?format=jpg&name=medium

We assumed that programming required intelligence. Some parts of developer's job is just dumb if you know the rules.

Wonder if there would be any difference in score if the tabular information was translated into textual content

FWIW, it seems to understand plantuml.

Input

here's some plantuml
@startuml
    
    participant "app: Application" as app
    participant "cm: ContentManager" as cm
    participant "item: DownloadItem" as item
    
    activate app
    activate cm
    
    note over app: User enters media info page
    
    note over app: Check if item exists
    app->cm: findItem(itemId)
    cm->cm: lookup(itemId)
    
    alt item found
        cm-->app: item
    else not found
        cm-->app: null
        app->cm: createItem(itemId, contentURL)
        cm->item: new(itemId, contentURL)
        activate item
        cm-->app: item
    
        app->cm: loadMetadata()
        note over cm
            Download and parse manifest, save in db
        end note
        cm-->app: onTracksAvailable
        cm-->app: onDownloadMetadata
        note over app: * See //track-selection// flow
    end group
    
    note over app: app is ready to start downloading
    app->item: startDownload()
    
    
    @enduml

Response:
This plantuml diagram describes a sequence of events in which the application interacts with a ContentManager and a DownloadItem. When the user enters the media info page, the app checks to see if the item already exists. If it does, the app retrieves the item from the ContentManager. If the item does not exist, the app creates a new DownloadItem with the specified itemId and contentURL, and then loads the metadata. Once the metadata is loaded, the app is ready to start downloading the item.

A minor note on the following quote in README.md:
"Question 1, point 6: initializes a new Game each time instead of calling methods on this (this works, and actually seems a little more sensible, but the text of the scoring guidelines seem to imply I should dock points here)."

This doesn't quite work due to inheritance. A subclass of Game, say Chess, with an overridden play() method would not have its playManyTimes(int) method work as intended (if theplayManyTimes(int) was not overriden).