Setting up Autoscoring

Once you have an evaluator trained or chosen a pretrained evaluator, you can use the evaluator via our SDK. This lets you add the evaluator into your CI/CD pipeline, or your own manual tests.

Finding the unique identifier

Every evaluator has a unique identifier (codename) that you need to pass into the sdk. This codename can be found on the profile page. In this example, the code name is grade-3-math.

Example

autoscore.py

from pearwise import PearWise

# Initialise Pearwise API
api_key = "YOUR_API_KEY"
pear = PearWise(api_key)

user_input = "What is AI?"
output = "AI, or Artificial Intelligence, refers to the development of computer systems and software that can perform tasks that typically require human intelligence."

response = pear.autoscore(user_input, output, "grade-3-math")
print("Status Code:", response.status_code)
print("Response JSON:", response.text)