Benchmarks with Pydantic Evals#
Companion notebooks for the YouTube video.
Test a model’s Python skills from Jupyter, with inference running locally on Android through PythonHere. Use Pydantic Evals to run a Python quiz, check answers, and report results, then build custom evaluators for code-reasoning tasks and programming challenges. For longer experiments, Task Runner generates answers in the background on Android so you can fetch and evaluate them later.