Simple-Eval

Welcome to the simple LLM evaluation framework.

You can use simpleval to evaluate your LLMs with a simple and easy-to-use CLI.

🚀 Head over to Quickstart to get started.

---

What's In The Box? 🎁

The built-in evaluation uses the "LLM as a Judge" technique, implemented with various LLM providers—learn more about it here.

Why Simple-Eval?

It is dead simple and easily customizable.

Often using an evaluation platform is an overkill, and other frameworks did not match the simple requirements we had. And so, simpleval was born.

Further Reading 📚

If you want to deep dive into the "LLM As a Judge" evaluation technique, some relevant papers are:

• LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods

• A Survey on LLM-as-a-Judge

Happy reading!