09-10, 13:30–16:30 (Asia/Jerusalem), Reichman
Language: עברית
A very deep dive, keyboard-first, into the details of different approaches to ensure our code does what we think it does.
Part I - xUnit tests
- lets write a test
- choosing good test cases
- simple
- independent
- carve the problem space at the joints
- hug the seams
- advanced features:
- setup/teardown/fixture
- parametrization
- mocks
- putting it to practice
- war stories and design exercises
- things that are hard to test: UI, nondeterministic code
Part II - testing prompts with statistical tests (evaluations)
- lets use an LLM to process a freeform request
- how to think about "correctness"
- train/test split
- quality measures:
- deterministic measurements
- accuracy curve, false positives and false negatives
- human as judge
- strong LLM as judge
- using a weak LLM to approximate confidence
- production monitoring
Part III - other approaches
- BDD
- FIT
- golden/snapshot testing
- exercise: tailoring a testing approach for a deterministic chatbot
- my own contribution to the field: bless tests
- the basic technique
- designing good harnesses
- advanced blessing techniques
- using it in a team
- lessons learned applying it in the field
- exercise: testing a complex algorithm with the blesstest library
- final design exercise
Intermediate
Target audience –Developers, Testers/QA
Sometimes developer, sometimes CTO, sometimes consultant, sometimes investor. Always a technologists at heart, on a quest to improve the craft of writing software.