I mostly test local and AI models nowadays, but will try to also upload old scripts.
#Script: testinglocalmodel.py
##Is a little program I am using to check local models I downloaded via Ollama
Building a Test Matrix: To test the output of your local models, you can create a matrix that evaluates several criteria:
Accuracy: Does the content correctly describe the technical processes and concepts? Clarity: Is the information presented in an understandable and logical manner? Completeness: Does the output cover all aspects outlined in the input instructions? Language Proficiency: How well does the model handle the German language in terms of grammar, syntax, and vocabulary? Relevance: Is the content relevant to the topics of cybersecurity and technical writing? Formatting: Does the model correctly use markdown or other formatting as instructed? You can rate these criteria on a scale (e.g., 1-5) for each section generated by the model and then calculate an overall score for each document. This will help you assess the model's performance in these specific areas.