Evaluation

#3
by islameissa - opened

Hi. First, Thank you very much for your contributions.
I didn't get the evaluation part that you posted and I would appreciate some more details.
The famous NVFP4-PTQ version of the model is the one by NVIDIA and in their model card it scored GPQA-D: 91.5
In your model card, away for less tokens, you mentioned that you QAD method scored higher than NVFP4 PTQ while the number is actually less = 89.9
Did you try both with the same harness, settings ... etc and you found that QAD NVFP4 scored better?
Thanks

Sign up or log in to comment