Spaces:

AIML-TUDA
/

VerifiableRewardsForScalableLogicalReasoning

Sleeping

Lukas Helff commited on Jun 25

Commit

0f1c352

1 Parent(s): 1fe4885

make eval config obligatory

Files changed (1) hide show

VerifiableRewardsForScalableLogicalReasoning.py CHANGED Viewed

@@ -261,7 +261,10 @@ class VerifiableRewardsForScalableLogicalReasoning(evaluate.Metric):
                 'predictions': datasets.Value('string'),
                 'references': {
                     'validation_program': datasets.Value('string'),
-                    'evaluation_config': datasets.Value("dict", id=None)
                 },
             }),
             codebase_urls=["https://github.com/AIML-TUDA/SLR-Bench"],

                 'predictions': datasets.Value('string'),
                 'references': {
                     'validation_program': datasets.Value('string'),
+                    'evaluation_config': {
+                        'positive_predicate': datasets.Value('string'),
+                        'negative_predicate': datasets.Value('string')
+                    }
                 },
             }),
             codebase_urls=["https://github.com/AIML-TUDA/SLR-Bench"],