def predict_loop(self, model, eval_dataloader, output_file=None):
        """on-the-fly prediction on a single gpu."""
        self.full_scores = []
        model.eval()
        model = model.to(0)
        with torch.no_grad():
            for data in eval_dataloader:
                data = self.to_ctx(data)
                outputs = model(**data)
                outputs.update(data)
                self(outputs)
        return self.finalize(output_file)
on-the-fly prediction on a single gpu.
understand_icl/fairseq/examples/MMPT/mmpt/evaluators/predictor.py