def preprocess_data(self, train_mode):
    """Main function where the preprocessing of the data starts.

    Args:
      train_mode: string, whether we are in train, dev or test mode
    """

    # Set the raw data directory and data.
    self.raw_data_dir = os.path.join('/'.join(self._data_dir.split('/')[:-1]),
                                     'raw_data')
    self.raw_data = os.path.join(self._raw_data_dir, 'ConvAI2')
    self.zipped_data = os.path.join(self._raw_data_dir, 'convai2.tar.gz')

    # Create the download url.
    self.url = 'http://parl.ai/downloads/convai2/convai2_fix_723.tgz'

    # Check at which part of the pipeline are we at.
    self.data_pipeline_status(train_mode)
Main function where the preprocessing of the data starts.

    Args:
      train_mode: string, whether we are in train, dev or test mode
    
tensor2tensor/data_generators/dialog_personachat.py