def build_tokens_types_paddings_from_text(row, tokenizer, max_seq_length):
    """Build token types and paddings, trim if needed, and pad if needed."""

    title_ids = tokenizer.tokenize(row['title'])
    context_ids = tokenizer.tokenize(row['text'])

    # Appending the title of the context at front
    extended_context_ids = title_ids + [tokenizer.sep_id] + context_ids

    context_ids, context_types, context_pad_mask = \
        build_tokens_types_paddings_from_ids(extended_context_ids, 
            max_seq_length, tokenizer.cls, tokenizer.sep, tokenizer.pad)

    return context_ids, context_types, context_pad_mask
Build token types and paddings, trim if needed, and pad if needed.
training/DeepSpeed-Domino/megatron/data/orqa_wiki_dataset.py