Function llamapun::util::data_helpers::heading_from_node_aux
source · [−]pub fn heading_from_node_aux(
node: RoNode,
tokenizer: &Tokenizer,
context: &mut Context
) -> Option<String>
Expand description
Provides a string for a given heading node, using DNM-enabled word-tokenization TODO: This is a low-level auxiliary function, we may need to build more user-facing interfaces if it becomes more widely useful