tf.strings.unicode_split_with_offsets

Splits each string into a sequence of code points with start offsets.

This op is similar to tf.strings.decode(...), but it also returns the start offset for each character in its respective string. This information can be used to align the characters with the original byte sequence.

Returns a tuple (chars, start_offsets) where:

  • chars[i1...iN, j] is the substring of input[i1...iN] that encodes its jth character, when decoded using input_encoding.
  • start_offsets[i1...iN, j] is the start byte offset for the jth character in input[i1...iN], when decoded using input_encoding.

input An N dimensional potentially ragged string tensor with shape [D1...DN]. N must be statically known.
input_encoding String name for the unicode encoding that should be used to decode each string.
errors Specifies the response when an input string can't be converted using the indicated encoding. One of: