The Token Spectrum: From Words to Meanings in AI