Search results

Documentation: <|endoftext|> cannot be parsed unless encode function given extra parameters, this is not documented anywhere

Status: Open.

#155 In dqbd/tiktoken;

· aeisenberg opened on May 25, 2026

Add "bun" export condition so standalone bun compile can embed tiktoken_bg.wasm

Status: Open.

#154 In dqbd/tiktoken;

· CastIronPlatypus opened on Apr 19, 2026

OpenAI Harmony tokenzier

Status: Open.

#152 In dqbd/tiktoken;

· vladfaust opened on Nov 5, 2025

[Feature] Add benchmarks (wasm vs js implementation). Benchmark here

Status: Open.

#147 In dqbd/tiktoken;

· kravetsone opened on May 23, 2025

Not compatable with iife due to top-level awaits

Status: Open.

#145 In dqbd/tiktoken;

· douglasg14b opened on May 9, 2025

Fetch request inside library could not be completed from some locations

Status: Open.

#138 In dqbd/tiktoken;

· opravdin opened on Mar 12, 2025

not able to find: text-embedding-3-small | npm module | "^1.0.18"

Status: Open.

#127 In dqbd/tiktoken;

· MwSpaceLLC opened on Feb 1, 2025

recursive use of an object detected which would lead to unsafe aliasing in rust

Status: Open.

#119 In dqbd/tiktoken;

· mogery opened on Aug 22, 2024

React native error SyntaxError: Invalid RegExp: Invalid escape] in react native expo.

Status: Open.

#110 In dqbd/tiktoken;

· djaffer opened on Jul 16, 2024

missing example code for js-tiktoken

Status: Open.

#102 In dqbd/tiktoken;

· coding22 opened on May 9, 2024

Performance

Status: Open.

#99 In dqbd/tiktoken;

· steebchen opened on Apr 15, 2024

Add decode_with_offsets function (or PR guidance request)

Status: Open.

#94 In dqbd/tiktoken;

· Sam1320 opened on Mar 11, 2024