Skip to content

Releases: yethee/tiktoken-php

0.7.0

12 Oct 17:49
16fa104
Compare
Choose a tag to compare

Added

  • Add support o1 models (#18, #19)
  • Add support chatgpt-4o models

0.6.0: Add support embedding models

22 Aug 13:03
780b874
Compare
Choose a tag to compare

Added

  • Added support for models:
    • babbage-002
    • davinci-002
    • text-embedding-3-small
    • text-embedding-3-large

0.5.1

17 Jul 10:14
94a4ac1
Compare
Choose a tag to compare

Fixed

  • Fix invalidate cache of vocab when the hash is mismatch.

0.5.0

14 May 11:39
ab25c89
Compare
Choose a tag to compare

Added

  • Added support for encoding o200k_base (GPT-4o) (#14)
  • Invalidate cache of vocab by checksum

Changed

  • [BC break] Changed signature of method VocabLoader::load

    - public function load(string $uri): Vocab;
    + public function load(string $uri, string|null $checksum = null): Vocab;

0.4.0: Improve performance

30 Apr 14:35
f512721
Compare
Choose a tag to compare

Changed

  • Improved performance of converting text into tokens (PR #10)

Removed

  • [BC break] Removed method EncodeUtil::fromBytes()

0.3.0

10 Jan 10:54
c84f066
Compare
Choose a tag to compare

Added

  • Added Encoder::encodeInChunks() method, which allows encode a text in chunks.

0.2.0: Use cache by default to avoid performance issues

17 Sep 13:53
d718336
Compare
Choose a tag to compare

Cache is enabled by default to avoid performance issues (see #2 and #5)