Skip to content

encoder.json/vocab.bpe show up in every project that uses SK #2679

@stephentoub

Description

@stephentoub

Bringing in the Microsoft.SemanticKernel nuget package causes these files to show up in the consuming application:
image
and they end up in the output directory for the application, regardless of whether the app is using the tokenizer or not.
image

I'd opened #1800 to turn them into assembly resources instead, so that they'd simply be part of the assembly and not separate files, but it was closed due to a lack of a decision about what to do with it.

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

Status

Sprint: Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions