Xin 14036ffea6
feat: enhance FlexSearch encoding for CJK support (#553)
- Added support for CJK (Chinese, Japanese, Korean) languages in FlexSearch encoding.
- Introduced `isCJK` function to detect language and select appropriate encoding method.
- Implemented `encodeCJK` and `encodeDefault` functions for different tokenization strategies.
2025-01-18 18:54:54 +00:00
..