Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch translit-tokenizer Excluding Merge-Ins
This is equivalent to a diff from 1e51bffe77 to 7b6de5c35d
2012-05-04
| ||
13:22 | Minor tweaks to the hebrew transliteration tables. (Leaf check-in: 7b6de5c35d user: drh tags: translit-tokenizer) | |
02:58 | Add an experimental tokenizer to FTS3/4: one that transliterates latin, greek, cyrillic, and hebrew characters into pure ascii. (check-in: 930115693a user: drh tags: translit-tokenizer) | |
2012-05-01
| ||
14:21 | Update a test in io.test to account for [05f98d4eec]. (check-in: bfa61e781c user: dan tags: trunk) | |
2012-04-27
| ||
16:38 | Fix a minor deviation from the coding style guidelines. (check-in: 1e51bffe77 user: drh tags: trunk) | |
01:09 | Enhance the processing of ORDER BY clauses on compound queries to better match terms of the order by against expressions in the result set, in order to enable better query optimization. (check-in: a49e909c87 user: drh tags: trunk) | |
Changes to Makefile.in.
︙ | |||
161 162 163 164 165 166 167 | 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 | - - + + + | # Object files for the SQLite library (non-amalgamation). # LIBOBJS0 = alter.lo analyze.lo attach.lo auth.lo \ backup.lo bitvec.lo btmutex.lo btree.lo build.lo \ callback.lo complete.lo ctime.lo date.lo delete.lo \ expr.lo fault.lo fkey.lo \ |
︙ | |||
313 314 315 316 317 318 319 320 321 322 323 324 325 326 | 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 | + | $(TOP)/ext/fts3/fts3_hash.h \ $(TOP)/ext/fts3/fts3_icu.c \ $(TOP)/ext/fts3/fts3_porter.c \ $(TOP)/ext/fts3/fts3_snippet.c \ $(TOP)/ext/fts3/fts3_tokenizer.h \ $(TOP)/ext/fts3/fts3_tokenizer.c \ $(TOP)/ext/fts3/fts3_tokenizer1.c \ $(TOP)/ext/fts3/fts3_tokenizer2.c \ $(TOP)/ext/fts3/fts3_write.c SRC += \ $(TOP)/ext/icu/sqliteicu.h \ $(TOP)/ext/icu/icu.c SRC += \ $(TOP)/ext/rtree/rtree.h \ $(TOP)/ext/rtree/rtree.c |
︙ | |||
851 852 853 854 855 856 857 858 859 860 861 862 863 864 | 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 | + + + | $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_snippet.c fts3_tokenizer.lo: $(TOP)/ext/fts3/fts3_tokenizer.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenizer.c fts3_tokenizer1.lo: $(TOP)/ext/fts3/fts3_tokenizer1.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenizer1.c fts3_tokenizer2.lo: $(TOP)/ext/fts3/fts3_tokenizer2.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_tokenizer2.c fts3_write.lo: $(TOP)/ext/fts3/fts3_write.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/fts3/fts3_write.c rtree.lo: $(TOP)/ext/rtree/rtree.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c $(TOP)/ext/rtree/rtree.c |
︙ |
Changes to ext/fts3/README.tokenizers.
1 2 3 4 5 6 7 8 9 10 11 12 13 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 | - + | 1. FTS3 Tokenizers When creating a new full-text table, FTS3 allows the user to select the text tokenizer implementation to be used when indexing text by specifying a "tokenize" clause as part of the CREATE VIRTUAL TABLE statement: CREATE VIRTUAL TABLE <table-name> USING fts3( <columns ...> [, tokenize <tokenizer-name> [<tokenizer-args>]] ); The built-in tokenizers (valid values to pass as <tokenizer name>) are |
︙ |
Changes to ext/fts3/fts3.c.
︙ | |||
3539 3540 3541 3542 3543 3544 3545 | 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 3566 3567 3568 3569 3570 3571 3572 3573 3574 3575 3576 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 | - - - - + + + + + + + + + | static void hashDestroy(void *p){ Fts3Hash *pHash = (Fts3Hash *)p; sqlite3Fts3HashClear(pHash); sqlite3_free(pHash); } /* |
︙ |
Added ext/fts3/fts3_tokenizer2.c.