Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch branch-3.8.10 Excluding Merge-Ins
This is equivalent to a diff from d33510f222 to a3a3160989
2015-07-01
| ||
17:34 | Fix a typo in the zero-malloc memory allocator documentation. (Leaf check-in: a3a3160989 user: drh tags: branch-3.8.10) | |
17:34 | Fix a typo in the zero-malloc memory allocator documentation. (check-in: fb40525c2c user: drh tags: trunk) | |
2015-06-26
| ||
19:17 | Make search.tcl more robust in the face of malformed MATCH queries. (check-in: f2337fb9a2 user: drh tags: branch-3.8.10) | |
2015-05-15
| ||
14:53 | Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. (check-in: 643e08dfb8 user: drh tags: branch-3.8.10) | |
2015-05-11
| ||
19:12 | Update fts3.html with a description of the matchinfo 'b' option. (check-in: 2dad241898 user: dan tags: trunk) | |
17:09 | Updates to the dbstat documentation to explain how it can be used to get information about attached databases other than "main". (check-in: d33510f222 user: drh tags: trunk) | |
16:03 | Fix a typo on the mostdeployed.html page. (check-in: 7da9280ceb user: drh tags: trunk) | |
Changes to pages/changes.in.
︙ | ︙ | |||
14 15 16 17 18 19 20 21 22 23 24 25 26 27 | set nChng 0 proc chng {date desc {options {}}} { global nChng aChng set aChng($nChng) [list $date $desc $options] incr nChng } chng {2015-05-09 (3.8.10.1)} { <li>Make [sqlite3_compileoption_used()] responsive to the [SQLITE_ENABLE_DBSTAT_VTAB] compile-time option. <li>Fix a harmless warning in the [command-line shell] on some versions of MSVC. <li>Fix minor issues with the [dbstat virtual table]. <li>SQLITE_SOURCE_ID: "2015-05-09 12:14:55 05b4b1f2a937c06c90db70c09890038f6c98ec40" | > > > > > > > > > > > | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | set nChng 0 proc chng {date desc {options {}}} { global nChng aChng set aChng($nChng) [list $date $desc $options] incr nChng } chng {2015-05-20 (3.8.10.2)} { <li>Fix an index corruption issue introduced by [version 3.8.7]. An index with a TEXT key can be corrupted by an [INSERT] into the corresponding table if the table has two nested triggers that convert the key value to INTEGER and back to TEXT again. Ticket [https://www.sqlite.org/src/info/34cd55d68e0e6e7c9a0711aab81a2ee3c354b4c0|34cd55d68e0] <li>SQLITE_SOURCE_ID: "2015-05-20 18:17:19 2ef4f3a5b1d1d0c4338f8243d40a2452cc1f7fe4" <li>SHA1 for sqlite3.c: 638abb77965332c956dbbd2c8e4248e84da4eb63 } {patchagainst 1 patchagainst 2} chng {2015-05-09 (3.8.10.1)} { <li>Make [sqlite3_compileoption_used()] responsive to the [SQLITE_ENABLE_DBSTAT_VTAB] compile-time option. <li>Fix a harmless warning in the [command-line shell] on some versions of MSVC. <li>Fix minor issues with the [dbstat virtual table]. <li>SQLITE_SOURCE_ID: "2015-05-09 12:14:55 05b4b1f2a937c06c90db70c09890038f6c98ec40" |
︙ | ︙ |
Changes to pages/fileformat2.in.
︙ | ︙ | |||
654 655 656 657 658 659 660 | are the more significant and bits taken from the later bytes. </p> <p>The format of a cell depends on which kind of b-tree page the cell appears on. The following table shows the elements of a cell, in order of appearance, for the various b-tree page types.</p> <blockquote><dl> | | | | | | | | | | 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 | are the more significant and bits taken from the later bytes. </p> <p>The format of a cell depends on which kind of b-tree page the cell appears on. The following table shows the elements of a cell, in order of appearance, for the various b-tree page types.</p> <blockquote><dl> <dt><p>Table B-Tree Leaf Cell (header 0x0d):</p></dt> <dd><p><ul> <li>A varint which is the total number of bytes of payload, including any overflow <li>A varint which is the integer key, a.k.a. "[rowid]" <li>The initial portion of the payload that does not spill to overflow pages. <li>A 4-byte big-endian integer page number for the first page of the overflow page list - omitted if all payload fits on the b-tree page. </ul></p></dd> <dt><p>Table B-Tree Interior Cell (header 0x05):</p></dt> <dd><p><ul> <li>A 4-byte big-endian page number which is the left child pointer. <li>A varint which is the integer key </ul></p></dd> <dt><p>Index B-Tree Leaf Cell (header 0x0a):</p></dt> <dd><p><ul> <li>A varint which is the total number of bytes of key payload, including any overflow <li>The initial portion of the payload that does not spill to overflow pages. <li>A 4-byte big-endian integer page number for the first page of the overflow page list - omitted if all payload fits on the b-tree page. </ul></p></dd> <dt><p>Index B-Tree Interior Cell (header 0x02):</p></dt> <dd><p><ul> <li>A 4-byte big-endianpage number which is the left child pointer. <li>A varint which is the total number of bytes of key payload, including any overflow <li>The initial portion of the payload that does not spill to overflow pages. <li>A 4-byte big-endian integer page number for the first page of the overflow page list - omitted if all payload fits on the b-tree page. </ul></p></dd> </dl></blockquote> <p>The information above can be recast into a table format as follows:</p> <tcl>hd_fragment cellformat {cell format summary}</tcl> <center> <i>B-tree Cell Format</i> <table border=1 width="80%"> <tr><th rowspan=2>Datatype <th colspan=4>Appears in... <th rowspan=2>Description <tr><th>Table Leaf (0x0d) <th>Table Interior (0x05) <th>Index Leaf (0x0a) <th>Index Interior (0x02) <tr><td align=center valign=top>4-byte integer <td align=center valign=top> <td align=center valign=top>✔ <td align=center valign=top> <td align=center valign=top>✔ <td align=left>Page number of left child <tr><td align=center valign=top>varint |
︙ | ︙ |
Changes to pages/fts5.in.
︙ | ︙ | |||
159 160 161 162 163 164 165 | Within an FTS expression a <b>string</b> may be specified in one of two ways: <ul> <li> <p>By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character. | < | > | | < > > > > > > | | < | | 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | Within an FTS expression a <b>string</b> may be specified in one of two ways: <ul> <li> <p>By enclosing it in double quotes ("). Within a string, any embedded double quote characters may be escaped SQL-style - by adding a second double-quote character. <li> <p>As an FTS5 bareword that is not "AND", "OR" or "NOT" (case sensitive). An FTS5 bareword is a string of one or more consecutive characters that are all either: <ul> <li> Non-ASCII range characters (i.e. unicode codepoints greater than 127), or <li> One of the 52 upper and lower case ASCII characters, or <li> One of the 10 decimal digit ASCII characters, or <li> The underscore character (unicode codepoint 96). </ul> Strings that include any other characters must be quoted. </ul> <p> FTS queries are made up of <b>phrases</b>. A phrase is an ordered list of one or more tokens. A string is transformed into a phrase by passing it to the FTS table tokenizer. Two phrases can be concatenated into a single large phrase using the "+" operator. For example, assuming the tokenizer |
︙ | ︙ |
Changes to pages/index.in.
︙ | ︙ | |||
68 69 70 71 72 73 74 | </td> <td width="20"></td><td bgcolor="#044a64" width="1"></td><td width="20"></td> <td valign="top"> <h3>Current Status</h3> <p><ul> | | | > | 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | </td> <td width="20"></td><td bgcolor="#044a64" width="1"></td><td width="20"></td> <td valign="top"> <h3>Current Status</h3> <p><ul> <li><a href="releaselog/3_8_10_2.html">Version 3.8.10.2</a> of SQLite is recommended for all new development. Upgrading from all prior releases is recommended. </li> </ul></p> <h3>Common Links</h3> <p><ul> <li> <a href="features.html">Features</a> </li> |
︙ | ︙ |
Changes to pages/lang.in.
︙ | ︙ | |||
400 401 402 403 404 405 406 | Section {ATTACH DATABASE} attach {attached *ATTACH} RecursiveBubbleDiagram attach-stmt </tcl> <p> ^The ATTACH DATABASE statement adds another database file to the current [database connection]. | > > > | | 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 | Section {ATTACH DATABASE} attach {attached *ATTACH} RecursiveBubbleDiagram attach-stmt </tcl> <p> ^The ATTACH DATABASE statement adds another database file to the current [database connection]. Database files that were previously attached can be removed using the [DETACH DATABASE] command. <p>^The filename for the database to be attached is the value of the expression that occurs before the AS keyword. ^The filename of the database follows the same semantics as the filename argument to [sqlite3_open()] and [sqlite3_open_v2()]; the special name "[:memory:]" results in an [in-memory database] and an empty string results in a new temporary database. ^The filename argument can be a [URI filename] if URI filename processing is enable on the database connection. The default behavior is for |
︙ | ︙ |
Changes to pages/malloc.in.
︙ | ︙ | |||
358 359 360 361 362 363 364 | <h4>3.1.5 Experimental memory allocators</h4> <p>The name "memsys5" used for the zero-malloc memory allocator implies that there are several additional memory allocators available, and indeed there are. The default memory allocator is "memsys1". The debugging memory allocator is "memsys2". Those have already been covered.</p> | | | 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 | <h4>3.1.5 Experimental memory allocators</h4> <p>The name "memsys5" used for the zero-malloc memory allocator implies that there are several additional memory allocators available, and indeed there are. The default memory allocator is "memsys1". The debugging memory allocator is "memsys2". Those have already been covered.</p> <p>If SQLite is compiled with [SQLITE_ENABLE_MEMSYS3] then another zero-malloc memory allocator, similar to memsys5, is included in the source tree. The memsys3 allocator, like memsys5, must be activated by a call to [sqlite3_config]([SQLITE_CONFIG_HEAP],...). Memsys3 uses the memory buffer supplied as its source for all memory allocations. The difference between memsys3 and memsys5 is that memsys3 uses a different memory allocation algorithm that seems to work well in practice, but which does not provide mathematical |
︙ | ︙ |
Changes to pages/news.in.
︙ | ︙ | |||
14 15 16 17 18 19 20 21 22 23 24 25 26 27 | hd_puts "<h3>$date - $title</h3>" regsub -all "\n( *\n)+" $text "</p>\n\n<p>" txt regsub -all {[Tt]icket #(\d+)} $txt \ {<a href="http://www.sqlite.org/cvstrac/tktview?tn=\1">\0</a>} txt hd_resolve "<blockquote>$txt</blockquote>" hd_puts "<hr width=\"50%\">" } newsitem {2015-05-09} {Release 3.8.10.1} { <p>The 3.8.10 release did not add the new [SQLITE_ENABLE_DBSTAT_VTAB] compile-time option to the [sqlite3_compileoption_used()] interface. This patch release fixes that omission. And while we are at it, the associated [dbstat virtual table] was enhanced slightly and a harmless compiler warning was fixed. | > > > > > > > > > > > > > > > > | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | hd_puts "<h3>$date - $title</h3>" regsub -all "\n( *\n)+" $text "</p>\n\n<p>" txt regsub -all {[Tt]icket #(\d+)} $txt \ {<a href="http://www.sqlite.org/cvstrac/tktview?tn=\1">\0</a>} txt hd_resolve "<blockquote>$txt</blockquote>" hd_puts "<hr width=\"50%\">" } newsitem {2015-05-20} {Release 3.8.10.2} { <p>Yikes! Index corruption after a sequence of valid SQL statements! <p>It has been many years since anything like [https://www.sqlite.org/src/info/34cd55d6|this bug] has snuck into an official SQLite release. But for the pasts seven months ([version 3.8.7] through [version 3.8.10.1]) if you do an INSERT into a carefully crafted schema in which there are two nested triggers that convert an index key value from TEXT to INTEGER and then back to TEXT again, the INTEGER value might get inserted as the index key instead of the correct TEXT, resulting in index corruption. This patch release adds a single line of code to fix the problem. <p>If you do actually encounter this problem, running [REINDEX] on the damaged indexes will clear it. } newsitem {2015-05-09} {Release 3.8.10.1} { <p>The 3.8.10 release did not add the new [SQLITE_ENABLE_DBSTAT_VTAB] compile-time option to the [sqlite3_compileoption_used()] interface. This patch release fixes that omission. And while we are at it, the associated [dbstat virtual table] was enhanced slightly and a harmless compiler warning was fixed. |
︙ | ︙ |
Changes to pages/pragma.in.
︙ | ︙ | |||
1536 1537 1538 1539 1540 1541 1542 | } } </tcl> </ul></td></tr></table> <p>Notes: <ol> <li>Pragmas whose names are marked through in the list above | > | | 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 | } } </tcl> </ul></td></tr></table> <p>Notes: <ol> <li>Pragmas whose names are marked through in the list above are deprecated. They are not maintained. They continue to exist for historical compatibility only. Do not use the deprecated pragmas in new applications. Remove deprecated pragmas from existing applications at your earliest opportunity.</blockquote> <li>These pragmas are used for debugging SQLite and are only available when SQLite is compiled using [SQLITE_DEBUG]. <li>These pragmas are used for testing SQLite and are not recommended for use in application programs.</ol></p> |
︙ | ︙ |
Changes to pages/rescode.in.
︙ | ︙ | |||
550 551 552 553 554 555 556 | <li> Process A starts a read transaction on the database and does one or more SELECT statement. Process A keeps the transaction open. <li> Process B updates the database, changing values previous read by process A. <li> Process A now tries to write to the database. But process A's view of the database content is now obsolete because process B has modified the database file after process A read from it. Hence | | | 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 | <li> Process A starts a read transaction on the database and does one or more SELECT statement. Process A keeps the transaction open. <li> Process B updates the database, changing values previous read by process A. <li> Process A now tries to write to the database. But process A's view of the database content is now obsolete because process B has modified the database file after process A read from it. Hence process A gets an SQLITE_BUSY_SNAPSHOT error. </ol> } RESCODE SQLITE_CANTOPEN_NOTEMPDIR {SQLITE_CANTOPEN | (1<<8)} { The SQLITE_CANTOPEN_NOTEMPDIR error code is no longer used. } RESCODE SQLITE_CANTOPEN_ISDIR {SQLITE_CANTOPEN | (2<<8)} { The SQLITE_CANTOPEN_ISDIR error code is an [ext-v-prim|extended error code] |
︙ | ︙ |
Changes to pages/testing.in.
︙ | ︙ | |||
668 669 670 671 672 673 674 | "-g -fprofile-arcs -ftest-coverage" and then the test program is run. Then "gcov -b" is run to generate a coverage report. The coverage report is verbose and inconvenient to read, so the gcov-generated report is processed using some simple scripts to put it into a more human-friendly format. This entire process is automated using scripts, of course. | | | | | | > > > | > > > > > > > | | 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 | "-g -fprofile-arcs -ftest-coverage" and then the test program is run. Then "gcov -b" is run to generate a coverage report. The coverage report is verbose and inconvenient to read, so the gcov-generated report is processed using some simple scripts to put it into a more human-friendly format. This entire process is automated using scripts, of course. <p>Note that running SQLite with gcov is not a test of SQLite — it is a test of the test suite. The gcov run does not test SQLite because the -fprofile-args and -ftest-coverage options cause the compiler to generate different code. The gcov run merely verifies that the test suite provides 100% branch test coverage. The gcov run is a test of the test - a meta-test. <p>After gcov has been run to verify 100% branch test coverage, then the test program is recompiled using delivery compiler options (without the special -fprofile-arcs and -ftest-coverage options) and the test program is rerun. This second run is the actual test of SQLite. <p>It is important to verify that the gcov test run and the second real test run both give the same output. Any differences in output indicate either the use of undefined or indeterminate behavior in the SQLite code (and hence a bug), or a bug in the compiler. Note that SQLite has, over the previous decade, encountered bugs in each of GCC, Clang, and MSVC. Compiler bugs, while rare, do happen, which is why it is so important to test the code in an as-delivered configuration. <tcl>hd_fragment thoughts1</tcl> <h3>7.6 Experience with full test coverage</h3> <p>The developers of SQLite have found that full coverage testing is an extremely effective method for locating and preventing bugs. Because every single branch instruction in SQLite core code is covered by test cases, the developers can be confident that changes made in one part of the code do not have unintended consequences in other parts of the code. The many new features and performance improvements that have been added to SQLite in recent years would not have been possible without the availability full-coverage testing.</p> <p>Maintaining 100% MC/DC is laborious and time-consuming. The level of effort needed to maintain full-coverage testing is probably not cost effective for a typical application. However, we think that full-coverage testing is justified for a [most widely deployed|very widely deployed] infrastructure library like SQLite, and especially for a database library which by its very nature "remembers" past mistakes. <tcl>hd_fragment dynamicanalysis</tcl> <h2>8.0 Dynamic Analysis</h2> <p>Dynamic analysis refers to internal and external checks on the SQLite code which are performed while the code is live and running. Dynamic analysis has proven to be a great help in maintaining the |
︙ | ︙ |
Changes to pages/th3.in.
︙ | ︙ | |||
31 32 33 34 35 36 37 | (Test coverage of the operating-system specific [VFSes] and extensions such as FTS and RTREE is less than 100%). </p></li> </ul> <p>TH3 was originally written for validation testing only, but has subsequently been used for development testing and debugging as well, and has proven very helpful in those roles. A full-coverage | | | | | | | | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | (Test coverage of the operating-system specific [VFSes] and extensions such as FTS and RTREE is less than 100%). </p></li> </ul> <p>TH3 was originally written for validation testing only, but has subsequently been used for development testing and debugging as well, and has proven very helpful in those roles. A full-coverage test takes less than five minutes on a workstation and hence serves as a fast regression test during day-to-day maintenance of the SQLite code base.</p> <h2>2.0 Operation</h2> <p>TH3 is a test program generator. The output of TH3 is a program implemented in C-code and intended to be linked against the SQLite library under test. The generated test program is compiled and run on the target platform in order to verify correct operation of SQLite on that platform.</p> <p>The inputs to TH3 are test modules written in C or SQL and small configuration files that determine how to initialize SQLite. The TH3 package includes over eleven hundred test modules and more than three dozen configuration files. New modules and configurations can be added to customize TH3 for specialized applications. Each time TH3 is run, it reads a subset of the available test modules and configuration files to generate a custom C program that performs all of the specified tests under all specified configurations. A complete test of SQLite normally involves running TH3 multiple times to generate multiple test programs covering different aspects of SQLite's operation, then linking all test programs against a common SQLite library and running them separately on the target platform. SQLite will be found to work if all test programs pass.</p> <p>There are no arbitrary limits in TH3. One could generate a single test program that contained all test modules and all configuration files. However, such a test program might be too large to deploy on embedded platforms. Hence, TH3 provides the ability to break the library of test modules up into smaller, more easily digested pieces.</p> <p>Each individual test module might contain dozens, hundreds, or thousands of separate tests. The test modules can be written in C or as scripts of SQL or a mixture of the two. About two-thirds of the existing test modules are |
︙ | ︙ | |||
117 118 119 120 121 122 123 | <p>Once the test program is generated, it is run with no arguments to perform the tests. Progress information as well as error diagnostics appear on standard output. (Alternative output arrangements can be made using a compile-time option for embedded devices that lack a standard output channel.) The program returns zero if there are no errors and non-zero if any problems were detected.</p> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > | > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > | 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 | <p>Once the test program is generated, it is run with no arguments to perform the tests. Progress information as well as error diagnostics appear on standard output. (Alternative output arrangements can be made using a compile-time option for embedded devices that lack a standard output channel.) The program returns zero if there are no errors and non-zero if any problems were detected.</p> <p>Typical output from a single TH3 test program run looks like this: <blockquote><pre> With SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e -DSQLITE_COVERAGE_TEST -DSQLITE_NO_SYNC -DSQLITE_SYSTEM_MALLOC -DSQLITE_THREADSAFE=1 Config-begin c1. Begin c1.pager08 End c1.pager08 Begin c1.build33 End c1.build33 Begin c1.orderby01 End c1.orderby01 <i>... 15014 lines of output omitted ....</i> Begin 64k.syscall01 End 64k.syscall01 Begin 64k.build01 End 64k.build01 Begin 64k.auth01 End 64k.auth01 Config-end 64k. TH3 memory used: 6373738 Config-begin wal1. Begin wal1.wal37 End wal1.wal37 Config-end wal1. TH3 memory used: 100961 All 226 VDBE coverage points reached th3: 0 errors out of 1442264 tests in 213.741 seconds. 64-bit little-endian th3: SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e </pre></blockquote> <p>The output begins with a report of the [SQLITE_SOURCE_ID] (cross-checked again [sqlite3_sourceid()]) for the SQLite under test and the compile-time options used as reported by [sqlite3_compileoption_get()]. The output concludes with a summary of the test results and a repeat of the [SQLITE_SOURCE_ID]. If any errors are detected, additional lines detail the problem. The error reporting lines always begin with a single space character so that they can be quickly extracted from large output files using: <blockquote><pre> grep "^ " </pre></blockquote> <p>The default output shows the beginning and end of each configuration and test module combination. In the example above "c1" and "64k" are configurations and "pager08", "build33", "orderby01", etc. are test modules. Compile-time and run-time options are available to increase or decrease the amount of output. The output can be increased by showing each test case within each test module. The output can be decreased by degrees: omitting test modules starts and stops, omitting configuration starts and stops, and finally by omitting all output. <h3>3.1 Test Automation Scripts</h3> <p>TH3 comes with additional TCL scripts that help automate the testing process on workstations. The "th3make" script automatically runs "mkth3.tcl" and "gcc" and then runs the resulting test program and checks the results. Arguments to th3make include all of the "*.test" test modules and "*.cfg" configurations that are to be included in the test. Additional options to th3make can cause the test program to be compiled using different compilers (GCC, Clang, MSVC), to use different output verbosity levels, to run the test program under valgrind, to check the output for coverage using gcov, and so forth. The th3make script also accepts "*.rc" filenames as arguments. These *.rc files are just collections of other arguments that are commonly used together for a single purpose. For example, the "quick.rc" file contains a set of eight arguments to th3make that run a fast (3-minute) full-coverage test. This allows the operator to type "./th3make quick.rc" as a short-cut to typing out all of the required command-line options. The following are a few of the more than 40 available *.rc files: <ul> <li><b>alignment</b><i>N</i><b>.rc</b> - These files contain -D options to the compiler that are used by various notable downstreams. <li><b>cov.rc</b> - Options for measuring test coverage <li><b>extensions.rc</b> - Options to enable [FTS4], [R-Trees], and [SQLITE_STAT4|STAT4]. <li><b>fast.rc</b> - Run mosts tests, including those not needed for coverage, skipping only soak tests, using delivery compiler options (ex: -O3) <li><b>memdebug.rc</b> - like test.rc but also enable [SQLITE_MEMDEBUG|-DSQLITE_MEMDEBUG]. <li><b>min.rc</b> - Run the minimum set of tests needed for 100% coverage. <li><b>quick.rc</b> - Run all tests required for 100% coverage tests using -Os and [SQLITE_DEBUG|-DSQLITE_DEBUG]. <li><b>test.rc</b> - Run the same tests as in fast.rc but without compiler optimization and enabling options like [SQLITE_DEBUG|-DSQLITE_DEBUG] and -DSQLITE_ENABLE_EXPENSIVE_ASSERT. <li><b>test-ex.rc</b> - long-running soak tests. </ul> <p>The TH3 repository also includes the "multitest.tcl" script, another TCL script used to automate TH3 testing on workstations. Multitest.tcl automatically compiles SQLite, then runs ./th3make repeatedly with a variety of alignments, and captures the output in a succinct summary screen. A typical multitest.tcl run generates output that looks like this: <blockquote><pre> file mkdir sqlite3bld cd sqlite3bld exec sh /home/drh/sqlite/sqlite/configure file copy -force config.h ../config.h exec make clean sqlite3.c file rename sqlite3.c ../sqlite3.c aa4f0f90c9c77424943e026a2ecee4a6c7f9e0d3 ../sqlite3.c file rename sqlite3.h ../sqlite3.h exec make clean sqlite3.c OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 file rename sqlite3.c ../sqlite3udl.c 0d3bbc92c433f940253bb2c7c19de7783133929d ../sqlite3udl.c exec make clean sqlite3.c OPTS=-DSQLITE_SMALL_STACK=1 file rename sqlite3.c ../sqlite3ss.c fcf6963e94096324461076d3b9e9dc1888e066e1 ../sqlite3ss.c cd .. ******************************************************************************* t01: quick.rc.................................................. Ok (00:04:00) t02: cov.rc.................................................... Ok (00:04:40) t03: quick.rc extensions.rc -D_HAVE_SQLITE_CONFIG_H............ Ok (00:05:22) t04: cov.rc -DSQLITE_ENABLE_STAT4 -D_HAVE_SQLITE_CONFIG_H...... Ok (00:05:20) t05: test.rc ../th3private/*.test.............................. Ok (00:00:17) t06: test.rc ../th3private/*.test -DSQLITE_ENABLE_STAT4........ Ok (00:00:43) t07: quick.rc -DSQLITE_TEST_REALLOC_STRESS -funsigned-char..... Ok (00:04:56) t08: quick.rc -DSQLITE_THREADSAFE=0 -fsigned-char.............. Ok (00:03:12) t09: quick.rc sqlite3ss.c -DSQLITE_MAX_ATTACHED=125............ Ok (00:04:04) t10: quick.rc -DSQLITE_RUNTIME_BYTEORDER....................... Ok (00:03:58) t11: quick.rc -DSQLITE_DIRECT_OVERFLOW_READ.................... Ok (00:04:01) t12: fast.rc................................................... Ok (00:14:19) t13: fast.rc alignment1.rc -m32................................ Ok (00:20:51) t14: fast.rc alignment2.rc sqlite3udl.c........................ Ok (00:16:06) t15: fast.rc alignment4.rc..................................... Ok (00:12:55) t16: fast.rc alignment5.rc..................................... Ok (00:14:58) t17: fast.rc alignment6.rc..................................... Ok (00:14:31) t18: fast.rc alignment7.rc..................................... Ok (00:16:06) t19: fast.rc alignment8.rc sqlite3udl.c........................ Ok (00:24:09) t20: test.rc alignment1.rc..................................... Ok (00:49:27) t21: test.rc alignment2.rc sqlite3udl.c........................ Ok (00:38:43) t22: test.rc alignment4.rc -m32 CC=clang....................... Ok (00:39:49) t23: test.rc alignment5.rc..................................... Ok (00:36:33) t24: test.rc alignment6.rc..................................... Ok (00:33:53) t25: test.rc alignment7.rc..................................... Ok (00:42:16) t26: test.rc alignment8.rc sqlite3udl.c........................ Ok (01:05:22) t27: memdebug.rc extensions.rc................................. Ok (01:35:56) t28: fast.rc -fsanitize=undefined.............................. Ok (00:15:09) t29: min.rc -O3 -valgrind...................................... Ok (01:26:10) t30: min.rc -O3 -valgrind extensions.rc........................ Ok (01:47:12) t31: test-ex.rc................................................ Ok (03:20:18) ******************************************************************************* 0 failures on 31 th3make runs and 166721387 tests in (16:25:29) SQLite 3.8.10 2015-05-05 18:52:54 04afa3febee32854fbb09ef8d4ffffd432119716 </pre></blockquote> <p>As can be seen above, a single run of multitest.tcl invokes th3make dozens times and takes between 12 and 24 hours. The middle section of the output shows the arguments to each individual th3make run and the result and elapse time for that th3make. All build products and output for the separate th3make runs are captures in subdirectories for post-test analysis. The two-line summary at the bottom shows the total number of errors and tests over all th3make runs and the total elapse time, together with the [SQLITE_SOURCE_ID] information for the version of SQLite that was tested. This summary information is recorded in the <a href="https://www.sqlite.org/checklists/3081000/index#c6">release checklist</a> during testing. <h2>4.0 Test Coverage</h2> <p>Using one particular subset of the available TH3 test modules (the "cov1" tests) SQLite obtained [test coverage | 100% branch test coverage] and 100% [MC/DC] as measured by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov] |
︙ | ︙ |
Changes to search/search.tcl.
︙ | ︙ | |||
192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 | # Count the '"' characters in $::A(q). If there is an odd number of # occurences, add a " to the end of the query so that fts3 can parse # it without error. if {[regexp -all \x22 $::A(q)] % 2} { append ::A(q) \x22 } set ::TITLE "Results for: \"[htmlize $::A(q)]\"" #db func rank rank #db func erank erank # If the user has clicked the "Lucky" button and the query returns one or # more results, redirect the browser to the highest ranked result. If the # query returns zero results, fall through and display the "No results" # page as if the user had clicked "Search". # if {[info exists ::A(s)] && $::A(s) == "Lucky"} { set url [db one { SELECT url FROM page, pagedata WHERE page MATCH $::A(q) AND page.docid = pagedata.docid ORDER BY rank(matchinfo(page), nk, nt, nc) DESC }] if {$url != ""} { cgi_redirect $url } } | > > > > > > > > > > < < < < | < > > | 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 | # Count the '"' characters in $::A(q). If there is an odd number of # occurences, add a " to the end of the query so that fts3 can parse # it without error. if {[regexp -all \x22 $::A(q)] % 2} { append ::A(q) \x22 } set ::TITLE "Results for: \"[htmlize $::A(q)]\"" # Set nRes to the total number of documents that the user's query matches. # set rc [catch { set nRes [db one { SELECT count(*) FROM page WHERE page MATCH $::A(q) }] }] if {$rc} { set ::A(q) "\"$::A(q)\"" set nRes [db one { SELECT count(*) FROM page WHERE page MATCH $::A(q) }] } #db func rank rank #db func erank erank # If the user has clicked the "Lucky" button and the query returns one or # more results, redirect the browser to the highest ranked result. If the # query returns zero results, fall through and display the "No results" # page as if the user had clicked "Search". # if {[info exists ::A(s)] && $::A(s) == "Lucky"} { set url [db one { SELECT url FROM page, pagedata WHERE page MATCH $::A(q) AND page.docid = pagedata.docid ORDER BY rank(matchinfo(page), nk, nt, nc) DESC }] if {$url != ""} { cgi_redirect $url } } # If nRes is 0, then the user's query returned zero results. Return a short # message to that effect. # if {$nRes == 0} { return [subst { No results for: <b>[htmlize $::A(q)]</b> }] } set score 0 catch {set score $::A(score)} # Set iStart to the index of the first result to display. Results are # indexed starting at zero from most to least relevant. # set iStart [expr {([info exists ::A(i)] ? $::A(i) : 0)*10}] # HTML markup used to highlight keywords within FTS3 generated snippets. |
︙ | ︙ |