Index: pages/changes.in ================================================================== --- pages/changes.in +++ pages/changes.in @@ -44,15 +44,19 @@
  • Lines that begin with "#" and that are not in the middle of an SQL statement are interpreted as comments.
  • Added the --append option to the ".backup" command.
  • Added the ".dbconfig" command.

    Performance: -

  • [UPDATE] avoids writing database pages that do not actually change. - For example, "UPDATE t1 SET x=25 WHERE y=?" becomes a no-op if the - value in column x is already 25. Similarly, - when doing [UPDATE] on records that span multiple pages, only write - the subset of pages that contain the changed value(s). +
  • [UPDATE] avoids unnecessary low-level disk writes when the contents + of the database file do not actually change. + For example, "UPDATE t1 SET x=25 WHERE y=?" generates no extra + disk I/O if the value in column x is already 25. Similarly, + when doing [UPDATE] on records that span multiple pages, only + the subset of pages that actually change are written to disk. + This is a low-level performance optimization only and does not + affect the behavior of TRIGGERs or other higher level SQL + structures.
  • Queries that use ORDER BY and LIMIT now try to avoid computing rows that cannot possibly come in under the LIMIT. This can greatly improve performance of ORDER BY LIMIT queries, especially when the LIMIT is small relative to the number of unrestricted output rows.
  • The [OR optimization] is allowed to proceed Index: pages/compile.in ================================================================== --- pages/compile.in +++ pages/compile.in @@ -1035,28 +1035,28 @@ to the sorter. But in some cases, for example if some of the output columns consists of large BLOB values, the size of the each record can be large, which means that the sorter has to either use more memory, and/or write more content to temporary storage.

    - When SQLITE_ENABLE_SORTER_PREFERENCES is enabled, the records passed to + When SQLITE_ENABLE_SORTER_REFERENCES is enabled, the records passed to the sorter often contain only a [ROWID] value. Such records are much smaller. This means the sorter has much less "payload" to deal with and can run faster. After sorting has occurred, the ROWID is used to look up the output column values in the original table. That requires another search into the table, and could potentially result in a slowdown. Or, it might be a performance win, depending on how large the values are.

    - Even when the SQLITE_ENABLE_SORTER_PREFERENCES compile-time option is on, + Even when the SQLITE_ENABLE_SORTER_REFERENCES compile-time option is on, sorter references are still disabled by default. To use sorter references, the application must set a sorter reference size threshold using the [sqlite3_config]([SQLITE_CONFIG_SORTERREF_SIZE]) interface at start-time.

    Because the SQLite developers do not know whether the - SQLITE_ENABLE_SORTER_PERFERENCES option will help or hurt performance, + SQLITE_ENABLE_SORTER_REFERENCES option will help or hurt performance, it is disabled by default at this time (2018-05-04). It might be enabled by default in some future release, depending on what is learned about its - inpact on performance. + impact on performance. } COMPILE_OPTION {SQLITE_ENABLE_STMT_SCANSTATUS} { This option enables the [sqlite3_stmt_scanstatus()] interface. The [sqlite3_stmt_scanstatus()] interface is normally omitted from the build Index: pages/fileformat2.in ================================================================== --- pages/fileformat2.in +++ pages/fileformat2.in @@ -14,11 +14,14 @@ contained in a single file on disk called the "main database file".

    During a transaction, SQLite stores additional information in a second file called the "rollback journal", or if SQLite is in [WAL mode], a write-ahead log file. -If the application or + +hd_fragment hotjrnl {hot journal} {hot journal files} +

    Hot Journals

    +

    If the application or host computer crashes before the transaction completes, then the rollback journal or write-ahead log contains information needed to restore the main database file to a consistent state. When a rollback journal or write-ahead log contains information necessary for recovering the state of the database, they are called a "hot journal" or "hot WAL file". Index: pages/howtocorrupt.in ================================================================== --- pages/howtocorrupt.in +++ pages/howtocorrupt.in @@ -71,24 +71,43 @@ hd_fragment delhotjrnl {deleting a hot journal}

    Deleting a hot journal

    SQLite normally stores all content in a single disk file. However, -while performing a transaction, information necessary to roll back that -transaction following a crash or power failure is stored in auxiliary -journal files. These journal files have the same name as the +while performing a transaction, information necessary to recover the +database following a crash or power failure is stored in auxiliary +journal files. Such journal files are described as [hot journal|"hot"]. +The journal files have the same name as the original database file with the addition of -journal or -wal suffix.

    SQLite must see the journal files in order to recover from a crash -or power failure. If the journal files are moved, deleted, or renamed +or power failure. If the [hot journal files] are moved, deleted, or renamed after a crash or power failure, then automatic recovery will not work and the database may go corrupt.

    Another manifestation of this problem is [database corruption caused by inconsistent use of 8+3 filenames].

    +hd_fragment roguejrnl {separating a database from its journal} +

    Mispairing database files and hot journals

    + +

    The previous example is a specific case of a more general problem: +The state of an SQLite database is controlled by both the +database file and the journal file. In a quiescent state, the journal +file does not exist and only the database file matters. +But if the journal file does exist, it must be +kept together with the database to avoid corruption. The following +actions are all likely to lead to corruption: +

    File locking problems

    SQLite uses file locks on the database file, and on the [write-ahead log] or [WAL] file, to coordinate access between concurrent Index: pages/lang.in ================================================================== --- pages/lang.in +++ pages/lang.in @@ -3444,10 +3444,14 @@ updated if the validDate for the newly inserted value is newer than the entry already in the table. If the table already contains an entry with the same name and a current validDate, then the WHERE clause causes the DO UPDATE to become a no-op. +

    Limitations

    + +

    UPSERT does not currently work for [virtual tables]. + ############################################################################## Section {ON CONFLICT clause} conflict {{conflict clause} {ON CONFLICT}} @@ -3857,11 +3861,11 @@ WHERE ancestor_of_alice.name=family.name AND died IS NULL ORDER BY born; -hd_fragment rcex2 +hd_fragment rcex3

    Queries Against A Graph

    A version control system (VCS) will typically store the evolving versions of a project as a directed acyclic graph (DAG). Call each version of the project a "checkin". A single Index: pages/lockingv3.in ================================================================== --- pages/lockingv3.in +++ pages/lockingv3.in @@ -190,12 +190,11 @@ is participating in the current transaction) no master journal is created and the normal rollback journal contains an empty string in the place normally reserved for recording the name of the master journal.

    -hd_fragment hotjrnl {hot journal} -

    A rollback journal is said to be hot +

    A rollback journal is said to be [hot journal|hot] if it needs to be rolled back in order to restore the integrity of its database. A hot journal is created when a process is in the middle of a database update and a program or operating system crash or power failure prevents the update from completing. Index: pages/prosupport.in ================================================================== --- pages/prosupport.in +++ pages/prosupport.in @@ -65,18 +65,18 @@ {{Buy Now!} https://www.hwaci.com/cgi-bin/sams-step1} support_entry 3 {Technical Support} { High-priority email and phone support directly from the SQLite developers. Guaranteed response time available as an option. -} {$8K-$35K/year} \ +} {$8K-50K/year} \ {{More Info} https://www.hwaci.com/sw/sqlite/tech.html} \ {{Request A Quote} https://www.hwaci.com/sw/sqlite/contact.html} support_entry 4 {SQLite Consortium Membership} { Premium enterprise support including on-site visits and access to all proprietary extensions and test suites. -} {$75K/year} \ +} {$85K/year} \ {{More Info} https://www.hwaci.com/sw/sqlite/member.html} \ {{Request A Quote} https://www.hwaci.com/sw/sqlite/contact.html} generate_support_html Index: pages/vtab.in ================================================================== --- pages/vtab.in +++ pages/vtab.in @@ -263,15 +263,15 @@ struct sqlite3_module { int iVersion; int (*xCreate)(sqlite3*, void *pAux, - int argc, char **argv, + int argc, char *const*argv, sqlite3_vtab **ppVTab, char **pzErr); int (*xConnect)(sqlite3*, void *pAux, - int argc, char **argv, + int argc, char *const*argv, sqlite3_vtab **ppVTab, char **pzErr); int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*); int (*xDisconnect)(sqlite3_vtab *pVTab); int (*xDestroy)(sqlite3_vtab *pVTab); @@ -352,11 +352,11 @@ hd_fragment xcreate {sqlite3_module.xCreate} {xCreate}

    The xCreate Method

    int (*xCreate)(sqlite3 *db, void *pAux, - int argc, char **argv, + int argc, char *const*argv, sqlite3_vtab **ppVTab, char **pzErr);

    The xCreate method is called to create a new instance of a virtual table @@ -551,11 +551,11 @@ hd_fragment xconnect {sqlite3_module.xConnect} {xConnect}

    The xConnect Method

    int (*xConnect)(sqlite3*, void *pAux, - int argc, char **argv, + int argc, char *const*argv, sqlite3_vtab **ppVTab, char **pzErr);

    The xConnect method is very similar to [xCreate].