Documentation Source Text

Changes On Branch 3.17
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch 3.17 Excluding Merge-Ins

This is equivalent to a diff from ebd7ef38cc to 1b710ef1fc

2017-03-20
12:26
Merge changes from the 3.17.0 release. (check-in: 3ff90dd03b user: drh tags: trunk)
12:25
Include a link to the draft change log on Pre-Release Snapshots on the download page. (Leaf check-in: 1b710ef1fc user: drh tags: 3.17)
2017-03-16
12:49
Mention the requirement of Fossil 2.0 in getthecode.html. (check-in: 36cbd238b5 user: drh tags: 3.17)
2017-03-06
19:56
Merge fixes from the 3.17 branch. (check-in: 02cadace0c user: drh tags: trunk)
2017-02-22
12:43
Fix typos in the documentation reported by Karol Swietlicki. (check-in: d24b5395d0 user: drh tags: 3.17)
2017-02-15
22:44
Fix a harmless typo in the rollback journal checksum algorithm description. (check-in: ebd7ef38cc user: drh tags: trunk)
22:43
Correct the rollback journal checksum algorithm description. (check-in: b69eaea914 user: drh tags: trunk)

Changes to document_header.tcl.

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
          <li><a href='${path}docs.html'>Documentation</a>
          <li><a href='${path}download.html'>Download</a>
          <li><a href='${path}support.html'>Support</a>
          <li><a href='${path}prosupport.html'>Purchase</a>
        </ul>
      </div>
      <div class="searchmenu" id="searchmenu">
        <form method="GET" action="search">
          <span class="desktoponly">Search for:</span> <input type="text" name="q">
          <input type="submit" value="Go">
        </form>
      </div>
    </div>
  }]








|







46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
          <li><a href='${path}docs.html'>Documentation</a>
          <li><a href='${path}download.html'>Download</a>
          <li><a href='${path}support.html'>Support</a>
          <li><a href='${path}prosupport.html'>Purchase</a>
        </ul>
      </div>
      <div class="searchmenu" id="searchmenu">
        <form method="GET" action="${path}search">
          <span class="desktoponly">Search for:</span> <input type="text" name="q">
          <input type="submit" value="Go">
        </form>
      </div>
    </div>
  }]

Changes to pages/compile.in.

41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
</tcl>

<h1>Recommended Compile-time Options</h1>

<p>The following compile-time options are recommended for applications that
are able to use them, in order to minimized the number of CPU cycles and
the bytes of memory used by SQLite.
Note all of these compile-time options are usable by every application.
For example, the SQLITE_THREADSAFE=0 option is only usable by applications
that never access SQLite from more than one thread at a time.  And the
SQLITE_OMIT_PROGESS_CALLBACK option is only usable by applications that
doe not use the [sqlite3_progress_handler()] interface.  And so forth.

<p>It is impossible to test every possible combination of compile-time
options for SQLite.  But the following set of compile-time options is







|







41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
</tcl>

<h1>Recommended Compile-time Options</h1>

<p>The following compile-time options are recommended for applications that
are able to use them, in order to minimized the number of CPU cycles and
the bytes of memory used by SQLite.
Not all of these compile-time options are usable by every application.
For example, the SQLITE_THREADSAFE=0 option is only usable by applications
that never access SQLite from more than one thread at a time.  And the
SQLITE_OMIT_PROGESS_CALLBACK option is only usable by applications that
doe not use the [sqlite3_progress_handler()] interface.  And so forth.

<p>It is impossible to test every possible combination of compile-time
options for SQLite.  But the following set of compile-time options is

Changes to pages/download.in.

99
100
101
102
103
104
105
106


107
108
109
110
111
112
113
set Caution #fff1c8

Heading {Pre-release Snapshots} {} $Caution


Product {snapshot/sqlite-snapshot-DATE.tar.gz} {
  The [amalgamation] source code, the [command-line shell] source code,
  configure/make scripts for unix, and a Makefile.msc for Windows.


}
Product {snapshot/sqlite-amalgamation-DATE.zip} {
  The [amalgamation]: complete source code a single "sqlite3.c" file.
}
#  The [amalgamation] as of VERSION.
#  See the <a href="http://www.sqlite.org/draft/releaselog/current.html">pending
#  change log</a> for details.







|
>
>







99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
set Caution #fff1c8

Heading {Pre-release Snapshots} {} $Caution


Product {snapshot/sqlite-snapshot-DATE.tar.gz} {
  The [amalgamation] source code, the [command-line shell] source code,
  configure/make scripts for unix, and a Makefile.msc for Windows. See the
  [https://www.sqlite.org/draft/releaselog/current.html|change log] for
  more information.
}
Product {snapshot/sqlite-amalgamation-DATE.zip} {
  The [amalgamation]: complete source code a single "sqlite3.c" file.
}
#  The [amalgamation] as of VERSION.
#  See the <a href="http://www.sqlite.org/draft/releaselog/current.html">pending
#  change log</a> for details.

Changes to pages/fileformat2.in.

1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
<p>^If M is -1 in the initial journal header, then the number of page records
that follow is computed by computing how many page records will fit in
the available space of the remainder of the journal file.</p>

<tcl>hd_fragment walformat {WAL format}</tcl>
<h1>The Write-Ahead Log</h1>

<p>Beginning with [version 3.7.0] (dateof:3.7.0), 
SQLite supports a new transaction
control mechanism called "[WAL | write-ahead log]" or "[WAL]".
^When a database is in WAL mode, all connections to that database must
use the WAL.  ^A particular database will use either a rollback journal
or a WAL, but not both at the same time.
^The WAL is always located in the same directory as the database
file and has the same name as the database file but with the string







|







1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
<p>^If M is -1 in the initial journal header, then the number of page records
that follow is computed by computing how many page records will fit in
the available space of the remainder of the journal file.</p>

<tcl>hd_fragment walformat {WAL format}</tcl>
<h1>The Write-Ahead Log</h1>

<p>Beginning with [version 3.7.0] ([dateof:3.7.0]), 
SQLite supports a new transaction
control mechanism called "[WAL | write-ahead log]" or "[WAL]".
^When a database is in WAL mode, all connections to that database must
use the WAL.  ^A particular database will use either a rollback journal
or a WAL, but not both at the same time.
^The WAL is always located in the same directory as the database
file and has the same name as the database file but with the string

Changes to pages/getthecode.in.

67
68
69
70
71
72
73
74
75










76
77

78
79
80
81
82

83
84
85
86
87
88
89
90
"ZIP archive" link to download the complete source tree.

<tcl>hd_fragment {clone} {clone the entire repository}</tcl>
<h1>Cloning The Complete Development History</h1>

<p>To clone the entire history of SQLite, first go to the
[http://www.fossil-scm.org/download.html] page and grab a precompiled binary
for the Fossil version control program.  (Or get the source code on the
same page and compile it yourself.)  Fossil is a completely stand-alone










program, so install it simply by putting the "fossil" or "fossil.exe"
executable someplace on your $PATH or %PATH%.  Then enter:


<codeblock>
fossil clone http://www.sqlite.org/cgi/src sqlite.fossil
</codeblock>


<p>That command will make a copy of the complete development history of
SQLite into the "sqlite.fossil" file on your computer.  Making this copy
takes about a minute and uses about 32 megabytes of transfer.  After
making the copy, "open" the repository by typing:

<codeblock>
fossil open sqlite.fossil
</codeblock>







|
|
>
>
>
>
>
>
>
>
>
>

|
>





>
|







67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
"ZIP archive" link to download the complete source tree.

<tcl>hd_fragment {clone} {clone the entire repository}</tcl>
<h1>Cloning The Complete Development History</h1>

<p>To clone the entire history of SQLite, first go to the
[http://www.fossil-scm.org/download.html] page and grab a precompiled binary
for the Fossil version control program.  Or get the source code on the
same page and compile it yourself.

<p>As of 2017-03-12, you must use Fossil version
2.0 or later for the following instructions to work.  
The SQLite repository started using
artifacts named using SHA3 hashes instead of SHA1 hashes on that date,
and Fossil 2.0 or later is needed in order to understand the new SHA3
hashes.  To find out what version of Fossil you are running, 
type "fossil -v".</p>

<p>Fossil is a completely stand-alone
program, so install it simply by putting the "fossil" or "fossil.exe"
executable someplace on your $PATH or %PATH%.  After you have Fossil
installed, do this:

<codeblock>
fossil clone http://www.sqlite.org/cgi/src sqlite.fossil
</codeblock>

<p>The command above
will make a copy of the complete development history of
SQLite into the "sqlite.fossil" file on your computer.  Making this copy
takes about a minute and uses about 32 megabytes of transfer.  After
making the copy, "open" the repository by typing:

<codeblock>
fossil open sqlite.fossil
</codeblock>

Changes to pages/malloc.in.

499
500
501
502
503
504
505
506
507
508
509
510
511
512
513

<tcl>hd_fragment pagecache {pagecache memory allocator}</tcl>
<h2> Page cache memory</h2>

<p>In most applications, the database page cache subsystem within 
SQLite uses more dynamically allocated memory than all other parts
of SQLite combined.  It is not unusual to see the database page cache
consumes over 10 times more memory than the rest of SQLite combined.</p>

<p>SQLite can be configured to make page cache memory allocations from
a separate and distinct memory pool of fixed-size
slots.  This can have two advantages:</p>

<ul>
<li><p>







|







499
500
501
502
503
504
505
506
507
508
509
510
511
512
513

<tcl>hd_fragment pagecache {pagecache memory allocator}</tcl>
<h2> Page cache memory</h2>

<p>In most applications, the database page cache subsystem within 
SQLite uses more dynamically allocated memory than all other parts
of SQLite combined.  It is not unusual to see the database page cache
consume over 10 times more memory than the rest of SQLite combined.</p>

<p>SQLite can be configured to make page cache memory allocations from
a separate and distinct memory pool of fixed-size
slots.  This can have two advantages:</p>

<ul>
<li><p>
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726

<a name="heaplimit"></a>
<h2> Setting memory usage limits</h2>

<p>The [sqlite3_soft_heap_limit64()] interface can be used to set an
upper bound on the total amount of outstanding memory that the
general-purpose memory allocator for SQLite will allow to be outstanding
at one time.  If attempts are made to allocate more memory that specified
by the soft heap limit, then SQLite will first attempt to free cache
memory before continuing with the allocation request.  The soft heap
limit mechanism only works if [memory statistics] are enabled and
it works best
if the SQLite library is compiled with the [SQLITE_ENABLE_MEMORY_MANAGEMENT]
compile-time option.</p>

<p>The soft heap limit is "soft" in this sense:  If SQLite is not able
to free up enough auxiliary memory to stay below the limit, it goes
ahead and allocations the extra memory and exceeds its limit.  This occurs
under the theory that it is better to use additional memory than to fail
outright.</p>

<p>As of SQLite [version 3.6.1] ([dateof:3.6.1]), 
the soft heap limit only applies to the
general-purpose memory allocator.  The soft heap limit does not know
about or interact with the [scratch memory allocator], 







|









|







702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726

<a name="heaplimit"></a>
<h2> Setting memory usage limits</h2>

<p>The [sqlite3_soft_heap_limit64()] interface can be used to set an
upper bound on the total amount of outstanding memory that the
general-purpose memory allocator for SQLite will allow to be outstanding
at one time.  If attempts are made to allocate more memory than specified
by the soft heap limit, then SQLite will first attempt to free cache
memory before continuing with the allocation request.  The soft heap
limit mechanism only works if [memory statistics] are enabled and
it works best
if the SQLite library is compiled with the [SQLITE_ENABLE_MEMORY_MANAGEMENT]
compile-time option.</p>

<p>The soft heap limit is "soft" in this sense:  If SQLite is not able
to free up enough auxiliary memory to stay below the limit, it goes
ahead and allocates the extra memory and exceeds its limit.  This occurs
under the theory that it is better to use additional memory than to fail
outright.</p>

<p>As of SQLite [version 3.6.1] ([dateof:3.6.1]), 
the soft heap limit only applies to the
general-purpose memory allocator.  The soft heap limit does not know
about or interact with the [scratch memory allocator], 

Changes to pages/news.in.

43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
One of the performance optimizations added in 3.16.0 caused triggers
and foreign keys to malfunction for the [REPLACE] statement on
[WITHOUT ROWID] tables that lack secondary indexes.  This patch
release fixes the problem.  See ticket 
[https://www.sqlite.org/src/info/30027b613b4|30027b613b4] for details.
}

newsitem {2016-01-03} {Release 3.16.1} {
SQLite [version 3.16.1] fixes a bug in the row-value logic for UPDATE
statements inside of triggers.  The bug has been there since row-values
were added by release 3.15.0, but was not discovered until just a few
minutes after the 3.16.0 release was published, and so it was not fixed
by 3.16.0.  This patch release is version 3.16.0 with the row-value bug fix.
}








|







43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
One of the performance optimizations added in 3.16.0 caused triggers
and foreign keys to malfunction for the [REPLACE] statement on
[WITHOUT ROWID] tables that lack secondary indexes.  This patch
release fixes the problem.  See ticket 
[https://www.sqlite.org/src/info/30027b613b4|30027b613b4] for details.
}

newsitem {2017-01-03} {Release 3.16.1} {
SQLite [version 3.16.1] fixes a bug in the row-value logic for UPDATE
statements inside of triggers.  The bug has been there since row-values
were added by release 3.15.0, but was not discovered until just a few
minutes after the 3.16.0 release was published, and so it was not fixed
by 3.16.0.  This patch release is version 3.16.0 with the row-value bug fix.
}

Changes to pages/pragma.in.

243
244
245
246
247
248
249
250
251
252
253
254
255
256
257

Pragma {automatic_index} {
    <p>^(<b>PRAGMA automatic_index;
     <br>PRAGMA automatic_index = </b><i>boolean</i><b>;</b></p>

    <p>Query, set, or clear the [automatic indexing] capability.)^
    <p>[Automatic indexing] is enabled by default as of 
    [version 3.7.17] (dateof:3.7.17]),
    but this might change in future releases of SQLite.
}

Pragma {auto_vacuum} {
    <p><b>PRAGMA DB.auto_vacuum;<br>
          PRAGMA DB.auto_vacuum = </b>
           <i>0 | NONE | 1 | FULL | 2 | INCREMENTAL</i><b>;</b></p>







|







243
244
245
246
247
248
249
250
251
252
253
254
255
256
257

Pragma {automatic_index} {
    <p>^(<b>PRAGMA automatic_index;
     <br>PRAGMA automatic_index = </b><i>boolean</i><b>;</b></p>

    <p>Query, set, or clear the [automatic indexing] capability.)^
    <p>[Automatic indexing] is enabled by default as of 
    [version 3.7.17] ([dateof:3.7.17]),
    but this might change in future releases of SQLite.
}

Pragma {auto_vacuum} {
    <p><b>PRAGMA DB.auto_vacuum;<br>
          PRAGMA DB.auto_vacuum = </b>
           <i>0 | NONE | 1 | FULL | 2 | INCREMENTAL</i><b>;</b></p>

Changes to pages/wal.in.

153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
of any current reader.  The checkpoint has to stop at that point because
otherwise it might overwrite part of the database file that the reader
is actively using.  The checkpoint remembers (in the wal-index) how far
it got and will resume transferring content from the WAL to the database
from where it left off on the next invocation.</p>

<p>Thus a long-running read transaction can prevent a checkpointer from
making progress.  But presumably every read transactions will eventually
end and the checkpointer will be able to continue.</p>

<p>Whenever a write operation occurs, the writer checks how much progress
the checkpointer has made, and if the entire WAL has been transferred into
the database and synced and if no readers are making use of the WAL, then
the writer will rewind the WAL back to the beginning and start putting new
transactions at the beginning of the WAL.  This mechanism prevents a WAL







|







153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
of any current reader.  The checkpoint has to stop at that point because
otherwise it might overwrite part of the database file that the reader
is actively using.  The checkpoint remembers (in the wal-index) how far
it got and will resume transferring content from the WAL to the database
from where it left off on the next invocation.</p>

<p>Thus a long-running read transaction can prevent a checkpointer from
making progress.  But presumably every read transaction will eventually
end and the checkpointer will be able to continue.</p>

<p>Whenever a write operation occurs, the writer checks how much progress
the checkpointer has made, and if the entire WAL has been transferred into
the database and synced and if no readers are making use of the WAL, then
the writer will rewind the WAL back to the beginning and start putting new
transactions at the beginning of the WAL.  This mechanism prevents a WAL