Index: pages/cli.in ================================================================== --- pages/cli.in +++ pages/cli.in @@ -914,11 +914,11 @@ are considered to be part of the .archive command. For example, the following commands are equivalent: sqlite3 new_archive.db -Acv file1 file2 file3 -sqlite3 new_archive.db ".ar -tv file1 file2 file3" +sqlite3 new_archive.db ".ar -cv file1 file2 file3"

Long and short style options may be mixed. For example, the following are equivalent: Index: pages/cpu.in ================================================================== --- pages/cpu.in +++ pages/cpu.in @@ -5,11 +5,11 @@

Overview

The graph below shows the number of CPU cycles used by SQLite on a standard workload, for all versions of SQLite going back about 9 years. -Recent version so SQLite use less then a third of the CPU cycles +Recent versions of SQLite use less than a third of the CPU cycles compared to older versions.

This article describes how the SQLite developers measure CPU usage, what those measurements actually mean, and the techniques used by Index: pages/howtocorrupt.in ================================================================== --- pages/howtocorrupt.in +++ pages/howtocorrupt.in @@ -229,10 +229,27 @@ the unix OS interface will attempt to resolve symbolic links and open the database file by its canonical name. Prior to version 3.10.0, opening a database file through a symbolic link was similar to opening a database file that had multiple hard links and resulted in undefined behavior.

+ +hd_fragment fork +

Carrying an open database connection across a fork()

+ +

Do not open an SQLite database connection, then fork(), then try +to use that database connection in the child process. All kinds of +locking problems will result and you can easily end up with a corrupt +database. SQLite is not designed to support that kind of behavior. +Any database connection that is used in a child process must be opened +in the child process, not inherited from the parent. + +

Do not even call [sqlite3_close()] on a database connection from a +child process if the connection was opened in the parent. It is safe +to close the underlying file descriptor, but the [sqlite3_close()] +interface might invoke cleanup activities that will delete content out +from under the parent, leading to errors and perhaps even database +corruption.

Failure to sync

In order to guarantee that database files are always consistent, SQLite will occasionally ask the operating system to flush all pending writes to Index: pages/lang.in ================================================================== --- pages/lang.in +++ pages/lang.in @@ -837,11 +837,11 @@ hd_fragment uniqueidx {unique index}

^If the UNIQUE keyword appears between CREATE and INDEX then duplicate index entries are not allowed. ^Any attempt to insert a duplicate entry will result in an error. ^For the purposes of unique indices, all NULL values -are considered to different from all other NULL values and are thus unique. +are considered different from all other NULL values and are thus unique. This is one of the two possible interpretations of the SQL-92 standard (the language in the standard is ambiguous) and is the interpretation followed by PostgreSQL, MySQL, Firebird, and Oracle. Informix and Microsoft SQL Server follow the other interpretation of the standard.

@@ -932,11 +932,11 @@ Expression Affinity Column Declared Type TEXT "TEXT" NUMERIC "NUM" INTEGER "INT" REAL "REAL" - NONE "" (empty string) + BLOB (a.k.a "NONE") "" (empty string) )^

^(A table created using CREATE TABLE AS has no PRIMARY KEY and no constraints of any kind. The default value of each column is NULL. The default collation sequence for each column of the new table is BINARY.)^ Index: pages/whentouse.in ================================================================== --- pages/whentouse.in +++ pages/whentouse.in @@ -64,10 +64,14 @@ performance, reduced cost and complexity, and improved reliability. See technical notes [file-format benefits|"aff_short.html"] and [application file-format|"appfileformat.html"] and [faster than the filesystem|"fasterthanfs.html"] for more information. +This use case is closely related to the +data transfer format and +data container use cases below. + hd_fragment website {using SQLite for websites}

  • Websites

    SQLite works great as the database engine for most low to @@ -152,14 +156,36 @@ an issue. Concurrency is also improved by "database sharding": using separate database files for different subdomains. For example, the server might have a separate SQLite database for each user, so that the server can handle hundreds or thousands of simultaneous connections, but each SQLite database is only used by one connection.

    +
    • + +hd_fragment wireproto {data transfer format} +

  • Data transfer format

    +

    Because an SQLite database is a single compact file in a +[file format|well-defined cross-platform format], it is often used +as a container for transfering content from one system to another. +The sender gathers content into an SQLite database file, transfers +that one file to the receiver, then the receiver uses SQL to extract +the content as needed. +

    An SQLite database facilitates data transfer between systems even +when the endpoints have different word sizes and/or byte orders. +The data can be a complex mix of large binary blobs, text, and small +numeric or boolean values. The data format can be easily extended +by adding new tables and/or columns, without breaking legacy receivers. +The SQL query language means that receivers are not required to parse +the entire transfer all at once, but can instead query the +received content as needed. The data format is "transparent" in the +sense that it is easily decoded for human viewing using +a variety of universally available, open-source tools, from multiple +vendors.

    • +hd_fragment container {data container}

  • File archive and/or data container

    The [SQLite Archive] idea shows how SQLite can be used as a substitute for ZIP archives or Tarballs. @@ -304,11 +330,11 @@

  • High Concurrency

    SQLite supports an unlimited number of simultaneous readers, but it will only allow one writer at any instant in time. -For many situations, this is not a problem. Writer queue up. Each application +For many situations, this is not a problem. Writers queue up. Each application does its database work quickly and moves on, and no lock lasts for more than a few dozen milliseconds. But there are some applications that require more concurrency, and those applications may need to seek a different solution.

    ADDED pages/whynotgit.in Index: pages/whynotgit.in ================================================================== --- /dev/null +++ pages/whynotgit.in @@ -0,0 +1,175 @@ +Why SQLite Does Not Use Git + + + +

    Introduction

    + +

    +SQLite does not use the +[https://git-scm.org|Git] version control system. +SQLite uses +[https://fossil-scm.org/|Fossil] instead. +Fossil and Git are both block-chain version-control systems. +They are both "distributed". They both store content has a +sequence of immutable check-ins identified by a cryptographic +hash. Git is wildly popular, to the point that many younger +developers are familiar with nothing else. And yet, the developers +of SQLite prefer Fossil. This article tries to explain why. + +

    A Few Reasons Why SQLite Does Not Use Git

    + +
      +

    1. +With Git, it is very difficult to find the successors (decendents) of +a check-in. + +

      +Git allows you to go backwards in time. Given the latest +check-in on a branch, Git lets you see all the ancestors of that +check-in. But Git makes it very difficult to move in the other +direction. Given some historical check-in, it is quite challenging +in Git to find out what came next. It can be done, but it is sufficiently +difficult and slow that nobody ever does it. There is no button in +GitHub that shows the descendents of a check-in. + +

      +This is a deal-breaker, a show-stopper. The principle maintainer +of SQLite cannot function effectively without being able to view +the successors of a check-in. This one issue is sufficient reason +to not use Git, in the view of the designer of SQLite. + +

    2. +The mental model for Git is needlessly complex and consequently +distracts attention from software under development. A user of Git +needs to keep all of the following in mind: +

        +
      1. The working directory +
      2. The "index" or staging area +
      3. The local head +
      4. The local copy of the remote head +
      5. The actual remote head +
      +

      +Git contains commands (or options on commands) for moving and +comparing content between all of these locations. + +

      In contrast, +Fossil users only need to think about their working directory and +the check-in they are working on. That is 60% less distraction. +Every developer has a finite number of brain-cycles. Fossil +requires fewer brain-cycles to operate, thus freeing up +intellectual resources to focus on the software under development. + +

    3. +Git does not track branch history. This makes review of historical +branches tedious. + +

      +As an example, consider display of a single historical +branch of SQLite as rendered by GitHub and by Fossil: + +

        +
      • GitHub: [https://github.com/mackyle/sqlite/commits/prefer-coroutine-sort-subquery] +
      • Fossil: [https://sqlite.org/src/timeline?r=prefer-coroutine-sort-subquery] +
      + +

      +The Fossil view clearly shows that the branch was eventually merged back into +trunk. It shows where the branch started, and it shows two occasions where changes +on trunk were merged into the branch. GitHub shows none of this. In fact, the +GitHub display is mostly useless in trying to figure out what happened. + +

    4. +Git lacks native wiki and bug tracking. +If you want these essential features, you have to install additional software +such as GitLab, or else use a third-party service such as GitHub. And even +then, the wiki and bug reports are centralized, not distributed. + +

    5. +Git requires more administrative support. +Setting up a website for a project to use Git requires a lot more software, +and a lot more work, than setting up a similar site with an integrated package +like Fossil. + +

    6. Nobody really understands Git. +
      +(credit: [https://xkcd.com/1597/]) +

    + +

    See Also

    + +

    Other pages that talk about Fossil and Git include: +

    + +

    A Git-User's Guide To Accessing SQLite Source Code

    + +

    +If you are a devoted Git user and have no intention of changing, +you can still easily access SQLite. This section gives some hints +on how to do so. + +

    GitHub Mirrors

    + +

    +There is a mirror of the SQLite source tree on GitHub at +[https://github.com/mackyle/sqlite]. This mirror is maintained +by user "mackyle" who is unaffiliated with, and unknown to, +the official SQLite development team. We do not know mackyle, +but we observe that he does a terrific job of keeping his mirror +current, and so if you want to access the SQLite source code on +GitHub, his mirror is the recommended source. + +

    Web Access

    + +

    +The [https://sqlite.org/src/timeline|SQLite Fossil Repository] contains links +for downloading a Tarball, ZIP Archive, or [SQLite Archive] for any +historical version of SQLite. The URLs for these downloads are +simple and can be incorporated easily into automated tools. The format is: + +

    +https://sqlite.org/src/tarball/VERSION/sqlite.tar.gz +
    + +

    +Simply replace VERSION with some description of the version to be +downloaded. The VERSION can be a prefix of the cryptographic hash +name of a specific check-in, or the name of a branch (in which case the +most recent version of the branch is fetched) or a tag for a specific +check-in like "version-3.23.0". + +

    Fossil Access

    + +

    +Fossil is really easy to install and use. Here are the steps for unix. +(Windows is similar.) + +

      +
    1. +Download the self-contained Fossil executable from +[https://fossil-scm.org/fossil/uv/download.html] and put the executable +somewhere on your $PATH. +
    2. mkdir ~/fossils +
    3. fossil clone https://fossil-scm.org/fossil ~/fossils/sqlite.fossil +
    4. mkdir ~/sqlite; cd ~/sqlite +
    5. fossil open ~/sqlite.fossil . +
    + +

    +Subsequently, to update your check-out, simply type: + +

    +fossil up VERSION +
    + +

    +Use "trunk" for version to get the latest trunk version of SQLite. +Or use a prefix of a cryptographic hash name, or the name of some branch +or tag. + +

    +Use the "fossil ui" command from within the ~/sqlite checkout to +bring up a local copy of the website.