/ Changes On Branch IN-operator-improvements
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch IN-operator-improvements Excluding Merge-Ins

This is equivalent to a diff from ba78265429 to 01f60027ad

2014-08-05
19:16
Add the ability to evaluate IN operators as a sequence of comparisons as an alternative to the long-standing algorithm of building a lookup table. Use the new implementation in circumstances where it is likely to be faster, such as when the RHS of the IN changes between successive evaluations. (check-in: 9528682168 user: drh tags: trunk)
11:04
Rename the internal Schema.flags field to Schema.schemaFlags. (check-in: 5ae80b3c8f user: drh tags: trunk)
00:53
Improved VdbeCoverage() macros. A few minor simplifications to generated VDBE code. (Closed-Leaf check-in: 01f60027ad user: drh tags: IN-operator-improvements)
2014-08-04
21:26
Part of the change in the previous check-in was incorrect and can result in an incorrect UPDATE for WITHOUT ROWID tables. This check-in fixes the problem. (check-in: ee5f6eae57 user: drh tags: IN-operator-improvements)
15:12
Changed my mind: This opcode name changes mere creates unnecessary diff marks between older and newer versions without significantly improving readability. Was: Rename the IsNull opcode to IfNull and rename the NotNull opcode to IfNotNull. (Closed-Leaf check-in: 2f724cbac9 user: drh tags: deadend)
2014-08-02
21:03
Enhancements to the code generator for the IN operator that result in much faster queries in some cases, for example when the RHS of the IN operator changes for each row of a large table scan. (check-in: 436e884215 user: drh tags: IN-operator-improvements)
20:44
Remove (newly) incorrect preprocessor check to fix build on WinRT. (check-in: ba78265429 user: mistachkin tags: trunk)
2014-08-01
21:12
A better comment on the generated code for the NULL-in-RHS-of-IN detection logic. (check-in: 9bc1c730a3 user: drh tags: trunk)

Changes to src/expr.c.

1364
1365
1366
1367
1368
1369
1370



1371
1372
1373
1374
1375
1376
1377
  if( op==TK_REGISTER ) op = p->op2;
  switch( op ){
    case TK_INTEGER:
    case TK_STRING:
    case TK_FLOAT:
    case TK_BLOB:
      return 0;



    default:
      return 1;
  }
}

/*
** Return TRUE if the given expression is a constant which would be







>
>
>







1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
  if( op==TK_REGISTER ) op = p->op2;
  switch( op ){
    case TK_INTEGER:
    case TK_STRING:
    case TK_FLOAT:
    case TK_BLOB:
      return 0;
    case TK_COLUMN:
      assert( p->pTab!=0 );
      return p->iColumn>=0 && p->pTab->aCol[p->iColumn].notNull==0;
    default:
      return 1;
  }
}

/*
** Return TRUE if the given expression is a constant which would be
1487
1488
1489
1490
1491
1492
1493


















1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512


1513
1514
1515
1516
1517
1518
1519
  j1 = sqlite3VdbeAddOp1(v, OP_Rewind, iCur); VdbeCoverage(v);
  sqlite3VdbeAddOp3(v, OP_Column, iCur, 0, regHasNull);
  sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
  VdbeComment((v, "first_entry_in(%d)", iCur));
  sqlite3VdbeJumpHere(v, j1);
}



















/*
** This function is used by the implementation of the IN (...) operator.
** The pX parameter is the expression on the RHS of the IN operator, which
** might be either a list of expressions or a subquery.
**
** The job of this routine is to find or create a b-tree object that can
** be used either to test for membership in the RHS set or to iterate through
** all members of the RHS set, skipping duplicates.
**
** A cursor is opened on the b-tree object that is the RHS of the IN operator
** and pX->iTable is set to the index of that cursor.
**
** The returned value of this function indicates the b-tree type, as follows:
**
**   IN_INDEX_ROWID      - The cursor was opened on a database table.
**   IN_INDEX_INDEX_ASC  - The cursor was opened on an ascending index.
**   IN_INDEX_INDEX_DESC - The cursor was opened on a descending index.
**   IN_INDEX_EPH        - The cursor was opened on a specially created and
**                         populated epheremal table.


**
** An existing b-tree might be used if the RHS expression pX is a simple
** subquery such as:
**
**     SELECT <column> FROM <table>
**
** If the RHS of the IN operator is a list or a more complex subquery, then







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



















>
>







1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
  j1 = sqlite3VdbeAddOp1(v, OP_Rewind, iCur); VdbeCoverage(v);
  sqlite3VdbeAddOp3(v, OP_Column, iCur, 0, regHasNull);
  sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
  VdbeComment((v, "first_entry_in(%d)", iCur));
  sqlite3VdbeJumpHere(v, j1);
}


#ifndef SQLITE_OMIT_SUBQUERY
/*
** The argument is an IN operator with a list (not a subquery) on the 
** right-hand side.  Return TRUE if that list is constant.
*/
static int sqlite3InRhsIsConstant(Expr *pIn){
  Expr *pLHS;
  int res;
  assert( !ExprHasProperty(pIn, EP_xIsSelect) );
  pLHS = pIn->pLeft;
  pIn->pLeft = 0;
  res = sqlite3ExprIsConstant(pIn);
  pIn->pLeft = pLHS;
  return res;
}
#endif

/*
** This function is used by the implementation of the IN (...) operator.
** The pX parameter is the expression on the RHS of the IN operator, which
** might be either a list of expressions or a subquery.
**
** The job of this routine is to find or create a b-tree object that can
** be used either to test for membership in the RHS set or to iterate through
** all members of the RHS set, skipping duplicates.
**
** A cursor is opened on the b-tree object that is the RHS of the IN operator
** and pX->iTable is set to the index of that cursor.
**
** The returned value of this function indicates the b-tree type, as follows:
**
**   IN_INDEX_ROWID      - The cursor was opened on a database table.
**   IN_INDEX_INDEX_ASC  - The cursor was opened on an ascending index.
**   IN_INDEX_INDEX_DESC - The cursor was opened on a descending index.
**   IN_INDEX_EPH        - The cursor was opened on a specially created and
**                         populated epheremal table.
**   IN_INDEX_NOOP       - No cursor was allocated.  The IN operator must be
**                         implemented as a sequence of comparisons.
**
** An existing b-tree might be used if the RHS expression pX is a simple
** subquery such as:
**
**     SELECT <column> FROM <table>
**
** If the RHS of the IN operator is a list or a more complex subquery, then
1534
1535
1536
1537
1538
1539
1540







1541
1542
1543
1544
1545
1546
1547
** to be unique - either because it is an INTEGER PRIMARY KEY or it
** has a UNIQUE constraint or UNIQUE index.
**
** When IN_INDEX_MEMBERSHIP is used (and the b-tree will be used 
** for fast set membership tests) then an epheremal table must 
** be used unless <column> is an INTEGER PRIMARY KEY or an index can 
** be found with <column> as its left-most column.







**
** When the b-tree is being used for membership tests, the calling function
** might need to know whether or not the RHS side of the IN operator
** contains a NULL.  If prRhsHasNull is not a NULL pointer and 
** if there is any chance that the (...) might contain a NULL value at
** runtime, then a register is allocated and the register number written
** to *prRhsHasNull. If there is no chance that the (...) contains a







>
>
>
>
>
>
>







1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
** to be unique - either because it is an INTEGER PRIMARY KEY or it
** has a UNIQUE constraint or UNIQUE index.
**
** When IN_INDEX_MEMBERSHIP is used (and the b-tree will be used 
** for fast set membership tests) then an epheremal table must 
** be used unless <column> is an INTEGER PRIMARY KEY or an index can 
** be found with <column> as its left-most column.
**
** If the IN_INDEX_NOOP_OK and IN_INDEX_MEMBERSHIP are both set and
** if the RHS of the IN operator is a list (not a subquery) then this
** routine might decide that creating an ephemeral b-tree for membership
** testing is too expensive and return IN_INDEX_NOOP.  In that case, the
** calling routine should implement the IN operator using a sequence
** of Eq or Ne comparison operations.
**
** When the b-tree is being used for membership tests, the calling function
** might need to know whether or not the RHS side of the IN operator
** contains a NULL.  If prRhsHasNull is not a NULL pointer and 
** if there is any chance that the (...) might contain a NULL value at
** runtime, then a register is allocated and the register number written
** to *prRhsHasNull. If there is no chance that the (...) contains a
1632
1633
1634
1635
1636
1637
1638
















1639
1640
1641
1642
1643
1644
1645
            sqlite3SetHasNullFlag(v, iTab, *prRhsHasNull);
          }
          sqlite3VdbeJumpHere(v, iAddr);
        }
      }
    }
  }

















  if( eType==0 ){
    /* Could not find an existing table or index to use as the RHS b-tree.
    ** We will have to generate an ephemeral table to do the job.
    */
    u32 savedNQueryLoop = pParse->nQueryLoop;
    int rMayHaveNull = 0;







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
            sqlite3SetHasNullFlag(v, iTab, *prRhsHasNull);
          }
          sqlite3VdbeJumpHere(v, iAddr);
        }
      }
    }
  }

  /* If no preexisting index is available for the IN clause
  ** and IN_INDEX_NOOP is an allowed reply
  ** and the RHS of the IN operator is a list, not a subquery
  ** and the RHS is not contant or has two or fewer terms,
  ** then it is not worth creating an ephermeral table to evaluate
  ** the IN operator so return IN_INDEX_NOOP.
  */
  if( eType==0
   && (inFlags & IN_INDEX_NOOP_OK)
   && !ExprHasProperty(pX, EP_xIsSelect)
   && (!sqlite3InRhsIsConstant(pX) || pX->x.pList->nExpr<=2)
  ){
    eType = IN_INDEX_NOOP;
  }
     

  if( eType==0 ){
    /* Could not find an existing table or index to use as the RHS b-tree.
    ** We will have to generate an ephemeral table to do the job.
    */
    u32 savedNQueryLoop = pParse->nQueryLoop;
    int rMayHaveNull = 0;
1931
1932
1933
1934
1935
1936
1937
1938

1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952











































1953
1954
1955

1956
1957
1958
1959
1960
1961
1962
1963
1964
1965

1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014

2015
2016
2017
2018
2019
2020
2021

  /* Compute the RHS.   After this step, the table with cursor
  ** pExpr->iTable will contains the values that make up the RHS.
  */
  v = pParse->pVdbe;
  assert( v!=0 );       /* OOM detected prior to this routine */
  VdbeNoopComment((v, "begin IN expr"));
  eType = sqlite3FindInIndex(pParse, pExpr, IN_INDEX_MEMBERSHIP,

                             destIfFalse==destIfNull ? 0 : &rRhsHasNull);

  /* Figure out the affinity to use to create a key from the results
  ** of the expression. affinityStr stores a static string suitable for
  ** P4 of OP_MakeRecord.
  */
  affinity = comparisonAffinity(pExpr);

  /* Code the LHS, the <expr> from "<expr> IN (...)".
  */
  sqlite3ExprCachePush(pParse);
  r1 = sqlite3GetTempReg(pParse);
  sqlite3ExprCode(pParse, pExpr->pLeft, r1);












































  /* If the LHS is NULL, then the result is either false or NULL depending
  ** on whether the RHS is empty or not, respectively.
  */

  if( destIfNull==destIfFalse ){
    /* Shortcut for the common case where the false and NULL outcomes are
    ** the same. */
    sqlite3VdbeAddOp2(v, OP_IsNull, r1, destIfNull); VdbeCoverage(v);
  }else{
    int addr1 = sqlite3VdbeAddOp1(v, OP_NotNull, r1); VdbeCoverage(v);
    sqlite3VdbeAddOp2(v, OP_Rewind, pExpr->iTable, destIfFalse);
    VdbeCoverage(v);
    sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfNull);
    sqlite3VdbeJumpHere(v, addr1);

  }

  if( eType==IN_INDEX_ROWID ){
    /* In this case, the RHS is the ROWID of table b-tree
    */
    sqlite3VdbeAddOp2(v, OP_MustBeInt, r1, destIfFalse); VdbeCoverage(v);
    sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, destIfFalse, r1);
    VdbeCoverage(v);
  }else{
    /* In this case, the RHS is an index b-tree.
    */
    sqlite3VdbeAddOp4(v, OP_Affinity, r1, 1, 0, &affinity, 1);

    /* If the set membership test fails, then the result of the 
    ** "x IN (...)" expression must be either 0 or NULL. If the set
    ** contains no NULL values, then the result is 0. If the set 
    ** contains one or more NULL values, then the result of the
    ** expression is also NULL.
    */
    assert( destIfFalse!=destIfNull || rRhsHasNull==0 );
    if( rRhsHasNull==0 ){
      /* This branch runs if it is known at compile time that the RHS
      ** cannot contain NULL values. This happens as the result
      ** of a "NOT NULL" constraint in the database schema.
      **
      ** Also run this branch if NULL is equivalent to FALSE
      ** for this particular IN operator.
      */
      sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable, destIfFalse, r1, 1);
      VdbeCoverage(v);
    }else{
      /* In this branch, the RHS of the IN might contain a NULL and
      ** the presence of a NULL on the RHS makes a difference in the
      ** outcome.
      */
      int j1;

      /* First check to see if the LHS is contained in the RHS.  If so,
      ** then the answer is TRUE the presence of NULLs in the RHS does
      ** not matter.  If the LHS is not contained in the RHS, then the
      ** answer is NULL if the RHS contains NULLs and the answer is
      ** FALSE if the RHS is NULL-free.
      */
      j1 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, r1, 1);
      VdbeCoverage(v);
      sqlite3VdbeAddOp2(v, OP_IsNull, rRhsHasNull, destIfNull);
      VdbeCoverage(v);
      sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse);
      sqlite3VdbeJumpHere(v, j1);

    }
  }
  sqlite3ReleaseTempReg(pParse, r1);
  sqlite3ExprCachePop(pParse);
  VdbeComment((v, "end IN expr"));
}
#endif /* SQLITE_OMIT_SUBQUERY */







|
>














>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
|
>
|
|
|
|
|
|
|
|
|
|
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>







1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114

  /* Compute the RHS.   After this step, the table with cursor
  ** pExpr->iTable will contains the values that make up the RHS.
  */
  v = pParse->pVdbe;
  assert( v!=0 );       /* OOM detected prior to this routine */
  VdbeNoopComment((v, "begin IN expr"));
  eType = sqlite3FindInIndex(pParse, pExpr,
                             IN_INDEX_MEMBERSHIP | IN_INDEX_NOOP_OK,
                             destIfFalse==destIfNull ? 0 : &rRhsHasNull);

  /* Figure out the affinity to use to create a key from the results
  ** of the expression. affinityStr stores a static string suitable for
  ** P4 of OP_MakeRecord.
  */
  affinity = comparisonAffinity(pExpr);

  /* Code the LHS, the <expr> from "<expr> IN (...)".
  */
  sqlite3ExprCachePush(pParse);
  r1 = sqlite3GetTempReg(pParse);
  sqlite3ExprCode(pParse, pExpr->pLeft, r1);

  /* If sqlite3FindInIndex() did not find or create an index that is
  ** suitable for evaluating the IN operator, then evaluate using a
  ** sequence of comparisons.
  */
  if( eType==IN_INDEX_NOOP ){
    ExprList *pList = pExpr->x.pList;
    CollSeq *pColl = sqlite3ExprCollSeq(pParse, pExpr->pLeft);
    int labelOk = sqlite3VdbeMakeLabel(v);
    int r2, regToFree;
    int regCkNull = 0;
    int ii;
    assert( !ExprHasProperty(pExpr, EP_xIsSelect) );
    if( destIfNull!=destIfFalse ){
      regCkNull = sqlite3GetTempReg(pParse);
      sqlite3VdbeAddOp3(v, OP_BitAnd, r1, r1, regCkNull);
    }
    for(ii=0; ii<pList->nExpr; ii++){
      r2 = sqlite3ExprCodeTemp(pParse, pList->a[ii].pExpr, &regToFree);
      if( regCkNull && sqlite3ExprCanBeNull(pList->a[ii].pExpr) ){
        sqlite3VdbeAddOp3(v, OP_BitAnd, regCkNull, r2, regCkNull);
      }
      if( ii<pList->nExpr-1 || destIfNull!=destIfFalse ){
        sqlite3VdbeAddOp4(v, OP_Eq, r1, labelOk, r2,
                          (void*)pColl, P4_COLLSEQ);
        VdbeCoverageIf(v, ii<pList->nExpr-1);
        VdbeCoverageIf(v, ii==pList->nExpr-1);
        sqlite3VdbeChangeP5(v, affinity);
      }else{
        assert( destIfNull==destIfFalse );
        sqlite3VdbeAddOp4(v, OP_Ne, r1, destIfFalse, r2,
                          (void*)pColl, P4_COLLSEQ); VdbeCoverage(v);
        sqlite3VdbeChangeP5(v, affinity | SQLITE_JUMPIFNULL);
      }
      sqlite3ReleaseTempReg(pParse, regToFree);
    }
    if( regCkNull ){
      sqlite3VdbeAddOp2(v, OP_IsNull, regCkNull, destIfNull); VdbeCoverage(v);
      sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse);
    }
    sqlite3VdbeResolveLabel(v, labelOk);
    sqlite3ReleaseTempReg(pParse, regCkNull);
  }else{
  
    /* If the LHS is NULL, then the result is either false or NULL depending
    ** on whether the RHS is empty or not, respectively.
    */
    if( sqlite3ExprCanBeNull(pExpr->pLeft) ){
      if( destIfNull==destIfFalse ){
        /* Shortcut for the common case where the false and NULL outcomes are
        ** the same. */
        sqlite3VdbeAddOp2(v, OP_IsNull, r1, destIfNull); VdbeCoverage(v);
      }else{
        int addr1 = sqlite3VdbeAddOp1(v, OP_NotNull, r1); VdbeCoverage(v);
        sqlite3VdbeAddOp2(v, OP_Rewind, pExpr->iTable, destIfFalse);
        VdbeCoverage(v);
        sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfNull);
        sqlite3VdbeJumpHere(v, addr1);
      }
    }
  
    if( eType==IN_INDEX_ROWID ){
      /* In this case, the RHS is the ROWID of table b-tree
      */
      sqlite3VdbeAddOp2(v, OP_MustBeInt, r1, destIfFalse); VdbeCoverage(v);
      sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, destIfFalse, r1);
      VdbeCoverage(v);
    }else{
      /* In this case, the RHS is an index b-tree.
      */
      sqlite3VdbeAddOp4(v, OP_Affinity, r1, 1, 0, &affinity, 1);
  
      /* If the set membership test fails, then the result of the 
      ** "x IN (...)" expression must be either 0 or NULL. If the set
      ** contains no NULL values, then the result is 0. If the set 
      ** contains one or more NULL values, then the result of the
      ** expression is also NULL.
      */
      assert( destIfFalse!=destIfNull || rRhsHasNull==0 );
      if( rRhsHasNull==0 ){
        /* This branch runs if it is known at compile time that the RHS
        ** cannot contain NULL values. This happens as the result
        ** of a "NOT NULL" constraint in the database schema.
        **
        ** Also run this branch if NULL is equivalent to FALSE
        ** for this particular IN operator.
        */
        sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable, destIfFalse, r1, 1);
        VdbeCoverage(v);
      }else{
        /* In this branch, the RHS of the IN might contain a NULL and
        ** the presence of a NULL on the RHS makes a difference in the
        ** outcome.
        */
        int j1;
  
        /* First check to see if the LHS is contained in the RHS.  If so,
        ** then the answer is TRUE the presence of NULLs in the RHS does
        ** not matter.  If the LHS is not contained in the RHS, then the
        ** answer is NULL if the RHS contains NULLs and the answer is
        ** FALSE if the RHS is NULL-free.
        */
        j1 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, r1, 1);
        VdbeCoverage(v);
        sqlite3VdbeAddOp2(v, OP_IsNull, rRhsHasNull, destIfNull);
        VdbeCoverage(v);
        sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse);
        sqlite3VdbeJumpHere(v, j1);
      }
    }
  }
  sqlite3ReleaseTempReg(pParse, r1);
  sqlite3ExprCachePop(pParse);
  VdbeComment((v, "end IN expr"));
}
#endif /* SQLITE_OMIT_SUBQUERY */
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
      assert( TK_NOTNULL==OP_NotNull ); testcase( op==TK_NOTNULL );
      sqlite3VdbeAddOp2(v, OP_Integer, 1, target);
      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
      testcase( regFree1==0 );
      addr = sqlite3VdbeAddOp1(v, op, r1);
      VdbeCoverageIf(v, op==TK_ISNULL);
      VdbeCoverageIf(v, op==TK_NOTNULL);
      sqlite3VdbeAddOp2(v, OP_AddImm, target, -1);
      sqlite3VdbeJumpHere(v, addr);
      break;
    }
    case TK_AGG_FUNCTION: {
      AggInfo *pInfo = pExpr->pAggInfo;
      if( pInfo==0 ){
        assert( !ExprHasProperty(pExpr, EP_IntValue) );







|







2724
2725
2726
2727
2728
2729
2730
2731
2732
2733
2734
2735
2736
2737
2738
      assert( TK_NOTNULL==OP_NotNull ); testcase( op==TK_NOTNULL );
      sqlite3VdbeAddOp2(v, OP_Integer, 1, target);
      r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
      testcase( regFree1==0 );
      addr = sqlite3VdbeAddOp1(v, op, r1);
      VdbeCoverageIf(v, op==TK_ISNULL);
      VdbeCoverageIf(v, op==TK_NOTNULL);
      sqlite3VdbeAddOp2(v, OP_Integer, 0, target);
      sqlite3VdbeJumpHere(v, addr);
      break;
    }
    case TK_AGG_FUNCTION: {
      AggInfo *pInfo = pExpr->pAggInfo;
      if( pInfo==0 ){
        assert( !ExprHasProperty(pExpr, EP_IntValue) );

Changes to src/pragma.c.

1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806

    /* Code that appears at the end of the integrity check.  If no error
    ** messages have been generated, output OK.  Otherwise output the
    ** error message
    */
    static const int iLn = VDBE_OFFSET_LINENO(2);
    static const VdbeOpList endCode[] = {
      { OP_AddImm,      1, 0,        0},    /* 0 */
      { OP_IfNeg,       1, 0,        0},    /* 1 */
      { OP_String8,     0, 3,        0},    /* 2 */
      { OP_ResultRow,   3, 1,        0},
    };

    int isQuick = (sqlite3Tolower(zLeft[0])=='q');

    /* If the PRAGMA command was of the form "PRAGMA <db>.integrity_check",
    ** then iDb is set to the index of the database identified by <db>.







<
|
|







1790
1791
1792
1793
1794
1795
1796

1797
1798
1799
1800
1801
1802
1803
1804
1805

    /* Code that appears at the end of the integrity check.  If no error
    ** messages have been generated, output OK.  Otherwise output the
    ** error message
    */
    static const int iLn = VDBE_OFFSET_LINENO(2);
    static const VdbeOpList endCode[] = {

      { OP_IfNeg,       1, 0,        0},    /* 0 */
      { OP_String8,     0, 3,        0},    /* 1 */
      { OP_ResultRow,   3, 1,        0},
    };

    int isQuick = (sqlite3Tolower(zLeft[0])=='q');

    /* If the PRAGMA command was of the form "PRAGMA <db>.integrity_check",
    ** then iDb is set to the index of the database identified by <db>.
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
          sqlite3VdbeAddOp3(v, OP_Concat, 3, 2, 7);
          sqlite3VdbeAddOp2(v, OP_ResultRow, 7, 1);
        }
#endif /* SQLITE_OMIT_BTREECOUNT */
      } 
    }
    addr = sqlite3VdbeAddOpList(v, ArraySize(endCode), endCode, iLn);
    sqlite3VdbeChangeP2(v, addr, -mxErr);
    sqlite3VdbeJumpHere(v, addr+1);
    sqlite3VdbeChangeP4(v, addr+2, "ok", P4_STATIC);
  }
  break;
#endif /* SQLITE_OMIT_INTEGRITY_CHECK */

#ifndef SQLITE_OMIT_UTF16
  /*
  **   PRAGMA encoding







|
|
|







1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
          sqlite3VdbeAddOp3(v, OP_Concat, 3, 2, 7);
          sqlite3VdbeAddOp2(v, OP_ResultRow, 7, 1);
        }
#endif /* SQLITE_OMIT_BTREECOUNT */
      } 
    }
    addr = sqlite3VdbeAddOpList(v, ArraySize(endCode), endCode, iLn);
    sqlite3VdbeChangeP3(v, addr, -mxErr);
    sqlite3VdbeJumpHere(v, addr);
    sqlite3VdbeChangeP4(v, addr+1, "ok", P4_STATIC);
  }
  break;
#endif /* SQLITE_OMIT_INTEGRITY_CHECK */

#ifndef SQLITE_OMIT_UTF16
  /*
  **   PRAGMA encoding

Changes to src/select.c.

537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
static void codeOffset(
  Vdbe *v,          /* Generate code into this VM */
  int iOffset,      /* Register holding the offset counter */
  int iContinue     /* Jump here to skip the current record */
){
  if( iOffset>0 ){
    int addr;
    sqlite3VdbeAddOp2(v, OP_AddImm, iOffset, -1);
    addr = sqlite3VdbeAddOp1(v, OP_IfNeg, iOffset); VdbeCoverage(v);
    sqlite3VdbeAddOp2(v, OP_Goto, 0, iContinue);
    VdbeComment((v, "skip OFFSET records"));
    sqlite3VdbeJumpHere(v, addr);
  }
}

/*







<
|







537
538
539
540
541
542
543

544
545
546
547
548
549
550
551
static void codeOffset(
  Vdbe *v,          /* Generate code into this VM */
  int iOffset,      /* Register holding the offset counter */
  int iContinue     /* Jump here to skip the current record */
){
  if( iOffset>0 ){
    int addr;

    addr = sqlite3VdbeAddOp3(v, OP_IfNeg, iOffset, 0, -1); VdbeCoverage(v);
    sqlite3VdbeAddOp2(v, OP_Goto, 0, iContinue);
    VdbeComment((v, "skip OFFSET records"));
    sqlite3VdbeJumpHere(v, addr);
  }
}

/*

Changes to src/sqliteInt.h.

3594
3595
3596
3597
3598
3599
3600

3601
3602
3603

3604
3605
3606
3607
3608
3609
3610
3611
3612
/*
** Allowed return values from sqlite3FindInIndex()
*/
#define IN_INDEX_ROWID        1   /* Search the rowid of the table */
#define IN_INDEX_EPH          2   /* Search an ephemeral b-tree */
#define IN_INDEX_INDEX_ASC    3   /* Existing index ASCENDING */
#define IN_INDEX_INDEX_DESC   4   /* Existing index DESCENDING */

/*
** Allowed flags for the 3rd parameter to sqlite3FindInIndex().
*/

#define IN_INDEX_MEMBERSHIP  0x0001  /* IN operator used for membership test */
#define IN_INDEX_LOOP        0x0002  /* IN operator used as a loop */
int sqlite3FindInIndex(Parse *, Expr *, u32, int*);

#ifdef SQLITE_ENABLE_ATOMIC_WRITE
  int sqlite3JournalOpen(sqlite3_vfs *, const char *, sqlite3_file *, int, int);
  int sqlite3JournalSize(sqlite3_vfs *);
  int sqlite3JournalCreate(sqlite3_file *);
  int sqlite3JournalExists(sqlite3_file *p);







>



>
|
|







3594
3595
3596
3597
3598
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
3614
/*
** Allowed return values from sqlite3FindInIndex()
*/
#define IN_INDEX_ROWID        1   /* Search the rowid of the table */
#define IN_INDEX_EPH          2   /* Search an ephemeral b-tree */
#define IN_INDEX_INDEX_ASC    3   /* Existing index ASCENDING */
#define IN_INDEX_INDEX_DESC   4   /* Existing index DESCENDING */
#define IN_INDEX_NOOP         5   /* No table available. Use comparisons */
/*
** Allowed flags for the 3rd parameter to sqlite3FindInIndex().
*/
#define IN_INDEX_NOOP_OK     0x0001  /* OK to return IN_INDEX_NOOP */
#define IN_INDEX_MEMBERSHIP  0x0002  /* IN operator used for membership test */
#define IN_INDEX_LOOP        0x0004  /* IN operator used as a loop */
int sqlite3FindInIndex(Parse *, Expr *, u32, int*);

#ifdef SQLITE_ENABLE_ATOMIC_WRITE
  int sqlite3JournalOpen(sqlite3_vfs *, const char *, sqlite3_file *, int, int);
  int sqlite3JournalSize(sqlite3_vfs *);
  int sqlite3JournalCreate(sqlite3_file *);
  int sqlite3JournalExists(sqlite3_file *p);

Changes to src/update.c.

434
435
436
437
438
439
440
441

442
443
444
445
446
447
448
    if( aToOpen[iDataCur-iBaseCur] ){
      assert( pPk!=0 );
      sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelBreak, regKey, nKey);
      VdbeCoverageNeverTaken(v);
    }
    labelContinue = labelBreak;
    sqlite3VdbeAddOp2(v, OP_IsNull, pPk ? regKey : regOldRowid, labelBreak);
    VdbeCoverage(v);

  }else if( pPk ){
    labelContinue = sqlite3VdbeMakeLabel(v);
    sqlite3VdbeAddOp2(v, OP_Rewind, iEph, labelBreak); VdbeCoverage(v);
    addrTop = sqlite3VdbeAddOp2(v, OP_RowKey, iEph, regKey);
    sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelContinue, regKey, 0);
    VdbeCoverage(v);
  }else{







|
>







434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
    if( aToOpen[iDataCur-iBaseCur] ){
      assert( pPk!=0 );
      sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelBreak, regKey, nKey);
      VdbeCoverageNeverTaken(v);
    }
    labelContinue = labelBreak;
    sqlite3VdbeAddOp2(v, OP_IsNull, pPk ? regKey : regOldRowid, labelBreak);
    VdbeCoverageIf(v, pPk==0);
    VdbeCoverageIf(v, pPk!=0);
  }else if( pPk ){
    labelContinue = sqlite3VdbeMakeLabel(v);
    sqlite3VdbeAddOp2(v, OP_Rewind, iEph, labelBreak); VdbeCoverage(v);
    addrTop = sqlite3VdbeAddOp2(v, OP_RowKey, iEph, regKey);
    sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelContinue, regKey, 0);
    VdbeCoverage(v);
  }else{

Changes to src/vdbe.c.

112
113
114
115
116
117
118






119
120
121
122
123
124
125
** feature is used for test suite validation only and does not appear an
** production builds.
**
** M is an integer, 2 or 3, that indices how many different ways the
** branch can go.  It is usually 2.  "I" is the direction the branch
** goes.  0 means falls through.  1 means branch is taken.  2 means the
** second alternative branch is taken.






*/
#if !defined(SQLITE_VDBE_COVERAGE)
# define VdbeBranchTaken(I,M)
#else
# define VdbeBranchTaken(I,M) vdbeTakeBranch(pOp->iSrcLine,I,M)
  static void vdbeTakeBranch(int iSrcLine, u8 I, u8 M){
    if( iSrcLine<=2 && ALWAYS(iSrcLine>0) ){







>
>
>
>
>
>







112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
** feature is used for test suite validation only and does not appear an
** production builds.
**
** M is an integer, 2 or 3, that indices how many different ways the
** branch can go.  It is usually 2.  "I" is the direction the branch
** goes.  0 means falls through.  1 means branch is taken.  2 means the
** second alternative branch is taken.
**
** iSrcLine is the source code line (from the __LINE__ macro) that
** generated the VDBE instruction.  This instrumentation assumes that all
** source code is in a single file (the amalgamation).  Special values 1
** and 2 for the iSrcLine parameter mean that this particular branch is
** always taken or never taken, respectively.
*/
#if !defined(SQLITE_VDBE_COVERAGE)
# define VdbeBranchTaken(I,M)
#else
# define VdbeBranchTaken(I,M) vdbeTakeBranch(pOp->iSrcLine,I,M)
  static void vdbeTakeBranch(int iSrcLine, u8 I, u8 M){
    if( iSrcLine<=2 && ALWAYS(iSrcLine>0) ){
5596
5597
5598
5599
5600
5601
5602
5603
5604
5605

5606
5607
5608
5609
5610
5611
5612
5613

5614
5615
5616
5617
5618
5619
5620
5621
5622
5623
5624
5625
5626
5627
5628
5629
5630
5631
5632
5633
5634
5635
  VdbeBranchTaken( pIn1->u.i>0, 2);
  if( pIn1->u.i>0 ){
     pc = pOp->p2 - 1;
  }
  break;
}

/* Opcode: IfNeg P1 P2 * * *
** Synopsis: if r[P1]<0 goto P2
**

** If the value of register P1 is less than zero, jump to P2. 
**
** It is illegal to use this instruction on a register that does
** not contain an integer.  An assertion fault will result if you try.
*/
case OP_IfNeg: {        /* jump, in1 */
  pIn1 = &aMem[pOp->p1];
  assert( pIn1->flags&MEM_Int );

  VdbeBranchTaken(pIn1->u.i<0, 2);
  if( pIn1->u.i<0 ){
     pc = pOp->p2 - 1;
  }
  break;
}

/* Opcode: IfZero P1 P2 P3 * *
** Synopsis: r[P1]+=P3, if r[P1]==0 goto P2
**
** The register P1 must contain an integer.  Add literal P3 to the
** value in register P1.  If the result is exactly 0, jump to P2. 
**
** It is illegal to use this instruction on a register that does
** not contain an integer.  An assertion fault will result if you try.
*/
case OP_IfZero: {        /* jump, in1 */
  pIn1 = &aMem[pOp->p1];
  assert( pIn1->flags&MEM_Int );
  pIn1->u.i += pOp->p3;
  VdbeBranchTaken(pIn1->u.i==0, 2);
  if( pIn1->u.i==0 ){







|
|

>
|
<
<
<




>












<
<
<







5602
5603
5604
5605
5606
5607
5608
5609
5610
5611
5612
5613



5614
5615
5616
5617
5618
5619
5620
5621
5622
5623
5624
5625
5626
5627
5628
5629
5630



5631
5632
5633
5634
5635
5636
5637
  VdbeBranchTaken( pIn1->u.i>0, 2);
  if( pIn1->u.i>0 ){
     pc = pOp->p2 - 1;
  }
  break;
}

/* Opcode: IfNeg P1 P2 P3 * *
** Synopsis: r[P1]+=P3, if r[P1]<0 goto P2
**
** Register P1 must contain an intger.  Add literal P3 to the value in
** register P1 then if the value of register P1 is less than zero, jump to P2. 



*/
case OP_IfNeg: {        /* jump, in1 */
  pIn1 = &aMem[pOp->p1];
  assert( pIn1->flags&MEM_Int );
  pIn1->u.i += pOp->p3;
  VdbeBranchTaken(pIn1->u.i<0, 2);
  if( pIn1->u.i<0 ){
     pc = pOp->p2 - 1;
  }
  break;
}

/* Opcode: IfZero P1 P2 P3 * *
** Synopsis: r[P1]+=P3, if r[P1]==0 goto P2
**
** The register P1 must contain an integer.  Add literal P3 to the
** value in register P1.  If the result is exactly 0, jump to P2. 



*/
case OP_IfZero: {        /* jump, in1 */
  pIn1 = &aMem[pOp->p1];
  assert( pIn1->flags&MEM_Int );
  pIn1->u.i += pOp->p3;
  VdbeBranchTaken(pIn1->u.i==0, 2);
  if( pIn1->u.i==0 ){

Changes to test/in4.test.

227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
  SELECT * FROM t3 WHERE x IN (10);
} {10 10 10}
do_execsql_test in4-3.44 {
  EXPLAIN
  SELECT * FROM t3 WHERE x IN (10);
} {~/OpenEphemeral/}
do_execsql_test in4-3.45 {
  SELECT * FROM t3 WHERE x NOT IN (10,11);
} {1 1 1}
do_execsql_test in4-3.46 {
  EXPLAIN
  SELECT * FROM t3 WHERE x NOT IN (10,11);
} {/OpenEphemeral/}
do_execsql_test in4-3.47 {
  SELECT * FROM t3 WHERE x NOT IN (10);
} {1 1 1}
do_execsql_test in4-3.48 {
  EXPLAIN
  SELECT * FROM t3 WHERE x NOT IN (10);







|



|







227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
  SELECT * FROM t3 WHERE x IN (10);
} {10 10 10}
do_execsql_test in4-3.44 {
  EXPLAIN
  SELECT * FROM t3 WHERE x IN (10);
} {~/OpenEphemeral/}
do_execsql_test in4-3.45 {
  SELECT * FROM t3 WHERE x NOT IN (10,11,99999);
} {1 1 1}
do_execsql_test in4-3.46 {
  EXPLAIN
  SELECT * FROM t3 WHERE x NOT IN (10,11,99999);
} {/OpenEphemeral/}
do_execsql_test in4-3.47 {
  SELECT * FROM t3 WHERE x NOT IN (10);
} {1 1 1}
do_execsql_test in4-3.48 {
  EXPLAIN
  SELECT * FROM t3 WHERE x NOT IN (10);

Changes to test/tkt-80e031a00f.test.

156
157
158
159
160
161
162




163
164
165
166
167
168
169
do_execsql_test tkt-80e031a00f.319 {SELECT 'c' NOT IN t7} 0
do_execsql_test tkt-80e031a00f.320 {SELECT 'c' IN t7n} 1
do_execsql_test tkt-80e031a00f.321 {SELECT 'd' NOT IN t7n} 0
do_execsql_test tkt-80e031a00f.322 {SELECT 'b' IN t8} 1
do_execsql_test tkt-80e031a00f.323 {SELECT 'c' NOT IN t8} 0
do_execsql_test tkt-80e031a00f.324 {SELECT 'c' IN t8n} 1
do_execsql_test tkt-80e031a00f.325 {SELECT 'd' NOT IN t8n} 0




#
# Row 4:
do_execsql_test tkt-80e031a00f.400 {SELECT 1 IN (2,3,4,null)} {{}}
do_execsql_test tkt-80e031a00f.401 {SELECT 1 NOT IN (2,3,4,null)} {{}}
do_execsql_test tkt-80e031a00f.402 {SELECT 'a' IN ('b','c',null,'d')} {{}}
do_execsql_test tkt-80e031a00f.403 {SELECT 'a' NOT IN (null,'b','c','d')} {{}}
do_execsql_test tkt-80e031a00f.404 {SELECT 1 IN t4n} {{}}







>
>
>
>







156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
do_execsql_test tkt-80e031a00f.319 {SELECT 'c' NOT IN t7} 0
do_execsql_test tkt-80e031a00f.320 {SELECT 'c' IN t7n} 1
do_execsql_test tkt-80e031a00f.321 {SELECT 'd' NOT IN t7n} 0
do_execsql_test tkt-80e031a00f.322 {SELECT 'b' IN t8} 1
do_execsql_test tkt-80e031a00f.323 {SELECT 'c' NOT IN t8} 0
do_execsql_test tkt-80e031a00f.324 {SELECT 'c' IN t8n} 1
do_execsql_test tkt-80e031a00f.325 {SELECT 'd' NOT IN t8n} 0
do_execsql_test tkt-80e031a00f.326 {SELECT 'a' IN (NULL,'a')} 1
do_execsql_test tkt-80e031a00f.327 {SELECT 'a' IN (NULL,'b')} {{}}
do_execsql_test tkt-80e031a00f.328 {SELECT 'a' NOT IN (NULL,'a')} 0
do_execsql_test tkt-80e031a00f.329 {SELECT 'a' NOT IN (NULL,'b')} {{}}
#
# Row 4:
do_execsql_test tkt-80e031a00f.400 {SELECT 1 IN (2,3,4,null)} {{}}
do_execsql_test tkt-80e031a00f.401 {SELECT 1 NOT IN (2,3,4,null)} {{}}
do_execsql_test tkt-80e031a00f.402 {SELECT 'a' IN ('b','c',null,'d')} {{}}
do_execsql_test tkt-80e031a00f.403 {SELECT 'a' NOT IN (null,'b','c','d')} {{}}
do_execsql_test tkt-80e031a00f.404 {SELECT 1 IN t4n} {{}}