Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch wal-read-change Excluding Merge-Ins
This is equivalent to a diff from 64abb65d4d to ab93024da7
2015-08-28
| ||
16:18 | When searching the wal file for a frame, do not search that part that was already checkpointed when the transaction was opened. (check-in: a84cf4f5d3 user: dan tags: trunk) | |
15:50 | Merge latest trunk into this branch. (Closed-Leaf check-in: ab93024da7 user: dan tags: wal-read-change) | |
09:27 | Merge latest trunk changes with this branch. (check-in: 57bc0194f4 user: dan tags: begin-concurrent) | |
03:48 | Add the json_check() function, which returns its argument if the argument is well-formed JSON or which throws an error otherwise. (check-in: 64abb65d4d user: drh tags: trunk) | |
03:33 | Enhance the json_insert(), json_replace(), and json_set() functions with the ability to add JSON instead of text if the argument is text and if the PATH begins with '$$' instead of just '$'. (check-in: 44f103d886 user: drh tags: trunk) | |
2015-08-13
| ||
20:23 | When searching the wal file for a frame, do not search that part that was already checkpointed when the snapshot being read was at the head of the wal file. (check-in: 90760e7232 user: dan tags: wal-read-change) | |
Changes to src/wal.c.
︙ | ︙ | |||
424 425 426 427 428 429 430 431 432 433 434 435 436 437 | u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ u8 truncateOnCommit; /* True to truncate WAL file on commit */ u8 syncHeader; /* Fsync the WAL header if true */ u8 padToSectorBoundary; /* Pad transactions out to the next sector */ WalIndexHdr hdr; /* Wal-index header for current transaction */ const char *zWalName; /* Name of WAL file */ u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ #ifdef SQLITE_DEBUG u8 lockError; /* True if a locking error has occurred */ #endif }; | > | 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 | u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ u8 truncateOnCommit; /* True to truncate WAL file on commit */ u8 syncHeader; /* Fsync the WAL header if true */ u8 padToSectorBoundary; /* Pad transactions out to the next sector */ WalIndexHdr hdr; /* Wal-index header for current transaction */ u32 minFrame; /* Ignore wal frames before this one */ const char *zWalName; /* Name of WAL file */ u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ #ifdef SQLITE_DEBUG u8 lockError; /* True if a locking error has occurred */ #endif }; |
︙ | ︙ | |||
2292 2293 2294 2295 2296 2297 2298 | ** that the log file may have been wrapped by a writer, or that frames ** that occur later in the log than pWal->hdr.mxFrame may have been ** copied into the database by a checkpointer. If either of these things ** happened, then reading the database with the current value of ** pWal->hdr.mxFrame risks reading a corrupted snapshot. So, retry ** instead. ** | | | | | | > > > > > > > > > > > > > > > | 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 | ** that the log file may have been wrapped by a writer, or that frames ** that occur later in the log than pWal->hdr.mxFrame may have been ** copied into the database by a checkpointer. If either of these things ** happened, then reading the database with the current value of ** pWal->hdr.mxFrame risks reading a corrupted snapshot. So, retry ** instead. ** ** Before checking that the live wal-index header has not changed ** since it was read, set Wal.minFrame to the first frame in the wal ** file that has not yet been checkpointed. This client will not need ** to read any frames earlier than minFrame from the wal file - they ** can be safely read directly from the database file. ** ** Because a ShmBarrier() call is made between taking the copy of ** nBackfill and checking that the wal-header in shared-memory still ** matches the one cached in pWal->hdr, it is guaranteed that the ** checkpointer that set nBackfill was not working with a wal-index ** header newer than that cached in pWal->hdr. If it were, that could ** cause a problem. The checkpointer could omit to checkpoint ** a version of page X that lies before pWal->minFrame (call that version ** A) on the basis that there is a newer version (version B) of the same ** page later in the wal file. But if version B happens to like past ** frame pWal->hdr.mxFrame - then the client would incorrectly assume ** that it can read version A from the database file. However, since ** we can guarantee that the checkpointer that set nBackfill could not ** see any pages past pWal->hdr.mxFrame, this problem does not come up. */ pWal->minFrame = pInfo->nBackfill+1; walShmBarrier(pWal); if( pInfo->aReadMark[mxI]!=mxReadMark || memcmp((void *)walIndexHdr(pWal), &pWal->hdr, sizeof(WalIndexHdr)) ){ walUnlockShared(pWal, WAL_READ_LOCK(mxI)); return WAL_RETRY; }else{ |
︙ | ︙ | |||
2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 | Wal *pWal, /* WAL handle */ Pgno pgno, /* Database page number to read data for */ u32 *piRead /* OUT: Frame number (or zero) */ ){ u32 iRead = 0; /* If !=0, WAL frame to return data from */ u32 iLast = pWal->hdr.mxFrame; /* Last page in WAL for this reader */ int iHash; /* Used to loop through N hash tables */ /* This routine is only be called from within a read transaction. */ assert( pWal->readLock>=0 || pWal->lockError ); /* If the "last page" field of the wal-index header snapshot is 0, then ** no data will be read from the wal under any circumstances. Return early ** in this case as an optimization. Likewise, if pWal->readLock==0, | > | 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 | Wal *pWal, /* WAL handle */ Pgno pgno, /* Database page number to read data for */ u32 *piRead /* OUT: Frame number (or zero) */ ){ u32 iRead = 0; /* If !=0, WAL frame to return data from */ u32 iLast = pWal->hdr.mxFrame; /* Last page in WAL for this reader */ int iHash; /* Used to loop through N hash tables */ int iMinHash; /* This routine is only be called from within a read transaction. */ assert( pWal->readLock>=0 || pWal->lockError ); /* If the "last page" field of the wal-index header snapshot is 0, then ** no data will be read from the wal under any circumstances. Return early ** in this case as an optimization. Likewise, if pWal->readLock==0, |
︙ | ︙ | |||
2408 2409 2410 2411 2412 2413 2414 | ** (aPgno[iFrame]==pgno): ** This condition filters out normal hash-table collisions. ** ** (iFrame<=iLast): ** This condition filters out entries that were added to the hash ** table after the current read-transaction had started. */ | > | | | 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 | ** (aPgno[iFrame]==pgno): ** This condition filters out normal hash-table collisions. ** ** (iFrame<=iLast): ** This condition filters out entries that were added to the hash ** table after the current read-transaction had started. */ iMinHash = walFramePage(pWal->minFrame); for(iHash=walFramePage(iLast); iHash>=iMinHash && iRead==0; iHash--){ volatile ht_slot *aHash; /* Pointer to hash table */ volatile u32 *aPgno; /* Pointer to array of page numbers */ u32 iZero; /* Frame number corresponding to aPgno[0] */ int iKey; /* Hash slot index */ int nCollide; /* Number of hash collisions remaining */ int rc; /* Error code */ rc = walHashGet(pWal, iHash, &aHash, &aPgno, &iZero); if( rc!=SQLITE_OK ){ return rc; } nCollide = HASHTABLE_NSLOT; for(iKey=walHash(pgno); aHash[iKey]; iKey=walNextHash(iKey)){ u32 iFrame = aHash[iKey] + iZero; if( iFrame<=iLast && iFrame>=pWal->minFrame && aPgno[aHash[iKey]]==pgno ){ assert( iFrame>iRead || CORRUPT_DB ); iRead = iFrame; } if( (nCollide--)==0 ){ return SQLITE_CORRUPT_BKPT; } } |
︙ | ︙ |
Changes to test/wal6.test.
︙ | ︙ | |||
188 189 190 191 192 193 194 195 196 197 | } {} db eval {SELECT test4('3.3.2')} do_test 3.x { db2 close } {} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 | } {} db eval {SELECT test4('3.3.2')} do_test 3.x { db2 close } {} #------------------------------------------------------------------------- # Check that if a wal file has been partially checkpointed, no frames are # read from the checkpointed part. # reset_db do_execsql_test 4.1 { PRAGMA page_size = 1024; PRAGMA journal_mode = wal; CREATE TABLE t1(a, b); CREATE TABLE t2(a, b); PRAGMA wal_checkpoint = truncate; } {wal 0 0 0} do_test 4.2 { execsql { INSERT INTO t1 VALUES(1, 2) } file size test.db-wal } [wal_file_size 1 1024] do_test 4.3 { sqlite3 db2 test.db execsql { BEGIN; INSERT INTO t2 VALUES(3, 4); } execsql { PRAGMA wal_checkpoint = passive } db2 } {0 1 1} do_test 4.3 { execsql { COMMIT } db2 close hexio_write test.db-wal 0 [string repeat 00 2000] sqlite3 db2 test.db } {} do_test 4.4.1 { catchsql { SELECT * FROM t1 } db2 } {0 {1 2}} do_test 4.4.2 { catchsql { SELECT * FROM t2 } db2 } {1 {database disk image is malformed}} finish_test |