Bug #9853
ProgressiveResults invalidate badly when changing the record on the edge of a bracket
100%
History
#1 Updated by Alexandru Lungu over 1 year ago
def var i as int.
def var n as int init 30.
def buffer b1 for book.
do transaction:
for each book:
delete book.
end.
do i = 1 to n: // to have multiple progressive brackets...
create book.
assign
book.isbn = string(i)
book.book-id = i
book.book-title = string(i).
release book.
end.
for each customer: delete customer. end.
create customer.
customer.customer = 1.
release customer.
end.
for each book no-lock:
// message book.isbn.
if book.book-id = 11 then
do transaction:
find first customer exclusive-lock.
customer.customer = customer - 1.
release customer.
find b1 where b1.book-id = book.book-id no-error.
if avail b1 then do:
b1.book-id = b1.book-id * -1.
b1.isbn = b1.isbn + "foo".
end.
end.
end.
Is an example that can reproduce the issue with FWD 15680. With later versions, the for each doesn't generate ProgressiveResults, so this test doesn't reproduce anymore. The FOR EACH should be replaced with an OPEN QUERY:
def var i as int.
def var n as int init 30.
def buffer b1 for book.
do transaction:
for each book:
delete book.
end.
do i = 1 to n: // to have multiple progressive brackets...
create book.
assign
book.isbn = string(i)
book.book-id = i
book.book-title = string(i).
release book.
end.
for each customer: delete customer. end.
create customer.
customer.customer = 1.
release customer.
end.
open query q for each book no-lock.
get first q.
get next q.
get next q.
get next q.
get next q.
get next q.
get next q.
get next q.
get next q.
get next q.
get next q.
do transaction:
find first customer exclusive-lock.
customer.customer = customer - 1.
release customer.
find b1 where b1.book-id = book.book-id no-error.
if avail b1 then do:
b1.book-id = b1.book-id * -1.
b1.isbn = b1.isbn + "foo".
end.
end.
get next q.
The goal is to make the edge record "disappear" without invalidating the query so that ProgressiveResults will no longer match the last record from the bracket with the first record of the next bracket. The invalidation leads to a bad state of the query.
#2 Updated by Alexandru Lungu over 1 year ago
- Status changed from New to WIP
- Assignee set to Alexandru Lungu
- % Done changed from 0 to 80
I investigated this case. It is about a fail-safe of ProgressiveResults to invalidate if things look not-alright. But this is not expected by the caching, which presumes that all data now relies in-memory.
I have a fix to check if the query was invalidated in the caching time. If so, simply do not presume the results are cached. I extended the case of #9853 to check the next books after invalidation and they are given in order. Everything is good.
It will take a bit to commit due to slowness in devsrv01.
#3 Updated by Alexandru Lungu over 1 year ago
- % Done changed from 80 to 100
- Status changed from WIP to Review
Committed 9853a/rev. 15838.
Constantin, please review.
#4 Updated by Constantin Asofiei over 1 year ago
I don't really understand the fix.
The line Results cached = cacheResults(old, true); where Results old = adapter.getResults(); will iterate (and consume) all the records in the adapter. Now, if we are not in preselect mode, we assume caching the results made the query go into dynamic mode.
But, the condition to enter this block is if (cacheOnReval || isPreselect()) - so, can the preselect be false, when entering this code?
Also, even so, what happens with the adapter's results if 'went into dynamic mode'?
#5 Updated by Alexandru Lungu over 1 year ago
But, the condition to enter this block is if (cacheOnReval || isPreselect()) - so, can the preselect be false, when entering this code?
It becomes false when calling cacheResults. In the process of iterating all results, we detect a mismatch in order so the query is invalidated. At that point, null is returned by ProgressiveResults simulating an "QueryOffEnd" and Results cached will contain only some records until the invalidation occurred.
Also, even so, what happens with the adapter's results if 'went into dynamic mode'?
It stays null because invalidation triggered AdaptiveQuery.resetResults, making results null and the adapter disconnected.
Motivation: The original problem was that the results were made null, but the adapter was set to the inconsistent Results cached. This is an invalid state. The adapter should have stayed disconnected in order to pick up a new results.
#6 Updated by Constantin Asofiei over 1 year ago
Can we rely on errorCondition instead of !isPreselect()? That's set like this:
AdaptiveQuery.resultsChanged() line: 2603 ProgressiveResults.fireResultsChanged(boolean) line: 1387 ProgressiveResults.fireResultsChanged() line: 1366 ProgressiveResults.getResults(ProgressiveResults$Locator) line: 1294 ProgressiveResults.moveTo(int, boolean) line: 1077 ProgressiveResults.moveTo(int) line: 925 ProgressiveResults.next() line: 483 AdaptiveQuery(PreselectQuery).cacheResults(Results, boolean) line: 5977 AdaptiveQuery.sessionEvent(SessionListener$Event) line: 2308 Persistence$Context.notifySessionEvent(SessionListener$Event) line: 6234 Persistence$Context.commit() line: 5607
#7 Updated by Alexandru Lungu over 1 year ago
Yes, this is what is happening in next. Would you prefer errorCondition? I guess it is less confusing than seeing isPreselect and then right after !isPreselect conditions.
#8 Updated by Constantin Asofiei over 1 year ago
Alexandru Lungu wrote:
Would you prefer
errorCondition? I guess it is less confusing than seeingisPreselectand then right after!isPreselectconditions.
Yes, is more explicit that we do not cache because something happened while trying to cache.
#9 Updated by Alexandru Lungu over 1 year ago
Committed 9853a/15839 and 15840, replacing !isPreselect with errorCondition. Please review.
#10 Updated by Constantin Asofiei over 1 year ago
Alexandru Lungu wrote:
Committed 9853a/15839 and 15840, replacing !isPreselect with errorCondition. Please review.
I'm OK with the change.
Also can confirm that #9777 is fixed.
#11 Updated by Alexandru Lungu over 1 year ago
- Status changed from Review to Internal Test
#12 Updated by Alexandru Lungu over 1 year ago
- Status changed from Internal Test to Test
Branch 9853a was merged to trunk rev 15841 and archived.