FWD » Core Development » Database

In a customer application, there are patterns where FIND is used to get a record, and from that record only its recid or a very small number of fields are used (while the denormalized table has 100s of fields). In FWD, this brings the entire record from the database (when found).

• at runtime, a mode (when activated) will allow tracking and reporting buffer's field usage (how many fields are read and written? from a buffer instance/location)
  • report the location where the buffer is defined
  • only permanent tables?
  • what about BUFFER-COPY or BUFFER-COMPARE?
• is it possible to expand FWD to allow a FIELDS clause at a FIND?
• at conversion time, determine if a FIND can be augmented with a FIELDS clause, to retrieve only the read fields (what about the written fields?)
  • what about buffers which get changed and then flushed (this should already work for the FOR EACH ... FIELDS case)?
  • what about extent fields - 4GL doesn't allow subscripted fields, like f1[1] in the FIELDS clause; and what about if the subscript for the field is not a constant, when accessed - use the entire field?
  • exclude buffers which are passed as arguments, or its handle is retrieved (via buffer b:handle), or used in BUFFER-COPY or BUFFER-COMPARE (other cases?)

#4400 is a mechanism to identify every query back to its original definition in source code. We could do a similar thing to track back to the buffer definition. We could pass a UUID as a parameter to the buffer definition.

• at runtime, a mode (when activated) will allow tracking and reporting buffer's field usage (how many fields are read and written? from a buffer instance/location)
  • report the location where the buffer is defined

I understand that this mode is 'collector' when the data is gathered and saved (persistently) somewhere. Some kind of cold run. When the flag is off, the collector stops, but the RAQs (I assume) will fetch only partial records, according to that information. The key here is when to switch to warm mode, to be sure the paths where the other columns of the table are not used, otherwise we will end with multiple requests instead of one, which could make things worse. I am assuming this happens automatically.

Otherwise we can 'preheat' the server offline with a run in which the data is collected from all known locations and provided as a separate resource for the production run, where this special mode is always off.

Just FYI: I also thought of this a while ago, but there was an issue with caching. The more variance we have in the FIELDS, then less DMOs can be cached to satisfy the FIELDS. In fact, incomplete DMOs are not cache-able or at least they can't be safely used from within the cache. In other words, the more FIELDS, the less session cache hits we will see, resulting is more reloads of DMOs. Empirically, I say that EXCEPT is rather used to skip heavy columns like extends, raw, longchar, etc. Maybe we can consider 3 types of loading. Instead of projection and full, we can consider projection, light and full. Light is basically the DMO without extents, raw, longchar, etc. that are usually excepted.