FWD » Core Development » User Interface

The widget variant of the CURRENT-ITERATION attribute is used by ADM2 (specifically smart.p, procedure applyEntry). At present, we incorrectly convert this form to the BUFFER object handle variant of the attribute.

From the Progress Language Reference for v11.0:

CURRENT-ITERATION attribute (widget objects)

   A handle for the current iteration of the frame or dialog box.

   Data type: HANDLE
   Access: Readable/Writeable

Applies to: DIALOG-BOX widget, FRAME widget

This attribute is a read-only attribute for dialog boxes.

The example code that references this attribute (this code is licensed as open source using the Possenet license, so it is OK to post here):

PROCEDURE applyEntry :
/*------------------------------------------------------------------------------
  Purpose:     Applies "ENTRY" to the first enabled and visible object 
               in the default frame (unless pcField is specified), 
               or in the first child which is a Frame.
  Parameters:  INPUT pcField AS CHARACTER -- optional fieldname; if specified,
               (if this parameter is not blank or unknown), then
               the frame field of that name will be positioned to. 
------------------------------------------------------------------------------*/

  DEFINE INPUT PARAMETER pcField  AS CHARACTER NO-UNDO.

  DEFINE VARIABLE hContainer      AS HANDLE NO-UNDO.
  DEFINE VARIABLE hWidget         AS HANDLE NO-UNDO.

    IF pcField = ? THEN pcField = "":U.

    /* {get Prop var [handle]} */   ASSIGN ghProp = WIDGET-HANDLE(ENTRY(1, TARGET-PROCEDURE:ADM-DATA, CHR(1))) ghProp = ghProp:BUFFER-FIELD('ContainerHandle':U) hContainer = ghProp:BUFFER-VALUE  
 .
    IF VALID-HANDLE(hContainer) THEN
    DO:     
       DO WHILE hContainer:TYPE = "WINDOW":U: 
           hContainer = hContainer:FIRST-CHILD.    
       END.
       ASSIGN hWidget = hContainer:CURRENT-ITERATION
              hWidget = hWidget:FIRST-TAB-ITEM.

          DO WHILE VALID-HANDLE(hWidget) :          
            IF hWidget:SENSITIVE AND hWidget:VISIBLE AND
              (pcField = "":U OR hWidget:NAME = pcField) THEN
            DO:
               /* In case the new widget with focus is not in the
                  CURRENT-WINDOW, change CURRENT-WINDOW to be the
                  window of the new widget; otherwise it won't actually
                  get focus. */
               DO WHILE (VALID-HANDLE(hContainer) AND 
                 hContainer:TYPE NE "WINDOW":U):
                   hContainer = hContainer:PARENT.
               END.
               IF hContainer:TYPE = "WINDOW":U THEN
                 CURRENT-WINDOW = hContainer.

               APPLY "ENTRY":U TO hWidget.                
               RETURN.
            END.
            ELSE ASSIGN hWidget = hWidget:NEXT-TAB-ITEM.            
          END.          
    END.  

    RETURN.
END PROCEDURE.

This is somewhat consistent with my theory. On a 5-down frame, you would expect 5 sets of widgets, but it looks like there are 6 sets in this case. I suspect there may be a set used for the column headers (e.g. -----). The other thing that is a bit strange is the numbering, which starts at 1000 and runs straight through to 1005. I would have expected that there would be gaps. This suggests that what may be returned is the field-group "widget" instead of the "num" widget.

From the v11.0 4GL docs:

FIELD-GROUP widget

A field group is the hidden parent of field-level widgets and child frames owned by a parent frame or dialog box. Thus, field groups are the actual children of frames and dialog boxes. A frame contains the following field groups:

• A background field group (which includes the frame header)
• For a one-down frame or dialog box: a single data field group containing field-level widgets and child frames
• For a multiple-down frame: one data field group for each data iteration in the frame

A field group has no visible representation. You cannot explicitly define or create field groups. They are generated automatically when frames are defined or created.

Please make some changes to try to use the returned handle to find out more information about the returned widget. For example, the NAME, TYPE, ROW, COLUMN, VISIBLE, PARENT... attributes should be informative.

Well, the current-iteration is not linked with an iterating block; lets take this case:

def var i as int.

form i with frame f1 5 down.

i = 0.
display i with frame f1.
put screen row 1 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).

down 1 with frame f1.
i = 1.
display i with frame f1.
put screen row 2 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).

up 1 with frame f1.
i = 2.
display i with frame f1.
put screen row 3 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).

pause.

┌──────────┐                 1000 0
│         i│                 1002 1
│──────────│                 1000 2
│         2│
│         1│
│          │
│          │
│          │
└──────────┘

def var i as int.

form i with frame f1 5 down.

do i = 1 to 5:
  display i with frame f1.
  if i <> 5 then down 1 with frame f1.
end.

up 2 with frame f1.
put screen row 1 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).
pause.

scroll up with frame f1.
put screen row 2 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).
pause.

scroll up with frame f1.
put screen row 3 col 30 string(frame f1:current-iteration) + " " + string(frame f1:current-iteration:first-child:screen-value).
pause.

┌──────────┐                 1000 3
│         i│                 1002 4
│──────────│                 1004 5
│         3│
│         4│
│         5│
│          │
│          │
└──────────┘

PS: Greg, you are correct, CURRENT-ITERATION points to a FIELD-GROUP widget.

Vadim Gindin wrote:

P.S. About numbering. What problem do you see in it? I viewed same values for newly created widgets in earlier task about CREATE WINDOW: 1000, 1001 and so on.

I think this is because a handle for the static widget is not created until it is accessed...

Also, there is one more thing for you to find: why is there an extra row reported in your test? You could walk each field-group and check what widgets it contains (and their screen-value).

Well, the current-iteration is not linked with an iterating block

Yes, this is exactly how we have implemented down frames in P2J too. The link is simply that some iterating blocks will trigger an implicit DOWN statement if a down frame is scoped to that block. That is really the only linkage to block iteration.

Thus, I think the current-iteration is just a pointer to the current row in a down frame.

Exactly.

But I think that a non-down frame will probably also have a current iteration which is always the same field-group widget. Vadim, make sure you test that case.

I thought that you meant 1-DOWN by the phrase "non-down frame".

Yes, you have it right.

About the tests at note 10: your conclusions are wrong. If you look at the repeating IDs, you will see that there are 6 repeating IDs with type field-group, not 5; and the frame is a 5-down frame. More, you are complicating the tests; there is no need for using temp-tables, a simple repeat is enough; if you think temp-tables are of interest and can affect this feature, you can go ahead and use them to confirm behaviour after the initial analysis with simple tests. But analysis should move from simple to complex, not viceversa.

def var i as int.
def var j as int.
def var h as handle.

j = 1.
repeat i = 1 to 6:
  display i j with frame f1 2 down.
  h = frame f1:current-iteration.
  put screen row j col 30 string(h) + " " + string(h:type) + " " + h:first-child:screen-value.
  j = j + 1.
end.

/* output frame's direct children */
h = frame f1:first-child.
j = 1.
do while h <> ?:
  put screen row j col 50 string(h) + " " +  h:first-child:type + " " + h:first-child:screen-value + " " +
                          string(if h:first-child:type = "literal" then "" else  h:first-child:label) .

  h = h:next-sibling.
  j = j + 1.
end.

┌─────────────────────┐      1000 FIELD-GROUP 1  1000 FILL-IN  i
│         i          j│      1002 FIELD-GROUP 2  1002 FILL-IN 5 i
│────────── ──────────│      1004 FIELD-GROUP 3  1004 FILL-IN 6 i
│         5          5│      1000 FIELD-GROUP 4  1007 LITERAL ---------- ----------
│         6          6│      1002 FIELD-GROUP 5
└─────────────────────┘      1004 FIELD-GROUP 6

• a 2 down frame and there are 3 repeating field-groups: 1000, 1002 and 1004
• checking the direct children of frame f1, the 1000, 1002 and 1004 field-groups are reported.
• if we change the loop to output 7 rows of data (i = 1 to 7), then is output:
  

  ┌─────────────────────┐      1000 FIELD-GROUP 1  1000 FILL-IN 7 i
  │         i          j│      1002 FIELD-GROUP 2  1002 FILL-IN  i
  │────────── ──────────│      1004 FIELD-GROUP 3  1004 FILL-IN  i
  │         7          7│      1000 FIELD-GROUP 4  1007 LITERAL ---------- ----------
  │                     │      1002 FIELD-GROUP 5
  └─────────────────────┘      1004 FIELD-GROUP 6
                               1000 FIELD-GROUP 7
  

Please extract rules describing the output you encounter, and document them in detail (not just "this is the output"). Use a structure like:

test program: some-test.p (the program must be in repository, not always here)
output file: some-test.4gl.txt (you can save the terminal screen by selecting it all with the mouse and and then pressing middle-click in a text document; @screen@ can be used for screen capturing too)
summary: testing 2-down frames and displaying 6 rows of data
details:
- there are reported 3 field-groups, with only two of them containing data after the loop ends.
- all 3 field-groups are reported as direct children of the frame.
- the separator between frame's header and body is reported as a frame child, but is not reported by current-iteration (which is expected).

• if the loop is i = 1 to 2, then only 2 field groups exist; this suggests that the 3rd field-group is just an "auxiliary" widget group to output data and is created after all initial field-groups are already used and the frame is "full".
• if we pause on i = 3 and output the frame's direct children, I think the data will be in the 3rd field-group; this 3rd field group will not be displayed until the existing iteration ends (i.e. down is silently called)
• check how the VISIBLE flag is reported at various iterations of the REPEAT loop. I doubt that for a 2-down frame, all 3 field-groups will be visible at once; I suspect that they will be made visible when they are brought on the screen (via implicit down or display).
• check what happens with frames scoped to the external program (for which an explicit down is required in the loop):
  

  form i with frame f1 2 down.
  repeat i = 1 to 6:
  display i with frame f1.
  down 1 with frame f1. /* an explicit down is needed in this case. */
  end.
  

Greg: AFAIK, P2J uses n widget rows for displaying the body of a n-down frame; assuming that 4GL indeed uses n+1 widgets for a n-down frame (with n != 1), then adding the additional set of widgets to the down body will not be trivial. My suggestion for a first phase is to add support for CURRENT-ITERATION without changing the client-side frame drawing.

AFAIK, P2J uses n widget rows for displaying the body of a n-down frame;

Correct.

assuming that 4GL indeed uses n+1 widgets for a n-down frame (with n != 1), then adding the additional set of widgets to the down body will not be trivial.

Also correct. There are several issues:

1. Fully understanding when exactly the extra field group is created and when it is actually used.
2. Force fitting that behavior onto our more simple n-based approach. I can imagine that there are many loops in our code that may be very sensitive to this new behavior.

My suggestion for a first phase is to add support for CURRENT-ITERATION without changing the client-side frame drawing.

I completely agree. Vadim: this means that the output of our test programs would be expected to be different and that is OK so long as the basic CURRENT-ITERATION behavior is correct.

The work to determine the rules when the auxiliary field-group is created and how the field-group rows are made visible will not be trivial. I understand it might be a little frustrating to ignore some behaviour while implementing other, but for now we need CURRENT-ITERATION to work only in terms of reporting the field-group for the active row (where the data will be displayed). We will add (and investigate) the auxiliary field-group part later (this auxiliary field-group doesn't even have to do with current-iteration, is related to how a down frame implements its body; current-iteration is what helps us understand how the frame's body "evolves" for a down frame).

• fix conversion of the CURRENT-ITERATION attribute, as currently a handle.unwrapBuffer is emitted (which will not work for widgets). You will need to find the interface which shares some other common attributes/methods between Buffer and Widgets. If this does not exist, create a new one and adjust unwrap and P2J buffer/widget resource definitions accordingly.
• check if CURRENT-ITERATION can be used with widgets
• implement the runtime for CURRENT-ITERATION, in Frame's case (without the auxiliary widget support). The implementation will be in FrameWidget (as this is the P2J class which is associated with the legacy frame resource).
• search if there is some tracking of the current frame row on the server-side. If there isn't, determine a way to send this info from the P2J client to the server-side. It can be via a new ClientExports API or via StateSynchronizer. If possible, I would prefer StateSynchronizer which will cache the row index on the server-side, avoiding trips down to the client each time CURRENT-ITERATION is called.

Let me know if you have any questions. Also, post notes about your implementation decisions, so we can discuss them, before making the code changes.

I tried to convert a simple procedure and it emitted correct (at first saw) java code myFrameFrame.currentIteration(). The class Buffer also contains the method Buffer.currentIteration() with its own implementation. I suppose that there is no conversion rules changes will be needed. What do you think?

At this moment there is a simple rule for CURRENT-ITERATION (method_attributes.rules):

1322                <rule>ftype == prog.kw_cur_iter
1323                   <!-- TODO: this can be used with frame/dialogs too -->
1324                   <action>hwrap = "Buffer"</action>
1325                   <action>methodText = "currentIteration"</action>
1326                </rule>

Yes hwrap = "Buffer" seems as not suitable in the case of frame, but do we really need conversion changes here? If so, how to distinguish different usages of CURRENT-ITERATION - frame and buffer?

Vadim Gindin wrote:

I tried to convert a simple procedure and it emitted correct (at first saw) java code myFrameFrame.currentIteration().

I guess you've tried a case like frame myframe:current-iteration, right? This will work, but a:

h = frame myframe:handle.
h = h:current-iteration@.

Yes hwrap = "Buffer" seems as not suitable in the case of frame, but do we really need conversion changes here? If so, how to distinguish different usages of CURRENT-ITERATION - frame and buffer?

• add a new interface, IterableResource (to not collide with java.lang.Iterable), which will define APIs to access the CURRENT-ITERATION. This interface will be inherited by the Buffer and CommonFrame interfaces. Also, remove the currentIteration() from Buffer.
• replace hwrap = "Buffer" with hwrap = "IterableResource" and add the appropriate handle.unwrapIterableResource in handle class (see how handle.unwrapBuffer is defined); note that handle.unwrap APIs are sorted by name.

This will allow proper access to this attribute, regardless of how is accessed (via handle var or frame).

1. you are missing the methods_attributes.rules file with the hwrap = "IterableResource" change.
2. don't forget about history entries and copyright year update
3. Buffer - remove the Buffer.currentIteration definition, as this will be picked up from the IterableResource
4. CommonFrame - this one should extend IterableResource, not FrameWidget. The interface annotated with a LegacyAnnotation is what defines the legacy resource in P2J.
5. FrameWidget.currentIteration - this can call directly into GenericFrame.currentIteration, via the protected GenericWidget.frame field (which is accessible from FrameWidget). So you are correct, the real implementation will be in GenericFrame.
6. there is no need to add LegacyAttribute annotations at the currentIteration implementation in FrameWidget and GenericFrame; remove those.
7. handle.iterableResource - the implementation is OK

The first obstacle is the CURRENT-ITERATION for Buffer is an integer but for Frame/DialogBox the CURRENT-ITERATION is a handle. By the way I didn't find Buffer:CURRENT-ITERATION in docs.. To overcome this obstacle I think we should refuse of the interface IterableResource (as we see that it's not common) and define separate rules for Buffer and Frame if it possible and assign hwrap="Buffer" or hwrap="Frame". The alternative way is to change Buffer.currentIteration() signature to make it returning handle in spite of integer. What do you think?

I can suppose following implementation skeleton.
1) There is Progress function FRAME-LINE that returns number of current logical row in frame iteration. This function is implemented. See GenericFrame.frameLine(). We need to get a handle of FieldGroup of this current logical row and we can use frameLine() for it.
2) If we look at GenericFrame.getFirstChild() we'll see that FieldGroup is created on the server-side and I couldn't find any FieldGroup processing on the client-side. FieldGroup is created with widgets list.
So if we want to implement currentIteration() in the same way we need the widgets list of the current logical row of the frame.

Questions.
1) What do you think about this variant of implementation?
2) If it is OK, how to get the widgets list of the current logical row?

Vadim Gindin wrote:

The first obstacle is the CURRENT-ITERATION for Buffer is an integer but for Frame/DialogBox the CURRENT-ITERATION is a handle.

• move back one step and remove the IterableResource interface. For now we will assume that the CURRENT-ITERATION is supported only by frames, and we will ignore the BUFFER case.
• in methods_attributes.rules, change the hwrap assignment to hwrap = "Widget". Add a comment knowing that we force it this way knowingly.
• in the future, when we will need to add support for BUFFER:CURRENT-ITERATION, we will probably use some conversion-time hints to distinguish between these two cases. Add a comment at the hwrap = "Widget" assignment like: "we will need to use conversion-time hints to distinguish between BUFFER and FRAME usage". This will be needed because Java does not allow overloading by the return type (you can't have two methods with the same name and parameters, but different return types).

I can suppose following implementation skeleton.
1) There is Progress function FRAME-LINE that returns number of current logical row in frame iteration. This function is implemented. See GenericFrame.frameLine(). We need to get a handle of FieldGroup of this current logical row and we can use frameLine() for it.

• a named stream (via DISPLAY STREAM)
• the unnamed stream (redirect default output with a OUTPUT TO statement)
• the terminal, via unnamed stream (an OUTPUT TO TERMINAL statement).

If CURRENT-ITERATION behaves the same way as FRAME-LINE, then we can use FRAME-LINE as the index.

2) If we look at GenericFrame.getFirstChild() we'll see that FieldGroup is created on the server-side and I couldn't find any FieldGroup processing on the client-side. FieldGroup is created with widgets list.
2) If it is OK, how to get the widgets list of the current logical row?

After a quick look at GenericFrame, I think on server-side we keep only one field-group for any kind of frame. We should be keeping n field-groups, for a n-down frame... I think this was because in P2J, when a i.e. DISPLAY is used, this always targets the current row, thus we don't need to distinguish between rows on server-side; this is the client-side's job.

I'm thinking about an approach for field-groups on server-side, I'll get back to you. In the mean time, check the frame-line and current-iteration in the scenarios above.

Constantin notes:

Considering that CURRENT-ITERATION is used in the server project only in the adm2/smart.p and the usage is also dependent on the FIRST-/NEXT-TAB-ITEM attributes:

    IF VALID-HANDLE(hContainer) THEN
    DO:
       DO WHILE hContainer:TYPE = "WINDOW":U:
           hContainer = hContainer:FIRST-CHILD.
       END.
       ASSIGN hWidget = hContainer:CURRENT-ITERATION
              hWidget = hWidget:FIRST-TAB-ITEM.

          DO WHILE VALID-HANDLE(hWidget) :
            IF hWidget:SENSITIVE AND hWidget:VISIBLE AND
              (pcField = "":U OR hWidget:NAME = pcField) THEN
            DO:
               /* In case the new widget with focus is not in the
                  CURRENT-WINDOW, change CURRENT-WINDOW to be the
                  window of the new widget; otherwise it won't actually
                  get focus. */
               DO WHILE (VALID-HANDLE(hContainer) AND
                 hContainer:TYPE NE "WINDOW":U):
                   hContainer = hContainer:PARENT.
               END.
               IF hContainer:TYPE = "WINDOW":U THEN
                 CURRENT-WINDOW = hContainer.

               APPLY "ENTRY":U TO hWidget.
               RETURN.
            END.
            ELSE ASSIGN hWidget = hWidget:NEXT-TAB-ITEM.
          END.
    END.

Can 2126 be re-assigned for M11 (to which NEXT/PREV-TAB-ITEM are assigned)? My thought is, even if we manage to bring CURRENT-ITERATION to a working state (i.e. for 1-down frames should work properly), this code will not work without the other attributes.

Vadim, using the test at note 15, check how the body of a down frame evolves in the cases I've mentioned at note 22. We need to determine the rules describing when a widget row is created for a down frame.

Constantin Asofiei wrote:

Vadim, using the test at note 15, check how the body of a down frame evolves in the cases I've mentioned at note 22. We need to determine the rules describing when a widget row is created for a down frame.

I only tried to move frame widgets printing before DISPLAY statement. The difference is not big. Only screen values was changed, but widgets row are stayed as earlier.

There is 2 variants of when they could be created: during iteration processing or during DISPLAY processing. When I placed DISPLAY statement at the end of iteration I found that widget row is not changed. It means, that dynamic widgets creation is going during iteration processing. Moreover, this variant is seemed more straightforward.

What else can I do to find when a widget row is created?

Vadim Gindin wrote:

What else can I do to find when a widget row is created?

Check your findings with cases when the frame is not used in a iterating loop, i.e. DISPLAY and DOWN are explicitly called, like in the test from note 6.

In the test on note 15, as the frame is scoped to the repeat block, an implicit down is executed on each iteration; test what happens if the frame is scoped to the external procedure and a down is needed in the repeat loop, like this:

form i with frame f1 5 down.
repeat i = 1 to 10:
   display i with frame f1.

   down with frame f1.
end.

More, enhance your tests checking which resource is saved in h after a h = i:handle in frame f1. statement is executed (both with iterating and non-iterating block).

I'm repeating that i:handle peculiarities are the same in iterating and non-iterating tests. All tests I have are committed to bzr in the folder current-iteration

Vadim, good findings, but you are misinterpreting something. When you are printing the frame's children in no-loop-ihandle.p:

  h = frame f1:first-child.
  do while h <> ?:
    put screen row j col 80 string(h) + " " +  h:first-child:type + " " + h:first-child:screen-value + " " + string(if h:first-child:type = "literal" then  "" else  h:first-child:label).

    h = h:next-sibling.
    j = j + 1.
  end.

To summarize, we can assume (for down frames):¶

1. accessing a frame's widget via its name (i.e. i in frame f1) will always point to the corresponding widget, in the field-group returned by current-iteration.
2. the frame starts with only one field-group (for the first row). When advancing to the next row, if a new field-group is not already created for the row, it will be created.
3. there is a field-group associated with the separator between the label and the frame body, but no field-group for the actual labels:
   - please test the NO-LABELS and SIDE-LABELS clauses for the frame phrase, and see how this field group is reported.
   - I assume there is only one LITERAL with the separator text for all widgets, not one LITERAL for each widget - test how this is reported when the frame has more than one widget.

What else we need to determine:¶

1. is FRAME-LINE always in sync with current-iteration?
2. what is the behavior when the frame is redirected to a stream (named, unnamed, or redirected to terminal).
3. when the frame is not scoped to a repeating block, what happens when i.e. DOWN 2 is called, and the down body is not yet fully initialized (i.e the next two rows are not yet created)? Does this create 2 rows for frame-line + 1 and frame-line + 2 or just one, for frame-line + 2?

About an implementation approach:¶

The problem in P2J is that on both server-side and client-side we have only one set of widgets, used to display the data on each row. Even if on client-side the frame's down body contains n sets of widgets (each widget on a row being a distinct instance), the same set of widget IDs is used for each row.

• on client-side, although the down body for a frame (saved in Frame.downBody) contains n rows, the widget IDs are the same for each row (as noted in Frame.fillDownBodyParts, only the first row is real and the rest are "artificial"). This will not make it possible to properly implement widget-level statements and attributes for a down frame, like HIDE i IN FRAME f1 or the VISIBLE attribute, as we can't distinguish between the widget on row 1 and the widget on row 2 by their ID.
• as a side note, the widget ID on client-side is different from the resource ID on server-side, so keep this in mind.
• on server-side, we don't maintain the down body; all we have is a single set of widgets. Conversion-wise is OK (as in 4GL, at a certain time only one widget can be referenced via its name in a widget phrase like i IN FRAME f1), so we don't need to change how the frame is converted; the widget getters in the frame definition interface are OK as they are. What we need to change is which widget is returned by the widget getter (i.e. a frame.widgetI() call), depending on the row on which the frame is currently standing.
• we need to link the down body on the server side with the widgets on the client side. But the client side currently requires that the Frame.downBody contain a number of widgets equal to the frame's down value; if possible, I don't want to change this. Instead, I think we should try to implement this "fixed down body" behavior on server-side too, with a tweak: we maintain some artificial widgets for rows which are "pending to be created". Same on client-side: it starts with only one real row, the others being artificial widgets. As new rows are added, they are replaced with real widgets (which have server-side counterparts). But we need to be careful to still make these "real" widgets linked to only one widget, so that i.e. the label is processed only once (there might be more to take care beside the label).

Please read carefully these notes and let me know what you think. Also, continue testing to find the rules describing when the extra field-group is created for the down frame, as is best to implement this too now than later. Keep in mind we will need to test various attributes for the widgets in a row (like VISIBLE, HIDDEN, SCREEN-VALUE, etc), so create tests for these too. Once the rules describing all of this start to make sense, please try to move into writing testcases which encode the output you've attached to the previous notes (via i.e. IF i in frame f1:screen-value <> "1" then message "something is wrong"), instead of using PUT SCREEN. I understand this will make the tests more complex and we will still have to do some side-by-side comparison (for checking how the frame is drawn), but it will make it easy for us to find problems and regressions.

As I can understand this, it sounds reasonable. I just have several questions about details. See below.

Constantin Asofiei wrote:
..

I've been thinking how to leverage existing implementation (to minimize code changes both on server-side and client-side), but I don't think we can get away without some major changes. Some findings/notes/ideas:

• on client-side, although the down body for a frame (saved in Frame.downBody) contains n rows, the widget IDs are the same for each row (as noted in Frame.fillDownBodyParts, only the first row is real and the rest are "artificial"). This will not make it possible to properly implement widget-level statements and attributes for a down frame, like HIDE i IN FRAME f1 or the VISIBLE attribute, as we can't distinguish between the widget on row 1 and the widget on row 2 by their ID.

Why it is needed to have downBody array with only the first row real? Do I correctly understand: two widgets in different rows will have the same ID value? Is that the reason why it is not possible to get correct widget by ID? But at the same time we know the current-line so first of all we can get correct row and then we can get the widget. Will it work?

• as a side note, the widget ID on client-side is different from the resource ID on server-side, so keep this in mind.
• on server-side, we don't maintain the down body; all we have is a single set of widgets. Conversion-wise is OK (as in 4GL, at a certain time only one widget can be referenced via its name in a widget phrase like i IN FRAME f1), so we don't need to change how the frame is converted; the widget getters in the frame definition interface are OK as they are. What we need to change is which widget is returned by the widget getter (i.e. a frame.widgetI() call), depending on the row on which the frame is currently standing.
• we need to link the down body on the server side with the widgets on the client side. But the client side currently requires that the Frame.downBody contain a number of widgets equal to the frame's down value; if possible, I don't want to change this. Instead, I think we should try to implement this "fixed down body" behavior on server-side too, with a tweak: we maintain some artificial widgets for rows which are "pending to be created". Same on client-side: it starts with only one real row, the others being artificial widgets. As new rows are added, they are replaced with real widgets (which have server-side counterparts). But we need to be careful to still make these "real" widgets linked to only one widget, so that i.e. the label is processed only once (there might be more to take care beside the label).

Do you mean that we should implement something like downBody on server side and make it synchronized with it's "brother" on the client side? If it is so we should includes all array of widget to light object ClientState. It will make this object not so light ) as it is now. Isn't it?

What do you mean when you are saying "the frame starts with only one field-group (for the first row). When advancing to the next row, if a new field-group is not already created for the row, it will be created."

Here is the first screen.

┌──────────┐ 1) 1000 FIELD-GROUP 1                        1000 FILL-IN 1 i
│         i│                                              1002 LITERAL ----------
│──────────│ 2) 1004 FIELD-GROUP 2  1000:visible:yes      1000 FILL-IN 1 i
│         1│                                              1004 FILL-IN 2 i
│         2│                                              1002 LITERAL ----------

Yes, the frame starts from only one field-group - for the first row (1000), but when it goes to the next row it creates another field-group for it (1004) and so on until the frame became full. The next frame starts also with new field-group (1006) and common amount of field-groups is becoming DOWN-size + 1. Then created field-group set is used in cyclic order i.e. next field-group for the next row will be the first created field-group (1000) and so on (1004, 1006...).

Field-group content is changes. It starts from the only one Fill-in field and LITERAL. But further this field-group is being extended by one Fill-in field every iteration until those count will become DOWN-size + 1. At this step extending of field-group stops and it's size stay unchanged for next iterations.

Vadim Gindin wrote:

Why it is needed to have downBody array with only the first row real?

The problems are in the code on P2J client-side, which draws the frame. This currently assumes the down body is fixed, so we need to have a fixed number of widgets at any time. I want to avoid rewriting the frame drawing code too much, because that part is hugely complicated. The idea was that initially only the first row is real; as real widgets are added for the other rows, the fake ones will be replaced with real ones.

Do I correctly understand: two widgets in different rows will have the same ID value?

No, you understood wrong. This is what we are trying to move away from: currently, on P2J client side, the same widget ID is used by all rows. This is not correct in 4GL terms, as a frame body has unique widgets for each row and doesn't reuse widgets.

Is that the reason why it is not possible to get correct widget by ID? But at the same time we know the current-line so first of all we can get correct row and then we can get the widget. Will it work?

When performing frame-level operations (drawing, hiding, etc), the protocol between the server and client side uses widget IDs (which, as I noted before, are not resource IDs). Thus, when i.e. performing HIDE i IN FRAME f1 in a down frame, we need to determine the widget ID for i in current-iteration and send that to the client. And if we reuse the ID, we will not be able to distinguish between i on current-iteration and i on another row.

Do you mean that we should implement something like downBody on server side and make it synchronized with it's "brother" on the client side?

Yes, you got it. We need to sync the downBody on server and client side.

If it is so we should includes all array of widget to light object ClientState. It will make this object not so light ) as it is now. Isn't it?

We will not need to send entire frame def back to the client. We will only need to send something like "for row i, create real widgets instead of fake ones and send back the IDs".

Vadim Gindin wrote:

What do you mean when you are saying "the frame starts with only one field-group (for the first row). When advancing to the next row, if a new field-group is not already created for the row, it will be created."

• 1, 2 - first iteration
• 3, 1 - second iteration (2 is the "extra")
• 2, 3 - third iteration (1 is the "extra")

This looks like 4GL keeps a pointer in a circular list, to the field-group displayed on the first row. Please confirm/infirm this with testing; let me know if you have other ideas about this.

Something else I would like for you to check: what happens if CLEAR FRAME is executed after the frame is on its second "fill" (the extra field-group was created and now we are displayed again from row 1?) Is the field-group order affected?

Here are next tests.

1. Test: frame_line.p
Description: The goal is to find out is the FRAME-LINE function and CURRENT-ITERATION attribute are synchronous. We are just running repeat cycle and output both of them.
Result:

┌──────────┐       1) 1 1000
│         i│       2) 2 1000
│──────────│       3) 1 1001
│         9│       4) 2 1002
│          │       5) 1 1000
└──────────┘       6) 2 1001
                   7) 1 1002
                   8) 2 1000
                   9) 1 1001

2. named, unnamed streams and redirecting to terminal
I only say that it works in the same way (several field-groups created for every frame) only for redirecting to terminal variant. For other variants - there only one field-group is created during execution.
I will commit procedure and output results later. At this moment is not simple for me to compose good all information with DISPLAY statement in spite of PUT SCREEN and to format overall output

3. DOWN-2
Test: down2.p
Goal: Find out is the Progress4GL creates or process all rows in the case where DOWN is explicitly called without repeat cycle and it's size more than one, so it skips some rows count. Find out how such DOWN statement affects the frame output.
Details: Executing the following sequence of statements DOWN1, UP1, DOWN2, DOWN1.
Results:

┌──────────┐  0) 1000 FIELD-GROUP no_parent
│         i│  1) 1001 FIELD-GROUP 1000
│──────────│  2) 1000 FIELD-GROUP no_parent
│         2│  3) 1002 FIELD-GROUP 1001
│         1│  4) 1003 FIELD-GROUP 1002
│         3│
│         4│
└──────────┘

i)   current-iteration:handle   current-iteration:handle:type    current-iteration:prev-sibling:handle

Normal order (all @DOWN 1@)                  Order with @DOWN 2@ 
1001                                         1001
1005                                         1005   
1001                                         1001 
1005                                         1007  
1007                                         1001

Please commmit the tests for note 36/37 to bzr.

About cyclic order of field groups I confirm it. All tests show it.

OK, this means things are starting clear up.

About your DOWN 2 findings: this means that if we do a DOWN n for a frame (where n is the number of rows), field-groups are created for all rows, right? And if DOWN n+1 is executed, the extra field-group should be created too.

About Fill-in fields - they have equal IDs at all iterations: 1000, 1004, 1006. May be they are reused?

No, they are not reused in terms that a new resource is created with the same ID. The behaviour should be this: for each row, its field-group is created when that line is reached via down. When the row's field-group is created, its "columns" (i.e. widgets for each frame element) are created too. When frame reaches a cetain row i.e. a second time, it will determine which field-group will be displayed and use that field-group for the row. The widgets inside a field-group are not moved around, they remain associated with that field-group.

1. CLEAR FRAME

I need to take a look at your testcase first. But I think you should test using other values for the frame's down, beside 2-down.

2. NO-LABELS:

Check what happens if the field's label is changed, using the fill-in for a certain row; do all the fill-ins report the same label? Does the UI change?

additional LITERAL

What I was referring to was to use a frame with two widgets (i.e. FORM i j) and check how the separator is reported: in a single LITERAL (with its value something like ---- ----) or something else.

The only unclear moment is the algorithm of Fill-in filling with values for me.

I'm not sure what you are refering to. Do you refer to implementing this behavior in P2J or to interpreting 4GL runtime (when fill-in's get "populated" with data)?

Constantin Asofiei wrote:

The only unclear moment is the algorithm of Fill-in filling with values for me.

I'm not sure what you are refering to. Do you refer to implementing this behavior in P2J or to interpreting 4GL runtime (when fill-in's get "populated" with data)?

I'm referring to interpreting 4GL runtime.

I suppose I can formulate some rule, interpreting the rules, that Progress uses to fill fields with values.
We have 2 DOWN frame and the repeat cycle. At every iteration frame contains 1 or 2 numbers in following order (each column - is iteration):

1st iteration    2nd iteration    3rd iteration   4th iteration      5th iteration     6th iteration
1                 1                 3                3                5                5
                  2                                  4                                 6

1st iteration    2nd iteration    DOWN call    3rd iteration   4th iteration     DOWN call    5th iteration
1000  1          1000  1                       1000            1000  4                        1000
1004             1004  2                       1004            1004                           1004  5
1006             1006                          1006  3         1006  3                        1006

1st iteration    2nd iteration    DOWN call    3rd iteration   4th iteration     DOWN call    5th iteration
1000  1          1000  1                       1006  3         1006  3                        1004  5
1004             1004  2                       1000            1004  4                        1006  
1006             1006                          1004            1000                           1000

As we are getting close to the time for implementation, I want to mention something.

I would like to consider an implementation where the only changes are on the server side. I like most of the concepts that Constantin previously laid out for implementation ideas. The only difference is that I would like to explore the idea of having a completely fake set of widgets on the server side that duplicate this multiple field-group/sets of widgets behavior. But I don't see a strong reason to actually map all of these to different widgets on the client side. I prefer an approach where multiuple virtual widgets on the server resolve to the correct widget on the client. The client side logic for down processing is really complex. I am worried that if we try to change it to tightly couple that with the server side, that we will break things badly.

Vadim Gindin wrote:

I just don't understand the logic of field-groups. Why authors decided to use spare field-group?! I just don't see any necessity of it. Of course I'm not a guru in Progress..
Moreover I reread the doc "Progress Programming Handbook" Chapter "Field-group widgets". It is also helpful..

The field-groups are used to group widgets together (as their names say). It is a lot easier to i.e. hide/remove the field-group instead of going through each and every one. More, I think there might be cases in 4GL when the child is visible but the parent is not - thus the child is not displayed, because the parent is not visible. You can also look at them as similar to the panel in swing/awt.

I wrote test to find out how UP statement behaves and how the field-groups are created.

Your test is stuck in an edge case, that's why the results are confusing. Make the frame i.e. 10 down and for the first iterate call, pass -4 as the UP step. As the frame is on the top row, it will go on the last row after the UP, not on the 7th row. This is what I've meant: when the UP/DOWN lands outside of frame, going past the edge will stop on the last/first row (depending on which direction you go); it will ignore the fact that there are some reminder steps.

Constantin Asofiei wrote:

The field-groups are used to group widgets together (as their names say). It is a lot easier to i.e. hide/remove the field-group instead of going through each and every one. More, I think there might be cases in 4GL when the child is visible but the parent is not - thus the child is not displayed, because the parent is not visible. You can also look at them as similar to the panel in swing/awt.

That is clear. I only meant the spare field-group here. Sorry I formulated it poorly.

I wrote test to find out how UP statement behaves and how the field-groups are created.

Your test is stuck in an edge case, that's why the results are confusing. Make the frame i.e. 10 down and for the first iterate call, pass -4 as the UP step. As the frame is on the top row, it will go on the last row after the UP, not on the 7th row. This is what I've meant: when the UP/DOWN lands outside of frame, going past the edge will stop on the last/first row (depending on which direction you go); it will ignore the fact that there are some reminder steps.

Now I understand.
I modified my test, stayed only one UP call and tried it with different values. Interesting thing, the frame always was staying unchanged. Only the second UP call moved the value to the last row as you said right. Where the first call wrote the value, to the spare field-group? Docs also mentioned a background field-group. It's completely unclear for me how to discover it and distinguish it from a spare field-group. OK, It was just a excursus.. The third sequential UP call displayed it's value using the step from previous UP call (from the second). The fourth UP call displayed it's value using the step from the third call and so forth. It seems that is the strange logic. Take a look at this sequence of calls and its results.

run iterate (input 0, input -1). 
/* no value is displayed in a frame. 
   Probably the value have got into spare field-group 
   and the next call makes this spare group visible.
   Moreover, the step will be 1 independently of the value, 
   specified in the call.
 */

/* each followed call uses the step from the previous call */
run iterate (input 1, input -4).
run iterate (input 2, input -2).
run iterate (input 3, input -3).
/* frame is full. See the first result below */

run iterate (input 4, input -6).
/* this call shows the next frame that is empty (as the first call makes) */

message "next up frame".
run iterate (input 5, input -1).
run iterate (input 6, input -3).
run iterate (input 7, input -1).
run iterate (input 8, input -1).
/* frame is full. See the second result below */

The first argument is a screen value i, that will be displayed in the first column (in the right part). The second argument is the step. Negative step means UP call (positive step - DOWN). It will be displayed in the fourth column.

Here first frame and the correspondent output. The fourth column shows current step. The first column shows the screen-value i.

┌──────────┐            0:       1   1000()        -1      1001 : 2    -1
│       val│            1:       5   1002()        -4      1003 : 4    -5
│──────────│            2:       7   1004()        -2      1001 : 2    -7
│         4│            3:       10   1006()        -3      1009 : 10    -10
│          │          
│         3│
│          │
│         2│
│          │
│          │
│          │
│         1│
└──────────┘

We can see here (1st and 4th columns) that for 1 the step is always 1 as it is transition to the next frame. For 2 the step is 4. For 3 the step is 2. For 4 the step is 3.

Here is the second frame and the correspondent output:

┌──────────┐            0:       1   1000()        -1      1001 : 2    -1
│       val│            1:       5   1002()        -4      1003 : 4    -5
│──────────│            2:       7   1004()        -2      1001 : 2    -7
│          │            3:       10   1006()        -3      1009 : 10    -10
│          │            4:       1   1008()        -6      1001 : 2    -16
│          │            5:       2   1000()        -1      1000 : 1    -17
│          │            6:       5   1010()        -3      1003 : 4    -20
│         8│            7:       6   1002()        -1      1002 : 3    -21
│         7│            8:       7   1012()        -1      1001 : 2    -22
│          │         
│         6│
│         5│
└──────────┘

And here we can see that for 5 step is -1 (transition). For 6 the step is 1. For 7 the step is -3 and for 8 the step is 1.

That's it. What do you think?

Marasmus... there is no shift. It's my mistake. It because of wrong order of operations: displaying the value, executing DOWN call, printing info. I'm sorry for that.

Vadim Gindin wrote:

There are an array of predefined FIELD-GROUP ids and we are calculating actual index (fgidx) in this array.

Do not do this. Resource IDs are not the same in 4GL and P2J; also, resource IDs are not "fixed" - depending on the order of accessing the field-group or other resources as handles, the IDs may be different. Think of another way to mark a certain field-group: if you need to identify it (without using screen-value), maybe the private-data can help; you can populate all field-groups for the frame (i.e. fill all rows with data, then clear the frame so it is repositioned back to row 1), then iterate through all field-groups and set private-data to some unique identifier (maybe their position in the frame's children).

I assume it is possible to access the PRIVATE-DATA for a FIELD-GROUP widget in 4GL code, which means using that attribute for our own internal P2J purposes could cause problems down the road. Instead, I prefer to just add a data member for this purpose.

I tried to find out what algorithm is used to generate new field-groups and to reuse existed. The case 3 in my previous note was about how the new field-groups are generated. If I will mark them after they are created I won't be able to find the way they were generated..

I found that I can use the NAME attribute for that purpose (mark the field-group). This attribute has unknown value for FIELD-GROUP widgets. Further I can use the following algorithm:
1. I'm running any of my "current-iteration" procedures.
2. If I've met some FIELD-GROUP first time (NAME attribute is ?) I will assign the next generated name to it, for example "grp1", "grp2" and so on.
3. If the FIELD-GROUP already has the NAMaaE attribute with some value not equal to ?, it will mean that this FIELD-GROUP is now reused.

What do you think about this attribute?

The NAME can still be interrogated from converted 4GL code, right? That means there could be code that relies upon the "trick" of checking for unknown value in a name to avoid some processing on those widgets.

I think it is best to just create a separate data member that is only used for our internal purposes.

Greg: Vadim I think is talking about a way to identify a field-group without using the handle ID in 4GL code, not in P2J runtime.

Vadim: what are your concerns about field-group creation? Did you find that there might be empty rows between two field-groups? Or something else? BTW, you can use an extent var of type handle in 4GL, if you want to save them (in the order of their creation).

Vadim Gindin wrote:

Field-group creation is seemed straightforward except UP direction.

• a 10 down frame
• use UP 1 calls to reach row 7.
• use DOWN 1 calls to reach row 3.
• how are the rows reused?

Well, I finished the test for the case of any sequence of DOWN and UP calls.
Test: mixedupdown1.p
Description: The source is the array of steps (inp1, inp2 or other). Concrete source array is specified below in the cycle which runs iterate procedure.
Result: The test outputs some information. See at the columns 2 and 4. Column 2 contains real generated field-groups and column 4 contains emulated field-group and the values of these columns should be equal.

Remained case is: initialization of field-group set. As you know at the start of frame filling field-groups are created when it really needs. It goes until generated field-groups count is equal to frame-size + 1. So mixedupdown1.p works for all cases when all frame-size + 1 field-groups are generated and remained case is the algorithm of field-group generation until there are frame-size + 1 field-groups. This case is strongly confusing.
Test: fgset_init.p
Description: I just tried to find out that algorithm of initial field-group generation and I tried to output all frame field-groups after each initial iteration.
Results:
1. Strange, but access sibling attributes (PREV-SIBLING, NEXT-SIBLING) of field-groups itself distorted generated groups. It also true for FIRST-CHILD attribute.
For example:

Just current-iteration. Field-groups are:
1000, 1001, 1002, 1003, ...
with FIRST-CHILD output:
1000, 1002, 1003, 1004

I'm trying to identify when the Progress creates auxiliary field-group. There are possible 2 variants:
1. Progress creates auxiliary row right after it creates the current-row.
2. Progress creates auxiliary row when DOWN or UP statement (PREV-SIBLING, NEXT-SIBLING, FIRST-CHILD or some others) is called.
But I don't have possibility to really find out how the Progress does it. The statements that can be used to go through field-groups itself distort target field-group set.

Short example. Suppose I want to execute DOWN 2 statement. After it is executed I want to know what field-groups are contained in a frame. When I will try to traverse trough frame field-groups I will inevitably distort target set.

What should I do further?

1. Strange, but access sibling attributes (PREV-SIBLING, NEXT-SIBLING) of field-groups itself distorted generated groups. It also true for FIRST-CHILD attribute.

• there is a global counter for handle IDs, which starts on 1000
• all static resources are maintained in a different data structure; when you want to access the handle for a static resource, 4GL will assign the next available handle ID to it and map this ID to the static resource. The key here is "access": a handle ID is not assigned until you actually access that resource's handle.
• the idea here is that the assigned ID depends on the order on which you access the handles for static resources. Thus, accessing resource R1 then R3 will map i.e. R1->1000 and R3->1001, but if you access R1, R2, R3 you will have R1->1000, R2->1001, R3->1002.

I'm looking through your tests, maybe I get lucky and find a pattern about all the frame row creation stuff. I'll get back to you hopefully today.

• each frame starts with only one field-group; frame initialy is positioned on row 1, which is associated with the (currently) single available field-group;
• assume that field-group FG is the field for current-iteration. If we are moving UP, assume we are moving to the left (prev) sibling and if we are moving DOWN, assume we are moving right (next) sibling. The frame's children (i.e. field-groups) are kept in a linked-list. Left-sibling of the first element is assumed to be the last element and right-sibling of the last element is assumed to be the first element (to emulate a circular linked list).
• all rows which have an associated field-group are marked so (there will be a mapping row-to-fieldGroup, in a R2FG map)
• when frame is cleared (explicit or implicit), the R2FG map is cleared too.
• when we need to move UP to target row TR:
  - if TR has an associated field-group, that one will be used
  - if TR has no associated field-group:
  1. if FG's left-side (circular) field-group has no associated row, then it will be used.
  2. if FG's left-side (circular) field-group has an associated row, then a new field-group will be created and added: = to the end of the frame's children, if FG (for current-iteration) is the first element in the list = at the left-side of FG, if FG has a left-side sibling.
• when we need to move DOWN to target row TR:
  - if TR has an associated field-group, that one will be used
  - if TR has no associated field-group:
  1. if the right-side (circular) field-group has no associated row, then it will be used.
  2. if the right-side (circular) field-group has an associated row, then a new field-group will be created and added at the right-side of FG. But keep in mind that this will not become the first element, will remain the last element in the list.

Note that the field-group associated with the label separator (which has a single LITERAL child, with a -----------like screen-value) is always the last child of the frame.

About that pesky extra field-group 4GL keeps adding. I think this can be explained with the fact that the field-group is created BEFORE the implicit frame clear is executed. Thus, as we are in a state when all field-groups are referenced by rows and all rows have an associated field-group, we have no choice but apply the "add new field-group (depending on direction)" rule. This field-group will be associated with the next row, after the implicit frame clear. Once the frame is cleared, all fieldGroup-to-row associations are removed too, so is no longer needed to create a new field-group, next time we call UP or DOWN.

• assume we are on frame-line FL
• if FL + steps lands outside of frame, then:
  - a number of steps - 1 steps are executed, and field-groups are created for each intermediate row, as if UP 1 or DOWN 1 is executed steps - 1 times.
  - one more step is executed, to land on the last row (for UP) or first row (for DOWN). during this time, the implicit frame clear is done and a the extra field-group be created.
• if FL + steps lands inside the frame, then field-groups are created for each intermediate row, as if UP 1 or DOWN 1 is executed steps times.

See the up_down_field_group_test.p test I've added to bzr: the frame's children (without the LITERAL one) are shown after each UP or DOWN statement. The field-group's name tells you when it was accessed first (i.e. if the frame-group's name is unknown, then it means it was created just now). Adjust the step calls in up_down_field_group_test.p as you need. See in the following output how the resource's handle ID (last column) follows the internal counter (i.e. if lines are ordered by the internal counter, the ID will be ordered too):

      FIELD-GROUP   0 1647
      FIELD-GROUP   1 1651
      FIELD-GROUP   2 1653
      FIELD-GROUP   3 1655
207   FIELD-GROUP  13 1675
206   FIELD-GROUP  12 1673
205   FIELD-GROUP  11 1671
204   FIELD-GROUP  10 1669
203   FIELD-GROUP   9 1667
202   FIELD-GROUP   8 1665
201   FIELD-GROUP   7 1663
200   FIELD-GROUP   6 1661
      FIELD-GROUP   5 1659
      FIELD-GROUP   4 1657

Let me know if you see something wrong with my analysis or you have questions.

The case with UP 1 and down 1 looks good. The question is why have you used additional display statement before the loop in the function step? If steps = 1 the function will output 2 values to the frame..

I found some confusing results with placement of values in the rows of the frame. I changed the down-size to 5 and steps to 3 -7 11 and here what I got.

Here are 2 neighbor iterations.

┌──────────┐       2     FIELD-GROUP   0 1000
│         i│       1     FIELD-GROUP   1 1004
│──────────│       0     FIELD-GROUP   2 1006
│         2│
│         1│
│         0│
│          │
│          │
└──────────┘

┌──────────┐       2     FIELD-GROUP   0 1000
│         i│       106   FIELD-GROUP   1 1004
│──────────│       1     FIELD-GROUP   2 1006
│         2│
│       106│
│         1│
│          │
│          │
└──────────┘

It's a first run of step(1, -7) command. Take a look at some "shift" of the value "1" from the second row to the third. FIELD-GROUP are not affected. This effect appears not every time. Here is another example:

┌──────────┐       101   FIELD-GROUP   0 1000
│         i│       102   FIELD-GROUP   1 1004
│──────────│       103   FIELD-GROUP   2 1006
│       100│       104   FIELD-GROUP   3 1008
│       101│       100   FIELD-GROUP   4 1010
│       102│
│       103│
│       104│
└──────────┘

┌──────────┐       210   FIELD-GROUP   0 1000
│         i│       102   FIELD-GROUP   1 1004
│──────────│       103   FIELD-GROUP   2 1006
│         2│       104   FIELD-GROUP   3 1008
│       210│       2     FIELD-GROUP   4 1010
│       102│
│       103│
│       104│
└──────────┘

Nevertheless your algorithm looks good for the UP/DOWN-1 and looks to what I wrote in mixedupdown1.p.

For the case of UP/DOWN <> 1 I would objected. In my understanding when the step crosses the border amount of real steps is calculated in that way.
1. If the step crosses the top border of the frame than real steps count will be equal to currentRowIndex where index is 1-based as it is in the Progress. I.e. after that step current row will be the last row in the previous frame.
2. If the step crosses the bottom border of the frame than real steps count will be equal to DOWN-size - currentRowIndex + 1 where index is 1-based as it is in the Progress. I.e. after that step the current row will be the first row in the next frame.

What do you think?

Vadim, please commit the exact testcase you used to capture the screens, I can't reproduce your screens easily.

Sorry, I already committed it. The file is up_down_field_group_test1.p

Vadim Gindin wrote:

The question is why have you used additional display statement before the loop in the function step? If steps = 1 the function will output 2 values to the frame..

Because I wanted to know on which field-group I am BEFORE I start moving UP or DOWN.

It's a first run of step(1, -7) command. Take a look at some "shift" of the value "1" from the second row to the third.

OK, I've confused you because there is a display k before we start the UP or DOWN, which is the actual case number. So, the shift you see is not a real shift. Replace:

 display k @ i with frame f1.

 display k * 100 @ i with frame f1.

1. If the step crosses the top border of the frame than real steps count will be equal to currentRowIndex where index is 1-based as it is in the Progress. I.e. after that step current row will be the last row in the previous frame.

Correct.

2. If the step crosses the bottom border of the frame than real steps count will be equal to DOWN-size - currentRowIndex + 1 where index is 1-based as it is in the Progress. I.e. after that step the current row will be the first row in the next frame.

Correct.

I don't understand what are your concerns about down size other than 1. Change the program as I noted above, and also change the step: you will see that new field-groups will be added following the same rules: when moving UP, always add to the left-side; when moving DOWN, always to the right-side.

1. there is no IterableResource file in the update
2. CommonFrame: as is extending IterableResource, the currentIteration method in this interface needs to be removed.
3. GenericFrame.downWorker: the field-group's prev/next-sibling needs to be updated only when we create the field-group (as this is the time when it is put in place). Later on, someone else may become its prev/next-sibling, but this is because someone was added on his left/right side, not that it was moved. Also, when linking them, although this acts like a circular list, keep in mind that the first and last field-group have no prev or next sibling, accordingly. So, you need a way of knowing which element is first in the list and which is last.

1. for a first approach:
   - assume the field-groups contain the same widgets (i.e. what you are currently doing)
   - only UP 1, DOWN 1 or implicit down is called
   - ignore the LITERAL widget for the frame header separator
2. once first phase seems to work, add full support for the UP n and DOWN n statements and the LITERAL widget.
3. once the field-group creation is working, we can move to the hard part: maintaining virtual copies of the frame's widgets, per each "row field-group". The P2J Client side needs to remain unchanged (or as few changes as possible), we don't want to add a widget for each row, as this will bring major changes in the drawing code. What this means: GenericFrame.n2w will be used for actual drawing, so we don't want to save these field-groups in the rowsFieldGroups map; instead, we will need to use copies of them, as the field-group's children. Some careful consideration will be needed so that SCREEN-VALUE and other widget-level functions use the "row field-group"-level data and not the "frame-line"-level data (i.e. currently, they all target the frame-line, so we can't access the widget data from another line).

Vadim, when posting test scenarios, please make sure the bzr test is setup exactly the same way you used to produce the intermediate screens.
From your cases:

1. I don't understand the logic of the case where all steps are -1. Here are two intermediate screens.
[...]
Why did the field-group 5 became not the last in the field-group list?

I think this is because the frame is cleared AFTER the next row is computed and populated; thus, when all rows are referencing a field-group and there are the sibling field-group is "occupied", a new field-group will be created and added to the list, depending on the location of the current field-group and the direction (up or down). See these rules from note 62:

2. if FG's left-side (circular) field-group has an associated row, then a new field-group will be created and added: = to the end of the frame's children, if FG (for current-iteration) is the first element in the list = at the left-side of FG, if FG has a left-side sibling.

2. if the right-side (circular) field-group has an associated row, then a new field-group will be created and added at the right-side of FG. But keep in mind that this will not become the first element, will remain the last element in the list.

This is the reason the maximum number of existing field-groups for a frame is not the same as the frame's body. And I think your problem is because in GenericFrame.downWorker you are computing the target field-group AFTER the down/up is performed, not before.

It is two intermediate screens of the test case bp_mixedupdow1.p with input steps array inp2 (you can specify it in lines 42 and 43). Look only at the second column.

I need the exact setup of the bp_mixedupdow1.p used to capture the intermediate screens, to comment on this. Is not obvious just looking at the screens.

Constantin Asofiei wrote:

..
I need the exact setup of the bp_mixedupdow1.p used to capture the intermediate screens, to comment on this. Is not obvious just looking at the screens.

Sorry. I just wanted to say that bp_mixedupdown1.p is intended to be ran for one of predefined tests - input arrays of steps defined at the start of procedure. Concrete input array of steps is specified in the line 42 And I talked about input array inp2.

Anyway I made bp_mixedupdown2.p for the input array inp2. Please use that script.

OK, I guess this test tries to predict which field-group will be next, after a UP/DOWN call? I think your problem is the same as the up_down_field_group_test2.p problem; for this screen:

┌──────────┐  1:       grp1        -1      grp1 :        0[1]
│       val│  2:       grp2        -1      grp5 :        4[4]
│──────────│  3:       grp1        -1      grp4 :        3[3]
│          │  4:       grp3        -1      grp3 :        2[2]
│          │  5:       grp4        -1      grp2 :        1[1]
│         7│  6:       grp5        -1      grp1 :        0[4]
│         6│  7:       grp2        -1      grp5 :        4[3]
└──────────┘

grp1
grp2 7
grp5 6
grp4
grp3

If you print the state of the field-groups after each iteration, as you do for up_down_field_group_test2.p, things will make sense: when positioned on grp5, an UP -1 is performed; this will look at the left-side of grp5, which is grp2. As grp2 is free, it will be used to output 7.

You are right. I made needed corrections and now it works for case "-1". At this moment I'm fixing another border case: up/down step (not equal to 1) when field-group set is not yet formed. In this case intermediate field-groups must been created. I'm debugging this case. Hope, I will upload next update tomorrow with all up/down cases work.

• method_attributes.rules is missing a header entry
• BaseEntity
  - please explain the reason behind "firstAsLast". What I don't understand is the if it initially assumed to be inserted as first. phrase from the parameter's javadoc.
  - getNextSibling - BaseEntity is always a HandleChain instance, so if (this instanceof HandleChain) will always return true.
• CommonFrame
  - place extends IterableResource on its own row
  - CommonFrame.currentIteration needs to be removed
• FrameWidget
  - there are no real changes in the file, so you can discard all of them
• GenericFrame
  - when multiple classes from the same package are imported, you need to import the entire package, via *
  - wrapWidgetsToFieldGroup - the javadoc is incorrect, there is no asLast parameter
  - there is no need to add constants for error messages, you can inline them. So please remove INCORRECT_NUMBER_OF_LINES and GROUP_HANDLE_IS_NULL.
  - currentIteration: you are throwing an IllegalStateException if a field-group is not found for the current line, with the "There are no correspondant field-group in a frame" message. Please change the message to String.format("There is no correspondent field-group for line %d in frame %s [%d].", line, frameName, frameId).
  - clear: I don't think the logic is correct. clear corresponds for the CLEAR FRAME statement and clearAll corresponds to CLEAR FRAME frameName ALL statement. Only CLEAR ALL touches the entire frame, and currentFieldGroup I think must become the field-group for the first row. You need some testcases to determine which field-group is associated to the first row, after a CLEAR ALL statement.
  - getName: in 4GL, the equivalent to java null is the unknown value. So, when working with BDT instances (which map legacy 4GL data types), you need to use toStringMessage, to get the string representation of a BDT instance and isUnknown to check if the value is unknown. So, for FIELD-GROUP:NAME, looks like it can't accept an unknown value, but the initial value for this attribute IS unknown. Thus, you need to:
  1. add FieldGroup.name(character), which shows the appropriate error if the passed argument is unknown
  2. in GenericFrame.getName, use widget.name().toStringMessage(), if the widget is a FieldGroup.
  - you need to test how shared frames work with current-iteration: is current-iteration inherited for shared frames, and back to the shared frame definition?
  - downWorker - there is this line int down = dynamicDown; // TODO =0 at first start at this moment. It's wrong. - what is the state of this TODO?
  - actualizeFieldGroups - please rename it to updateFieldGroups. Also, the javadoc mentions that only DOWN statement can create multiple field-groups (when step is not 1), and this applies to UP too, when step is not 1.
  - instead of using getResource() != null (in several methods), use the isUnknown() API.
  - isFGAssigned - you can avoid calling asWidgetHandle for each entry and thus creating multiple handle instances, by comparing the FieldGroup instances directly: extract the FieldGroup from the groupHandle parameter before the for loop, and compare the entry's FieldGroup with the received one.
  - I think there is a problem with conditional UP or DOWN statements. downWorker always readjusts the field-groups, regardless of the type of down. Takes this example, which uses a conditional down:
  

  def var i as int.
  form i with frame f1 4 down.
  do i = 1 to 4:
    display i with frame f1.
    down with frame f1.
  end.
  message "frame is full, not cleared, on row" frame-line(f1).
  display i with frame f1.
  

  
  Although the frame is reported as being on row 5, it does not get cleared until one more display or explicit down is performed. I think downWorker and actualizeFieldGroups need to keep track of the state of the cond flag too.
• HandleChain
  - please format the code at lines 153-155 according to standard
  - this test at line 154 before.getPrevSibling().getResource() == null, sugests that the before argument can be unknown: this should never be allowed, as it will result in an inconsistent chain.
  - if the c'tor or method signature exceeds the max line length, then the parameters need to be placed one per line, as in:
  

     public HandleChain(boolean     dynamic,
                        boolean     saveFirstLast, 
                        HandleChain before,
                        boolean     firstAsLast)
  

  
  Also, column-aligning the parameter names will make the code more readable.
  This applies to some other methods in your code, too.
  - please order the private instance methods according to standard (they are last in the class, before inner classes). Check your other code changes, too.

1. clean up the code (formatting, javadoc, address review notes, etc) and post the update here.
2. check the shared frame cases: I suspect the current-iteration will be inherited from the "master" frame to the shared frame and back, but we need to confirm this. You will need two programs: one program with a DEFINE NEW SHARED FRAME f1... and another program, ran from the first program, with a DEFINE SHARED FRAME f1.... Check the syntax for the DEFINE FRAME statement.
3. fix the CLEAR ALL, CLEAR, the implicit down and the conditional up/down cases: the frame clear from a conditional down I think is triggered directly at the P2J client, so you will need to start digging into P2J client code.
4. once these are fixed, you can go ahead and think about the widget virtual copies in each field-group, only on server-side.

Constantin Asofiei wrote:

..

1. check the shared frame cases: I suspect the current-iteration will be inherited from the "master" frame to the shared frame and back, but we need to confirm this. You will need two programs: one program with a DEFINE NEW SHARED FRAME f1... and another program, ran from the first program, with a DEFINE SHARED FRAME f1.... Check the syntax for the DEFINE FRAME statement.
   ..

You are right. I've added 2 tests down_shared.p and down_shared_ext.p The first procedure calls the second procedure. Both procedures have same print_frame procedure logic and outputs the same output. It means that the shared frame inherits the current-iteration form the procedure where that frame was defined.

Vadim, about 0324a.zip: GenericFrame and other files don't have the proper history number (is the same as the previous one) - please double check all files.

GenericFrame imports couldn't become simpler because of possible ambiguity of classes ErrorManager or LogHelper or BlockManager.Condition wich causes compile errors.

Actually, your changes require only the import java.util.Map.Entry; line, the other import lines can remain unchanged, as they are in the previous version of GenericFrame.

downWorker down = dynamicDown comments. The problem is following. dynamicDown is calculated on the server-side to prevent client calling. At the first iteration dynamicDown is equal to 0 independently of real down-size, but this behavior doesn't affect field-groups updating algorithm. So we can leave it "as is". What do you think?

OK, what you are saying is that dynamicDown is updated only after the real frame down is computed on the P2J client side (after initial frame display/view), and is 0 until so. But, the frame view can be triggered by a DOWN (or UP), too, so when downWorker is called in this case:

def var i as int.
form i with frame f1 with down.
down 5 with frame f1.
display i with frame f1.

About before object - as a parameter of constructor I allow it to be null to specify the case when I want to add new object been created would added as the last element in chain.

Is OK allowing before to be null. And is OK to test for

before.getPrevSibling() == null

before.getPrevSibling().getResource() == null

1. Yes, Constantin. I really skipped this case. In your example dynamicDown=0 indeed. The problem appears when display is called after down statement or display is called before current-iteration. In the first case the place where the problem is born is GenericFrame.downWorker (where I placed TODO). In the second case the problem is born in GenericFrame.currentIteration. It calls frameLineWorker() that returns 0. Could you advice me something in this situation? The comments in the code tell that there is a try to avoid client call in this case, but we need to know real down-size at the start to make field-group algorithm work correctly.
2. About before.getPrevSibling().isUnknown(). Take a look at the implemenation of methods HandleChain.getPrevSibling() and HandleChain.getNextSibling(). They create the new handle even if prevSibling/nextSibling is null. It forces me to check isUnknown() too.

I forgot to add, that your short test case shows the problem dynamicDown=0 in another aspect. This wrong behavior leads to incorrect display of the current value:

Progress4GL   Java impl
┌──────────┐ ┌──────────┐
│         i│ │         i│
│──────────│ │──────────│
│          │ │         0│
│          │ │          │
│          │ │          │
│          │ │          │
│          │ │          │
│         0│ │          │
│          │ │          │
│          │ │          │
│          │ │          │
│          │ │          │
└──────────┘ └──────────┘

Vadim Gindin wrote:

I forgot to add, that your short test case shows the problem dynamicDown=0 in another aspect. This wrong behavior leads to incorrect display of the current value:

OK, this suggests that we may need to force a "view frame" before (or at the beginning) of downWorker. This forced view will be done only if this is the very first view (I think testing for "dynamicDown is zero" should be enough). The view will layout the frame and compute the dynamicDown value, on P2J client side; after the view, when the P2J client returns control back to the P2J server, it will sync the client-side state with the server-side state, via the StateSynchronized.applyChanges (which is implemented by LogicalTerminal on server-side).

Vadim Gindin wrote:

1. Yes, Constantin. I really skipped this case. In your example dynamicDown=0 indeed. The problem appears when display is called after down statement or display is called before current-iteration. In the first case the place where the problem is born is GenericFrame.downWorker (where I placed TODO). In the second case the problem is born in GenericFrame.currentIteration. It calls frameLineWorker() that returns 0. Could you advice me something in this situation? The comments in the code tell that there is a try to avoid client call in this case, but we need to know real down-size at the start to make field-group algorithm work correctly.

See the previous reply: the forced view should solve this case too.

1. About before.getPrevSibling().isUnknown(). Take a look at the implemenation of methods HandleChain.getPrevSibling() and HandleChain.getNextSibling(). They create the new handle even if prevSibling/nextSibling is null. It forces me to check isUnknown() too.

You are correct, forgot about that. So is the null equality still needed, if getPrevSibling and getNextSibling always returns something not-null?

1. Server side can initiate a call to the client-side by LogicalTerminal methods. How can client-side initiate a call to the server-side?
2. Virtual widgets. How they differ from real widgets? May be real widgets have IDs and virtual widgets don't? When ID is assigned to newly created widget?

Vadim Gindin wrote:

I'm thinking about virtual copies of widgets on the server side. I need some advice. I'm planning to create new field GenericWidget[][] downBody in GenericFrame class that will be an "alter ego" of Frame.downBody. If I correctly understand I should fill this array with virtual widgets at the start.

Yes, the server-side will need the "alter ego" downBody, but the widgets will not be populated by the client-side and the client-side will have no real knowledge about these widgets.

1. Server side can initiate a call to the client-side by LogicalTerminal methods. How can client-side initiate a call to the server-side?

The client-side can use the ThinClient.server field, which is a network proxy to the server-side LogicalTerminal, which statically implements the ServerExports interface.

1. Virtual widgets. How they differ from real widgets? May be real widgets have IDs and virtual widgets don't? When ID is assigned to newly created widget?

How I image the virtual widgets working: on server-side they need to be real resources, as their state can be changed/checked individually. Thus, each of them will have their own resource ID (as 4GL does). The tricky part is, when accessing these "virtual widgets" from server side: I think is best to intercept these calls via an anonymous proxy and redirect them to the client-side, so that the appropriate widget (from Frame.downBody) is accessed/changed. I guess this access will depend on the row to which the field-group is associated and also each individual widget's internal ID (the ID used by the server-side and client-side to identify a widget in a frame, set at the ComponentConfig.id field - this is not the resource ID, is an internal ID used by P2J).

1. merge 0401a.zip with the latest bzr revision
2. explain and address the remaining TODOs added by you in GenericFrame
3. put the revised 0401a.zip into runtime and conversion regression testing. This will help us isolate any problems with this update early on. And if it passes, I think is best to release it to bzr.
4. experiment and create testcases to see what happens when a down frame is used and widget state is changed, for a widget which is not the one on the current frame-line:
   - what happens if the HIDDEN or VISIBLE state is changed?
   - same for SCREEN-VALUE?
   - what happens when changing the widget's LABEL attribute, when the FRAME is set with or without SIDE-LABELS? When side-labels are use, looks like the row's FIELD-GROUP is added some LITERAL widgets for each widget, with the widget's label.
   - what happens if the cases above are done from a trigger? When are the changes pushed to the screen - when the trigger ends, or is it needed an explicit view of the frame, to update the screen?
   - whatever else you can think of.

The test can look like, but not limited to:

def var h as handle.
def var i as int.
def var j as int.
form i j with 10 down frame f1.

do i = 1 to 9:
   display i j with frame f1.
   if i = 1 then h = frame f1:current-iteration.
   down 1 with frame f1.
end.

h:first-child:screen-value = "11".

up 1 with frame f1.
frame f1:current-iteration:first-child:screen-value = "22".

view frame f1.

up 1 with frame f1.
frame f1:current-iteration:first-child:visible = false. /* does this update the screen right away or on next view? if update is done right away, does this affect the entire frame (i.e. the screen buffer is pushed to the client for the entire frame), or just this widget? */

• after merging, the history entry numbers for changes which are in bzr can't be changed. This translates to: after merging, your history entry needs to be last. i.e. in BufferImpl instead of this:
  

  ** 035 VIG 20140314 Changed currentIteration() method signature. 
  ** 036 VMN 20140330 Added LockType errors processing.
  

  
  you must have this:
  

  ** 035 VMN 20140330 Added LockType errors processing.
  ** 036 VIG 20140314 Changed currentIteration() method signature. 
  

• you have some code which doesn't follow the coding standards. See the conditions in GenericFrame lines 10073, 10152, 10160, 10175: the operator needs to be right-aligned and the operands need to be left-aligned.
• GenericFrame.frameLineWorker - isn't the comment related to how actually FRAME-LINE function works? More, I think FRAME-LINE follows the CURRENT-ITERATION attribute, so this is why it can return the frame-down + 1 value.
• GenericFrame.downWorker - this code I think reads better with Math.min and Math.max:
  

        // correct if it really needed
        savedFrameLine = savedFrameLine > dynamicDown ? dynamicDown : savedFrameLine;
        savedFrameLine = savedFrameLine < 0 ? 1 : savedFrameLine;
  

Vadim Gindin wrote:

It happen! The fixed update passed regression testing. I also made corrections corresponding to previous note. Please take a look. Can I commit it?

• conditionUpDown
  

  // TODO frameLine can be incorrect here (not in a frame)
           // int currentRow = frameLine;
  

• printFieldGroups and updateFieldGroups - TODO delete it. - please post how the debug info looks like, as it might be OK to leave this API behind, and not just delete it.
• getFGRow - TODO: use bidirectional map

Constantin Asofiei wrote:
..

Great. Before committing, I need to know what is the state of the TODOs which are left behind the code, in GenericFrame:

• conditionUpDown
  [...]

The point is that updateFieldGroup methods takes currentRow that must be valid row number, but the frameLine returned from correspondent method can be out of the frame borders. So we can't use it as a parameter for updateFieldGroup. I'll delete TODO

• printFieldGroups and updateFieldGroups - TODO delete it. - please post how the debug info looks like, as it might be OK to leave this API behind, and not just delete it.

Here is an example:

3
 gr1 [null, gr2]      gr2 [gr1, gr3]      gr3 [gr2, gr4]      gr4 [gr3, gr5]      gr5 [gr4, UN]      UN [gr5, null]     
1 -> gr5, 

3
 gr1 [null, gr2]      gr2 [gr1, gr3]      gr3 [gr2, gr4]      gr4 [gr3, gr5]      gr5 [gr4, UN]      UN [gr5, null]     
1 -> gr5, 2 -> gr1, 

3
 gr1 [null, gr2]      gr2 [gr1, gr3]      gr3 [gr2, gr4]      gr4 [gr3, gr5]      gr5 [gr4, UN]      UN [gr5, null]     
1 -> gr5, 2 -> gr1, 3 -> gr2, 

Here are groups of several triplets of strings.
The first line in a triplet is a number of lines to step. Can be < 0.
The second row displays groups each with it's neighbors in square brackets [left, right].
The third row is a mapping from the row number to field-group.

• getFGRow - TODO: use bidirectional map

The field rows2fg is used in both directions: get the value by the key, and vice versa - the key by value. So bidirectional map (such as some implementation of org.apache.commons.collections.BidiMap) would be better here instead of usual HashMap, but this class is in an external jar file. I'm not sure if I can use it.

Vadim Gindin wrote:

• conditionUpDown
  [...]

The point is that updateFieldGroup methods takes currentRow that must be valid row number, but the frameLine returned from correspondent method can be out of the frame borders. So we can't use it as a parameter for updateFieldGroup. I'll delete TODO

OK, but leave behind a comment with what you noted here (that frameLine can't be used because we need valid row numbers).

• printFieldGroups and updateFieldGroups - TODO delete it. - please post how the debug info looks like, as it might be OK to leave this API behind, and not just delete it.

Here are groups of several triplets of strings.

OK, add this to the printFieldGroups's javadoc, and remove both TODOs

• getFGRow - TODO: use bidirectional map

The field rows2fg is used in both directions: get the value by the key, and vice versa - the key by value. So bidirectional map (such as some implementation of org.apache.commons.collections.BidiMap) would be better here instead of usual HashMap, but this class is in an external jar file. I'm not sure if I can use it.

OK, for this update leave the TODO behind. As the commons-collections jar is already in the project, it doesn't add any new dependencies if using BidiMap; but which implementation do you want to use?

About bidirectional map implementation. That is the moment I also wanted to ask you advice about. There are possible two choices:

1. Simple choice is TreeMap (no external jars dependencies) and use its appropriate containsKey and containsValue methods. The minus of this variant is doubled map storage (there are two maps are stored keyToVal and valToKey If I remember correctly).
2. TreeBidiMap implementation of BidiMap (from common-collections.jar). It holds the common map storage for both cases.

I suppose that the storage capacity is not critical here and we can just use the first variant. What do you think?

Vadim Gindin wrote:

1. Simple choice is TreeMap (no external jars dependencies) and use its appropriate containsKey and containsValue methods. The minus of this variant is doubled map storage (there are two maps are stored keyToVal and valToKey If I remember correctly).

I'm not sure how you can use containsKey and containsValue to retrieve the key associated with a certain value.

2. TreeBidiMap implementation of BidiMap (from common-collections.jar). It holds the common map storage for both cases.

Does this require the values to be Comparable? Because when I hear Tree, I think of something sorted...

What do you think?

This is for a future update. In the end, it might be simpler to just add an additional map, fg2rows, which is maintained alongside rows2fg.

At this time: please make the changes in note 93 related to javadoc/comments, submit the update for review, and you can release it after I take a final look at it.

Committed to bzr rev #10522.

Tests about virtual widgets (сommitted to current-iteration folder).

1. The frame without side labels. Changing widget attributes from different rows.
Test: virtual-widgets.p
Description: 10-down frame without side-labels, filled by values from 1 to 9. Several attributes from different widgets (from different rows) are changed: "screen-value", "hidden", "label", "visible".
Result: all changes was applied immediately and only for corresponding row. Only label attribute changed the column label.

┌─────────────────────┐
│         5lab       j│
│────────── ──────────│
│        11          0│
│         2          0│
│                    0│
│         4          0│
│         5          0│
│         6          0│
│         7          0│
│                    0│
│        22          0│
│                     │
└─────────────────────┘

2. The frame with side labels. Changing widget attributes from different rows.
Test: virtual-widgets-slabels.p
Description: 10-down frame with side-labels, filled by values from 1 to 9. Several attributes from different widgets (from different rows) are changed: "screen-value", "hidden", "label", "visible".
Result: all changes was applied immediately and only for corresponding row. When trying to change the label attribute the following error is happen:
**Unable to set SCREEN-VALUE. LITERAL widget does not fit in FRAME f1. (4054)
I don't understand why it happen.

┌───────────────────────────┐
│i: 11         j: 0         │
│i: 2          j: 0         │
│              j: 0         │
│i: 4          j: 0         │
│i: 5          j: 0         │
│i: 6          j: 0         │
│i: 7          j: 0         │
│              j: 0         │
│i: 22         j: 0         │
│                           │
└───────────────────────────┘

3. The frame without side labels. Changing widget attributes from different rows. The place: F4-trigger
Test: virtual-widgets-trigger.p
Description: same as issue №1. The difference is the changes are applied when user pressed F4.
Result after pressing F4 the same as in the issue №1

4. The frame with side labels. Changing widget attributes from different rows. The place: F4-trigger
Test: virtual-widgets-trigger-slabels.p
Description: same as issue №2. The difference is the changes are applied when user pressed F4.
Result after pressing F4 the same as in the issue №2. The same error is happen.

Vadim Gindin wrote:

Tests about virtual widgets (сommitted to current-iteration folder).

These tests look good, and give a good indication of what is going on: the changes reflect the widget referenced by the handle var or by CURRENT-ITERATION. What we are primarily interested is the case when widget's handle is saved in a var (i.e. firsth, secondh, thirdh in your tests) - these need to be implemented in P2J.

Result: ... When trying to change the label attribute the following error is happen:
**Unable to set SCREEN-VALUE. LITERAL widget does not fit in FRAME f1. (4054)
I don't understand why it happen.

I suspect this is because your test changed the length of the label, as you used 11 (two letters) - try using a single-letter string. I suspect this is a limitation from 4GL, to not modify the layout of the frame (as changing the label width will cause the entire frame to relayout...).

Conclusion: investigate how widgets are kept on the client-side (in WidgetRegistry, Frame, and other client-side classes) and find a way of virtualizing access to this data, from the server-side.

• ScreenBuffer and ScreenDefinition classes - data structures are kept only per-widget (usually the one on CURRENT-ITERATION field-grou), and not per-each-row-widget
• GenericFrame.screenBuf - this is always in sync with CURRENT-ITERATION row
• GenericFrame.screenBufferCache - this is a data structure which keeps data per each row (an exception to the rule)
• GenericFrame.frameDef - this is always in sync with CURRENT-ITERATION row

• set/getScreenValue - this works as if the widget on the CURRENT-ITERATION field-group is accessed
• pushScreenDefinition (is called each time a widget's attribute is changed (i.e. VISIBLE/HIDDEN/LABEL/etc)) - same, this works as if the widget on the CURRENT-ITERATION field-group is accessed.

• when a widget from CURRENT-ITERATION is accessed or <widget>:handle IN FRAME is called, then a handle with a GenericWidget unique to that row/field-group must be returned.
• when communicating with the client-side via a widget outside of CURRENT-ITERATION field-group, the client-side needs to receive both the widget ID (returned by WidgetConfig.getId) and the ROW to which the widget is associated. My thought at this time is that ScreenBuffer, ScreenDefinition and the widgets all should keep an info about which frame row is targeted, and send this to the client-side. But there is a problem on the client-side too: in a down-frame, each widget has only ONE config, thus i.e. the VISIBLE flag can be set in only once, and not for each row-level widget.

At this time, I think we should look into how the CURRENT-ITERATION and widget-handles are used in the GUI source code and determine which widget attributes/methods are accessed via handles, in down frames. Because if only SCREEN-VALUE is checked and only the widgets on CURRENT-ITERATION are used, we can avoid the complex changes and limit to only what the GUI code actually uses.

Greg: I think we should wait for the delivery of the GUI sources before continuing, and run the reports on that; because, to add full-compatibility, then I don't think we can get away without major changes on the client-side, too.

Greg: I think we should wait for the delivery of the GUI sources before continuing, and run the reports on that; because, to add full-compatibility, then I don't think we can get away without major changes on the client-side, too.

Agreed.

In the customer's GUI application, this attribute it used 915 times in 888 files. However, all usage is of this same pattern:

      ASSIGN curr-widget = FRAME F-Main:CURRENT-ITERATION. /* Field group*/
      ASSIGN curr-widget = curr-widget:FIRST-CHILD. /* First field */
      DO WHILE VALID-HANDLE (curr-widget):
         ...
      END.

In other words, all usage is just to get the field-group out of the frame. I think we support this already.

Constantin: Please correct me if I am wrong.

I was looking for a testcase to deal with a frame-col issue, and found this one uast/current_iteration/clear-frame.p is not working. It returns an error Lead attributes in a chained-attribute expression (a:b:c) must be type HANDLE or a user-defined type and valid (not UNKNOWN). (10068) when getting first-child. In debugger, the handle is 1001. In 4GL it should have found the literal with handle 1002.

Expected 4GL output:

┌──────────┐  1) 1000 FIELD-GROUP 1  <?>              1000 FILL-IN 1 i
│         i│                                          1002 LITERAL ----------
│──────────│  2) 1004 FIELD-GROUP 2  1000:visible:yes 1000 FILL-IN 1 i
│         1│                                          1004 FILL-IN 2 i
│         2│                                          1002 LITERAL ----------
└──────────┘

FWD output:

┌──────────┐  1) 1000 FIELD-GROUP 1  <?>              1000 FILL-IN 1 i
│         i│  2) 1002 FIELD-GROUP 2  1000:visible:no  1000 FILL-IN 2 i
│──────────│                                          1002 FILL-IN 2 i
│         1│
│         2│
└──────────┘

I didn't want to pass through this testcase without adding this information.

• field-groups are 'static' in a frame - once they are built, they are not deleted and built again. I have a fix for this, but it affects CURRENT-ITERATION and I need to test it. Plus some issues how CURRENT-ITERATION behaves when the frame is cleared. Also, I really don't recall why the initial implementation had to re-build the field-groups.
• I have a deja-vu that we've discussed (or I've noticed) before about an additional FIELD-GROUP created in a DOWN frame, on which CURRENT-ITERATION 'stays' when 'going out of frame' (i.e. past last row to go back to first row). But I can't find notes about this. My problem about this additional FIELD-GROUP is that it affects CURRENT-ITERATION.

The refactoring is in 3821c/13120.

• OE shows a 'hidden' field-group when going beyond the last row (or above the first row) - this field-group is moved as the first row in the list.
• literal field-group: I've added some code to populate it with widgets, so some tests pass, but this is most likely incomplete
• accessing a widget's SCREEN-VALUE or other attributes, for a widget not part of CURRENT-ITERATION (one can always save the handle of the widget while in CURRENT-ITERATION, and after that alter the frame). This is not working properly in FWD, the widget will always be the one in CURRENT-ITERATION
• the tests in testcases/uast/current-iteration while they are interesting, they are not working properly in FWD. Mostly because of the hidden field-group, FIELD-GROUP attributes like ROW, SCREEN-VALUE for a widget not part of CURRENT-ITERATION, etc.