FWD » Core Development » Runtime Infrastructure

After switching between embedded windows with opening modal dialogs the JS Embedded application client can fall in this state when the currently active embedded window is not displayed and "Loading..." is only visible. In this state menus don't work. The JS client has this suspected logs where id = -653 becomes visible probably before this window will be shown or restack.

remote side has received an unknown message, and will be discarded: {"target":"OverlayPanel","visible":true,"id":-653} p2j.embedded.js:1485:10
914: 8d done in 1  p2j.socket.js:3225:7
918: 8c done in 0  p2j.socket.js:3225:7
919: 9c done in 0  p2j.socket.js:3225:7
922: 96 done in 3  p2j.socket.js:3225:7
restack request  p2j.screen.js:3320:7
10  p2j.screen.js:3323:10
-653  p2j.screen.js:3323:10
restack  p2j.screen.js:3370:7
Object { id: 1, win: Object }  p2j.screen.js:3373:10
Object { id: 10, win: Object }  p2j.screen.js:3373:10
Object { id: -653, win: Object }

[11/06/2017 00:39:38 MSK] (ErrorManager:SEVERE) {00000018:00000046:bogus} Invalid handle.  Not initialized or points to a deleted object. (3135)

Nov 06, 2017 12:38:08 AM com.goldencode.p2j.ui.client.FontManager getBrowseCellBottomTextOffset
WARNING: There is no formula for browse cell bottom text offset for the font 'Microsoft Sans Serif', using default formula.
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=178]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,0.05] Dimension[7.0,0.62] label = null value = Single  [ Single]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=180]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=181]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,0.91] Dimension[8.0,0.62] label = null value = Double  [ Double]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=183]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=184]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,3.48] Dimension[5.6,0.62] label = null value = Twin    [ Twin]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=186]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=187]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,4.33] Dimension[13.4,0.62] label = null value = Luxury  [ Luxury Suite]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=189]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=190]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=194]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,0.05] Dimension[19.0,0.62] label = null value = Double  [ Double - Sea View]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=196]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=197]
INFO: Drawing with widget not attached to a window instance: FillIn Point[6.8,0.91] Dimension[18.6,0.62] label = null value = Double  [ Double - King Bed]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=199]
INFO: Drawing with widget not attached to a window instance: ImageGuiImpl [id=200]
java.lang.RuntimeException: No renderer is registered for id = -378
    at com.goldencode.p2j.ui.client.gui.driver.GuiPrimitivesImpl.getWindowEmulator(GuiPrimitivesImpl.java:261)
    at com.goldencode.p2j.ui.client.gui.driver.web.GuiWebDriver.raiseMouseEvent(GuiWebDriver.java:1053)
    at com.goldencode.p2j.ui.client.gui.driver.web.GuiWebSocket.processBinaryMessage(GuiWebSocket.java:1671)
    at com.goldencode.p2j.ui.client.driver.web.WebClientProtocol$1.run(WebClientProtocol.java:373)
    at com.goldencode.p2j.ui.client.driver.web.WebTaskWorker.run(WebTaskWorker.java:127) 

Working on this issue within the new task branch 3366a.

In this case the server Conversation thread is blocked.

Name: Conversation [00000008:bogus]
State: WAITING on java.lang.Object@1674a2a7
Total blocked: 2,679  Total waited: 2,680

Stack trace: 
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
com.goldencode.p2j.net.Conversation.block(Conversation.java:391)
com.goldencode.p2j.net.Conversation.waitMessage(Conversation.java:348)
com.goldencode.p2j.net.Queue.transactImpl(Queue.java:1170)
com.goldencode.p2j.net.Queue.transact(Queue.java:641)
com.goldencode.p2j.net.BaseSession.transact(BaseSession.java:271)
com.goldencode.p2j.net.HighLevelObject.transact(HighLevelObject.java:211)
com.goldencode.p2j.net.RemoteObject$RemoteAccess.invokeCore(RemoteObject.java:1473)
com.goldencode.p2j.net.InvocationStub.invoke(InvocationStub.java:145)
com.sun.proxy.$Proxy17.displayError(Unknown Source)
com.goldencode.p2j.ui.LogicalTerminal.displayError(LogicalTerminal.java:2852)
com.goldencode.p2j.ui.ErrorWriterServer.displayError(ErrorWriterServer.java:99)
com.goldencode.p2j.util.ErrorManager.displayError(ErrorManager.java:2217)
com.goldencode.p2j.util.ErrorManager.displayError(ErrorManager.java:2197)
com.goldencode.p2j.util.ErrorManager.recordOrThrowError(ErrorManager.java:1243)
com.goldencode.p2j.util.handle$InvalidAttributeAccess.invoke(handle.java:3072)
com.sun.proxy.$Proxy18.getResourceType(Unknown Source)
com.goldencode.hotel.FwdEmbeddedDriver.lambda$inMsgFwdShowWindow$9(FwdEmbeddedDriver.java:343)
com.goldencode.hotel.FwdEmbeddedDriver$$Lambda$96/1784525300.body(Unknown Source)
com.goldencode.p2j.util.Block.body(Block.java:604)
com.goldencode.p2j.util.BlockManager.processBody(BlockManager.java:6985)
com.goldencode.p2j.util.BlockManager.topLevelBlock(BlockManager.java:6776)
com.goldencode.p2j.util.BlockManager.internalProcedure(BlockManager.java:369)
com.goldencode.p2j.util.BlockManager.internalProcedure(BlockManager.java:355)
com.goldencode.hotel.FwdEmbeddedDriver.inMsgFwdShowWindow(FwdEmbeddedDriver.java:265)
sun.reflect.GeneratedMethodAccessor799.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.goldencode.p2j.util.ControlFlowOps$InternalEntryCaller.invokeImpl(ControlFlowOps.java:5385)
com.goldencode.p2j.util.ControlFlowOps$InternalEntryCaller.invoke(ControlFlowOps.java:5363)
com.goldencode.p2j.util.ControlFlowOps.invokeImpl(ControlFlowOps.java:4424)
com.goldencode.p2j.util.ControlFlowOps.invoke(ControlFlowOps.java:3275)
com.goldencode.p2j.ui.LogicalTerminal.lambda$invoke$4(LogicalTerminal.java:14850)
com.goldencode.p2j.ui.LogicalTerminal$$Lambda$41/20526825.get(Unknown Source)
com.goldencode.p2j.ui.LogicalTerminal.invokeOnServer(LogicalTerminal.java:15558)
com.goldencode.p2j.ui.LogicalTerminal.invoke(LogicalTerminal.java:14850)
sun.reflect.GeneratedMethodAccessor408.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.goldencode.p2j.util.MethodInvoker.invoke(MethodInvoker.java:124)
com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:757)
com.goldencode.p2j.net.Conversation.block(Conversation.java:412)
com.goldencode.p2j.net.Conversation.waitMessage(Conversation.java:348)
com.goldencode.p2j.net.Queue.transactImpl(Queue.java:1170)
com.goldencode.p2j.net.Queue.transact(Queue.java:641)
com.goldencode.p2j.net.BaseSession.transact(BaseSession.java:271)
com.goldencode.p2j.net.HighLevelObject.transact(HighLevelObject.java:211)
com.goldencode.p2j.net.RemoteObject$RemoteAccess.invokeCore(RemoteObject.java:1473)
com.goldencode.p2j.net.InvocationStub.invoke(InvocationStub.java:145)
com.sun.proxy.$Proxy17.waitFor(Unknown Source)
com.goldencode.p2j.ui.LogicalTerminal.waitFor(LogicalTerminal.java:6400)
com.goldencode.p2j.ui.LogicalTerminal.waitFor(LogicalTerminal.java:6170)
com.goldencode.hotel.Emain.lambda$execute$0(Emain.java:43)
com.goldencode.hotel.Emain$$Lambda$39/1228925533.body(Unknown Source)
com.goldencode.p2j.util.Block.body(Block.java:604)
com.goldencode.p2j.util.BlockManager.processBody(BlockManager.java:6985)
com.goldencode.p2j.util.BlockManager.topLevelBlock(BlockManager.java:6776)
com.goldencode.p2j.util.BlockManager.externalProcedure(BlockManager.java:343)
com.goldencode.p2j.util.BlockManager.externalProcedure(BlockManager.java:317)
com.goldencode.hotel.Emain.execute(Emain.java:41)

Sergey Ivanovskiy wrote:

In this case the server Conversation thread is blocked.
[...]

There is a displayError in the stacktrace - is there anything shown in the app?

No. There is no any Error dialog and Loading.. message is displayed only. The web client main thread is at this point

Name: main
State: TIMED_WAITING on com.goldencode.p2j.ui.client.TypeAhead@5ac42f7f
Total blocked: 6,864  Total waited: 6,812

Stack trace: 
java.lang.Object.wait(Native Method)
com.goldencode.p2j.ui.client.TypeAhead.getKeystroke(TypeAhead.java:405)
com.goldencode.p2j.ui.chui.ThinClient.modalEventLoopWorker(ThinClient.java:8091)
com.goldencode.p2j.ui.chui.ThinClient.modalEventLoop(ThinClient.java:7983)
com.goldencode.p2j.ui.chui.ThinClient.lambda$messageBox$20(ThinClient.java:7958)
com.goldencode.p2j.ui.chui.ThinClient$$Lambda$148/1949363603.run(Unknown Source)
com.goldencode.p2j.ui.chui.ThinClient.withIndependentEventList(ThinClient.java:8024)
com.goldencode.p2j.ui.chui.ThinClient.messageBox(ThinClient.java:7930)
com.goldencode.p2j.ui.chui.ThinClient.displayErrorMessage(ThinClient.java:7770)
com.goldencode.p2j.ui.ErrorWriterInteractive.displayError(ErrorWriterInteractive.java:118)
com.goldencode.p2j.util.ErrorManager.displayError(ErrorManager.java:2217)
com.goldencode.p2j.ui.chui.ThinClient.displayError(ThinClient.java:7631)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.goldencode.p2j.util.MethodInvoker.invoke(MethodInvoker.java:124)
com.goldencode.p2j.net.Dispatcher.processInbound(Dispatcher.java:757)
com.goldencode.p2j.net.Conversation.block(Conversation.java:412)
com.goldencode.p2j.net.Conversation.waitMessage(Conversation.java:348)
com.goldencode.p2j.net.Queue.transactImpl(Queue.java:1170)
com.goldencode.p2j.net.Queue.transact(Queue.java:641)
com.goldencode.p2j.net.BaseSession.transact(BaseSession.java:271)
com.goldencode.p2j.net.HighLevelObject.transact(HighLevelObject.java:211)
com.goldencode.p2j.net.RemoteObject$RemoteAccess.invokeCore(RemoteObject.java:1473)
com.goldencode.p2j.net.InvocationStub.invoke(InvocationStub.java:145)
com.sun.proxy.$Proxy10.invoke(Unknown Source)
com.goldencode.p2j.ui.client.gui.driver.web.GuiWebSocket.lambda$null$1(GuiWebSocket.java:1931)
com.goldencode.p2j.ui.client.gui.driver.web.GuiWebSocket$$Lambda$27/1855610584.get(Unknown Source)
com.goldencode.p2j.ui.chui.ThinClient.invokeOnServer(ThinClient.java:6148)
com.goldencode.p2j.ui.client.gui.driver.web.GuiWebSocket.lambda$processBinaryMessage$2(GuiWebSocket.java:1931)
com.goldencode.p2j.ui.client.gui.driver.web.GuiWebSocket$$Lambda$14/755926024.run(Unknown Source)
com.goldencode.p2j.ui.chui.ThinClient.waitForEvent(ThinClient.java:14304)
com.goldencode.p2j.ui.chui.ThinClient.waitForWorker(ThinClient.java:12103)
com.goldencode.p2j.ui.chui.ThinClient.waitForWorker(ThinClient.java:11677)
com.goldencode.p2j.ui.chui.ThinClient.waitFor(ThinClient.java:11620)
com.goldencode.p2j.ui.chui.ThinClient.waitFor(ThinClient.java:11574)

Please debug the server and see what is the message when the client blocks. I assume server.log has no info which can help?

OK. Except this last error

[11/06/2017 20:45:05 MSK] (ErrorManager:SEVERE) {00000008:0000001E:bogus} Invalid handle.  Not initialized or points to a deleted object. (3135)

It seems that this issue is closely related to the embedded JS client because before this loading message appeared covering the active window the screen was splashed with the target canvas window and (?) this error dialog (?). It can be observed by the Inspector Browser tool. The web client is waiting the JS client, and the JS client looks like blocked by the overlay panel. It seems that the web client is not blocked.

It seems that this exception is due to this scenario. If Checkin dialog of the Available rooms embedded window has been cancelled after selecting room number in its combobox, then the remote side invokes this function

   function roomUtilizationEvent(params)
   {
      // start date is check-in date of stay/reservation
      var start = parseProgressDate(params[0]);

      // end date is day before check-out date of stay/reservation
      var end = moment(parseProgressDate(params[1])).subtract(1, "days").toDate();

      // up-call to collect utilization/capacity data for given date window
      getUtilizationByDate(formattedDate(start), formattedDate(end));

      // up-call to collect utilization data by room type for given date window; only invoke
      // if the currently displayed date window overlaps with the period that changed
      if (utilByTypeStartDate <= end && utilByTypeEndDate >= start)
      {
         getUtilizationByType(utilByTypeStartDate, utilByTypeEndDate);
      }
   };

Runnable error = () ->
               {
                  // raise error, could not convert!
                  String msg = "Remote called routine %s %s has mismatched parameters";
                  msg = String.format(msg, caller.getInternalEntryName(), name);

                  throw new ErrorConditionException(5729, msg);
               };

&Scoped-define SELF-NAME checkIn
&ANALYZE-SUSPEND _UIB-CODE-BLOCK _CONTROL checkIn fFrameWin
ON CHOOSE OF checkIn IN FRAME fMain /* Check-In... */
DO:
  IF NOT AVAIL(room) THEN DO:
     {no-selected-records.i "rooms"}
  END.

  DEF VAR date1 AS DATE.
  DEF VAR date2 AS DATE.
  date1 = startDate:INPUT-VALUE IN FRAME fMain.
  date2 = endDate:INPUT-VALUE IN FRAME fMain.

  DEF VAR needRefresh AS LOGICAL.
  DEF VAR rid AS ROWID.
  DEF VAR createdStayRid AS ROWID.
  DO TRANSACTION:
     RUN update-stay-dialog.w (INPUT rid, 
                               OUTPUT needRefresh, 
                               OUTPUT createdStayRid,
                               INPUT FALSE,
                               room.room-num,
                               date1,
                               date2,
                               ?).

     IF needRefresh THEN DO:
        RUN refresh-avail-rooms.
        publish "refresh-stays".
     END.
     ELSE DO:
        UNDO, LEAVE.
     END.
  END.
END.

Constantin, could you help to link this task to the task where the embedded architecture was developed: added new FWD 4GL extension P2J-REMOTE-CALL and defined this new application protocol. It helps to check the details of this architecture.

The work was done in #3209.

I lost that the original exception was thrown in the fwd-embedded-driver.p on show embedded window inMsg_fwdShowWindow in this code

   hwid = hwin:first-child.
   do while hwid <> ?:
      if hwid:type = "FRAME" 
      then do:
         hwid:visible = false.
      end.

      hwid = hwid:next-sibling.
   end.

The bug scenario can be reproduced if repeat this sequence rapidly.
1) On Available Rooms open Checkin dialog , select a room number, then cancel this dialog.
2) Then open another embedded window to add new room and repeat similar actions: select a room type and then cancel this Add room dialog.

The converted code is this

         while (_notUnknown(hwid))
         {
            if (_isEqual(hwid.unwrapType().getResourceType(), "FRAME"))
            {
               hwid.unwrapWidget().setVisible(new logical(false));
            }

            hwid.assign(hwid.unwrap().getNextSibling());
         }

I don't understand if it is possible that some handle in that iteration can be invalid. Why we don't use VALID-HANDLE(hwid) to replace hwid <> ? ?

Sergey Ivanovskiy wrote:

I don't understand if it is possible that some handle in that iteration can be invalid. Why we don't use VALID-HANDLE(hwid) to replace hwid <> ? ?

We might not be cleaning up properly the deleted frames. The next-sibling attribute must return ? when it reaches the end, so valid-handle can't be used.

Please check the code which is used for next-sibling (for FRAME and WINDOW) on server-side and try to find why an invalid resource might end up being reported by FWD.

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

I don't understand if it is possible that some handle in that iteration can be invalid. Why we don't use VALID-HANDLE(hwid) to replace hwid <> ? ?

We might not be cleaning up properly the deleted frames. The next-sibling attribute must return ? when it reaches the end, so valid-handle can't be used.

Please check the code which is used for next-sibling (for FRAME and WINDOW) on server-side and try to find why an invalid resource might end up being reported by FWD.

I don't understand it. Please explain why it must return ?. May be it help to proceed with this bug. The code from HandleChain.getNextSibling can return new handle() that is invalid.

Sergey Ivanovskiy wrote:

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

I don't understand if it is possible that some handle in that iteration can be invalid. Why we don't use VALID-HANDLE(hwid) to replace hwid <> ? ?

We might not be cleaning up properly the deleted frames. The next-sibling attribute must return ? when it reaches the end, so valid-handle can't be used.

Please check the code which is used for next-sibling (for FRAME and WINDOW) on server-side and try to find why an invalid resource might end up being reported by FWD.

I don't understand it. Please explain why it must return ?. May be it help to proceed with this bug. The code from HandleChain.getNextSibling can return new handle() that is invalid.

A handle var can be unknown (this is what hwid <> ? does, it tests for unknown value, and new handle() is an unknown value), but also can be not unknown, but invalid (think of a dynamic frame referenced by a handle var h - this can be not-unknown, but invalid, if the frame is invalid).

Please explain when this procedure must be invoked

TRIGGER PROCEDURE FOR Write|Delete OF stay.

publish "roomUtilization" from this-procedure (string(stay.start-date, "99/99/9999"),
                                               string(stay.end-date, "99/99/9999")).

TRIGGER PROCEDURE FOR Write|Delete OF reservation.

publish "roomUtilization" from this-procedure (string(reservation.start-date, "99/99/9999"),
                                               string(reservation.end-date, "99/99/9999")).

on message  p2j.embedded.js:212:34
p2j.embedded.receiveRemoteMessage  p2j.embedded.js:1284:7
p2j.embedded.publishCallback  p2j.embedded.js:1619:7
ehotel.roomUtilizationEvent: ?,?  ehotel.js:430:7
p2j.embedded.handleMessage  p2j.embedded.js:1298:10
p2j.embedded.getHandler: -148  p2j.embedded.js:360:7
p2j.embedded.defaultMessageCallback  p2j.embedded.js:388:7

The handle in #3366-12 is a static frame of gdialog##com.goldencode.hotel.ui.UpdateStayDialogGdialog$UpdateStayDialogGdialogDef and its FrameWidget.this.deleted property is set true. Please help where in the code I can set a watch on changes of this property?

Sergey Ivanovskiy wrote:

The handle in #3366-12 is a static frame of gdialog##com.goldencode.hotel.ui.UpdateStayDialogGdialog$UpdateStayDialogGdialogDef and its FrameWidget.this.deleted property is set true. Please help where in the code I can set a watch on changes of this property?

Please see this code in BaseEntity:

   public handle getNextSibling()
   {
      if (useHandleChainSiblings())
      {
         return super.getNextSibling((HandleChain) getParent());
      }
      else
      {
         GenericWidget<?> widget = (group == null) ? null : group.getNextChild(this);
         return new handle(widget);
      }
   }

Root frames are walked via HandleChain and child frames are walked via the BaseEntity.group. But I don't see any code which removes the frame from the associated Group, when it gets deleted. FrameWidget.delete() is removing it only from the HandleChain lists.

Can you debug into getNextSibling() and see how the UpdateStayDialogGdialog is walked (i.e. what parent it has)?

Sergey Ivanovskiy wrote:

Please explain when this procedure must be invoked
[...]
Now we can observe that 'roomUtilization event is thrown if we open "Checkin" dialog of "Available rooms" and then cancel it. The browser gets "roomUtilization" remote message with an undefined range as it follows from this sequence of logged calls

on message p2j.embedded.js:212:34
p2j.embedded.receiveRemoteMessage p2j.embedded.js:1284:7
p2j.embedded.publishCallback p2j.embedded.js:1619:7
ehotel.roomUtilizationEvent: ?,? ehotel.js:430:7
p2j.embedded.handleMessage p2j.embedded.js:1298:10
p2j.embedded.getHandler: -148 p2j.embedded.js:360:7
p2j.embedded.defaultMessageCallback p2j.embedded.js:388:7

Is it correct that on cancel Check-in dialog the write event is triggered ?

Staywtrg.write(Stay$Buf) line: 48    
Staywtrg.write(Buffer) line: 35    
DatabaseTriggerManager.fireTrigger(DatabaseEventType, TriggerData, Buffer, Buffer) line: 1119    
DatabaseTriggerManager.trigger(DatabaseEventType, Class<Buffer>, Buffer, Persistable) line: 752    
RecordBuffer.maybeFireWriteTrigger() line: 6606    
RecordBuffer.validate(boolean) line: 4858    
TransactionManager.processValidate() line: 4147    
BlockManager.lambda$processBody$0() line: 7019    
1912086003.run() line: not available    
BlockManager.executeStateManaged(Runnable) line: 9358    
BlockManager.processBody(Block, OnPhrase[]) line: 7019    
BlockManager.topLevelBlock(Object, BlockManager$TransactionType, BlockType, Block) line: 6776    
BlockManager.externalProcedure(Object, BlockManager$TransactionType, Block) line: 343    
BlockManager.externalProcedure(Object, Block) line: 317    
UpdateStayDialog.execute(rowid, logical, rowid, logical, integer, date, date, character) line: 669    
NativeMethodAccessorImpl.invoke0(Method, Object, Object[]) line: not available [native method]    
NativeMethodAccessorImpl.invoke(Object, Object[]) line: 62    
DelegatingMethodAccessorImpl.invoke(Object, Object[]) line: 43    
Method.invoke(Object, Object...) line: 498    
ControlFlowOps$ExternalProcedureCaller(ControlFlowOps$InternalEntryCaller).invokeImpl(String, Object...) line: 5385    
ControlFlowOps$ExternalProcedureCaller(ControlFlowOps$InternalEntryCaller).invoke(String, Object...) line: 5363    
ControlFlowOps.invokeImpl(List<Resolver>, handle, character, boolean, boolean, boolean, boolean, String, ArgumentResolver) line: 4424    
ControlFlowOps.invoke(handle, character, boolean, boolean, boolean, boolean, String, Object...) line: 3453    
ControlFlowOps.invokeImpl(AsyncRequestImpl, handle, handle, character, boolean, boolean, boolean, boolean, String, Object...) line: 4101    
ControlFlowOps.invokeImpl(handle, handle, character, boolean, boolean, boolean, boolean, String, Object...) line: 4030    
ControlFlowOps.invokeWithMode(character, String, Object...) line: 404    
ControlFlowOps.invokeWithMode(String, String, Object...) line: 386    
AvailRoomsFrame$TriggerBlock0.lambda$body$0() line: 1752    

The issue in #3366-20 looks like a new bug. Please help to investigate it separately.

Sergey Ivanovskiy wrote:

Is it correct that on cancel Check-in dialog the write event is triggered ?

I'm not sure. There is a CREATE stay. in the update-stay-dialog.w program, executed always, even if this program's CANCEL button is pressed. In avail-rooms-frame.w, there is a UNDO, LEAVE. call, which backs out the transaction.

Eric: when is a WRITE table-trigger executed? When the full transaction is committed or even for sub-transactions?

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

The handle in #3366-12 is a static frame of gdialog##com.goldencode.hotel.ui.UpdateStayDialogGdialog$UpdateStayDialogGdialogDef and its FrameWidget.this.deleted property is set true. Please help where in the code I can set a watch on changes of this property?

Please see this code in BaseEntity:
[...]

Root frames are walked via HandleChain and child frames are walked via the BaseEntity.group. But I don't see any code which removes the frame from the associated Group, when it gets deleted. FrameWidget.delete() is removing it only from the HandleChain lists.

Can you debug into getNextSibling() and see how the UpdateStayDialogGdialog is walked (i.e. what parent it has)?

Thank you, FrameWidget.delete() is called on cancel this dialog and this frame has unset parent, and its FrameWidget.this.config.parentId points to the main application window.

Constantin Asofiei wrote:

Eric: when is a WRITE table-trigger executed? When the full transaction is committed or even for sub-transactions?

A write trigger can execute in a more tightly nested scope than a full transaction. For example, if a new record is about to be pushed into the buffer, or if the buffer scope ends before the full transaction scope ends, a pending write trigger will fire.

This write trigger staywtrg.p is also executed if we press "Cancel" on the "Check-in" dialog. From these diagrams it follows that the write trigger is executed if we click on the "Add" button, but they don't describe "Cancel". Is this case valid?

Sergey Ivanovskiy wrote:

This write trigger staywtrg.p is also executed if we press "Cancel" on the "Check-in" dialog. From these diagrams it follows that the write trigger is executed if we click on the "Add" button, but they don't describe "Cancel". Is this case valid?

Yes, I think is valid, as the create stay. is committed in the sub-transaction, but only rolled back by the caller's full-transaction.

Please try this to fix it: in the roomUtilization JavaScript's side, check if either one of the two arguments is unknown - in this case, just return and don't do anything. You can compare directly with "?" string to check for unknown.

Sergey Ivanovskiy wrote:

This write trigger staywtrg.p is also executed if we press "Cancel" on the "Check-in" dialog. From these diagrams it follows that the write trigger is executed if we click on the "Add" button, but they don't describe "Cancel". Is this case valid?

Firing the write trigger on "Cancel" was never my intended design for the demo. The intention was that it fire only if a record is successfully added to the stay table.

Looks like Constantin already posted a proposed solution while I was writing this.

It looks that if an exception is thrown in FwdEmbeddedDriver.inMsgFwdShowWindow, then the embedded client and the web client and the server are mistakenly frozen since the server waits until the web client responds but the embedded client hides the screen with the current input focus because it switches into new screen. Should it be fixed so the managed exceptions don't block the work flow? LogicalTerminal.displayError can notify the embedded client about exceptions thrown in FwdEmbeddedDriver so that they don't block web and embedded clients.

In this particular case #3366-12 (#3366-18) I don't understand when this deleted frame FrameWidget is removed from its main window WindowWidget sibling chain. I suspect that it can be after the peer frame widget on the client side will be deleted. It can explain why usually this deleted frame FrameWidget is not present in its main window sibling chain due to concurrency of these events to show window and to update window configuration?

LogicalTerminal.displayError can notify the embedded client about exceptions thrown in FwdEmbeddedDriver that they don't block web and embedded clients.

Sergey Ivanovskiy wrote:

LogicalTerminal.displayError can notify the embedded client about exceptions thrown in FwdEmbeddedDriver so that they don't block web and embedded clients.

The error I think is more subtle in FWD. FWD, for one, it must not walk invalid frames - this is definitely a bug - regardless if errors are shown or not. Please try and fix this bug first.

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

LogicalTerminal.displayError can notify the embedded client about exceptions thrown in FwdEmbeddedDriver so that they don't block web and embedded clients.

The error I think is more subtle in FWD. FWD, for one, it must not walk invalid frames - this is definitely a bug - regardless if errors are shown or not. Please try and fix this bug first.

In the usual case WindowWidget.this.firstChild() returns its frame associated with the main window but in the bug presence the deleted frame is returned. I can't detect what happens here.

Please explain another question. The main-window.p is converted into MainWindow.java with new generated methods

   public handle fwdGetMainFrame()
   {
      return function("fwdGetMainFrame", handle.class, new Block((Body) () -> 
      {
         returnNormal(fmainFrame.asWidgetHandle());
      }));
   }

   public handle fwdGetWindow()
   {
      return function("fwdGetWindow", handle.class, new Block((Body) () -> 
      {
         returnNormal(wWin);
      }));
   }

   public handle fwdGetTabHandle()
   {
      return function("fwdGetTabHandle", handle.class, new Block((Body) () -> 
      {
         returnNormal(hFolder);
      }));
   }

I just realize that we probably have an issue in this WindowWidget.firstChild()

   public handle firstChild()
   {
      handle child = HandleChain.firstResource(this, LegacyResource.WINDOW);
      if (child.isUnknown())
      {
         child = HandleChain.firstResource(this, LegacyResource.FRAME);
      }

      return child;
   }

Sergey, please provide a step-by-step recreate for the #3366-12 issue, a final screenshot of the web client, the server stacktrace and JS errors (if any). I can't duplicate this issue.

Also, let me know which revision of Hotel GUI you are using (and if you made any JS changes).

Constantin Asofiei wrote:

Sergey, please provide a step-by-step recreate for the #3366-12 issue, a final screenshot of the web client, the server stacktrace and JS errors (if any). I can't duplicate this issue.

Also, let me know which revision of Hotel GUI you are using (and if you made any JS changes).

The bug scenario can be reproduced if repeat this sequence rapidly.

1) On Available Rooms open Checkin dialog , select a room number, then cancel this dialog.
2) Then open another embedded window to add new room and repeat similar actions: select a room type and then cancel this Add room dialog.
3) Repeat 1)

Sergey Ivanovskiy wrote:

The bug scenario can be reproduced if repeat this sequence rapidly.
2) Then open another embedded window to add new room and repeat similar actions: select a room type and then cancel this Add room dialog.

You mean open another Browser window, login into Hotel embedded mode and perform the action?

Sorry, I can't duplicate it at all. Are you using JS breakpoints by any chance?

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

The bug scenario can be reproduced if repeat this sequence rapidly.
2) Then open another embedded window to add new room and repeat similar actions: select a room type and then cancel this Add room dialog.

You mean open another Browser window, login into Hotel embedded mode and perform the action?

I meant to select another menu Rooms and to add new room and repeat similar actions: select a room type and then cancel this Add room dialog.

Sorry, I can't duplicate it at all. Are you using JS breakpoints by any chance?

No, I didn't set JS breakpoints.

OK, I've managed to duplicate it somehow, after some more random clicks... will let you know what I found.

Constantin Asofiei wrote:

OK, I've managed to duplicate it somehow, after some more random clicks... will let you know what I found.

It doesn't need to do it random, it can be reproduced in my environment if I followed #3366-35

OK, found it... it's a stupid bug, the window is actually a DIALOG-BOX. But, during creation, its TYPE is reported incorrectly, as FRAME - thus is registered in HandleChain in the FRAME first/last references, not DIALOG-BOX... and when it gets deleted, its TYPE is reported as DIALOG-BOX, thus it remains registered in the FRAME first/last references.

Constantin Asofiei wrote:

OK, found it... it's a stupid bug, the window is actually a DIALOG-BOX. But, during creation, its TYPE is reported incorrectly, as FRAME - thus is registered in HandleChain in the FRAME first/last references, not DIALOG-BOX... and when it gets deleted, its TYPE is reported as DIALOG-BOX, thus it remains registered in the FRAME first/last references.

You are cool!

Sergey Ivanovskiy wrote:

Please explain another question. The main-window.p is converted into MainWindow.java with new generated methods
[...]
Are these methods generated automatically? What rules control these cases to generate new methods and what are their meanings and usages?

Yes, these are generated automatically. If you are curious, see adm/adm_windows.xml how the decisions are made if a window can be used in embedded mode or not.

Sergey, I've added some fixes (including the DIALOG-BOX chaining) to 3366a rev 11197.

Please review them and check if you find any other issues.

The p2j.fonts.js bug is another weird one - in Firefox, if the font name contains double-quotes, it will error out bad. I've fixed it to just replace the double quotes with single quotes, but I haven't investigated further, if there are other repercussions to this change.

Tested rev 11197 and rev 11198 (certs fix) it seems that the issue was fixed.

Planning to rebase 3366a, then it will be ready for trunc.

3366a is rebased, rev 11200 is over 11197 trunc. Constantin, what do you think that #3366-29 and #3366-30 should be done or not in order to permit errors to be displayed in the embedded mode without blocking clients?

Sergey Ivanovskiy wrote:

Constantin, what do you think that #3366-29 and #3366-30 should be done or not in order to permit errors to be displayed in the embedded mode without blocking clients?

The issue here is more of a embedded app logic then a FWD problem. The code in fwd-embedded-driver.p should never raise errors; the 'blocking clients' issue was a side-effect of the fact that the overlay div was hiding the FWD iframe, so the user couldn't see the error dialog.

I'm not sure yet what would be the right approach; I don't want a notification only in case of errors. I'm thinking more of a 'if the embedded app is hiding the iframe and the last request was not completed yet but the FWD client is in a wait-for blocking mode, then the embedded app should be aware of this and show the iframe' approach (I know, it's a mouthful...).

Better translated, I think FWD should provide a notification to the embedded side each time it gets in a user-input blocking mode. And the embedded side can act accordingly (i.e. show the iframe if is hidden). A trick here is to not degrade performance.

On the other hand, did you had a chance to take a look at my p2j.font.js change? My experience with JS (and inter-browser compatibility) is not that great, and I might be missing something.

Constantin Asofiei wrote:

On the other hand, did you had a chance to take a look at my p2j.font.js change? My experience with JS (and inter-browser compatibility) is not that great, and I might be missing something.

I tested this change with Hotel GUI but with the current hotel_gui/abl/fwd-embedded-driver.p. Is it correct that fwd-embedded-driver.p should be placed into this folder abl under hotel_gui project manually? This change

=== modified file 'src/com/goldencode/p2j/ui/client/driver/web/res/p2j.fonts.js'
--- src/com/goldencode/p2j/ui/client/driver/web/res/p2j.fonts.js    2017-04-01 23:33:34 +0000
+++ src/com/goldencode/p2j/ui/client/driver/web/res/p2j.fonts.js    2017-11-09 19:24:55 +0000
@@ -4,7 +4,7 @@
 **
 ** Copyright (c) 2015-2017, Golden Code Development Corporation.
 **
-** -#- -I- --Date-- ------------------------------Description----------------------------------
+** -#- -I- --Date-- -------------------------------Description------------------------------------
 ** 001 CA  20150812 First version.
 ** 002 CA  20150818 Fixed param/return data types in the function javadoc. Added font maxWidth.
 **                  If a font width the same specifications already exists, re-use it from the 
@@ -13,6 +13,8 @@
 ** 003 CA  20151007 Fix for underline font support.
 ** 004 SBI 20160107 Added the fonts checker to detect if the target font family is available.
 **     CA  20160323 Added getFontMaxWidth and getTextWidths.
+** 005 CA  20171109 Fixed a problem with font naming in Firefox (never use double-quotes, just 
+*                   single quotes).
 */
 /*
 ** This program is free software: you can redistribute it and/or modify
@@ -464,6 +466,7 @@
       fontCtx.fillRect(0, 0, width, height);
       fontCtx.textBaseline = 'top';
       fontCtx.fillStyle = 'white';
+      font = font.replace(/'/g , '"');
       fontCtx.font = font;
       fontCtx.fillText(text, 0, 0);

@@ -542,6 +545,7 @@
       fontCtx.save();

       fontCtx.textBaseline = 'top';
+      font = font.replace(/'/g , '"');
       fontCtx.font = font;

       var width = fontCtx.measureText(text).width;
@@ -643,4 +647,4 @@
    }

    return me;
-}());
\ No newline at end of file
+}());

Sergey Ivanovskiy wrote:

Constantin Asofiei wrote:

On the other hand, did you had a chance to take a look at my p2j.font.js change? My experience with JS (and inter-browser compatibility) is not that great, and I might be missing something.

I tested this change with Hotel GUI but with the current hotel_gui/abl/fwd-embedded-driver.p. Is it correct that fwd-embedded-driver.p should be placed into this folder abl under hotel_gui project manually?

No. You need to to a 'ant deploy', which will pick up this file automatically.

[...]
Why this " can be an issue with Firefox? I don't understand this change.

Well, this is where I wanted your help. If you test the embedded mode on firefox without this fix, you will see it will fail.

In my environment I usually use Firefox. With the old code Hotel GUI embedded mode I didn't notice this issue. A year ago we had unresolved problems with fonts related to true type font format file. After that issue I didn't take part in the development related to open fonts. Can you help with links to the related tasks in order to recall this issue if you have this knowledge?

It seems that the bug issue was fixed. I am rechecking it now. Can this branch be merged into the trunc? What cases should be tested?

Committed rev 152 (Hotel GUI) applied this diff

=== modified file 'embedded/src/resources/ehotel.js'
--- embedded/src/resources/ehotel.js    2017-10-27 10:57:53 +0000
+++ embedded/src/resources/ehotel.js    2017-11-10 14:04:21 +0000
@@ -418,6 +418,11 @@
     */
    function roomUtilizationEvent(params)
    {
+      // staywrtrg.p can be triggered on cancel of Check-in dialog (with an empty record)
+      if (params[0] == "?" || params[1] == "?")
+      {
+         return;
+      }
       // start date is check-in date of stay/reservation
       var start = parseProgressDate(params[0]);

Sergey Ivanovskiy wrote:

In my environment I usually use Firefox. With the old code Hotel GUI embedded mode I didn't notice this issue. A year ago we had unresolved problems with fonts related to true type font format file. After that issue I didn't take part in the development related to open fonts. Can you help with links to the related tasks in order to recall this issue if you have this knowledge?

I wonder if this only happens when there are font files in the deploy/server/fonts/ directory.

The font work was done in #1794 and #2765.

Can this branch be merged into the trunc? What cases should be tested?

I will review the code.

I previously saw these other issues:

• The bootstrap menu "disappears" when you click on it. Then if you click on the iframe it comes back. While it is "gone", hovering the mouse over the "empty space" shows the menu item again because of the highlight.
• The tabulator in the bottom right was not loading correctly. If you resize the browser window, it is fixed temporarily. The next time you try to load it, it will fail again.

I'd like to address these ASAP.

Testing will need to be on all 4 browsers to see if there is anything wrong.

Greg Shah wrote:

Can this branch be merged into the trunc? What cases should be tested?

I will review the code.

I previously saw these other issues:

• The bootstrap menu "disappears" when you click on it. Then if you click on the iframe it comes back. While it is "gone", hovering the mouse over the "empty space" shows the menu item again because of the highlight.

This was fixed by Sergey in Hotel GUI rev 151.

• The tabulator in the bottom right was not loading correctly. If you resize the browser window, it is fixed temporarily. The next time you try to load it, it will fail again.

There should be a fix for this (related to resize) in Hotel GUI rev 149. Another issue I see is that the tabulator doesn't highlight the clicked row - is not always responding to the click.

Committed revision 11201 fixed the case if the current certificate request is still valid and the web server hasn't been started.

3366a rev 11202 fixed a bug in ConnectionManager related to -ct which was not stopping if first attempt was successful.

Code Review Task Branch 3366a Revision 11202

I'm fine with the changes.

We don't need full regression testing, but I want to make sure that the MAJIC RFQ use case is working. A manual test is fine.

The other testing needed is a thorough check on Hotel Embedded Mode and Virtual Desktop Mode in multiple browsers.

If we can include the certificate fixes in this branch that is also good.

It seems that I need to save certificates chain in the directory for the case when reuse CA certificates is true. What key can be used? There is a root certificate under

security/cas/{root alias}

security/cas/{{root alias}-1}
security/cas/{{root alias}-1}-2}
...............................

Constantin, please review the committed rev 11203. The certificate chain was lost because the used chain was built from the root CA certificate as a self-signed certificate.
Now, certificate chain should be stored in the directory with root CA certificate. If a root CA is a self-signed certificate, then this chain is absent.
For an example, if root-hotel has two intermediate certificates in its chain, then these entries will be added

security/cas/root-hotel/
security/cas/root-hotel-1/
security/cas/root-hotel-1-2/

Sergey Ivanovskiy wrote:

Now, certificate chain should be stored in the directory with root CA certificate. If a root CA is a self-signed certificate, then this chain is absent.

How do you handle the case where we use Let's Encrypt? The self-signed root CA, shouldn't this be dropped?

Constantin Asofiei wrote:

Sergey Ivanovskiy wrote:

Now, certificate chain should be stored in the directory with root CA certificate. If a root CA is a self-signed certificate, then this chain is absent.

How do you handle the case where we use Let's Encrypt? The self-signed root CA, shouldn't this be dropped?

Let's Encrypt certificates are loaded as any others private and public certificates signed by trusted CA authority. Let's Encrypt certificates are received independently via ACME client and then they are loaded and used to build another certificates. As you noted the JKS key store didn't contain the certificate chain. This fix should add correct certificate chain if this chain is present. For self-signed certificates the full chain is built from one element that is a CA certificate itself.

Ah, for Let's Encrypt certificates these keys are given from Let's Encrypt. Now they give the chain with only one intermediate certificate. Thus these entries are from my test

        <node class="container" name="cas">
          <node class="bytes" name="hotel-root">
            <node-attribute name="value" value="MIIFCTCCA/GgAwIBAgISA83BDRqqYhIkAah2Ckf3LA4xMA0GCSqGSIb3DQEBCwUAMEoxCzAJBgNVBAYTAlVTMRYwFAYDVQQKEw1MZXQncyBFbmNyeXB0MSMwIQYDVQQDExpMZXQncyBFbmNyeXB0IEF1dGhvcml0eSBYMzAeFw0xNzExMTAxNTM1NDNaFw0xODAyMDgxNTM1NDNaMB4xHDAaBgNVBAMTE2RlbW8uZ29sZGVuY29kZS5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCC/29LBFgOQVNvN6NxPNYkvC17tPa10AXvWC8TPRUGji/Y1QC/TABxWixd0+xldxvTSdgZglfFm6oY9iKdtztHbTPO/XN2Vwj8lssfDqanI7xaJmakY1WEo1HGSefJl/u+ireBJo67feseSel9tJ35HHfpyrjHIERBOhdjAej6iM3QQChTwDwqfKcGsL7390btTXxb6LE/Zqg5g7ISNVRqM4hePU2gT580iYBZFqPZmyKGDcgfIo/3cw6VByNsO81qAg9tkCngKUleUzWuUXCJYhTHmQf4ncIadFlyIBp2ar0bj1FxhbILKwN9GNBh3aB6N9x7f1KAihWTtn8y2635AgMBAAGjggITMIICDzAOBgNVHQ8BAf8EBAMCBaAwHQYDVR0lBBYwFAYIKwYBBQUHAwEGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFLHggqrIDoltbrO649mn0k1Mz553MB8GA1UdIwQYMBaAFKhKamMEfd265tE5t6ZFZe/zqOyhMG8GCCsGAQUFBwEBBGMwYTAuBggrBgEFBQcwAYYiaHR0cDovL29jc3AuaW50LXgzLmxldHNlbmNyeXB0Lm9yZzAvBggrBgEFBQcwAoYjaHR0cDovL2NlcnQuaW50LXgzLmxldHNlbmNyeXB0Lm9yZy8wHgYDVR0RBBcwFYITZGVtby5nb2xkZW5jb2RlLmNvbTCB/gYDVR0gBIH2MIHzMAgGBmeBDAECATCB5gYLKwYBBAGC3xMBAQEwgdYwJgYIKwYBBQUHAgEWGmh0dHA6Ly9jcHMubGV0c2VuY3J5cHQub3JnMIGrBggrBgEFBQcCAjCBngyBm1RoaXMgQ2VydGlmaWNhdGUgbWF5IG9ubHkgYmUgcmVsaWVkIHVwb24gYnkgUmVseWluZyBQYXJ0aWVzIGFuZCBvbmx5IGluIGFjY29yZGFuY2Ugd2l0aCB0aGUgQ2VydGlmaWNhdGUgUG9saWN5IGZvdW5kIGF0IGh0dHBzOi8vbGV0c2VuY3J5cHQub3JnL3JlcG9zaXRvcnkvMA0GCSqGSIb3DQEBCwUAA4IBAQBEom25ZSC1AHaH+/aFB7mBPRzouDy25yem4WwHGIAq124fdkgZbij6o7ZxrL3hy/ZumGey/95ObfBpn2Sv35eE3s9rkH0lgOzffX5A/yJ8mUGuJVh8w9VMLKt2i1T+JhC9yBOMxv0LlWAv5LL85llkpLGzRv2IqVEce2PlTlmf34pH5OQOFoJy6b2TQERwUGr8bdWsADKoKUDSSZyQnmyyvnFKSGQOPjMlZXRl/hkF1d0BKau/HnYKgDFEY2RcJzKqEGP9x27VXGc2JYqsHsa89H1V2j+kcfjKcydtZ7ORWrcF4tHilJqOecgIwXgUuwntO4mA0AC0Y6QIyWVU8pWK"/>
          </node>
          <node class="bytes" name="hotel-root-1">
            <node-attribute name="value" value="MIIEkjCCA3qgAwIBAgIQCgFBQgAAAVOFc2oLheynCDANBgkqhkiG9w0BAQsFADA/MSQwIgYDVQQKExtEaWdpdGFsIFNpZ25hdHVyZSBUcnVzdCBDby4xFzAVBgNVBAMTDkRTVCBSb290IENBIFgzMB4XDTE2MDMxNzE2NDA0NloXDTIxMDMxNzE2NDA0NlowSjELMAkGA1UEBhMCVVMxFjAUBgNVBAoTDUxldCdzIEVuY3J5cHQxIzAhBgNVBAMTGkxldCdzIEVuY3J5cHQgQXV0aG9yaXR5IFgzMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAnNMM8FrlLke3cl03g7NoYzDq1zUmGSXhvb418XCSL7e4S0EFq6meNQhY7LEqxGiHC6PjdeTm86dicbp5gWAf15Gan/PQeGdxyGkOlZHP/uaZ6WA8SMx+yk13EiSdRxta67nsHjcAHJyse6cF6s5K671B5TaYucv9bTyWaN8jKkKQDIZ0Z8h/pZq4UmEUEz9l6YKHy9v6Dlb2honzhT+Xhq+w3Brvaw2VFn3EK6BlspkENnWAa6xK8xuQSXgvopZPKiAlKQTGdMDQMc2PMTiVFrqoM7hD8bEfwzB/onkxEz0tNvjj/PIzark5McWvxI0NHWQWM6r6hCm21AvA2H3DkwIDAQABo4IBfTCCAXkwEgYDVR0TAQH/BAgwBgEB/wIBADAOBgNVHQ8BAf8EBAMCAYYwfwYIKwYBBQUHAQEEczBxMDIGCCsGAQUFBzABhiZodHRwOi8vaXNyZy50cnVzdGlkLm9jc3AuaWRlbnRydXN0LmNvbTA7BggrBgEFBQcwAoYvaHR0cDovL2FwcHMuaWRlbnRydXN0LmNvbS9yb290cy9kc3Ryb290Y2F4My5wN2MwHwYDVR0jBBgwFoAUxKexpHsscfrb4UuQdf/EFWCFiRAwVAYDVR0gBE0wSzAIBgZngQwBAgEwPwYLKwYBBAGC3xMBAQEwMDAuBggrBgEFBQcCARYiaHR0cDovL2Nwcy5yb290LXgxLmxldHNlbmNyeXB0Lm9yZzA8BgNVHR8ENTAzMDGgL6AthitodHRwOi8vY3JsLmlkZW50cnVzdC5jb20vRFNUUk9PVENBWDNDUkwuY3JsMB0GA1UdDgQWBBSoSmpjBH3duubRObemRWXv86jsoTANBgkqhkiG9w0BAQsFAAOCAQEA3TPXEfNjWDjdGBX7CVW+dla5cEilaUcne8IkCJLxWh9KEik3JHRRHGJouM2VcGfl96S8TihRzZvoroed6ti6WqEBmtzw3Wodatg+VyOeph4EYpr/1wXKtx8/wApIvJSwtmVi4MFU5aMqrSDE6ea73Mj2tcMyo5jMd6jmeWUHK8so/joWUoHOUgwuX4Po1QYz+3dszkDqMp4fklxBwXRsW10KXzPMTZ+sOPAveyxindmjkW8lGy+QsRlGPfZ+G6Z6h7mjem0Y+iWlkYcV4PIWL1iwBi8saCbGS5jN2p8M+X+Q7UNKEkROb3N6KOqkqm57TH2H3eDJAkSnh6/DNFu0Qg=="/>
          </node>
        </node>

Constantin, do you know the answer on this question that I don't understand. We got domain.crt (signed by Let's Encrypt) that has these usages

sbi@am:~/projects/3366a/server$ /usr/lib/jvm/default-java/bin/keytool -list -v -keystore root-ca-pk.store 
Enter keystore password:  
Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: hotel-root
Creation date: Nov 11, 2017
Entry type: PrivateKeyEntry
Certificate chain length: 2
Certificate[1]:
Owner: CN=demo.goldencode.com
Issuer: CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
Serial number: 3cdc10d1aaa62122401a8760a47f72c0e31
Valid from: Fri Nov 10 18:35:43 MSK 2017 until: Thu Feb 08 18:35:43 MSK 2018
Certificate fingerprints:
     MD5:  6D:AD:28:5B:E7:2F:E5:BC:FD:0C:BC:D0:4A:3B:84:A5
     SHA1: 45:C5:33:CC:B6:3F:A6:FB:6D:5B:E2:D4:39:F3:9E:31:D8:84:4E:FE
     SHA256: FC:AD:37:CF:6F:B8:E4:0C:DB:49:1B:CD:79:75:94:FE:50:28:CA:A4:4E:4A:C7:75:B5:F1:0F:23:C4:46:20:34
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: ocsp
   accessLocation: URIName: http://ocsp.int-x3.letsencrypt.org
, 
   accessMethod: caIssuers
   accessLocation: URIName: http://cert.int-x3.letsencrypt.org/
]
]

#2: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: A8 4A 6A 63 04 7D DD BA   E6 D1 39 B7 A6 45 65 EF  .Jjc......9..Ee.
0010: F3 A8 EC A1                                        ....
]
]

#3: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:false
  PathLen: undefined
]

#4: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: [2.23.140.1.2.1]
[]  ]
  [CertificatePolicyId: [1.3.6.1.4.1.44947.1.1.1]
[PolicyQualifierInfo: [
  qualifierID: 1.3.6.1.5.5.7.2.1
  qualifier: 0000: 16 1A 68 74 74 70 3A 2F   2F 63 70 73 2E 6C 65 74  ..http://cps.let
0010: 73 65 6E 63 72 79 70 74   2E 6F 72 67              sencrypt.org

], PolicyQualifierInfo: [
  qualifierID: 1.3.6.1.5.5.7.2.2
  qualifier: 0000: 30 81 9E 0C 81 9B 54 68   69 73 20 43 65 72 74 69  0.....This Certi
0010: 66 69 63 61 74 65 20 6D   61 79 20 6F 6E 6C 79 20  ficate may only 
0020: 62 65 20 72 65 6C 69 65   64 20 75 70 6F 6E 20 62  be relied upon b
0030: 79 20 52 65 6C 79 69 6E   67 20 50 61 72 74 69 65  y Relying Partie
0040: 73 20 61 6E 64 20 6F 6E   6C 79 20 69 6E 20 61 63  s and only in ac
0050: 63 6F 72 64 61 6E 63 65   20 77 69 74 68 20 74 68  cordance with th
0060: 65 20 43 65 72 74 69 66   69 63 61 74 65 20 50 6F  e Certificate Po
0070: 6C 69 63 79 20 66 6F 75   6E 64 20 61 74 20 68 74  licy found at ht
0080: 74 70 73 3A 2F 2F 6C 65   74 73 65 6E 63 72 79 70  tps://letsencryp
0090: 74 2E 6F 72 67 2F 72 65   70 6F 73 69 74 6F 72 79  t.org/repository
00A0: 2F                                                 /

]]  ]
]

#5: ObjectId: 2.5.29.37 Criticality=false
ExtendedKeyUsages [
  serverAuth
  clientAuth
]

#6: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
]

#7: ObjectId: 2.5.29.17 Criticality=false
SubjectAlternativeName [
  DNSName: demo.goldencode.com
]

#8: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: B1 E0 82 AA C8 0E 89 6D   6E B3 BA E3 D9 A7 D2 4D  .......mn......M
0010: 4C CF 9E 77                                        L..w
]
]

Certificate[2]:

For an example, the intermediate Let' Encrypt Authority X3 has these key usages

Certificate[2]:
Owner: CN=Let's Encrypt Authority X3, O=Let's Encrypt, C=US
Issuer: CN=DST Root CA X3, O=Digital Signature Trust Co.
Serial number: a0141420000015385736a0b85eca708
Valid from: Thu Mar 17 19:40:46 MSK 2016 until: Wed Mar 17 19:40:46 MSK 2021
Certificate fingerprints:
     MD5:  B1:54:09:27:4F:54:AD:8F:02:3D:3B:85:A5:EC:EC:5D
     SHA1: E6:A3:B4:5B:06:2D:50:9B:33:82:28:2D:19:6E:FE:97:D5:95:6C:CB
     SHA256: 25:84:7D:66:8E:B4:F0:4F:DD:40:B1:2B:6B:07:40:C5:67:DA:7D:02:43:08:EB:6C:2C:96:FE:41:D9:DE:21:8D
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
AuthorityInfoAccess [
  [
   accessMethod: ocsp
   accessLocation: URIName: http://isrg.trustid.ocsp.identrust.com
, 
   accessMethod: caIssuers
   accessLocation: URIName: http://apps.identrust.com/roots/dstrootcax3.p7c
]
]

#2: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: C4 A7 B1 A4 7B 2C 71 FA   DB E1 4B 90 75 FF C4 15  .....,q...K.u...
0010: 60 85 89 10                                        `...
]
]

#3: ObjectId: 2.5.29.19 Criticality=true
BasicConstraints:[
  CA:true
  PathLen:0
]

#4: ObjectId: 2.5.29.31 Criticality=false
CRLDistributionPoints [
  [DistributionPoint:
     [URIName: http://crl.identrust.com/DSTROOTCAX3CRL.crl]
]]

#5: ObjectId: 2.5.29.32 Criticality=false
CertificatePolicies [
  [CertificatePolicyId: [2.23.140.1.2.1]
[]  ]
  [CertificatePolicyId: [1.3.6.1.4.1.44947.1.1.1]
[PolicyQualifierInfo: [
  qualifierID: 1.3.6.1.5.5.7.2.1
  qualifier: 0000: 16 22 68 74 74 70 3A 2F   2F 63 70 73 2E 72 6F 6F  ."http://cps.roo
0010: 74 2D 78 31 2E 6C 65 74   73 65 6E 63 72 79 70 74  t-x1.letsencrypt
0020: 2E 6F 72 67                                        .org

]]  ]
]

#6: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_CertSign
  Crl_Sign
]

Greg, Constantin, please share your thoughts because from Let's Encrypt we can get only leaf server certificates (https://letsencrypt.org/docs/faq/). We can only ask for different domains. Can we separate our services by domain names: alias.demo.goldencode.com, fwd.demo.goldencode.com, demo.goldencode.com. Then we can request all these certificates and got them for usages in our secured services. Correct?

Another idea is to use Let's Encrypt certificates only for web clients and embedded web server. Others services should use certificates signed by the root self-signed certificate.

Sergey Ivanovskiy wrote:

Greg, Constantin, please share your thoughts because from Let's Encrypt we can get only leaf server certificates (https://letsencrypt.org/docs/faq/). We can only ask for different domains. Can we separate our services by domain names: alias.demo.goldencode.com, fwd.demo.goldencode.com, demo.goldencode.com. Then we can request all these certificates and got them for usages in our secured services. Correct?

I don't understand. demo.goldencode.com is a leaf server. And I have manually obtained certs from Let's Encrypt (as documented in #3240) and it worked. There was no need for additional sub-domain names.

Sergey Ivanovskiy wrote:

Another idea is to use Let's Encrypt certificates only for web clients and embedded web server. Others services should use certificates signed by the root self-signed certificate.

This is not suitable. There is a bug in one or more of the following:

• how we are requesting the certs
• the ACME4J code
• how we build the resulting key stores and directory entries

Do the current changes cause a problem with generating and processing the self signed certificates? Is there any reason that it can't be merged to trunk?

If not, please focus on the Hotel GUI testing to confirm that the other changes are OK. I want to get this into the trunk this weekend, even if the ACME cert issues cannot be resolved in time.

Greg Shah wrote:

Sergey Ivanovskiy wrote:

Greg, Constantin, please share your thoughts because from Let's Encrypt we can get only leaf server certificates (https://letsencrypt.org/docs/faq/). We can only ask for different domains. Can we separate our services by domain names: alias.demo.goldencode.com, fwd.demo.goldencode.com, demo.goldencode.com. Then we can request all these certificates and got them for usages in our secured services. Correct?

I don't understand. demo.goldencode.com is a leaf server. And I have manually obtained certs from Let's Encrypt (as documented in #3240) and it worked. There was no need for additional sub-domain names.

This certificate pair that you got can't be used to sign another certificates. It can be used only for the web and tls servers and clients.

Greg Shah wrote:

Sergey Ivanovskiy wrote:

Another idea is to use Let's Encrypt certificates only for web clients and embedded web server. Others services should use certificates signed by the root self-signed certificate.

This is not suitable. There is a bug in one or more of the following:

• how we are requesting the certs

There are no bugs here. The ACME client works properly.

• the ACME4J code

There are no bugs here. The ACME client works properly.

• how we build the resulting key stores and directory entries

The ACME certificates can't be used to sign another certificates. That is the reason. The code in 3366a consider the case when we have a signed certificate that can sign another ones.

Greg Shah wrote:

Do the current changes cause a problem with generating and processing the self signed certificates? Is there any reason that it can't be merged to trunk?
If not, please focus on the Hotel GUI testing to confirm that the other changes are OK. I want to get this into the trunk this weekend, even if the ACME cert issues cannot be resolved in time.

It should not be a problem to generate self signed certificates. There is no reason that it can't be merged to trunk. I would like to remove usage ACME certificates from com.goldencode.p2j.security.SSLCertGenUtil since the Let's Encrypt certificates can't be used as CA root certificates. The other changes are correct and AcmeClient works properly.

The ACME certificates can't be used to sign another certificates. That is the reason. The code in 3366a consider the case when we have a signed certificate that can sign another ones.

We were not expecting the ACME certificates to be usable as a signing authority. Why would we need that?

Greg/Sergey, I think we need to specify in webClient directory.cml a certificate alias to be used by all the web clients, and generate only this cert with let's encrypt.

For the other cases (FWD process, server, etc) I don't think we can t use let's encrypt easily (because each certificate would need a domain associated, as I understand).

Please review committed rev 11204 that removed ACME certificate usages as CA root certificates since these certificates has no Key_CertSign key usage.

Constantin Asofiei wrote:

Greg/Sergey, I think we need to specify in webClient directory.cml a certificate alias to be used by all the web clients, and generate only this cert with let's encrypt.

For the other cases (FWD process, server, etc) I don't think we can t use let's encrypt easily (because each certificate would need a domain associated, as I understand).

Yes, Let' Encrypt certificates can't be used for web clients because they are associated with concrete domain names. The wild card will be supported in 2018 according to information from Let's Encrypt.

Sergey Ivanovskiy wrote:

Greg Shah wrote:

Do the current changes cause a problem with generating and processing the self signed certificates? Is there any reason that it can't be merged to trunk?
If not, please focus on the Hotel GUI testing to confirm that the other changes are OK. I want to get this into the trunk this weekend, even if the ACME cert issues cannot be resolved in time.

It should not be a problem to generate self signed certificates. There is no reason that it can't be merged to trunk. I would like to remove usage ACME certificates from com.goldencode.p2j.security.SSLCertGenUtil since the Let's Encrypt certificates can't be used as CA root certificates. The other changes are correct and AcmeClient works properly.

OK, go ahead with this change.

Will ACME certs work for all the use cases that we expect?

• FWD Server secure socket for FWD client sessions and FWD server to FWD server sessions (com.goldencode.p2j.net.* and RemoteObject). This would include the connection from the spawner to launch a new FWD client.
• FWD Server's Embedded Jetty Instance (admin port which is also used for all other FWD server web usage)
• Embedded Application's Embedded Jetty Instance

What use cases are not supported?

Sergey Ivanovskiy wrote:

Constantin Asofiei wrote:

Greg/Sergey, I think we need to specify in webClient directory.cml a certificate alias to be used by all the web clients, and generate only this cert with let's encrypt.

For the other cases (FWD process, server, etc) I don't think we can t use let's encrypt easily (because each certificate would need a domain associated, as I understand).

Yes, Let' Encrypt certificates can't be used for web clients because they are associated with concrete domain names. The wild card will be supported in 2018 according to information from Let's Encrypt.

If these run on the same system (IP address) as the FWD server, then the FWD server's ACME cert can be used also for these embedded web servers.

If these run on a separate system, then we could process an ACME request on that system, right? It is OK for there to be a concrete domain name for that system.

Committed revision 11205 fixed my erroneous change 11204. Greg, Constantin, please help to design the architecture of Let's Encrypt usage in P2J. I think that P2J web server should also use these certificates if they are present in the directory under some node in /security/certificates/ and P2J web client if they are launched locally on the same machine as the P2J web server. I think that the most practical usage of Let's Encrypt is in Apache proxy server only. It seems that it should satisfy customer use cases, correct?

Greg Shah wrote:

Will ACME certs work for all the use cases that we expect?

• FWD Server secure socket for FWD client sessions and FWD server to FWD server sessions (com.goldencode.p2j.net.* and RemoteObject). This would include the connection from the spawner to launch a new FWD client.
• FWD Server's Embedded Jetty Instance (admin port which is also used for all other FWD server web usage)
• Embedded Application's Embedded Jetty Instance

What use cases are not supported?

It seems that only the usage in the Apache proxy server is supported now. We can get the target certificates automatically with help of AcmeClient and use them in Apache proxy SSL configuration.

        SSLCertificateFile    /etc/apache2/ssl/certs/apache.crt
        SSLCertificateKeyFile /etc/apache2/ssl/private/apache.key

        #   Server Certificate Chain:
        #   Point SSLCertificateChainFile at a file containing the
        #   concatenation of PEM encoded CA certificates which form the
        #   certificate chain for the server certificate. Alternatively
        #   the referenced file can be the same as SSLCertificateFile
        #   when the CA certificates are directly appended to the server
        #   certificate for convinience.
        #SSLCertificateChainFile /etc/apache2/ssl.crt/server-ca.crt

I think that P2J web server should also use these certificates if they are present in the directory under some node in /security/certificates/ and P2J web client if they are launched locally on the same machine as the P2J web server.

Yes, this was the intention. As noted in #3240, this is exactly what I did manually.

I think that the most practical usage of Let's Encrypt is in Apache proxy server only. It seems that it should satisfy customer use cases, correct?

I don't think so. The reverse proxy will not always be used. It is possible to put a non-proxy FWD installation directly on the internet. Also, any internal access to the non-proxyied web servers (FWD server or FWD clients) would still have the certificate issue even if the server was configured with the proper external domain name.

I just want to use these certs the same way we use the self-signed certs. By using them for all SSL usage on that same system it seems that they should work.

For now, just make the code safe for the trunk. It seems that the ACME work will take some more effort and testing. I don't want to deplay this branch getting into trunk.