FWD » Core Development » User Interface

My perception is that the browser/javascript drawing performance is very dependent upon the GPU of the system.

For example, Eric's system has some pretty generic Intel integrated chipset video and is seeing a particular customer application screen take roughly 2 seconds to load, where on that same system the Swing client is 1.5 seconds. But Eugenie sees the same application screen load much closer to 1 second for both Swing and the web client. Eugenie's system has a real GPU involved.

On Swing we use the "normal" Java2D drawing primitives. But in javascript we do some very unusual stuff, including drawing our own lines using pixels. On the javascript side, we do this because the "normal" drawing primitives result in anti-aliasing/fuzzy lines. To get crisp output we have to implement our own graphics primitives. But I suspect these are much more sensitive to hardware/acceleration. Perhaps they are just slow and it isn't about hardware sensitivity.

I would like us to explore this. The idea is to implement a configuration flag that lets us choose the drawing method. We can implement the normal/built-in 2D drawing as an alternative to our own primitives. I suspect the built-in primitives will be faster but with fuzzy output while our primitives will be crisp but slower.

Sergey: How hard is it to implement this idea so we can test it out?

It shouldn't be hard to implement because p2j_canvas_rendering.js uses Canvas API for filling all drawings and only stroked lines are drawn using pixels manipulations.
Planning to add this flag forbidPixelManipulation under webClient node. If this value is set, then all drawing primitives, except image drawing primitive won't use pixels manipulations. Please correct this flag name and its default value. Should pixels manipulations be used if this flag wasn't set (absent)?

Let's use disablePixelManipulation instead. The default value is false, which means that our current pixel manipulation approach will be used.

I encountered the new issue with OffscreenCanvas. If pixels manipulations are disabled, then line styles are not applied correctly. If OffscreenCanvas is changed to html5 canvas, then lines have assigned styles.

Sergey Ivanovskiy wrote:

Please review this patch. Can I commit these changes for 3821c?

Yes, no objection.

Committed revision 11711 (3821c) changed drawLineSegment so that if this flag, disablePixelManipulation, is set in the directory,

        <node class="container" name="webClient">
...............................................................
          <node class="boolean" name="disablePixelManipulation">
            <node-attribute name="value" value="TRUE"/>
          </node>
...............................................................

this.offscreenCtx.beginPath();
this.offscreenCtx.moveTo(x1 - this.origin.x, y1 - this.origin.y);
this.offscreenCtx.lineTo(x2 - this.origin.x, y2 - this.origin.y);
this.offscreenCtx.stroke();

Another note. Can you estimate the performance gain from disablePixelManipulation approach? On my machine I do not see the difference comparing to old one (which looks pretty effective for horizontal or vertical lines).

Eugenie Lyzenko wrote:

Disabled pixel manipulation give the following picture with many GUI line artifacts:

Yes, I observed them too. but I would like to estimate this approach with different environments so we can do correct solution. What are the actual performance bottlenecks of the web client.

Another note. Can you estimate the performance gain from disablePixelManipulation approach? On my machine I do not see the difference comparing to old one (which looks pretty effective for horizontal or vertical lines).

Yes, I can. For Google Chrome if disablePixelManipulation=true@, then 80 rectangles in one series are drawn for 5-6 ms, otherwise for 10-13 ms. And if disablePixelManipulation=true@, then 80 lines having the common point are drawn for 7-11 ms, otherwise for 130-140 ms. I agree that these changes don't improve the UI performance that it can be visible on your computer.

I agree that these changes don't improve the UI performance that it can be visible on your computer.

I think this is yet to be determined. A few weeks ago, Constantin has measured substantial time (1+ secs) in JS drawing for a customer screen that was taking 2.5 seconds in total. The amount of line and rectangle drawing is extreme in the UI, so a 2x to 10x improvement may very well be seen in real life. As I mentioned above, I suspect that people with better hardware are not seeing the performance cost of the JS drawing, whereas Eric and our customers (who have poor video hardware and slower CPUs) are seeing a real issue.

Sergey: I know you have written a test tool to display quite a lot of line output (horizontal, vertical and diagonal). Can this test be used for performance measurement? Or can it easily be modified to display timings and do enough drawing to usefully measure performance?

I'd like to include/checkin this code into our client. The idea is to provide a menu item in the virtual desktop web client, where this can be launched at any time. This will allow us to test multiple systems and browsers, including on systems in production. I see very little risk to this and lots of benefits.

I think the point here is how fast the BitBlt or line primitives work in graphics driver. Another word what is better hardware accelerated with driver+GPU combination. I have made double+ check. Really, no difference between bitmap based and vector lines drawing.

And what I'm surprising is CPU integrated graphics has such difference comparing to discrete GPU. Just because the i9XX chip integrated into CPU has also accelerated primitives implemented. May be sharing main memory for CPU and GPU is important.

I have some old notebook with integrated graphics so I could try to run web client on it to compare the timing for Swing and Web client.

Greg, what would you like to measure for our js web client. It can degrade performance if we add performance counters for primitive drawings. Please explain it more thoroughly.

It needs to fix incorrect lines when using drawings on the canvas, but I tried to check clipping regions and empty lines but it didn't help to detect its root causes.

It needs to fix incorrect lines when using drawings on the canvas. I tried to check clipping regions and empty lines but it didn't help to detect its root causes.

Sorry, incorrect lines are part of the clipping regions, I just did them visible via CanvasRenderer.prototype.clip

putImageData doesn't take into account clipping regions. Thus, if disablePixelManipulation=true, then some of these thin rectangular areas are not overwritten by putImageData. As a result we have incorrect lines.

what would you like to measure for our js web client. It can degrade performance if we add performance counters for primitive drawings. Please explain it more thoroughly.

I want a test which when executed uses a set of drawing primitives. This drawing should be intensive enough that the start and end of the test can be measured in some meaningful way to a user. This means that the test should take 1-2 seconds to execute on fast hardware. This may mean that the set of drawing primitives must be executed many times in order to have a long-enough usage of the CPU/GPU.

I do not expect that the individual primitives are measured. I care about the overall test time. When Eric runs it on his system and you run it on your system, we can compare the results.

Sergey Ivanovskiy wrote:

putImageData doesn't take into account clipping regions. Thus, if disablePixelManipulation=true, then some of these thin rectangular areas are not overwritten by putImageData. As a result we have incorrect lines.

Please do fix this.

On Chrome, when opening a widget-heavy customer screen, the this.offscreenCtx.putImageData(imageData, x, y); line is executed 6789 times in ~400ms. If I comment this line, the screen shows (albeit incorrectly) ~300ms faster than when this line is used.

We use putImageData for each and every line draw, and this includes drawing rectangles. If we could use a single putImageData, strokeRect should be ~60-70% faster.

We use putImageData for each and every line draw, and this includes drawing rectangles. If we could use a single putImageData, strokeRect should be ~60-70% faster.

I'd like to prove this using the proposed test program on different systems/browsers.

Assuming your point is correct, I have an idea (possibly crazy).

Instead of drawing into an offscreen canvas and blitting it (a.k.a. "flash" it) to the visible canvas, could we treat a large ImageData structure as the "off screen buffer"? When I say large, I mean it would be exactly the full size of the canvas.

ImageData structures can be long-lived. There is nothing special about them. We use this today for pixel level drawing. But it seems to me that we could batch our drawing in such a way as to make many edits to this large ImageData structure without calling putImageData() for each individual edit. Instead, we would make many edits and then at the end of batch moment, use putImageData() once to push the entire canvas contents.

Is it possible? Would it greatly help performance without causing issues?

Sergey: What is the status of having a drawing test page integrated with our client?

Constantin Asofiei wrote:

On Chrome, when opening a widget-heavy customer screen, the this.offscreenCtx.putImageData(imageData, x, y); line is executed 6789 times in ~400ms. If I comment this line, the screen shows (albeit incorrectly) ~300ms faster than when this line is used.

We use putImageData for each and every line draw, and this includes drawing rectangles. If we could use a single putImageData, strokeRect should be ~60-70% faster.

For rectangles painting I have doubts this can help. The rectangle is combination of 4 lines, 2 vertical and 2 horizontal with 1 pixel width. Every line need separate putImageData. If we get full image covered by rectangle to draw it back in one call then it will be difficult to fast draw the lines inside this large image (especially for vertical lines) - we will have to operate with single pixels. So the advantage of the single putImageData call can be eaten with over-complexity line drawing algorithm.

Every line need separate putImageData.

Why is this needed?

If we get full image covered by rectangle to draw it back in one call then it will be difficult to fast draw the lines inside this large image (especially for vertical lines) - we will have to operate with single pixels.

Yes, this is true. All drawing would have to be at the pixel level.

Greg Shah wrote:

We use putImageData for each and every line draw, and this includes drawing rectangles. If we could use a single putImageData, strokeRect should be ~60-70% faster.

I'd like to prove this using the proposed test program on different systems/browsers.

Assuming your point is correct, I have an idea (possibly crazy).

Instead of drawing into an offscreen canvas and blitting it (a.k.a. "flash" it) to the visible canvas, could we treat a large ImageData structure as the "off screen buffer"? When I say large, I mean it would be exactly the full size of the canvas.

ImageData structures can be long-lived. There is nothing special about them. We use this today for pixel level drawing. But it seems to me that we could batch our drawing in such a way as to make many edits to this large ImageData structure without calling putImageData() for each individual edit. Instead, we would make many edits and then at the end of batch moment, use putImageData() once to push the entire canvas contents.

Is it possible? Would it greatly help performance without causing issues?

If I understand properly your idea this is a kind of "double buffering" concept. We have two bitmaps one that is currently on screen and other that is the subject of current batch modification. When the batch painting is finished we swap the bitmaps by single putImageData(), right? In this case I think yes we can speed up the drawing performance. Or do you mean some other approach behind?

Crazy idea number 2: use WebGL. All of the modern browsers support a canvas drawing context using webgl. This is a much lower-level API that should be significantly faster than the the 2d context from canvas. WebGL is designed to move most of the work onto the GPU whereas the 2d context may or may not be well written for that usage.

https://stackoverflow.com/questions/28867297/why-webgl-is-faster-than-canvas

The significant down side is that we would have to re-write all our drawing primitives. This code will be more verbose and harder to support since it is operating on a low-level 3D capable API instead of the simplified 2D canvas API. We could possibly use something like PixiJS. This would provide a simpler 2D API but it also brings quite a bit of unnecessary "baggage" for scenes, animation and so forth. If one of these frameworks is simple enough to use, then we might at least try it to compare performance.

If I understand properly your idea this is a kind of "double buffering" concept. We have two bitmaps one that is currently on screen and other that is the subject of current batch modification. When the batch painting is finished we swap the bitmaps by single putImageData(), right?

Yes, this is the idea. We always draw into the same "off screen" ImageData so that it is the accumulation of all previous drawing.

Greg Shah wrote:

Every line need separate putImageData.

Why is this needed?

This is a kind of optimization. If the single line is either horizontal or vertical we use very effective algorithm of the bitmap pixel painting. For example 16 pixels line can be painted with 4 iterations while in pixel based approach it takes 16 iterations.

If we get full image covered by rectangle to draw it back in one call then it will be difficult to fast draw the lines inside this large image (especially for vertical lines) - we will have to operate with single pixels.

Yes, this is true. All drawing would have to be at the pixel level.

And we will loose optimized pixels rendering described above.

Greg Shah wrote:

Crazy idea number 2: use WebGL. All of the modern browsers support a canvas drawing context using webgl. This is a much lower-level API that should be significantly faster than the the 2d context from canvas. WebGL is designed to move most of the work onto the GPU whereas the 2d context may or may not be well written for that usage.

https://stackoverflow.com/questions/28867297/why-webgl-is-faster-than-canvas

The significant down side is that we would have to re-write all our drawing primitives. This code will be more verbose and harder to support since it is operating on a low-level 3D capable API instead of the simplified 2D canvas API. We could possibly use something like PixiJS. This would provide a simpler 2D API but it also brings quite a bit of unnecessary "baggage" for scenes, animation and so forth. If one of these frameworks is simple enough to use, then we might at least try it to compare performance.

Yes, I also think about concurrency here. Even considering usage of the Worker approach for JS:

https://medium.com/techtrument/multithreading-javascript-46156179cf9a

But this is the subject of discussion because it requires some message based communication between different JS blocks so possible performance benefit can be zeroed by another communication channel expenses. So I agree, the ideal case is to make hardware accelerated canvas with simple entry points for drawing primitives, line, text and bitmaps. In this case the performance boost could be comparable with the moment when GPU accelerators were first appeared on the market. It was like a magic, just ask draw line from point 1 to point 2 or fill the rectangle and forget about the rest. So may be WebGL is the best opportunity for this goal. Ideally it could make Web client faster than Swing due to higher level of native concurrency and direct usage of the GPU capabilities.

Greg Shah wrote:

If I understand properly your idea this is a kind of "double buffering" concept. We have two bitmaps one that is currently on screen and other that is the subject of current batch modification. When the batch painting is finished we swap the bitmaps by single putImageData(), right?

Yes, this is the idea. We always draw into the same "off screen" ImageData so that it is the accumulation of all previous drawing.

It was in my first modification where it can be possible to set a larger screen image and put all pixels on this image. But if we would want to draw all pixels from all drawing operations in a one batch and then to put it on the canvas, then we should control clipping regions manually. In my first changes that I reverted drawImage and drawLineSegment take into account clipping regions. The putImageData doesn't take into account clipping regions.

Greg Shah wrote:

If I understand properly your idea this is a kind of "double buffering" concept. We have two bitmaps one that is currently on screen and other that is the subject of current batch modification. When the batch painting is finished we swap the bitmaps by single putImageData(), right?

Yes, this is the idea. We always draw into the same "off screen" ImageData so that it is the accumulation of all previous drawing.

Since the order of drawing operations is important and there are operations implemented only by 2D context (like text rendering) you could draw to the ImageData structure only until such an context-exclusive operation comes up.

This idea has been already proposed by Sergey, see #4785-831 and related notes. But I'm not sure how far he went with it.

Greg Shah wrote:

what would you like to measure for our js web client. It can degrade performance if we add performance counters for primitive drawings. Please explain it more thoroughly.

I want a test which when executed uses a set of drawing primitives. This drawing should be intensive enough that the start and end of the test can be measured in some meaningful way to a user. This means that the test should take 1-2 seconds to execute on fast hardware. This may mean that the set of drawing primitives must be executed many times in order to have a long-enough usage of the CPU/GPU.

I do not expect that the individual primitives are measured. I care about the overall test time. When Eric runs it on his system and you run it on your system, we can compare the results.

Could it be a standalone html page?

Sergey Ivanovskiy wrote:

Greg Shah wrote:

what would you like to measure for our js web client. It can degrade performance if we add performance counters for primitive drawings. Please explain it more thoroughly.

I want a test which when executed uses a set of drawing primitives. This drawing should be intensive enough that the start and end of the test can be measured in some meaningful way to a user. This means that the test should take 1-2 seconds to execute on fast hardware. This may mean that the set of drawing primitives must be executed many times in order to have a long-enough usage of the CPU/GPU.

I do not expect that the individual primitives are measured. I care about the overall test time. When Eric runs it on his system and you run it on your system, we can compare the results.

Could it be a standalone html page?

I'm OK with that, just have a way to open it from the virtual screen. Are you using the FWD primitives in p2j.canvas_renderer and such, or something else?

Yes, I am using p2j.canvas_renderer.js and p2j.strokes.js and p2j.js. Why does it require to run this html page with the web client? It can add some useless code to the web client.

Greg Shah wrote:

Sergey Ivanovskiy wrote:

putImageData doesn't take into account clipping regions. Thus, if disablePixelManipulation=true, then some of these thin rectangular areas are not overwritten by putImageData. As a result we have incorrect lines.

Please do fix this.

At this moment I didn't succeed in fixing this issue. I observed strange effects when using canvas drawing. For an example, this drawing code

ctx.beginPath();
ctx.rect(20, 20, 100, 50);
ctx.clip();
ctx.stroke();

This idea has been already proposed by Sergey, see #4785-831 and related notes. But I'm not sure how far he went with it.

Sorry, to Sergey. I had not "caught up" with the idea yet.

there are operations implemented only by 2D context (like text rendering) you could draw to the ImageData structure only until such an context-exclusive operation comes up.

Yes, I agree this is a big issue. It makes the solution much more complex and fragile.

I think the webgl approach is likely to be a complete answer to the problem of "How do we get full rendering control AND blazing fast hardware accelerated speed?" The more I think about it, the more it "fits". I think in this approach there will be no anti-aliasing issues with line drawing.

Considering that we don't need animation, camera perspectives/light sources/scenes/3D or other advanced features, I think it is best to implement the (relatively) simple 2D drawing primitives directly in webgl instead of using a framework. We should at least start this way. Once a test page is ready, we can add alternate webgl implementations and compare the results on a wide range of browsers and hardware.

I also wonder if we can handle z-order more naturally using webgl. I suspect so, though it may not be worth it to add the complexity.

The one area where there will be some effort is text drawing. I think we will need to use the 2D canvas text rendering and integrate it. As that page mentions, if you use the 2D for drawing text it can't be covered by webgl elements, so the actual rendering might have to be done in webgl while the 2D canvas is still used for the core font processing/scaling.

In regard to concurrency, I want to avoid it for now. Web workers come with other issues and certainly add complexity. My intuition suggests that what we are drawing is far less taxing than a 3D game that implements high FPS. Yet, the webgl canvas is perfectly capable of rendering such high end results with little trouble. I think we are just hitting the limit of 2D canvas especially when combined with the line anti-aliasing "fix".

Sergey Ivanovskiy wrote:

Yes, I am using p2j.canvas_renderer.js and p2j.strokes.js and p2j.js. Why does it require to run this html page with the web client? It can add some useless code to the web client.

The idea here is for anyone to be able to easily open this 'performance test' page. You can build a standalone html page which calls FWD APIs from the JS driver to measure performance. We just need to allow users to have access to this tool easily - and using the virtual screen is the most straightforward way.

Ok. Following this way I need to add new js module p2j.perf_tests.js to index.html page

<script type="text/javascript" src="${webRoot}/client/p2j.perf_tests.js"></script>

Yes, that is OK.

Constantin Asofiei wrote:

On Chrome, when opening a widget-heavy customer screen, the this.offscreenCtx.putImageData(imageData, x, y); line is executed 6789 times in ~400ms. If I comment this line, the screen shows (albeit incorrectly) ~300ms faster than when this line is used.

We use putImageData for each and every line draw, and this includes drawing rectangles. If we could use a single putImageData, strokeRect should be ~60-70% faster.

What I think as another possible way of optimization is to understand why the line drawings are so huge (6789). Do we really see all of these lines on screen? If not we need to inspect the widget painting code to find out if we can keep only lines we really see. This can improve performance not only for Web client but Swing should also run faster. Actually I think for the screen you mentioned there are about ~20 different lines in total to draw. Certainly not 6789.

Most rectangles are drawn with 4 lines instead of the rectangle primitive. There are quite a few non-black lines drawn even on simple screens. So I would expect there to be more than just ~20 lines.

With that said, I wouldn't have expected 6700+ lines either. I suspect we may have a lot of "overdrawing" (drawing the same thing multiple times before the final output is visible). I think there are cases where we found some missing output and added unconditional repainting. There are probably other causes too.

It is probably worth investigating.

Please review this patch for p2j.logger.js to enable its functionality to the virtual desktop. Can I commit these changes?

Yes

If you provide a popup menu for the virtual desktop background for the performance test page, we should also add a "View Log" menu item and then remove the "special" button on the task bar.

Sergey: What is your timing on the test application? I had thought it would be something pretty quick.

Once that is available, please work on adding enough webgl primitives so that it can be tested with the same test app. I want to get some idea of the value of webgl before Monday, if possible.

So the clipping bug can be paused while that is worked.

When I am suggesting a popup menu, I don't mean for this to be in the 4GL. I am expecting a simple HTML5/JS based menu that can be used from the "desktop background" in virtual desktop mode. It is nothing related to converted 4GL. The menu is just there for some debugging aids, right?

Yes, you are correct. It is not hard to implement. I am using dojo menu widget for the virtual task bar.

Greg Shah wrote:

Sergey: What is your timing on the test application? I had thought it would be something pretty quick.

I was planning yesterday but finally spent some time for digging dojo api.

Once that is available, please work on adding enough webgl primitives so that it can be tested with the same test app. I want to get some idea of the value of webgl before Monday, if possible.

Yes, planning to reach this deadline.

Greg Shah wrote:

The one area where there will be some effort is text drawing. I think we will need to use the 2D canvas text rendering and integrate it. As that page mentions, if you use the 2D for drawing text it can't be covered by webgl elements, so the actual rendering might have to be done in webgl while the 2D canvas is still used for the core font processing/scaling.

I'm a bit worried about this. With the two canvases approach, we can't implement proper z-order. I.e. cases where text is rendered with lower z-order than other widgets.

Another potential issue is that the vertex shaders use a so called clip space coordinates that go from -1 to 1 in each direction. This will be a subject to rounding errors and so we may not always achieve pixel perfect output.

Hynek Cihlar wrote:

Greg Shah wrote:

The one area where there will be some effort is text drawing. I think we will need to use the 2D canvas text rendering and integrate it. As that page mentions, if you use the 2D for drawing text it can't be covered by webgl elements, so the actual rendering might have to be done in webgl while the 2D canvas is still used for the core font processing/scaling.

I'm a bit worried about this. With the two canvases approach, we can't implement proper z-order. I.e. cases where text is rendered with lower z-order than other widgets.

Yes, using a transparent overlay canvas we would need to clip the partially visible content to show only those parts that should be visible. This means tracking whether something is obscured or not.

Hynek Cihlar wrote:

Another potential issue is that the vertex shaders use a so called clip space coordinates that go from -1 to 1 in each direction. This will be a subject to rounding errors and so we may not always achieve pixel perfect output.

Btw. the coordinates are single precision float numbers (32 bit).

Greg Shah wrote:

Hynek Cihlar wrote:

Greg Shah wrote:

The one area where there will be some effort is text drawing. I think we will need to use the 2D canvas text rendering and integrate it. As that page mentions, if you use the 2D for drawing text it can't be covered by webgl elements, so the actual rendering might have to be done in webgl while the 2D canvas is still used for the core font processing/scaling.

I'm a bit worried about this. With the two canvases approach, we can't implement proper z-order. I.e. cases where text is rendered with lower z-order than other widgets.

Yes, using a transparent overlay canvas we would need to clip the partially visible content to show only those parts that should be visible. This means tracking whether something is obscured or not.

Please explain more thoroughly. There are two canvases: one of them is 2d and the other is webgl. Which one of them is transparent? 2d canvas renders only text and webgl renders only drawings.

There are two canvases: one of them is 2d and the other is webgl.

Yes.

Which one of them is transparent?

The 2d canvas. It is "above" or "in front" of the webgl canvas. This corresponds to the 2d canvas having a higher z-index in JS terms. It needs to be transparent because all other drawing for the webgl canvas must be visible.

2d canvas renders only text and webgl renders only drawings.

Yes.

Sergey Ivanovskiy wrote:

Please explain more thoroughly. There are two canvases: one of them is 2d and the other is webgl. Which one of them is transparent? 2d canvas renders only text and webgl renders only drawings.

For the initial implementation of the performance demo page we don't need text rendering, so only single canvas is enough.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Please explain more thoroughly. There are two canvases: one of them is 2d and the other is webgl. Which one of them is transparent? 2d canvas renders only text and webgl renders only drawings.

For the initial implementation of the performance demo page we don't need text rendering, so only single canvas is enough.

This part related to webgl won't be ready today. Now p2j.canvas_renderer.js module is dependent on 2d context. It requires to refactor this module.
Sadly, we don't have pure independent modules and sometimes it plays badly in the development. For an example, p2j.clipboard.js installs mouseup listener that is a part of copy, paste and cut functionality, but it blurs the dojo menus. It was unexpected and finally debugging helped me to detect the issue.

I did some reading on WebGL and some related technologies. Here are some unsorted notes I made.

• it is very low level, everything that is drawn must be broken into triangles (consider round shapes - round rectangles, ellipses).
• rounding of float coordinates will be an issue - the result will not be pixel perfect
• there is an overhead of setting up shaders and buffers, so the whole screen should be rendered in one batch - break up all the UI shapes into triangles, prepare and upload buffers (with vertices and textures), and render

Consider some 2D JS library capable to render to WebGL context. PixiJS seems to be an overkill for us, it focuses more on animations. But perhaps Two.js would be a good fit. It seems to be very lean and has all the needed primitives (including text rendering, but very basic, probably unusable).

I suggest we create two WebGL versions for the performance line drawing page. One in raw WebGL and another using Two.js (or similar).

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Please explain more thoroughly. There are two canvases: one of them is 2d and the other is webgl. Which one of them is transparent? 2d canvas renders only text and webgl renders only drawings.

For the initial implementation of the performance demo page we don't need text rendering, so only single canvas is enough.

This part related to webgl won't be ready today. Now p2j.canvas_renderer.js module is dependent on 2d context. It requires to refactor this module.

For simplicity, can't we just introduce new js file p2j.webgl_renderer.js and implement only the limited set of primitives for line drawing with compatible API? Without worrying about refactoring of p2j.canvas_renderer.js.

Yes, agree. Thus, we should be able to test 2d with pixels manipulation and two variants of webgl. Correct?

Sergey Ivanovskiy wrote:

Yes, agree. Thus, we should be able to test 2d with pixels manipulation and two variants of webgl. Correct?

Unless you are already working on it, I can prepare one of the webgl renderers.

I suggest we create two WebGL versions for the performance line drawing page. One in raw WebGL and another using Two.js (or similar).

Agreed. I'm OK with trying Two.js.

Yes, agree. Thus, we should be able to test 2d with pixels manipulation and two variants of webgl. Correct?

Yes. Actually, we want to test 4 cases:

• 2d with our manual pixel line drawing
• 2d with standard line drawing
• webgl raw
• webgl Two.js

Guys,

Thinking about Web driver line painting speed up I've got the idea for how to improve current bitmap based approach. While the putImageData() is still left for every line painting the preparation steps are significantly simplified. Consider the following new code in p2j.strokes.js:

   function DefaultPathRenderer(canvasRenderer, strokeColor)
   {
...
      const pix32Bit = new Uint32Array(imageData.buffer);
      var onePix = pix32Bit[0];

      /**
       * Prepares the given stroke to be applied to the line of pixels called a path.
       */
      this.beginStroke = function()
      {
      }

      /**
       * Apply the stroke to vertical or horizontal lines given by the absolute coordinates that
       * satisfy x1 <= x2 and y1 <= y2.
       * 
       * @param    {Number} x1
       *           X coordinate of the starting pixel to be drawn.
       * @param    {Number} y1
       *           Y coordinate of the starting pixel to be drawn.
       * @param    {Number} x2
       *           X coordinate of the ending pixel to be drawn.
       * @param    {Number} y2
       *           Y coordinate of the ending pixel to be drawn.
       */
      this.strokeLine = function(x1, y1, x2, y2)
      {
         // The default line width is 1 pixel.
         // Calculate the line with and height to draw a line as an image
         var lineWidth  = x2 - x1 + 1;
         var lineHeight = y2 - y1 + 1;
         // Create a line image
         var screenImage = canvasRenderer.createImageData(lineWidth, lineHeight);

         // create 32-bit reference pointer to screen byte buffer
         const data32 = new Uint32Array(screenImage.data.buffer);

         // Fill a line image with the given stroke color (32-bit operation)
         data32.fill(onePix);

         // Put the line image on the canvas
         canvasRenderer.putImageData(screenImage, x1, y1);
      }
...

As you can see there is no loop inside line painter. Only single array fill call. I tested on my system and this certainly gives some improvement.

Can I commit this change into 3821c? I think this can be more effective than previous version (difficult to estimate the exact value). At least for GPU (integrated or discrete) that has accelerated area fill operation.

Eric,

you could then test and compare with previous painter to see if there are any improvement on your system or not. If I'm right we could have the improvement immediately for every line painting (horizontal or vertical).

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Yes, agree. Thus, we should be able to test 2d with pixels manipulation and two variants of webgl. Correct?

Unless you are already working on it, I can prepare one of the webgl renderers.

OK, I didn't start this work yet. What renderer would you prefer to develop? Greg, do you agree?

Greg, do you agree?

Yes

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Unless you are already working on it, I can prepare one of the webgl renderers.

OK, I didn't start this work yet. What renderer would you prefer to develop?

In this case it needs to integrate p2j.webgl_renderer.js into p2j.perf_tests.js that will be committed sooner.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Yes, agree. Thus, we should be able to test 2d with pixels manipulation and two variants of webgl. Correct?

Unless you are already working on it, I can prepare one of the webgl renderers.

OK, I didn't start this work yet. What renderer would you prefer to develop? Greg, do you agree?

I've started some prototyping on the raw webgl. So I will take that.

As you can see there is no loop inside line painter. Only single array fill call. I tested on my system and this certainly gives some improvement.

I think it is worth testing this approach. Since this is just a low level JS array fill, I doubt that it would be accelerated be GPU hardware. But I can see that it may be slightly more efficient.

Please note that your approach does not set the current onePix bytes to the proper strokeColor[] RGB values and it does not properly set the alpha. In addition, using the Uint32Array brings an endian/byte ordering issue. It is likely that you would get this running on a little-endian system. Someone trying this later on a big-endian system will find it broken unless you handle the byte ordering properly.

Greg Shah wrote:

As you can see there is no loop inside line painter. Only single array fill call. I tested on my system and this certainly gives some improvement.

I think it is worth testing this approach. Since this is just a low level JS array fill, I doubt that it would be accelerated be GPU hardware. But I can see that it may be slightly more efficient.

Please note that your approach does not set the current onePix bytes to the proper strokeColor[] RGB values and it does not properly set the alpha. In addition, using the Uint32Array brings an endian/byte ordering issue. It is likely that you would get this running on a little-endian system. Someone trying this later on a big-endian system will find it broken unless you handle the byte ordering properly.

The stroke color is properly set up in the DefaultPathRenderer constructor (I have provided only changed part of the code previously):

   function DefaultPathRenderer(canvasRenderer, strokeColor)
   {
      var width = defaultStroke.getWidth();
      var image = canvasRenderer.createImageData(width, width);
      var imageData = image.data;

      for (var i = 0; i < 4 * width * width; i += 4)
      {
         imageData[i]     = strokeColor[0];
         imageData[i + 1] = strokeColor[1];
         imageData[i + 2] = strokeColor[2];
         imageData[i + 3] = 255;
      }

      const pix32Bit = new Uint32Array(imageData.buffer);
      var onePix = pix32Bit[0];
...

So we already do this work. In addition to Endian specific handling. Meaning nothing in this point is changed. I just removed extra double processing for some data. Moreover we do not even need to create image data for every horizontal or vertical line. We can create single template line with maximum possible size for given display only once while initializing p2j.strokes.js module for given path renderer.

Please post a patch so that we can more easily review the exact changes.

Moreover we do not even need to create image data for every horizontal or vertical line. We can create single template line with maximum possible size for given display only once while initializing p2j.strokes.js module for given path renderer.

Yes, I think this is useful too. I assume you mean that we would then use the variant putImageData(imageData, dx, dy, dirtyX, dirtyY, dirtyWidth, dirtyHeight) instead of the simpler one.

HTMLCanvasElement.getContext accepts an antialiasing option, but this doesn't fully work, see below.

You've set WebGLRenderingContext.getContextAttributes().antialias to false and it doesn't disable it?

It may be that you have to do it when you request the context (see this) as in canvas.getContext('webgl', { antialias: false });.

Greg Shah wrote:

HTMLCanvasElement.getContext accepts an antialiasing option, but this doesn't fully work, see below.

You've set WebGLRenderingContext.getContextAttributes().antialias to false and it doesn't disable it?

Sorry for not expressing myself clearly. When I wrote "this doesn't fully work" I meant changing the option does make a difference, just the output shapes are not as sharp as I would expect. With antialias set to false the lines are blurred as can be seen on the image Screenshot from 2020-10-16 16-24-10.png above. The other image in the note shows the output when antialias is set to true.

I added p2j.perf_tests.js in the committed rev 11721 (3821c). The context menu is opened for the virtual task bar on the right mouse click. I encountered technical difficulties related to the widget's layouts and to the clipboard functionality that doesn't work with dojo widgets because the FWD hidden textarea(id=copy) losses its focus. I will change the logs widget to the simple textarea dojo widget if the correct solution will not be found within this weekend.

It seems that anti-aliasing really should be able to be turned off. Right now it seems to be partially implemented when disabled. We would certainly want to figure out how to fully disable it or change how we are drawing if it is an artifact of our code.

Regardless of the status of the antialiasing is, let's still determine if webgl is faster or not.

OK, this confirms what we suspected. The webgl approach is a game changer. So the key question: can we eliminate the anti-aliasing?

One idea: test Two.js, Phaser.js... webgl frameworks to see if they have the ability to fully disable anti-aliasing/draw unaliased lines. If so, then let's see what they do under the covers to implement it.

I find it hard to believe that there is no way to draw a line in webgl without aliasing.

Greg Shah wrote:

One idea: test Two.js, Phaser.js... webgl frameworks to see if they have the ability to fully disable anti-aliasing/draw unaliased lines. If so, then let's see what they do under the covers to implement it.

OK, will do.

Hynek, could you share the code for your performance presentations? Could we insert these charts into p2j.perf_tests.js? What chart widgets are you using?

Committed revision 11724 (3821c) fixed "copy logs to the clipboard" functionality.

The reported antialiasing issue in WebGL seems to be just an interpolation between canvas resolution and physical screen resolution. WebGL (with the antialias option turned off) seems to perform not worse than our pixel manipulation.

I can also report, that Two.js is a lot slower for line drawing than any of the methods tested above (Raw WebGL, 2D lineTo and 2D pixel). One of the reasons is that it builds object representation of the whole scene. But also because it uses more generic approach to drawing (to handle all possible line styles) involving GL textures.

Sergey Ivanovskiy wrote:

Hynek, could you share the code for your performance presentations? Could we insert these charts into p2j.perf_tests.js? What chart widgets are you using?

I used LibreOffice Calc to chart the results. I let the script output the results in the console and I just copied these in the spreadsheet.

Great results! The solution is clear: implement raw webgl support. This is the top priority for the both of you (Hynek and Sergey). How do we split up the work?

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, could you share the code for your performance presentations? Could we insert these charts into p2j.perf_tests.js? What chart widgets are you using?

Sergey, it is attached. Just please bare in mind, it is still work in progress.

OK. I am studying OpenGL in general. It seems that I can't download webgl.js from Redmine, don't understand why this file downloaded by Firefox has a zero size but the attached file is not empty.

Greg Shah wrote:

Great results! The solution is clear: implement raw webgl support. This is the top priority for the both of you (Hynek and Sergey). How do we split up the work?

I will start with implementing the line primitives in GUI driver.

Thank you! I tested your test application with Firefox and Google Chrome. Google Chrome works properly, but Firefox throws exception:

Uncaught 
Exception
​
columnNumber: 0
​
data: null
​
filename: "file:///home/sbi/projects/3821c/webgl.js" 
​
lineNumber: 446
​
message: "" 
​
name: "NS_ERROR_NOT_IMPLEMENTED" 
​
result: 2147500033
​
stack: "main2@file:///home/sbi/projects/3821c/webgl.js:446:26\nonload@file:///home/sbi/projects/3821c/index.html:1:6\n" 
​
<prototype>: ExceptionPrototype { toString: toString(), name: Getter, message: Getter, … }
webgl.js:446
    main2 file:///home/sbi/projects/3821c/webgl.js:446
    onload file:///home/sbi/projects/3821c/index.html:1

Sorry, it is not webgl. It is a offscreen canvas.

Sergey Ivanovskiy wrote:

Sorry, it is not webgl. It is a offscreen canvas.

Yes, in createOffscreenCanvas uncomment the browser detection.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Sorry, it is not webgl. It is a offscreen canvas.

Yes, in createOffscreenCanvas uncomment the browser detection.

Also, comment out the call to drawLinesWithTwo, this requires two.js.

Here are some interesting results running index.html above on my Pixel 1 mobile phone. The times are in ms.

Number of lines    500    1000    1500    2000    2500    3000    3500    4000    4500    5000
Web GL    9.90000000456348    37.2999999963213    29.3000000092434    4.19999999576248    4.89999999990687    4.70000000495929    5.30000000435393    7.19999999273568    6.69999999809079    7.29999999748543
2D lineTo    73.2999999891035    32.3000000062166    45.5000000074506    35.6000000028871    39.9000000033993    30.5999999982305    34.2000000091502    34.4999999942956    41.4000000018859    45.8000000071479
2D pixel    206.700000009732    791.400000001886    1256.80000000284    2519.80000000913    3637.49999999709    5491.20000000403    7609.20000000624    17715.7000000007    12492.2999999981    33929.3999999936

Number of lines    2000    4000    6000    8000    10000    12000    14000    16000    18000    20000
Web GL    16.1000000080094    17.1000000118511    32.3999999964144    35.1000000082422    14.500000004773    19.5999999996275    21.0999999981141    24.0999999950873    28.9000000047963    30.1000000035856
2D lineTo    29.500000004191    44.2000000039116    59.8000000027241    68.699999988894    87.1000000042841    109.899999995832    126.600000003236    155.299999998533    164.6999999939    185.100000002421

This clearly shows that moving out of the current GUI implementation for mobile client will be mandatory to achieve good results.

It is preferred for us to build these changes in a manner that allows them to be checked in to 3821c. You can disable those paths for the real web client until they are ready but I'd really like to be able to run the test page on many different systems and browsers. Can we make that happen quickly?

I'd also like to hear an estimate of when we can see a working version of the client using webgl. I understand that getting the obscured text fully working will be a tricky part.

Finally, Eric mentioned (to me) a good point yesterday. He pointed out that we have some cases where we XOR a vertical line over some text. He was thinking about the line on the right side of a fill-in where the text gets clipped and that some themes XOR a line there to show the text as grey instead of black. The case I recall is the I-bar cursor which needs to XOR when over text. We will need to draw these parts on the overlay canvas. I think that means in some cases that we need to redraw some background parts from the lower webgl canvas under the XOR lines as well.

Greg Shah wrote:

It is preferred for us to build these changes in a manner that allows them to be checked in to 3821c. You can disable those paths for the real web client until they are ready but I'd really like to be able to run the test page on many different systems and browsers. Can we make that happen quickly?

Sergey, can you integrate the test in the client?

Greg Shah wrote:

Finally, Eric mentioned (to me) a good point yesterday. He pointed out that we have some cases where we XOR a vertical line over some text. He was thinking about the line on the right side of a fill-in where the text gets clipped and that some themes XOR a line there to show the text as grey instead of black. The case I recall is the I-bar cursor which needs to XOR when over text. We will need to draw these parts on the overlay canvas. I think that means in some cases that we need to redraw some background parts from the lower webgl canvas under the XOR lines as well.

Yes, all the XORing involving text must be drawn on the 2D canvas. The image rendered on the WebGL canvas (or part of it) can be copied to the 2D canvas.

Hynek Cihlar wrote:

Greg Shah wrote:

It is preferred for us to build these changes in a manner that allows them to be checked in to 3821c. You can disable those paths for the real web client until they are ready but I'd really like to be able to run the test page on many different systems and browsers. Can we make that happen quickly?

Sergey, can you integrate the test in the client?

Yes, I will follow your test.

It seems that the idea with two canvases that one of them has webgl context and the other has 2d context shouldn't work. The text on the transparent canvas isn't visible. The displayed text begins to fade if the css opacity style tends to zero. I missed this idea now. Could you clarify this idea? We can check https://stackoverflow.com/questions/8847899/how-to-draw-text-using-only-opengl-methods

Sergey Ivanovskiy wrote:

It seems that the idea with two canvases that one of them has webgl context and the other has 2d context shouldn't work. The text on the transparent canvas isn't visible. The displayed text begins to fade if the css opacity style tends to zero. I missed this idea now. Could you clarify this idea? We can check https://stackoverflow.com/questions/8847899/how-to-draw-text-using-only-opengl-methods

Please check the link Greg sent, https://webglfundamentals.org/webgl/lessons/webgl-text-canvas2d.html. The demo there shows the text with GL content just fine.

I wasn't sure how to draw all the UI elements with images in one draw pass in WebGL. Perhaps the following scheme could word. As mentioned above all (except text and images) GUI driver primitives must be broken into triangles. Then all the image primitive will combine into a single image. The single combined image with the calculated triangles (and some metadata to direct fragment shader to either use a specified color or a texture) will be uploaded to the GPU and drawn in a single draw.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

It seems that the idea with two canvases that one of them has webgl context and the other has 2d context shouldn't work. The text on the transparent canvas isn't visible. The displayed text begins to fade if the css opacity style tends to zero. I missed this idea now. Could you clarify this idea? We can check https://stackoverflow.com/questions/8847899/how-to-draw-text-using-only-opengl-methods

Please check the link Greg sent, https://webglfundamentals.org/webgl/lessons/webgl-text-canvas2d.html. The demo there shows the text with GL content just fine.

Thank you. Understand. This example just uses background-color : transparent; css property for the text canvas.

Hynek Cihlar wrote:

I wasn't sure how to draw all the UI elements with images in one draw pass in WebGL. Perhaps the following scheme could word. As mentioned above all (except text and images) GUI driver primitives must be broken into triangles. Then all the image primitive will combine into a single image. The single combined image with the calculated triangles (and some metadata to direct fragment shader to either use a specified color or a texture) will be uploaded to the GPU and drawn in a single draw.

It seems reasonable.

Sergey, I think I have acquired enough of implementation changes for the line drawing support. I'd like to start to integrate these in the GUI driver itself. My idea is to create new set of files to complement the existing GUI driver JS code. The idea is to keep the existing implementation, at least for the time until the new implementation is stable enough, and effectively have two JS GUI stacks. Do you see any issues with that?

Please review WebGL tests results and compare them with the current FWD 2D (committed rev 11732 (3281c)). On my computer WebGL works 10 times faster. It has only integrated ATI Radeon HD 3000 GPU supporting only 1GB shared memory with AMD FX-8320. Also the tested webgl code doesn't use batch operations so it draws each line or rectangle separately with new call to the graphics core.

Hynek Cihlar wrote:

Sergey, I think I have acquired enough of implementation changes for the line drawing support. I'd like to start to integrate these in the GUI driver itself. My idea is to create new set of files to complement the existing GUI driver JS code. The idea is to keep the existing implementation, at least for the time until the new implementation is stable enough, and effectively have two JS GUI stacks. Do you see any issues with that?

I have no objections.

Hynek, I don't know at this moment how to implement clipping functionality with WebGL. Should it be implemented at the vertex shader stage or another one? Do you have ideas?

Translations can be done at the vertex shader as it is shown https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html

Sergey Ivanovskiy wrote:

Hynek, I don't know at this moment how to implement clipping functionality with WebGL. Should it be implemented at the vertex shader stage or another one? Do you have ideas?

I'm afraid we will have to implement this ourselves. But except for text, which will be handled by 2D canvas anyway, this shouldn't be a major issue.

I tested rev.11735 (3821c) webgl test page within Hotel_GUI on Mac mini (OS Catalina v 10.15.6, Safari 14.0). If 2d canvas tests use only one flash operation per a test, then the performance improvements of webgl is not so great. If flash operation is executed after each primitive drawing in one test, then webgl looks 10 times faster. It is important to take this effect into account.

Sergey I checked in WebGL canvas renderer (p2j.canvas_renderer_webgl.js) in 3821c revision 11737. I added TODOs in places that must be worked on. The TODOs mentioning my initials are assigned to me. Please put your initials (and check in) in TODOs you will work on, before you actually start implementing so that we can coordinate our efforts.

OK. It would be helpful to list demos or docs from public webgl resources which solved one of these tasks. For an example, translations can be done using demo from https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html and drawImage can be implemented using textures https://webglfundamentals.org/webgl/lessons/webgl-image-processing.html, drawLine https://webglfundamentals.org/webgl/lessons/webgl-resizing-the-canvas.html.
I don't know how to implement xor operations for images and to draw styled lines. May be translations can help to implement styled lines.

Sergey Ivanovskiy wrote:

I don't know how to implement xor operations for images and to draw styled lines.

For the composite operations, we could simply grab the image data from screen canvas, use it as input texture and calculate the resulting color in fragment shader. But perhaps there is a simple solution.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I don't know how to implement xor operations for images and to draw styled lines.

For the composite operations, we could simply grab the image data from screen canvas, use it as input texture and calculate the resulting color in fragment shader. But perhaps there is a simple solution.

Are you planning to add inline shaders to p2j.canvas_renderer_webgl.js? Are you planning to use Interface Blocks to communicate data between shaders? This is an example

#version 430 core
// "offset" is an input vertex attribute
layout (location = 0) in vec4 offset;
layout (location = 1) in vec4 color;
// Declare VS_OUT as an output interface block
out VS_OUT
{
vec4 color;
// Send color to the next stage
} vs_out;
void main(void)
{
const vec4 vertices[3] = vec4[3](vec4( 0.25, -0.25, 0.5, 1.0),
vec4(-0.25, -0.25, 0.5, 1.0),
vec4( 0.25, 0.25, 0.5, 1.0));
// Add "offset" to our hard-coded vertex position
gl_Position = vertices[gl_VertexID] + offset;
// Output a fixed value for vs_color
vs_out.color = color;
}

#version 430 core
// Declare VS_OUT as an input interface block
in VS_OUT
{
vec4 color;
// Send color to the next stage
} fs_in;
// Output to the framebuffer
out vec4 color;
void main(void)
{
// Simply assign the color we were given by the vertex shader
// to our output
color = fs_in.color;
}

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I don't know how to implement xor operations for images and to draw styled lines.

For the composite operations, we could simply grab the image data from screen canvas, use it as input texture and calculate the resulting color in fragment shader. But perhaps there is a simple solution.

Are you planning to add inline shaders to p2j.canvas_renderer_webgl.js?

Yes, but feel free to add them yourself. The shaders will be very simple anyway, at least for the start.

Are you planning to use Interface Blocks to communicate data between shaders? This is an example

I'm not sure yet. Would this bring any benefits if we end up with a single vertex and single fragment shader?

Please, add rectangles structure that we can start simple drawing primitives: drawLine and drawRect.

The structure for now should be simple. Only to capture triangle vertices and colors. Every line and every rectangle will be composed by two triangles. So 6 vertices per single line and per single rectangle. Perhaps the structure could be a simple array of either primitive type to denote color or array of 12 numbers. Obviously there will be more metadata needed as we go, but let's start simple.

Hynek Cihlar wrote:

Are you planning to use Interface Blocks to communicate data between shaders? This is an example

I'm not sure yet. Would this bring any benefits if we end up with a single vertex and single fragment shader?

I would like that the vertex shader converts the original vertex coordinates to its clipping space like in this shader and for dotted style lines

   out vec4 vs_color;
   attribute vec2 a_position;
   uniform vec2 u_resolution;
   void main() {
     // convert the rectangle from pixels to 0.0 to 1.0
     vec2 zeroToOne = a_position / u_resolution;
     // convert from 0->1 to 0->2
     vec2 zeroToTwo = zeroToOne * 2.0;
     // convert from 0->2 to -1->+1 (clipspace)
     vec2 clipSpace = zeroToTwo - 1.0;
     gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
     //TODO calculate vs_color for dotted style
   };

   // fragment shaders don't have a default precision so we need
   // to pick one. mediump is a good default
   precision mediump float;
   uniform vec4 u_color;
   in vec4 vs_color;

   void main() {
     gl_FragColor = vs_color != 0 ? vs_color : u_color;
   };

Can this operation substitute a clipping rectangle? https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/scissor

Another way to use the vertex shader is to change the point size for the styled line via gl_PointSize. Please look at this example https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/By_example/Hello_vertex_attributes It seems it is analog of glPointSize that is absent for WebGL context interface.

Sergey Ivanovskiy wrote:

I would like that the vertex shader converts the original vertex coordinates to its clipping space like in this shader

Is there any advantage of converting the coordinates in a shader?

and for dotted style lines

Vertex shader outputs the output variable (or varying) for each vertex, but when the value gets in the fragment shader, it gets interpolated for each pixel of the generated primitive (a triangle for example). I'm not sure how single color variable could be used to calculate the dotted line. I was more thinking about a texture that would hold the desired pattern.

Sergey Ivanovskiy wrote:

Can this operation substitute a clipping rectangle? https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/scissor

Yes! This looks like it.

Sergey Ivanovskiy wrote:

Another way to use the vertex shader is to change the point size for the styled line via gl_PointSize. Please look at this example https://developer.mozilla.org/en-US/docs/Web/API/WebGL_API/By_example/Hello_vertex_attributes It seems it is analog of glPointSize that is absent for WebGL context interface.

AFAIK gl_PointSize can be assigned only when you draw POINTS, not TRIANGLES.

Yes, sadly. This WebGL gl.enable(cap); doesn't support GL_PROGRAM_POINT_SIZE that is defined with OpenGL. I couldn't assign a new value to gl_PointSize, since the vertex shader wasn't compiled with that changes.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I would like that the vertex shader converts the original vertex coordinates to its clipping space like in this shader

Is there any advantage of converting the coordinates in a shader?

It needs to compare which of these two ways proves the best performance results. The shader code should be executed by the graphics processor.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I would like that the vertex shader converts the original vertex coordinates to its clipping space like in this shader

Is there any advantage of converting the coordinates in a shader?

It needs to compare which of these two ways proves the best performance results. The shader code should be executed by the graphics processor.

I added your lines test under WebGL Tests>Vertex Shader Test and compared these two shaders. They show slight performance differences. On my computer the large number of lines are drawn slightly faster by WebGL if it uses the vertex shader that transforms coordinates. I committed these shaders from p2j.perf_tests.js into p2j.canvas_renderer_webgl.js. (rev. 11741(3821a)).

I would like to add translations to the vertex shader according to https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html.

Sergey Ivanovskiy wrote:

I would like to add translations to the vertex shader according to https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html.

Translation will have to be part of the input vertex stream in order to minimize the number of drawArrays calls. If you pass it in the shader as an uniform, then every time its value changes, you will have to reinitialize it and restart the shaders.

I checked in WebGL version of a strokes manager in 3821c, mostly empty at this time.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I would like to add translations to the vertex shader according to https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html.

Translation will have to be part of the input vertex stream in order to minimize the number of drawArrays calls. If you pass it in the shader as an uniform, then every time its value changes, you will have to reinitialize it and restart the shaders.

OK, I don't know another way to implement translations. Uniform variable is just visible to all shaders in the OpenGL pipeline.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I would like to add translations to the vertex shader according to https://webglfundamentals.org/webgl/lessons/webgl-2d-translation.html.

Translation will have to be part of the input vertex stream in order to minimize the number of drawArrays calls. If you pass it in the shader as an uniform, then every time its value changes, you will have to reinitialize it and restart the shaders.

OK, I don't know another way to implement translations.

We can pass the translation value using the same technique as the triangle vertices are passed in. With a buffer attribute.

Uniform variable is just visible to all shaders in the OpenGL pipeline.

The problem with an uniform, unless I interpreted the docs incorrectly, is that you can't change its value unless you also restart the shaders.

Yes, uniform variables are used for batch of drawing operations. In this case the vertex color should be passed in as an attribute value. It needs that this.rectangles array contains these vertices objects

this.rectangles = [vertex={x:x,
                           y:y,
                           color:color
                          }];

Sergey Ivanovskiy wrote:

Yes, uniform variables are used for batch of drawing operations. In this case the vertex color should be passed in as an attribute value. It needs that this.rectangles array contains these vertices objects
[...]
because we need to feed vertex shader with a vertex and its color. Correct?

Correct.

It seems that we need to pack data in the buffer similarly to https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/vertexAttribPointer

I encountered difficulties with shader code because GLSL/WebGl doesn't support in/out specifiers and uses varying https://webglfundamentals.org/webgl/lessons/webgl-shaders-and-glsl.html

Hynek, please look at the rev 11746 (3821c) where a vertex and its color are packed into one buffer

         var x1 = x;
         var x2 = x + w;
         var y1 = y;
         var y2 = y + h;
         var vertices = [
                         { x: x1, y: y1, color: c},
                         { x: x2, y: y1, color: c},
                         { x: x1, y: y2, color: c},
                         { x: x1, y: y2, color: c},
                         { x: x2, y: y1, color: c},
                         { x: x2, y: y2, color: c}
                         ];
         var buffer = new ArrayBuffer(12 * 6);
         //Fill array buffer
         var dv = new DataView(buffer);
         for (let i = 0; i < vertices.length; i++) {
            dv.setFloat32(12 * i, vertices[i].x, true);
            dv.setFloat32(12 * i + 4, vertices[i].y, true);
            dv.setUint8(12 * i + 8, vertices[i].color[0]);
            dv.setUint8(12 * i + 9, vertices[i].color[1]);
            dv.setUint8(12 * i + 10, vertices[i].color[2]);
            dv.setUint8(12 * i + 11, 255);
          }
         me.gl.bufferData(me.gl.ARRAY_BUFFER, dv, me.gl.STATIC_DRAW);

Sergey Ivanovskiy wrote:

If it is OK, then I will apply it to p2j.canvas_renderer_webgl.js.

Yes, this has been my idea. We will accumulate the vertices data (with color) from the renderer functions (drawLine, drawRectangle, drawDot, etc.) into a single array (until an operation comes which requires rasterized image or the array reaches certain size limit), pass it in GL buffer and execute the shaders.

Please review these changes revision 11749 (3821c).

The committed rev 11751 (3821c) cleaned the previous changes.

Sergey Ivanovskiy wrote:

The committed rev 11751 (3821c) cleaned the previous changes.

11749 up to 11751 look good.

Sergey, in order to have the working prototype one of us should focus on integrating what we already have and make it working. If you agree I can take that. Meanwhile you would implement the advanced line drawing features - sloped, connecting and styles lines.

Hynek Cihlar wrote:

Sergey, in order to have the working prototype one of us should focus on integrating what we already have and make it working. If you agree I can take that. Meanwhile you would implement the advanced line drawing features - sloped, connecting and styles lines.

OK. Please do it. Now we have different formats for rectangles and the returned vertex and color array from strokeLine.

@@ -533,7 +533,7 @@
     */
    function DefaultPathRenderer(canvasRenderer, basicStroke, strokeColor)
    {
-      var color = strokeColor;
+      var color = strokeColor.concat([255]); // add alpha channel
       var width = basicStroke.getWidth();

       /**
@@ -561,11 +561,11 @@
       {
          if (y1 == y2)
          {
-            return [x1, y1, x1, y1 + width, x2, y2, x2, y2, x2, y2 + width, x1, y1 + width, color];
+            return {x : x1, y : y1, width : x2 - x1 + 1, height : width, color: color};
          }
          else if (x1 == x2)
          {
-            return [x1, y1, x1 + width, y1, x2, y2, x2, y2, x2 + width, y2, x1 + width, y1, color];
+            return {x: x1, y : y1, width: width, height: y2 - y1 + 1, color: color};
          }
          else
          {

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, in order to have the working prototype one of us should focus on integrating what we already have and make it working. If you agree I can take that. Meanwhile you would implement the advanced line drawing features - sloped, connecting and styles lines.

OK. Please do it. Now we have different formats for rectangles and the returned vertex and color array from strokeLine.

Please check in your format changes.

I am working on the dotted and styled lines if you are not working on this part.

OK.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, in order to have the working prototype one of us should focus on integrating what we already have and make it working. If you agree I can take that. Meanwhile you would implement the advanced line drawing features - sloped, connecting and styles lines.

OK. Please do it. Now we have different formats for rectangles and the returned vertex and color array from strokeLine.

Please check in your format changes.

The reason I chose a simple array over an object was that there may be many of such entries. I assume array requires less storage.

Sergey, WebGLCanvasRenderer.rectangles must hold triangles, but currently it is assumed it contains rectangles in flashGL. I will change this. Also, instead of storing objects in rectangles I will change the format to arrays.

OK. Are you planning to commit these changes? Did you test drawing images with textures?

  const texture = gl.createTexture();
  gl.bindTexture(gl.TEXTURE_2D, texture);
  const level = 0;
  const internalFormat = gl.RGBA;
  const width = 1;
  const height = 1;
  const border = 0;
  const srcFormat = gl.RGBA;
  const srcType = gl.UNSIGNED_BYTE;
  const image = new Uint8Array([0, 0, 255, 255, ...]);  
  gl.texImage2D(gl.TEXTURE_2D, level, internalFormat,
                width, height, border, srcFormat, srcType,
                image);

For me using rectangles looks reasonable. Any case this code should be consistent with the rectangles format

WebGLCanvasRenderer.prototype.drawLineSegment = function(x1, y1, x2, y2, strokeRenderer)
{
   x1 = x1 + this.origin.x;
   x2 = x2 + this.origin.x;

   y1 = y1 + this.origin.y;
   y2 = y2 + this.origin.y;

   // intersect with clipping region
   // and normalize this region, thus (d[0],d[1]) defines the left top cone point and (d[2],d[3])
   // defines the right bottom cone point of the clipping region
   var d = this.cutLine(x1, y1, x2, y2);
   if (d === null)
   {
      return;
   }

   for (var i = 0; i < d.length; i++)
   {
      x1 = d[i][0];
      y1 = d[i][1];
      x2 = d[i][2];
      y2 = d[i][3];

      var glData = strokeRenderer.strokeLine(x1, y1, x2, y2);
      this.rectangles = this.rectangles.concat(glData);
   }
};

Sergey Ivanovskiy wrote:

I am confused now what you are planning to change and waiting for your commit.

See 3821c revision 11757. Btw. the rectangles fields I created should really have been triangles, sorry for the confusion.

It seems that with your changes this code

      this.strokeLine = function(x1, y1, x2, y2)
      {
         if (y1 == y2)
         {
            return [x1, y1, x1, y1 + width, x2, y2, x2, y2, x2, y2 + width, x1, y1 + width, color];
         }
         else if (x1 == x2)
         {
            return [x1, y1, x1 + width, y1, x2, y2, x2, y2, x2 + width, y2, x1 + width, y1, color];
         }
         else
         {
            // TODO: sloped line
         }
      }

   for (var i = 0; i < d.length; i++)
   {
      x1 = d[i][0];
      y1 = d[i][1];
      x2 = d[i][2];
      y2 = d[i][3];

      var glData = strokeRenderer.strokeLine(x1, y1, x2, y2);
      this.triangles = this.triangles.concat(glData);
   }

   for (var i = 0; i < numTriangles; i++)
   {
      var t = this.triangles[i];
      var x1 = t[0];
      var y1 = t[1];
      var x2 = t[2];
      var y2 = t[3];
      var x3 = t[4];
      var y3 = t[5];
      var c  = t[6];

Sergey Ivanovskiy wrote:

It seems that with your changes this code
returns 13 elements but these code expect that it is a triangle or array of triangles (of 7 elements)
doesn't work with

Yes, this will be resolved with my next check in.

Another question is about this.screenTexture? Did you test somehow this code with the webgl texture? The idea is reasonable. If I understand correctly, then dotted lines can be placed on the texture but what is the performance cost of this operation? A dotted line can be drawn as number of small lines or filled rectangles?

Sergey Ivanovskiy wrote:

Another question is about this.screenTexture? Did you test somehow this code with the webgl texture? The idea is reasonable. If I understand correctly, then dotted lines can be placed on the texture

Yes.

but what is the performance cost of this operation?

I think it is the matter of getting the texture in screen buffer.

A dotted line can be drawn as number of small lines or filled rectangles?

Perhaps we could cash all the different patterns (as a raster) and just replicate it in the texture.

Sergey, Hynek,

Can you please tell if the new WebGL approach is active now? Or to use it I need to configure something in directory.xml? Or it already works as default mode(or partially works)? I just need to know what is the picture renderer I'm looking at when starting Web client and if I can see the difference comparing to the previous renderer.

Eugenie, this renderer defined by p2j.canvas_renderer_webgl.js is not used by the js client. It is not ready.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Another question is about this.screenTexture? Did you test somehow this code with the webgl texture? The idea is reasonable. If I understand correctly, then dotted lines can be placed on the texture

Yes.

OK.

but what is the performance cost of this operation?

I think it is the matter of getting the texture in screen buffer.

Don't understand. Do you mean the time to fill rgba data for the texture?

A dotted line can be drawn as number of small lines or filled rectangles?

Perhaps we could cash all the different patterns (as a raster) and just replicate it in the texture.

It is not clear for me. I checked the fragment shader that draws chess like patterns with help of gl_FragCoord and mod built-in function but it is not clear that it is useable for creating dotted lines. Now I see only the available way to build the array of triangles to draw dotted lines.

You planned to integrate this renderer into the js client. For webgl testing I am using a standalone test page with p2j.perf_tests.js. But I think we need to integrate this renderer defined by p2j.canvas_renderer_webgl.js in order to prove the conception and to investigate unseen issues. This task should be done before the other tasks of implementing drawing primitives. The simple lines are implemented now so we need to evaluate this way ASAP.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

but what is the performance cost of this operation?

I think it is the matter of getting the texture in screen buffer.

Don't understand. Do you mean the time to fill rgba data for the texture?

A dotted line can be drawn as number of small lines or filled rectangles?

I mean the time to get the texture in GPU. But since this should be performed only one time per draw cycle, the performance hit of this should be negligible.

Perhaps we could cash all the different patterns (as a raster) and just replicate it in the texture.

It is not clear for me. I checked the fragment shader that draws chess like patterns with help of gl_FragCoord and mod built-in function but it is not clear that it is useable for creating dotted lines. Now I see only the available way to build the array of triangles to draw dotted lines.

I meant to fill a pattern in the texture. The pattern is just an array of color values that will be replicated in the target texture.

Sergey Ivanovskiy wrote:

You planned to integrate this renderer into the js client. For webgl testing I am using a standalone test page with p2j.perf_tests.js. But I think we need to integrate this renderer defined by p2j.canvas_renderer_webgl.js in order to prove the conception and to investigate unseen issues. This task should be done before the other tasks of implementing drawing primitives. The simple lines are implemented now so we need to evaluate this way ASAP.

Yes, this is what I'm working on currently. The line drawing algorithms can be implemented separately, there are no hard dependencies on renderer.

Please review DotsPathRenderer.strokeLine and DefaultPathRenderer.strokeLine in the committed rev. 11758 (3821c).

Sergey Ivanovskiy wrote:

Eugenie, this renderer defined by p2j.canvas_renderer_webgl.js is not used by the js client. It is not ready.

OK, understood. Thanks for clarification.

Sergey I need your help with implementing clipping. This is a real time priority as it blocks the prototype.

In the latest 3821c you fill find function clipTriangles. It is functionally equivalent to clipRegion. The function takes an array of triangles and according to current clipped region it returns new array of triangles. The part that needs to be implemented is marked with "TODO".

Please let me know when you accept the challenge.

Yes, I am working on this issue. Do we need to clip triangles in our main js thread if we are planning to use https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/scissor?

It seems that we need to clip rectangles and then to build triangles from them. I suppose that line drawing primitives returns triangulation of rectangles. Correct?

Sergey Ivanovskiy wrote:

It seems that we need to clip rectangles and then to build triangles from them.

Yes. The triangles must be clipped before their vertices are passed to the shaders.

I suppose that line drawing primitives returns triangulation of rectangles. Correct ?

Correct.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

It seems that we need to clip rectangles and then to build triangles from them.

Yes. The triangles must be clipped before their vertices are passed to the shaders.

Sergey, can you first implement (and check in) the simplest case for right triangles when their legs are perpendicular to either x or y axis?

I will commit sooner, sorry for this delay.

Sergey Ivanovskiy wrote:

I will commit sooner, sorry for this delay.

OK, no problem.

Please review this version committed revision 11763 (3821c). I am using cutLine, but it can be used an analog cutTriangle in order to reduce the number of iterations over clipping regions.

Sergey Ivanovskiy wrote:

Please review this version committed revision 11763 (3821c). I am using cutLine, but it can be used an analog cutTriangle in order to reduce the number of iterations over clipping regions.

cutLine returns array of arrays, the single line can be cut in multiple lines. But in clipTriangles just an array of coordinates is expected.

If you haven't please test the implementation and make sure it works.

Thanks, it has a gap that I didn't observe. Working on the fix.

Please look at this fixed version, the committed revision 11766 (3821c).

Sergey Ivanovskiy wrote:

Please look at this fixed version, the committed revision 11766 (3821c).

Sergey, it still has some issues. When I enable it, I get artifacts. I will review the implementation tomorrow. Thanks!

The WebGL prototype is nearly ready. See below a screenshot of the demo_widget.p test case rendered in WebGL.

Very cool!

I'd like to change this to single onscreen canvas approach with an offscreen canvas used for rendering the text and copying it in the onscreen one.

Understood. Do what you think is best.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Please look at this fixed version, the committed revision 11766 (3821c).

Sergey, it still has some issues. When I enable it, I get artifacts. I will review the implementation tomorrow. Thanks!

I think that we need to check the Precision test. This test should give the same results as for the pixels version of the 2d canvas renderer. The same artifacts can be seen with the 2d canvas mode without pixel manipulations. The Precision test for this version doesn't give the same results as for the pixels version of the 2d canvas renderer.

Sadly, this modern multi line string impacts the Eclipse outline view so it doesn't show all renderer's methods. Does Idea parse well this js module?

var vertexShaderSource = `
      attribute vec2 a_position; 
      attribute vec4 a_color;       
      uniform vec2 u_resolution; 
      varying vec4 v_color;
      varying vec2 v_texCoords;

      void main() 
      { 
        // convert the rectangle from pixels to 0.0 to 1.0 
        vec2 zeroToOne = a_position / u_resolution; 
        // convert from 0->1 to 0->2 
        vec2 zeroToTwo = zeroToOne * 2.0; 
        // convert from 0->2 to -1->+1 (clipspace) 
        vec2 clipSpace = zeroToTwo - 1.0; 
        gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1); 
        v_color = a_color;        
        v_texCoords = zeroToOne;
      }
      `;

Sergey Ivanovskiy wrote:

Sadly, this modern multi line string impacts the Eclipse outline view so it doesn't show all renderer's methods. Does Idea parse well this js module?
[...]

Idea parses this OK. If you move the shader variables at the very end of the file, will it help?

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Please look at this fixed version, the committed revision 11766 (3821c).

Sergey, it still has some issues. When I enable it, I get artifacts. I will review the implementation tomorrow. Thanks!

I think that we need to check the Precision test. This test should give the same results as for the pixels version of the 2d canvas renderer. The same artifacts can be seen with the 2d canvas mode without pixel manipulations. The Precision test for this version doesn't give the same results as for the pixels version of the 2d canvas renderer.

Clipping a triangle may result in multiple new triangles produced. Is this handled in clipTriangles?

Yes, the algorithm is quite simple. Each triangle is intersected with disjoint clipping rectangles and thus it can produce triangles for each of these intersections. Each intersection is defined by its vertexes and then each this intersection is split into triangles with common vertex.

Hynek, could you add an option to the directory to enable webgl rendering?

I found an issue with cutTriangleLine that is it can change the order of points. It needs to save the order of vertexes because it is important for clipTriangles. If we have a triangle ABC, then intersections with the given clipping region should save this order from A->B->C. I will commit it now.

Sergey Ivanovskiy wrote:

I found an issue with cutTriangleLine that is it can change the order of points. It needs to save the order of vertexes because it is important for clipTriangles. If we have a triangle ABC, then intersections with the given clipping region should save this order from A->B->C. I will commit it now.

Fixed in the committed revision 11769 (3821c).

I have a question why in this code we try to clip drawings two times. It should be enough to clip one time: cutLine or pushTriangles

WebGLCanvasRenderer.prototype.drawLineSegment = function(x1, y1, x2, y2, strokeRenderer)
{
   x1 = x1 + this.origin.x;
   x2 = x2 + this.origin.x;

   y1 = y1 + this.origin.y;
   y2 = y2 + this.origin.y;

   // intersect with clipping region
   // and normalize this region, thus (d[0],d[1]) defines the left top cone point and (d[2],d[3])
   // defines the right bottom cone point of the clipping region
   var d = this.cutLine(x1, y1, x2, y2);
   if (d === null)
   {
      return;
   }

   for (var i = 0; i < d.length; i++)
   {
      x1 = d[i][0];
      y1 = d[i][1];
      x2 = d[i][2];
      y2 = d[i][3];

      this.pushTriangles(strokeRenderer.strokeLine(x1, y1, x2, y2))
   }
};

Sergey Ivanovskiy wrote:

Hynek, could you add an option to the directory to enable webgl rendering?

Will do. If you need to enable WebGL rendering meanwhile, change this.canvasRenderer = new CanvasRenderer to this.canvasRenderer = new WebGLCanvasRenderer in p2j.screen.js.

I am not sure that clipTriangles is important for the general cases but the current implementation has gaps. One of the issue is that if a line is not vertical or horizontal, then cutLine returns incorrect line segment. The other issue is the cases when clipping rectangle has its vertices inside the target triangle. I will fix it tomorrow and will add a simple test.

Committed revision 11774-11776 (3821c) fixed clipTriangles() and added clipTrianglesTest.

Committed revision 11777-11778 (3821c) fixed cutLine in the case of slopped line given by (x1, y1) and (x2, y2).

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

but what is the performance cost of this operation?

I think it is the matter of getting the texture in screen buffer.

Don't understand. Do you mean the time to fill rgba data for the texture?

A dotted line can be drawn as number of small lines or filled rectangles?

I mean the time to get the texture in GPU. But since this should be performed only one time per draw cycle, the performance hit of this should be negligible.

Perhaps we could cash all the different patterns (as a raster) and just replicate it in the texture.

It is not clear for me. I checked the fragment shader that draws chess like patterns with help of gl_FragCoord and mod built-in function but it is not clear that it is useable for creating dotted lines. Now I see only the available way to build the array of triangles to draw dotted lines.

I meant to fill a pattern in the texture. The pattern is just an array of color values that will be replicated in the target texture.

It is not clear for me now. Even we can put drawing on the texture, how we can clean the part of the texture after another drawing. It looks like all drawings are present on the texture like the past and the present and the future before one's eyes.

Sergey Ivanovskiy wrote:

It is not clear for me now. Even we can put drawing on the texture, how we can clean the part of the texture after another drawing. It looks like all drawings are present on the texture like the past and the present and the future before one's eyes.

The texture is cleaned after every draw cycle (a call to Window.draw), see WebGLCanvasRenderer.clear).

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

It is not clear for me now. Even we can put drawing on the texture, how we can clean the part of the texture after another drawing. It looks like all drawings are present on the texture like the past and the present and the future before one's eyes.

The texture is cleaned after every draw cycle (a call to Window.draw), see WebGLCanvasRenderer.clear).

It seems that you have different code from the code base of 3821c

WebGLCanvasRenderer.prototype.clear = function()
{
   this.offscreenCtx.save();
   this.offscreenCtx.translate(-this.origin.x, -this.origin.y);
   this.offscreenCtx.clearRect(0, 0, this.offscreenCanvas.width, this.offscreenCanvas.height);
   this.offscreenCtx.translate(this.origin.x, this.origin.y);
   this.offscreenCtx.restore();

   this.triangles = [];
   this.screenTexture = new Uint8Array(this.webglCanvas.width * this.webglCanvas.height * 4);
}

      // fill the texture
      this.gl.texImage2D(
         this.gl.TEXTURE_2D,
         0,
         this.gl.RGBA,
         this.webglCanvas.width,
         this.webglCanvas.height,
         0,
         this.gl.RGBA,
         this.gl.UNSIGNED_BYTE,
         this.screenTexture
      );
      this.gl.texParameteri(this.gl.TEXTURE_2D, this.gl.TEXTURE_WRAP_S, this.gl.CLAMP_TO_EDGE);
      this.gl.texParameteri(this.gl.TEXTURE_2D, this.gl.TEXTURE_WRAP_T, this.gl.CLAMP_TO_EDGE);
      this.gl.texParameteri(this.gl.TEXTURE_2D, this.gl.TEXTURE_MIN_FILTER, this.gl.NEAREST);
      this.gl.texParameteri(this.gl.TEXTURE_2D, this.gl.TEXTURE_MAG_FILTER, this.gl.NEAREST);

      this.gl.bufferData(this.gl.ARRAY_BUFFER, dv, this.gl.STATIC_DRAW);

Don't mind this note because it occurs due to disablePixelManipulation

<node class="boolean" name="disablePixelManipulation">
  <node-attribute name="value" value="TRUE"/>
</node>

I observed strange effect that with webgl renderer p2j.perf-tests.js Java 2D Test works differently even it uses CanvasRenderer of p2j.canvas_renderer.js. It is a mess of code now.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

It is not clear for me now. Even we can put drawing on the texture, how we can clean the part of the texture after another drawing. It looks like all drawings are present on the texture like the past and the present and the future before one's eyes.

The texture is cleaned after every draw cycle (a call to Window.draw), see WebGLCanvasRenderer.clear).

It seems that you have different code from the code base of 3821c

I'm not sure what you mean. The code block you attached shows the texture cleanup, the line this.screenTexture = new Uint8Array(this.webglCanvas.width * this.webglCanvas.height * 4);.

Should we put this texture on the webgl canvas?

Do you mean screenTexture? This is already being put on the webgl canvas, see flashGL.

Sergey Ivanovskiy wrote:

Yes, screenTexture. I enabled webgl and recorded this video hotel-webgl-enabled.mkv with hotel_gui. Labels and text are displayed on the transparent 2d canvas, correct? You are working on displaying text on webgl canvas, correct?

I don't see this. Make sure you don't have any local changes, you have rebuilt and deployed FWD jars with ant deploy.prepare.

What is the value for graphicsCached must be? This question is due to p2j.screen.js has this code at the end of the drawing circle

if (this.clips == 0 && this.trans == 0 && graphicsCached)
{
         // save the offscreen canvas
         var img;

         if (this.offscreenCanvas instanceof HTMLCanvasElement)
         {
            img = new Image();
            img.src = this.offscreenCanvas.toDataURL("image/png");
         }
         else
         {
            // OffscreenCanvas : get ImageBitmap
            img = this.offscreenCanvas.transferToImageBitmap();
         }

         this.cacheDraw(md5, img);

         // clear the offscreen canvas for next drawing
         this.canvasRenderer.clear();
}

Sergey Ivanovskiy wrote:

What is the value for graphicsCached must be? This question is due to p2j.screen.js has this code at the end of the drawing circle

This is part of the Web UI caching facility. Note that this is not yet supported in WebGL rednerer and must be turned off.

I have no manual edits and still can't reproduce the correct screen. All drawings are still present at the screen after clicking on new tab of Hotel GUI.

Hynek, could you share your source files that I can reproduce your results too when text are cleared properly?

This diff has

=== modified file 'src/com/goldencode/p2j/ui/client/gui/driver/web/GuiWebSocket.java'
--- old/src/com/goldencode/p2j/ui/client/gui/driver/web/GuiWebSocket.java    2020-10-26 23:51:46 +0000
+++ new/src/com/goldencode/p2j/ui/client/gui/driver/web/GuiWebSocket.java    2020-11-05 11:40:53 +0000
@@ -324,7 +324,7 @@

       // window ID = 0 means the global cache
       this.perWidgetCache.addCacheExpiryListener(new DrawCacheExpiryListener(0));
-      this.graphicsCached = graphicsCached;
+      this.graphicsCached = false;//graphicsCached;
    }

Hynek, I found that the strokes manager in your test uses the 2d canvas pixels strokes manager so I tried to test p2j.strokes.webgl.js and implemented DotsPathRenderer following the same way as for the 2d offscreen canvas to update screenTexture with help of WebGLCanvasRenderer.prototype.drawImage = function(x, y, width, height, imgData) that you implemented for p2j.canvas_renderer_webgl.js. Also implemented the same code for

WebGLCanvasRenderer.prototype.strokeRect = function(x, y, width, height, color, fill)
{
   // TODO
   if (fill)
   {
      var x1 = x + this.origin.x;
      var x2 = x + width + this.origin.x;
      var y1 = y + this.origin.y;
      var y2 = y + height + this.origin.y;

      var clipped = this.clipRegion(x1, y1, x2, y2);

      if (clipped)
      {
         for (var i = 0; i < clipped.length; i++)
         {
            var clip = clipped[i];
            x1 = clip.left;
            y1 = clip.top;
            x2 = clip.right;
            y2 = clip.bottom;
            this.pushTriangles([[x1, y1, x2, y1, x1, y2, color[0], color[1], color[2], 255], [x1, y2, x2, y1, x2, y2, color[0], color[1], color[2], 255]]);
         }
      }
   }
   else
   {
      var renderer = this.strokesManager.getStrokePathRenderer(this, this.strokeStyleId,
            this.strokeWidth, color);

      renderer.beginStroke();
      this.drawLineSegment(x, y, x + width, y, renderer);
      this.drawLineSegment(x + width, y + 1, x + width, y + height, renderer);
      this.drawLineSegment(x + width - 1, y + height, x + 1, y + height, renderer);
      this.drawLineSegment(x, y + height, x, y + 1, renderer);// close the path
   }
};

Sergey Ivanovskiy wrote:

Hynek, I found that the strokes manager in your test uses the 2d canvas pixels strokes manager

Do you mean the attached index.html?

The result was not good to proceed this way. Do you have ideas?

Can you be more specific, what was not good?

I found this article https://mattdesl.svbtle.com/drawing-lines-is-hard that describes available WebGL methods to support line styles. It has some good idea to add new vertex attribute distance alone path in order to draw dashes.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I found that the strokes manager in your test uses the 2d canvas pixels strokes manager

Do you mean the attached index.html?

I meant your webgl.diff doesn't use p2j.strokes.webgl.js. So the test used p2j.strokes.js

The result was not good to proceed this way. Do you have ideas?

Can you be more specific, what was not good?

I meant that the performance issue if we would continue to draw lines on the texture. 2d canvas shows the better performance.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I found that the strokes manager in your test uses the 2d canvas pixels strokes manager

Do you mean the attached index.html?

I meant your webgl.diff doesn't use p2j.strokes.webgl.js. So the test used p2j.strokes.js

I'm not sure what you mean. webgl.diff doesn't contain changes related to stroke managers, except the js file include. If you look in WebGLCanvasRenderer you will find two strokes managers there. One is the old fashioned 2D and the new one Web GL, see the class fields strokesManager and glStrokesManager respectively. glStrokesManager is used during solid pixel-wide line drawing and rect primitives. I left the 2D strokes manager in as it may be still used in some cases, like the text caret.

The result was not good to proceed this way. Do you have ideas?

Can you be more specific, what was not good?

I meant that the performance issue if we would continue to draw lines on the texture. 2d canvas shows the better performance.

Do you have any tests that show the performance characteristics for both the texture approach and 2D canvas approach?

OK. I see glStrokesManager now. I just tested Hotel GUI and didn't do special performance tests.

Sergey Ivanovskiy wrote:

OK. I see glStrokesManager now. I just tested Hotel GUI and didn't do special performance tests.

If you think the texture based drawing is slower it would be worth to include it in your test suite and see how it performs with different work loads and systems.

Interesting information https://en.wikipedia.org/wiki/ANGLE_(software) that Firefox uses ANGLE as the default WebGL backend on Windows. Chrome uses ANGLE not only for WebGL, but also for its implementation of the 2D HTML5 canvas and for the graphics layer of the Google Native Client (which is OpenGL ES 2.0 compatible).

https://stackoverflow.com/questions/51384738/draw-a-ellipse-curve-in-fragment-shader (Example https://jsfiddle.net/7rh2eog1/5/)

Sergey Ivanovskiy wrote:

Please apply this attached diff and test Hotel GUI. I found that Hotel GUI Calendar popup didn't clear outlines of the selected day correctly.

Please commit in the repo.

Committed rev 11794 (3821c), sorry for this delayed commit.

Probably we need to follow this way https://cesium.com/blog/2013/04/22/robust-polyline-rendering-with-webgl/

How much more work is needed to have this be a functional solution which can be provided to a customer for testing?

Greg Shah wrote:

How much more work is needed to have this be a functional solution which can be provided to a customer for testing?

• Text rendering (2 MDs)
• Caching (0.5 MD)
• JS driver integration - event handling, configuration (like allowing to select the stack), etc. (2 MD)
• Text caret (1 MD)

I'll let comment Sergey on his assignments.

• Planning to evaluate ideas from https://github.com/spite/THREE.MeshLine/blob/master/src/THREE.MeshLine.js
  in order to move line styles into vertex and fragment shaders (2MD)
• Drawing primitives: ellipse, rounded rectangles (1MD )
• Tests for line styles, drawing primitives and bug fixing.(1MD)

Hynek, I tried to add two another buffers for these two new attributes, relative distance from 0 to 1 alone a line and its end point coordinates. If draw vertex arrays with gl.TRIANGLES, then all rectangular area between its left and right vertexes is filled. If draw vertex arrays with gl.TRIANGLES_STRIP, then the line is drawn by points (100 points in this example)

      var vertices = [];

      for (var i=0; i < 100; i++)
      {
         vertices.push({ x: x1, y: y1, color: c});
         vertices.push({ x: x1, y: y1, color: c});
         vertices.push({ x: x1, y: y1, color: c});
      }
      var buffer = new ArrayBuffer(12 * vertices.length);
      //Fill array buffer
      var dv = new DataView(buffer);
      for (let i = 0; i < vertices.length; i++) {
         dv.setFloat32(12 * i, vertices[i].x, true);
         dv.setFloat32(12 * i + 4, vertices[i].y, true);
         dv.setUint8(12 * i + 8, vertices[i].color[0]);
         dv.setUint8(12 * i + 9, vertices[i].color[1]);
         dv.setUint8(12 * i + 10, vertices[i].color[2]);
         dv.setUint8(12 * i + 11, 255);
       }
      me.gl.bindBuffer(me.gl.ARRAY_BUFFER, me.positionBuffer);
      me.gl.bufferData(me.gl.ARRAY_BUFFER, dv, me.gl.STATIC_DRAW);
      me.gl.enableVertexAttribArray(me.positionAttributeLocation);

      // Tell the attribute how to get data out of positionBuffer (ARRAY_BUFFER)
      var size = 2;          // 2 components per iteration
      var type = me.gl.FLOAT;   // the data is 32bit floats
      var normalize = false; // don't normalize the data
      var stride = 0;        // 0 = move forward size * sizeof(type) each iteration to get the next position
      var offset = 0;        // start at the beginning of the buffer
      me.gl.vertexAttribPointer(
         me.positionAttributeLocation, size, type, normalize, stride, offset);

      var endPos = [];
      var dist = []
      for (var i=0; i < 100; i++)
      {
         endPos.push(x2);
         endPos.push(y2);
         endPos.push(x2);
         endPos.push(y2);
         endPos.push(x2);
         endPos.push(y2);
         dist.push((i + 1)/100);
         dist.push((i + 1)/100);
         dist.push((i + 1)/100);
      }
      me.gl.bindBuffer(me.gl.ARRAY_BUFFER, me.endPosBuf);
      me.gl.bufferData(me.gl.ARRAY_BUFFER, new Float32Array(endPos), me.gl.STATIC_DRAW);
      me.gl.enableVertexAttribArray(me.endPosAttrLoc);
      var size = 2;          // 2 components per iteration
      var type = me.gl.FLOAT;   // the data is 32bit floats
      var normalize = false; // don't normalize the data
      var stride = 0;        // 0 = move forward size * sizeof(type) each iteration to get the next position
      var offset = 0;        // start at the beginning of the buffer
      me.gl.vertexAttribPointer(
         me.endPosAttrLoc, size, type, normalize, stride, offset);

      me.gl.enableVertexAttribArray(me.distAttrLoc);
      me.gl.bindBuffer(me.gl.ARRAY_BUFFER, me.distBuf);
      me.gl.bufferData(me.gl.ARRAY_BUFFER, new Float32Array(dist), me.gl.STATIC_DRAW);
      var size = 1;          // 1 components per iteration
      var type = me.gl.FLOAT;   // the data is 32bit floats
      var normalize = false; // don't normalize the data
      var stride = 0;        // 0 = move forward size * sizeof(type) each iteration to get the next position
      var offset = 0;        // start at the beginning of the buffer
      me.gl.vertexAttribPointer(
          me.distAttrLoc, size, type, normalize, stride, offset);

      var primitiveType = me.gl.TRIANGLES_STRIP;
      var offset = 0;
      var count = vertices.length;
      me.gl.drawArrays(primitiveType, offset, count);

I used this vertex shader in the previous example

   var vertexShaderSource = 
   "attribute vec2 a_position;\n" +
   "attribute vec4 a_color;\n" +
   "attribute mediump float a_dist;\n" +
   "attribute vec2 a_endpos;\n" +
   "uniform vec2 u_resolution;\n" +
   "varying vec4 vs_out;\n" +
   "//mediump float gl_PointSize;\n" +
   "vec4 convert(vec2 v)\n" +
   "{\n" +
   "  // convert the rectangle from pixels to 0.0 to 1.0\n" +
   "  vec2 zeroToOne = v / u_resolution;\n" +
   "  // convert from 0->1 to 0->2\n" +
   "  vec2 zeroToTwo = zeroToOne * 2.0;\n" +
   "  // convert from 0->2 to -1->+1 (clipspace)\n" +
   "  vec2 clipSpace = zeroToTwo - 1.0;\n" +
   "  return vec4(clipSpace * vec2(1, -1), 0, 1);\n" +
   "}\n" +
   "\n" +
   "void main() {\n" +
   "  vec4 startPos = convert(a_position);\n" +
   "  vec4 endPos = convert(a_endpos);\n" +
   "  gl_Position = mix(startPos, endPos, a_dist);\n" +
   "  vs_out = a_color / 255.0;\n" +
   "  gl_PointSize = 1.0;\n" +
   "}\n";

Thank you for the updates. Our objective is to complete this work before next Monday.

Sergey Ivanovskiy wrote:

Don't understand what is missed here.

What is the expected outcome?

Sergey Ivanovskiy wrote:

If draw vertex arrays with gl.TRIANGLES, then all rectangular area between its left and right vertexes is filled.

This should be expected. The triangles will be filled with the color of the vertices set in fragment shader (or the interpolated values).

If draw vertex arrays with gl.TRIANGLES_STRIP, then the line is drawn by points (100 points in this example)

I remember reading about TRIANGLE_STRIP somewhere that it is unreliable. Supposedly the rendering outcome may differ depending on the actual GPU stack.

Sergey Ivanovskiy wrote:

Hynek, now our vertical and horizontal lines extended asymmetrically towards the right and bottom sides of the webgl canvas
[...]
For the 2d canvas pixels operations are used with coordinates represented by integers but for the webgl canvas x1,y1,width,x2,y2 are floats. Should we use symmetric extension of primitive lines. I am not sure because it can affect drawings. Which one of these ways is more fitted? For symmetric line thickness we can change p2j.stroke.webgl.js using this diff.

Good point. We may need to increment the coordinates by 0,5 to get to the right pixel boundary. Also we should replicate the 2D canvas line "extension" behavior.

Hynek Cihlar wrote:

We may need to increment the coordinates by 0,5 to get to the right pixel boundary.

To clarify, I was thinking about 1-pixel lines. The offset should be half the line thickness, which seems to be what you did in the diff.

Hynek, please evaluate p2j.strokes.webgl.js (the committed revision 11800 (3821c)). Today planning to implement joints of lines for JOIN_MITER style and to fix unimplemented drawing primitives. Joints and dash style lines are done by the main js thread. I would change this way to delegate drawings of line joints and dash style lines to vertex and fragment shaders but I didn't succeed with the working demo.

Also I added webgl dependencies for p2j.perf_tests.js in index.html and did minor changes in p2j.canvas_renderer_webgl.js. (the committed revision 11801 (3821c))

Sergey Ivanovskiy wrote:

Hynek, please evaluate p2j.strokes.webgl.js (the committed revision 11800 (3821c)). Today planning to implement joints of lines for JOIN_MITER style and to fix unimplemented drawing primitives. Joints and dash style lines are done by the main js thread. I would change this way to delegate drawings of line joints and dash style lines to vertex and fragment shaders but I didn't succeed with the working demo.

I think it is OK to keep this logic in JS. We can always move this (or any other part) in shaders IF it turns out to be a significant performance penalty.

Sergey Ivanovskiy wrote:

Hynek, please evaluate p2j.strokes.webgl.js (the committed revision 11800 (3821c)). Today planning to implement joints of lines for JOIN_MITER style and to fix unimplemented drawing primitives. Joints and dash style lines are done by the main js thread. I would change this way to delegate drawings of line joints and dash style lines to vertex and fragment shaders but I didn't succeed with the working demo.

The changes seem reasonable.

In DefaultPathRenderer.strokeLine, does it make sense to handle the special cases of rectangular lines separately? I mean without the computation of normal vector, etc.

Is the implementation of DefaultPathRenderer.strokePoint final? If so, please remove the TODO.

Sergey Ivanovskiy wrote:

Also I added webgl dependencies for p2j.perf_tests.js in index.html and did minor changes in p2j.canvas_renderer_webgl.js. (the committed revision 11801 (3821c))

It looks good.

Committed revision 11806 (3821c) implemented draw3DRect, strokeEllipse, fixed cutTriangleLine and did minor changes for tests. Working on line joints and line styles. Planning to fix it within the first half of this day.

Committed revision 11807 (3821c) fixed repeated triangles for strokeEllipse.

Committed revision 11809 implemented JOINT_MITER style. Hynek, please review these changes and test with Hotel GUI dotted style lines that are shifted now. At this moment I didn't find a solution for the shifted dotted lines.

Sergey Ivanovskiy wrote:

Committed revision 11809 implemented JOINT_MITER style. Hynek, please review these changes and test with Hotel GUI dotted style lines that are shifted now. At this moment I didn't find a solution for the shifted dotted lines.

Sergey, please post a screenshot of the shift. No need to do full screen, just the affected widget with a bit of context is fine.

Hynek Cihlar wrote:

Sergey, there is an unexpected line shift. See the file attached, this shows demo_widgets.p.

This could be also caused by the rectangles below not filling the entire area. Also notice the lines not properly connected, look at the fill-in box for example.

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey, there is an unexpected line shift. See the file attached, this shows demo_widgets.p.

This could be also caused by the rectangles below not filling the entire area. Also notice the lines not properly connected, look at the fill-in box for example.

On the 2D canvas 1 pixel width lines are not symmetric along its direction. Each pixel is drawn at right and bottom sides of its position but webgl lines are symmetric and half of pixel is drawn below and upper its direction. So we have differences. 2D canvas don't use any joint styles. For webgl I implemented MITER_JOINT style for lines in the path for drawing polygons. It is not related to strokeRect and draw3DRect. Please evaluate committed rev 11812(3821c). It tried to fix strokeRect and draw3DRect. Please look at the p2j.perf_tests.js for drawPrimitivesTest. I tried to implement dotted style lines with fragment shader but confused myself because at fragment shader I got 3 vertices and the position of target point that should be colored or not. I tried to use the distance from one of these 3 points but I missed how WebGL interpolates points inside a triangle. May be you have an idea that I missed due to these tedious issues.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey, there is an unexpected line shift. See the file attached, this shows demo_widgets.p.

This could be also caused by the rectangles below not filling the entire area. Also notice the lines not properly connected, look at the fill-in box for example.

On the 2D canvas 1 pixel width lines are not symmetric along its direction. Each pixel is drawn at right and bottom sides of its position but webgl lines are symmetric and half of pixel is drawn below and upper its direction. So we have differences.

These differences must be cleared. WebGL renderer must give the same outputs. Can you handle this?

I tried to implement dotted style lines with fragment shader but confused myself because at fragment shader I got 3 vertices and the position of target point that should be colored or not. I tried to use the distance from one of these 3 points but I missed how WebGL interpolates points inside a triangle. May be you have an idea that I missed due to these tedious issues.

The only idea I had how this could be implemented was with a texture. The line pattern (dots for example) would be copied multiple times (depending on the target line length) in a flattened 1d array representing the image and rendered with drawImage.

3821c revision 11813 adds new webClient directory parameter renderer to specify web client renderer. Use the value webgl for WebGL renderer or anything else (or missing) for the Canvas 2D renderer.

Add the following in directory.xml under webClient container to enable WebGL (includes disabling of graphics caching as there are some issues with that atm).

          <node class="boolean" name="graphicsCached">
            <node-attribute name="value" value="FALSE"/>
          </node>
          <node class="string" name="renderer">
            <node-attribute name="value" value="webgl"/>
          </node>

I just hit the error Too many active webgl contexts in browser console when testing a large GUI app. This was when I was aggressively playing with menus. Apparently browsers have a limit on the number of active webgl contexts. Current design enables one webgl context for each top-level window. This will have to change.

Sergey Ivanovskiy wrote:

I tested HotelGUI and encountered that after clicking on the Floor combobox (Rooms tab) the following exception was thrown

This is from my most recent changes. I will look at it.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey, there is an unexpected line shift. See the file attached, this shows demo_widgets.p.

This could be also caused by the rectangles below not filling the entire area. Also notice the lines not properly connected, look at the fill-in box for example.

On the 2D canvas 1 pixel width lines are not symmetric along its direction. Each pixel is drawn at right and bottom sides of its position but webgl lines are symmetric and half of pixel is drawn below and upper its direction. So we have differences.

These differences must be cleared. WebGL renderer must give the same outputs. Can you handle this?

OK. I will recheck all these issues.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I tested HotelGUI and encountered that after clicking on the Floor combobox (Rooms tab) the following exception was thrown

This is from my most recent changes. I will look at it.

Please retest with 3821c revision 11814.

Hynek, I observed another issue related to tool tips windows that added their images to the background after they were hidden. Did you observe this issue?

Committed revision 11815 (3821a) fixed #4955-244 issue.

Committed revision 11818 (3821c) added missed fill polygon with correct clipping, improved line drawings by using 2 triangles. Please review. It seems that WebGLCanvasRenderer.prototype.renderClosedPath can be removed.

Image caching fixed in webgl renderer in 3821c revision 11819.

Hynek, you calculated the rectangular clipping region as

WebGLCanvasRenderer.prototype.drawImage = function(x, y, width, height, imgData, noTranslate, invert, noClip)
{
   var originX = this.origin.x;
   var originY = this.origin.y;

   if (noTranslate === true)
   {
      originX = 0;
      originY = 0;
   }

   var x1 = Math.round(x + originX);
   var x2 = x1 + width;
   var y1 = Math.round(y + originY);
   var y2 = y1 + height;

   var clipped = null;
   if (noClip === true)
   {
      clipped = [{left: x1, right: x2, top: y1, bottom: y2}];

var x2 = x1 + width;

var x2 = x1 + width - 1;

Committed rev 11820 (3821c) fixed drawRoundRect, strokeEllipse and drawPolygon.

Sergey Ivanovskiy wrote:

Hynek, you calculated the rectangular clipping region as
[...]
but the width and height of this region are one pixel more than the original width and height. Please explain it more thoroughly, why the left bound should be

I think you are right. I was assuming the coordinates were exclusive, but this doesn't seem to be the case. Thanks for spotting this.

Sergey Ivanovskiy wrote:

Did you observe this issue with Hotel GUI?

Yes, I think I've seen this, but haven't had a chance to look at this in more details.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, you calculated the rectangular clipping region as
[...]
but the width and height of this region are one pixel more than the original width and height. Please explain it more thoroughly, why the left bound should be

I think you are right. I was assuming the coordinates were exclusive, but this doesn't seem to be the case. Thanks for spotting this.

When I decrease the right and bottom coordinates by 1 the images get cut by 1 pixel on the right and bottom. As if the coordinates should really be exclusive. But according to the WebGL (and shader) specs, all vertices should be handled equal and be colored inclusively.

Sergey, any idea? I was thinking I was perhaps placing the actual image data in the buffers incorrectly, or a there could be a rounding issue.

I found a mistake in one pixel line drawings that leads to incorrect bounds of fill-in widgets. Please review the last change, the committed revision 11823 (3821c).
OK. Checking what was the issue. It seems that the rectangular area of given width and height should satisfy these equations for its left, right, top and bottom coordinates

bottom - top + 1 = height;
right - left + 1 = width;

drawingBufferWidth and drawingBufferHeight are actual frame buffer's width and height, the documentation said that "which may differ from the
width attribute of the HTMLCanvasElement if the implementation is unable to satisfy the requested width or height". And ImageData has its own width and height that can differ from canvas width and height.

For an example, for some versions of Chrome the copy image code should take this difference into account

CanvasRenderer.prototype.copyImage = function(img, data, imgData, width, height, imgDataOffset)
{
   var awidth  = img.width;
   var aheight = img.height;
   var x = 0;
   var y = 0;
   var pixelsInBytes = awidth * aheight * 4;
   var offset = imgDataOffset;
   for (var p = 0; p < pixelsInBytes; offset += 4)
   {
      data[p]     = imgData[offset];
      data[p + 1] = imgData[offset + 1];
      data[p + 2] = imgData[offset + 2];
      data[p + 3] = imgData[offset + 3];
      x++;
      // p = (y * awidth + x) * 4
      // p1 - p0 = [(y1 - y0) * awidth + (x1 - x0)] * 4 = 4 or 4 * (awidth - width) + 4 
      if (x == width)
      {
         y++;
         x = 0;
         p += ((awidth - width + 1) * 4);
      }
      else
      {
         p += 4; 
      }
   };

   return data;
};

Sergey Ivanovskiy wrote:

drawingBufferWidth and drawingBufferHeight are actual frame buffer's width and height

Good point!

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

drawingBufferWidth and drawingBufferHeight are actual frame buffer's width and height

Good point!

Sergey, could you review the places where canvas width and height is used and replace with drawingBufferWidth and drawingBufferHeight if needed?

I tried to fix #4955-263 but didn't succeed yet and tried to use drawingBufferWidth and drawingBufferHeight too. I will commit the related changes tomorrow. It seems that we shouldn't copy all imageData and should select a subset related to clipping area. It seems that imageData has dirty areas.

Committed rev 11826 (3821c) added scissor and used frame buffer drawingBufferWidth and drawingBufferHeight instead of canvas width and height

 WebGLCanvasRenderer.prototype.flashGL = function()
 {
+   var lastClip = this.lastClippedRegion(true);
+   if (lastClip)
+   {
+      for (var i = 0; i < lastClip.length; i++)
+      {
+         var rect = lastClip[i];
+         this.gl.scissor(rect.x, rect.y, rect.width, rect.height);
+      }
+   }

I tried to apply these changes for drawImage that replaced Math.round to Math.floor and to reduce clipping area width and height by 1

-   var x1 = Math.round(x + originX);
+   var x1 = Math.floor(x + originX);
    var x2 = x1 + width;
-   var y1 = Math.round(y + originY);
+   var y1 = Math.floor(y + originY);
    var y2 = y1 + height;

    var clipped = null;
    if (noClip === true)
    {
-      clipped = [{left: x1, right: x2, top: y1, bottom: y2}];
+      clipped = [{left: x1, right: x2 - 1, top: y1, bottom: y2 - 1}];
    }
    else
    {

It seems I found the root cause of #4955-268. The target image data was incorrectly placed by drawImage and it did incorrect clipping. I will commit sooner.

Sergey Ivanovskiy wrote:

Committed rev 11826 (3821c) added scissor and used frame buffer drawingBufferWidth and drawingBufferHeight instead of canvas width and height
[...]
These changes fixed #4955-256, but #4955-268 and #4955-263 can be reproduced with this version too.

Committed rev 11826 had incorrectly spelled identifier and I checked it again #4955-256 was not fixed. drawImage is not correct now but with my changes for drawImage closing tooltip in #4955-256 clears logins fields and their labels. Investigating root causes. Hynek, could you review drawImage

    var x2 = x1 + width;
    var y1 = Math.round(y + originY);
    var y2 = y1 + height;
+   
+   var cwidth = this.gl.drawingBufferWidth;
+   var tlen = this.screenTexture.length;
+
+   var startOffset = y1 * cwidth + x1;
+   for (var j = 0; j < height; j++)
+   {
+      var srcLine = invert === true ? width * (height - j - 1) : width * j;
+      var toffset = (startOffset + j * cwidth) * 4;
+      if (toffset >= tlen)
+      {
+         break;
+      }
+
+      var srcLineEnd = srcLine * 4 + Math.min(width * 4, tlen - toffset);
+      this.screenTexture.set(imgData.subarray(srcLine * 4, srcLineEnd), toffset);
+   }
+
+   this.pushTriangles([[x1, y1, x2, y1, x1, y2, 0, 255, 203, 255], [x1, y2, x2, y1, x2, y2, 0, 255, 203, 255]]);

Hynek, please review the committed revision 11831 (3821c). We should evaluate our approaches.

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

To convert from pixel coordinates to texture coordinates we can just use

    texcoordX = pixelCoordX / (width  - 1)
    texcoordY = pixelCoordY / (height - 1)

    texcoordX = pixelCoordX / (width)
    texcoordY = pixelCoordY / (height)

Sergey Ivanovskiy wrote:

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

Why do you think the current formula is not correct? If you are worried about filling the right pixels, the pixel coordinate divided by width (or height) should be shifted by half "pixel" right (or down) I think.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

Why do you think the current formula is not correct? If you are worried about filling the right pixels, the pixel coordinate divided by width (or height) should be shifted by half "pixel" right (or down) I think.

The lines/rectangles/text positions are calculated incorrectly. We have a lot of UI artifacts in custormer's application screens.

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

Why do you think the current formula is not correct? If you are worried about filling the right pixels, the pixel coordinate divided by width (or height) should be shifted by half "pixel" right (or down) I think.

The lines/rectangles/text positions are calculated incorrectly. We have a lot of UI artifacts in custormer's application screens.

Yes, I'm aware of it. There are multiple places where the coordinates may be calculated wrong, I'm just not certain this is the one.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

Why do you think the current formula is not correct? If you are worried about filling the right pixels, the pixel coordinate divided by width (or height) should be shifted by half "pixel" right (or down) I think.

If we consider a rectangle of 2 pixels width and height with 4 pixels, then these 4 pixels should have coordinates (0, 0), (1, 0), (0, 1) and (1, 1). If we divide pixels by the width, then they have (0, 0), (0.5, 0), (0, 0.5) and (0.5, 0.5). The shift by half pixel doesn't help to get (0, 0), (1, 0), (0, 1) and (1, 1). Thus it seems that the correct formula

texcoordX = pixelCoordX / (width  - 1)
texcoordY = pixelCoordY / (height - 1)

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I just found that texture coordinates are calculated from the pixels by this formula

Why do you think the current formula is not correct? If you are worried about filling the right pixels, the pixel coordinate divided by width (or height) should be shifted by half "pixel" right (or down) I think.

If we consider a rectangle of 2 pixels width and height with 4 pixels, then these 4 pixels should have coordinates (0, 0), (1, 0), (0, 1) and (1, 1). If we divide pixels by the width, then they have (0, 0), (0.5, 0), (0, 0.5) and (0.5, 0.5). The shift by half pixel doesn't help to get (0, 0), (1, 0), (0, 1) and (1, 1). Thus it seems that the correct formula
[...]

True. Both the pixel-wbgl translation formula in vertex shader as well as the draw primitives (like drawImage, etc.) must reflect this. If you want to fix this, please go ahead. Otherwise I will include this in my next commit.

I commented to show that this code needs to be fixed but the existed code was incorrect because it calculates var startOffset = y1 * cwidth + x1; from clipping regions.

I simplified dotted line style by adding these additional vertex attributes: a_dash, a_dash_pattern, a_normal and a_point.

   var vertexShaderSource = `
   attribute vec2 a_position;
   attribute vec4 a_color;
   attribute vec2 a_normal;
   attribute vec2 a_point;
   attribute float a_dash;
   attribute vec2 a_dash_pattern;

   uniform vec2 u_resolution;
   varying vec4 vs_out;
   varying vec2 p0;
   varying vec2 n;
   varying vec2 p;
   varying float dash;
   varying vec2 dash_pattern;
    //mediump float gl_PointSize;
   vec4 convert(vec2 v)
   {
     // convert the rectangle from pixels to 0.0 to 1.0
     vec2 zeroToOne = v / u_resolution;
     // convert from 0->1 to 0->2
     vec2 zeroToTwo = zeroToOne * 2.0;
     // convert from 0->2 to -1->+1 (clipspace)
     vec2 clipSpace = zeroToTwo - 1.0;

     vec4 res = vec4(clipSpace * vec2(1, -1), 0, 1);

     return res;
   }

   void main()
   {
     vec4 pos = convert(a_position);

     gl_Position = pos;
     p0 = a_position;
     p = a_point;
     n = a_normal;
     vs_out = a_color / 255.0;
     dash = a_dash;
     dash_pattern = a_dash_pattern;
     gl_PointSize = 1.0;
   }
   `;

   var fragmentShaderSource = `
   // fragment shaders don't have a default precision so we need
   // to pick one. mediump is a good default
   precision mediump float;
   varying vec4 vs_out;
   varying vec2 p0;
   varying vec2 p;
   varying vec2 n;
   varying float dash;
   varying vec2 dash_pattern;

   void main()
   {
      vec2 v = vec2(gl_FragCoord.x - p.x, gl_FragCoord.y - p.y);
      float d0 = abs(dot(n, v));
      float period = dash_pattern.x + dash_pattern.y;
      float m = d0 - floor(d0  / period) * period;
      if (dash == 1.0)
      {
         if (m <= dash_pattern.x)
         {
            gl_FragColor = vs_out;
         }
         else
         {
            discard;
         }
      }
      else
      {
         gl_FragColor = vs_out;
      }
   }
   `;

It seems that this is an example for drawImage, https://webglfundamentals.org/webgl/lessons/webgl-2d-drawimage.html

Hynek, I am planning to change shader code according to the rev 11832 (3821c). If you have changes in the shader code, then please commit them. We will fix drawing issues later when we will collect all issues.

Sergey Ivanovskiy wrote:

It seems that this is an example for drawImage, https://webglfundamentals.org/webgl/lessons/webgl-2d-drawimage.html

This is how we implement drawImage primitive also.

Sergey Ivanovskiy wrote:

Hynek, I am planning to change shader code according to the rev 11832 (3821c). If you have changes in the shader code, then please commit them. We will fix drawing issues later when we will collect all issues.

Please go ahead and check in your changes. I will merge later.

Status update.

Feature wise we should be complete AFAIK, Sergey please correct me if I'm wrong. But there are still draw issues.

One of them relates to my work on the limit of max number of active webgl contexts. With the introduction of single shared offscreen webgl canvas I broke image rendering in cached mode. The problem is that any image rendered through textures (including texts) is not blended properly and the rendered results look ugly. I suspect the images get somehow rendered multiple times. I'm looking at this. I haven't checked in my changes due to this issue.

Yes, the drawing primitives are complete. Now I am working to draw dash lines by the fragment shader. What was found is thin some vertical and horizontal lines of 0.5 width are not dashed by this method. I am trying to work arround this issue. But sloped thin lines are dashed.

Sergey Ivanovskiy wrote:

Yes, the drawing primitives are complete. Now I am working to draw dash lines by the fragment shader. What was found is thin some vertical and horizontal lines of 0.5 width are not dashed by this method. I am trying to work arround this issue. But sloped thin lines are dashed.

I came to opinion that there is a minimal width of 1 webgl pixel and that thin lines are not drawn by webgl triangle primitive and they can be drawn by the webgl line primitive.

Sergey Ivanovskiy wrote:

Sergey Ivanovskiy wrote:

Yes, the drawing primitives are complete. Now I am working to draw dash lines by the fragment shader. What was found is thin some vertical and horizontal lines of 0.5 width are not dashed by this method. I am trying to work arround this issue. But sloped thin lines are dashed.

I came to opinion that there is a minimal width of 1 webgl pixel and that thin lines are not drawn by webgl triangle primitive and they can be drawn by the webgl line primitive.

Sorry, I fixed Java2D test, now I am cleaning up and then will commit these changes.

Committed revision 11833 (3821c) added dotted style lines to fragment shader code.

Hynek, I don't understand why for

WebGLCanvasRenderer.prototype.strokeRect = function(x, y, width, height, color, fill)
{
   if (fill)
   {
      var x1 = x + this.origin.x;
      var x2 = x + width - 1 + this.origin.x;
      var y1 = y + this.origin.y;
      var y2 = y + height - 1 + this.origin.y;

      var clipped = this.clipRegion(x1, y1, x2, y2);

      if (clipped)
      {
         for (var i = 0; i < clipped.length; i++)
         {
            var clip = clipped[i];
            x1 = clip.left;
            y1 = clip.top;
            x2 = clip.right;
            y2 = clip.bottom;
            this.pushTriangles([[x1, y1, x2, y1, x1, y2, this.rawColor[0], this.rawColor[1], this.rawColor[2], 255],
                                [x1, y2, x2, y1, x2, y2, this.rawColor[0], this.rawColor[1], this.rawColor[2], 255]]);
            this.offscreenCtx.fillRect(x1 - this.origin.x, y1 - this.origin.y, x2 - x1 + 1, y2 - y1 + 1);
         }
      }
.................
}

Sergey Ivanovskiy wrote:

Hynek, I don't understand why for
[...]
this.offscreenCtx.fillRect(x1 - this.origin.x, y1 - this.origin.y, x2 - x1 + 1, y2 - y1 + 1); influences text drawings on the texture. Ignoring this code line affects text drawings. Text becomes displayed twice or more times at the shifted positions. If the strokeRect code is changed to be the same as for 2D canvas renderer, then the 2D and webgl stroke tests display the same patterns, but at the same moment the Hotel GUI becomes displayed with rectangles artifacts. We should clarify this issue somehow.

The reason is that strokeRect is used by the theme classes to erase widget background. I found this issue also, some widgets issue multiple text draw for the same location.

Hynek Cihlar wrote:

The reason is that strokeRect is used by the theme classes to erase widget background. I found this issue also, some widgets issue multiple text draw for the same location.

It seems that the fragment shader should be responsible for these effects.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

The reason is that strokeRect is used by the theme classes to erase widget background. I found this issue also, some widgets issue multiple text draw for the same location.

It seems that the fragment shader should be responsible for these effects.

Did you find anything interesting in there?

No, I experimented with the fragment shader by adding coordinates of the text top left corner to the text triangles and then changing coordinates of texture in order to smooth these effects. It seems that v_texCoords inside neighbour triangles can produce these effects. For an example, I used the following code with the similar effects

   "void main()\n" +
   "{\n" +
   "  vec2 v = vec2(p0.x - p.x, p0.y - p.y);\n" +
   "  float d0 = abs(dot(n, v));\n" +
   "  float period = dash_pattern.x + dash_pattern.y;\n" +
   "  float m = d0 - floor(d0  / period) * period;\n" +
   "  vec2 tex = p0 / u_resolution;// TODO\n" +
   "  if (v_color.x == 0.0 && v_color.y == 255.0 && v_color.z == 203.0)\n" +
   "  {\n" +
   "     vec4 c = texture2D(u_image, tex);\n" + // tex is used instead of v_texCoords
   "     //c.x = c.x / 255.0;\n" + 
   "     //c.y = c.y / 255.0;\n" +
   "     //c.z = c.z / 255.0;\n" +
   "     //c.w = 0.0;\n" +
   "     gl_FragColor = c;\n" +
   "  }\n" +

Guys,

Have you measured the performance difference for new webgl approach comparing to legacy canvas2d? Does the current embedded GUI test show the real difference (for stroke test especially)? What difference you have on your systems?

I have some results from my systems but I'm afraid they are too custom to be used as general estimation.

Greg, if you agree, I will commit my changes with this Chrome issue so that we can continue with the other issues, and return to this later. This will require Firefox to be used to get good results.

OK

I do wonder if this is a Chrome bug.

Please notify about the target branch updates. Meanwhile bug fixing we can dig into https://css-tricks.com/techniques-for-rendering-text-with-webgl/ and https://en.wikipedia.org/wiki/Texture_atlas
https://www.youtube.com/watch?v=qasFxsOCfpA

Eugenie Lyzenko wrote:

Guys,

Have you measured the performance difference for new webgl approach comparing to legacy canvas2d? Does the current embedded GUI test show the real difference (for stroke test especially)? What difference you have on your systems?

I have some results from my systems buy I'm afraid they are too custom to be used as general estimation.

This guy did it https://www.youtube.com/watch?v=qkDg-Y9iHBA

Sergey Ivanovskiy wrote:

Eugenie Lyzenko wrote:

Guys,

Have you measured the performance difference for new webgl approach comparing to legacy canvas2d? Does the current embedded GUI test show the real difference (for stroke test especially)? What difference you have on your systems?

I have some results from my systems buy I'm afraid they are too custom to be used as general estimation.

This guy did it https://www.youtube.com/watch?v=qkDg-Y9iHBA

I see only one demo in a 25 min show - with single pixel painting. But I see the problem - when calling for many but short operations to GPU then the time waiting for GPU to be back can overlap the time of doing the same operations set with CPU. This can be important in our case.

Eugenie Lyzenko wrote:

I see only one demo in a 25 min show - with single pixel painting. But I see the problem - when calling for many but short operations to GPU then the time waiting for GPU to be back can overlap the time of doing the same operations set with CPU. This can be important in our case.

This is why we minimize the interaction with the GPU. The whole screen is broken into an array of triangles, which is, together with textures and other metadata, sent to the GPU in one batch.

Hynek, it seems this doc https://delphic.me.uk/tutorials/webgl-text has an example of webgl drawing text that uses texture canvas explicitly without intermediate pixels array.

Sergey Ivanovskiy wrote:

Hynek, it seems this doc https://delphic.me.uk/tutorials/webgl-text has an example of webgl drawing text that uses texture canvas explicitly without intermediate pixels array.

Sergey, thanks for the link. I was considering this idea. The reason I didn't choose this approach was that the 2D canvas has no depth, all texts must be present at a single z-index. So for the cases where you have a widget with a label for example and the label is overlapped with another widget you would have to calculate a mask to hide invisible portions of the rendered texts. For non-rectangular areas this would be complicated.

The preamble to the following question https://stackoverflow.com/questions/50554803/triangulate-path-data-from-opentype-js-using-earcut/50581183#50581183 relates to our method of text rendering.

I split drawing batches at translate, clip invoking flashGL() and observed that Hotel GUI performed faster if drawing batches mode were split. It follows that collecting triangles can degrade webgl renderer. I an't sure that my experiments are fair.

Sergey Ivanovskiy wrote:

I split drawing batches at translate, clip invoking flashGL() and observed that Hotel GUI performed faster if drawing batches mode were split. It follows that collecting triangles can degrade webgl renderer. I an't sure that my experiments are fair.

Please post the diff here. How did you test?

Sergey, I checked in 3821c revision 11841. This, among others, introduces single active webgl context. Please review these changes, especially the changes in p2j.perf_tests.js.

Sergey, according to my tests, the artifacts seem to directly relate to WebGLCanvasRenderer.prototype.drawImage or the code path related to the function.

Please share these tests. OK. drawImage is suspected.

Sergey Ivanovskiy wrote:

Please share these tests. OK. drawImage is suspected.

These were not typical tests. I just selectively disabled the code paths where drawImage was used (caching and DRAW_IMAGE driver primitive) and observed whether/how it affected the artifacts.

Did you test this technique https://stackoverflow.com/questions/46806063/how-stencil-buffer-and-masking-work ? Should we apply it to clip areas for images?

Sergey Ivanovskiy wrote:

Did you test this technique https://stackoverflow.com/questions/46806063/how-stencil-buffer-and-masking-work ? Should we apply it to clip areas for images?

In principal I don't think we should need depth buffer to hide invisible portions of the screen. In fact we don't (or should not) have any invisible parts subjected to depth testing. We only draw to a simple 2D plane.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Did you test this technique https://stackoverflow.com/questions/46806063/how-stencil-buffer-and-masking-work ? Should we apply it to clip areas for images?

In principal I don't think we should need depth buffer to hide invisible portions of the screen. In fact we don't (or should not) have any invisible parts subjected to depth testing. We only draw to a simple 2D plane.

This example shows how drawing of two colored triangles is clipped with help of this method. The result is in the clipping quard area. If you set {stencil: true}, then you can evaluate this example https://jsfiddle.net/b1xznod6/

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Did you test this technique https://stackoverflow.com/questions/46806063/how-stencil-buffer-and-masking-work ? Should we apply it to clip areas for images?

In principal I don't think we should need depth buffer to hide invisible portions of the screen. In fact we don't (or should not) have any invisible parts subjected to depth testing. We only draw to a simple 2D plane.

This example shows how drawing of two colored triangles is clipped with help of this method. The result is in the clipping quard area. If you set {stencil: true}, then you can evaluate this example https://jsfiddle.net/b1xznod6/

I understand. We only ever need to clip rectangular areas, which we already handle (or should be able to handle) in JS code. I would consider stencil buffer if it provided a real functional benefit, like improved performance.

Another suspect is clipTriangles, this method clipTriangles is tested and you can check it with perf_tests.js. The triangles should built correctly. For Hotel GUI drawing artifacts is always some part of the large image that displays the hotel room type. It can be that the image data are aligned specially somehow.

For my desktop system these tests showed the following opposite results. I got for Firefox:

WebGL Stroke Test finished in 220 ms.
Stroke Test finished in 134 ms

WebGL Stroke Test finished in 436 ms.
Stroke Test finished in 273 ms.

For the "Performance Tests" menu item, it would be useful if it ran all the tests (both webgl and 2d canvas) and displayed the results in a simple table. I'm sure it is useful to be able to run the tests individually. But the most common usage will be to report on the results for a browser which allows comparison of webgl and 2d canvas. Today this is awkward because we must click through all the different tests and then carefully read the log messages.

In these tests possible hardware accelerations were turned off. If hardware possible acceleration is on for Chrome, then the result looks similar to your presented results. webgl performance is increased 299 ms and 2d performance is degraded 74000 ms.

Greg Shah wrote:

For the "Performance Tests" menu item, it would be useful if it ran all the tests (both webgl and 2d canvas) and displayed the results in a simple table. I'm sure it is useful to be able to run the tests individually. But the most common usage will be to report on the results for a browser which allows comparison of webgl and 2d canvas. Today this is awkward because we must click through all the different tests and then carefully read the log messages.

OK. Planning to add this summary table.

Sergey Ivanovskiy wrote:

In these tests possible hardware accelerations were turned off. If hardware possible acceleration is on for Chrome, then the result looks similar to your presented results. webgl performance is increased 299 ms and 2d performance is degraded 74000 ms.

So the conclusion is for browser HW acceleration option on/off for Chrome in Ubuntu:
1. The webgl painting does not almost depend on the usage of the acceleration within browser.
2. The canvas2d approach significantly degrades when acceleration is turning on.

Correct?

Hynek,

What about your system?

I'm asking because I need to provide stable recommendations for browser options tuning to get the best possible performance.

Yes, correct, for Chrome (Ubuntu) webgl performance is increased if HW acceleration is on.

scissor works only if scissor op is enabled explicitly by gl.enable(gl.SCISSOR_TEST); and it uses the last set rectangle during rendering (performing SCISSOR_TEST) so it can be completely removed from the code. It seems that the same is true for gl.DEPTH_TEST and gl.STENCIL_TEST.

Sergey Ivanovskiy wrote:

scissor works only if scissor op is enabled explicitly by gl.enable(gl.SCISSOR_TEST); and it uses the last set rectangle during rendering (performing SCISSOR_TEST) so it can be completely removed from the code. It seems that the same is true for gl.DEPTH_TEST and gl.STENCIL_TEST.

Another thing is that calling scissor during flash should have no effect as there should be no active clipping at the time.

Eugenie Lyzenko wrote:

Sergey Ivanovskiy wrote:

In these tests possible hardware accelerations were turned off. If hardware possible acceleration is on for Chrome, then the result looks similar to your presented results. webgl performance is increased 299 ms and 2d performance is degraded 74000 ms.

So the conclusion is for browser HW acceleration option on/off for Chrome in Ubuntu:
1. The webgl painting does not almost depend on the usage of the acceleration within browser.
2. The canvas2d approach significantly degrades when acceleration is turning on.

Correct?

Hynek,

What about your system?

I'm asking because I need to provide stable recommendations for browser options tuning to get the best possible performance.

On my system with dedicated GPU (AMD Radeon RX 590), browsers left in default settings the times are as follows:

Chrome with acceleration on (both WebGL and Canvas):
Stroke Test finished in about 12000 ms
WebGL Stroke Test finished in about 119 ms

Firefox:
Stroke Test finished in about 66 ms
WebGL Stroke Test finished in about 80 ms

Greg Shah wrote:

For the "Performance Tests" menu item, it would be useful if it ran all the tests (both webgl and 2d canvas) and displayed the results in a simple table. I'm sure it is useful to be able to run the tests individually. But the most common usage will be to report on the results for a browser which allows comparison of webgl and 2d canvas. Today this is awkward because we must click through all the different tests and then carefully read the log messages.

The output should also contain the browser and system version and other system setup info like resolution.

Also the suite should execute the individual test cases multiple times each run with increased number of draw primitives performed. This is to get the idea how well the individual scenarios scale on the system. Also the number of draw primitives executed should be part of the report output.

The suite should also provide an abort button which would abort currently executed test scenario. On some systems the scenario can take minutes to finish. With scaled up runs, this could get even longer.

Hynek Cihlar wrote:

On my system with dedicated GPU (AMD Radeon RX 590), browsers left in default settings the times are as follows:

> Chrome with acceleration on (both WebGL and Canvas):
> Stroke Test finished in about 12000 ms
> WebGL Stroke Test finished in about 119 ms
> 

When Use hardware acceleration when available is switched off, the Stroke Test time decreases to about 101 ms.

Here is what I have on my test environment (units are ms, not first but best try).

Windows 10 middle-high perf system, discrete NVIDIA G210, embedded performance score for stroke lines:

Ubuntu 18.04.5 middle-high perf system, discrete NVIDIA G210, embedded performance score for stroke lines:

Ubuntu 18.04.5 high perf system, discrete AMD Radeon pro WX 3200, embedded performance score for stroke lines:

Sergey, the number of executed draw primitives (lines, strokes, etc.) in p2j.perf_tests.js should match the actual number of executed primitives in one of the complex screens from a large customer GUI app. If the number of executed draw primitives is too low for example, the results will be biased by the mandatory and constant overhead involved in each draw cycle (in case of WebGL for example, this is the overhead resulting from GPU communication).

Hynek Cihlar wrote:

Sergey, I checked in 3821c revision 11841. This, among others, introduces single active webgl context. Please review these changes, especially the changes in p2j.perf_tests.js.

Now application windows and this test program use the same global context. Could it be changed so that the test program would use its own webgl context? The current issue is related to resize webgl context for each windows, it is a bad idea for the test program if it will change the webgl canvas size unexpectedly for the main application.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, I checked in 3821c revision 11841. This, among others, introduces single active webgl context. Please review these changes, especially the changes in p2j.perf_tests.js.

Now application windows and this test program use the same global context. Could it be changed so that the test program would use its own webgl context? The current issue is related to resize webgl context for each windows, it is a bad idea for the test program if it will change the webgl canvas size unexpectedly for the main application.

True, the performance test should use its own webgl context. I'll fix it.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, I checked in 3821c revision 11841. This, among others, introduces single active webgl context. Please review these changes, especially the changes in p2j.perf_tests.js.

Now application windows and this test program use the same global context. Could it be changed so that the test program would use its own webgl context? The current issue is related to resize webgl context for each windows, it is a bad idea for the test program if it will change the webgl canvas size unexpectedly for the main application.

True, the performance test should use its own webgl context. I'll fix it.

Actually, the single webgl canvas should work OK even for the performance tests. The resize is expected, each legacy window will make sure the size is correct before the drawing cycle begins.

Hynek Cihlar wrote:

Sergey, the number of executed draw primitives (lines, strokes, etc.) in p2j.perf_tests.js should match the actual number of executed primitives in one of the complex screens from a large customer GUI app. If the number of executed draw primitives is too low for example, the results will be biased by the mandatory and constant overhead involved in each draw cycle (in case of WebGL for example, this is the overhead resulting from GPU communication).

OK. I am changing now the program output presentation and planning to add number primitives and average results for several runs for each test run as you proposed.

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Now application windows and this test program use the same global context. Could it be changed so that the test program would use its own webgl context? The current issue is related to resize webgl context for each windows, it is a bad idea for the test program if it will change the webgl canvas size unexpectedly for the main application.

True, the performance test should use its own webgl context. I'll fix it.

Actually, the single webgl canvas should work OK even for the performance tests. The resize is expected, each legacy window will make sure the size is correct before the drawing cycle begins.

OK. Thus, the test program should do it similarly by invoking this method

WebGLCanvasRenderer.prototype.notifyWindowDrawStart = function(numOps)
{
   this.resize(this.screenCanvas.width, this.screenCanvas.height);
   this.offscreenCtx.clearRect(0, 0, this.offscreenCanvas.width, this.offscreenCanvas.height);
}

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Now application windows and this test program use the same global context. Could it be changed so that the test program would use its own webgl context? The current issue is related to resize webgl context for each windows, it is a bad idea for the test program if it will change the webgl canvas size unexpectedly for the main application.

True, the performance test should use its own webgl context. I'll fix it.

Actually, the single webgl canvas should work OK even for the performance tests. The resize is expected, each legacy window will make sure the size is correct before the drawing cycle begins.

OK. Thus, the test program should do it similarly by invoking this method
[...]
Correct?

I don't recall if the perf tests window is modal. If it is the call is not necessary, as you already resize the canvas. But calling the function before each draw cycle starts is a good idea in any case.

Hynek Cihlar wrote:

I don't recall if the perf tests window is modal.

perf_tests.js uses dijit/Dialog from dojo js library.

If it is the call is not necessary, as you already resize the canvas. But calling the function before each draw cycle starts is a good idea in any case.

You add new dependencies between p2j.screen and p2j.canvas_renderer_webgl via the global unique webgl canvas/context variables.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

I don't recall if the perf tests window is modal.

perf_tests.js uses dijit/Dialog from dojo js library.

If it is the call is not necessary, as you already resize the canvas. But calling the function before each draw cycle starts is a good idea in any case.

You add new dependencies between p2j.screen and p2j.canvas_renderer_webgl via the global unique webgl canvas/context variables.

Yes, the shared webgl canvas is stored in the p2j.screen object.

Eugenie recently summarized the following rules for best performance:

• Windows, Chrome - using webgl render, hardware acceleration enabled in browser.
• Windows, Firefox - using canvas2d render, hardware acceleration disabled in browser.
• Linux, Chrome - using webgl render, hardware acceleration enabled in browser.
• Linux, Firefox - using canvas2d render, hardware acceleration is not important.

I see multiple problems here.

• These results may change over time as we change FWD and as the browsers evolve.
• This relies upon each user (or organization if the systems are standardized) to remember these rules and ensure their systems always follow them.

I'm concerned that this will be error prone and will turn into a support cost for our customers. To resolve this, we could detect which approach is better at runtime and chose the renderer that is best at that time on that system/browser.

Looking at the results, I suspect that any scenario where 2D canvas is outperformance webgl is an area in which we can improve our webgl results. Ultimately, they are not doing anything in 2D canvas which we are not able to do at least as well in webgl. Text rendering might be an exception.

Hynek, please review these new batch tests for comparisons webgl and 2d renderer. committed rev 11848 (3821c). In these tests the number of repeated runs to calculate the average values through 10 runs is hardcoded and number of primitives is harcoded to be [100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]

Sergey Ivanovskiy wrote:

Hynek, please review these new batch tests for comparisons webgl and 2d renderer. committed rev 11848 (3821c). In these tests the number of repeated runs to calculate the average values through 10 runs is hardcoded and number of primitives is harcoded to be [100, 200, 300, 400, 500, 600, 700, 800, 900, 1000]

The data can be exported into csv via the test table context menu and then viewed and copied manually from the browser's web console. It was a workaround the dojo issue that dojo menus in this dialog attract the focus input and it makes problems for copying the data into the system clipboard. I didn't fix it yet.

Is it correct that the most priority task within this task is fixing drawing artifacts? It is suspected that these artifacts are related to drawImage. The problem here is that the similar function is used fro 2D canvas. The pixel to pixel mapping looks correct at least for the case when invert = false. I think if this mapping would be incorrect, then we were not able to watch icons and images in the Hotel GUI at all.

Sergey, the artifacts and the trimmed images are caused by the pixel-webgl coordinate transformation logic. According to my experiments, webgl will only draw the exact difference of the supplied coordinates. So, when you want to draw an area of 1 pixel for example, you will have to provide two triangles with the coordinates [x, y, x + 0.01, y, x, y + 0.01], [x, y + 0.01, x + 0.01, y, x + 0.01, y + 0.01] (assuming the pixel size in both directions is 0.01).

See this jsfiddle http://jsfiddle.net/jc7p85Lv/, where you can experiment with this. The example draws a black rectangle on a green webgl canvas. You can change the rectangle coordinates and size on line 42.

All the places where the coordinates are calculated from width or height must be changed to x2 = x1 + width instead of x2 = x1 + width - 1. This must be done in JS code as the shaders only receive coordinates without any further context about the actual geometry of the drawn shapes.

I'd like to see your comment on this. First I want to be sure I'm interpreting the webgl coordinate system right and second for you to review the impact on the line implementation.

Yes, agree. Your line test proves that the line of one pixel width should be drawn using triangles

function drawRect(x, y, w, h)
   {
      var x1 = x;
      var x2 = x + w;
      var y1 = 200 - y;
      var y2 = 200 - (y + h);

      return [x1, y1, x2, y1, x1, y2, x1, y2, x2, y1, x2, y2];
   }

Sergey Ivanovskiy wrote:

Yes, agree. Your line test proves that the line of one pixel width should be drawn using triangles
[...]
But it seems p2j.stroke.webgl.js builds the same triangles extending lines symmetrically for lines having their widths more than one pixel. Did you mean that clipping regions are calculated incorrectly?

So far I don't have any indication that clipping is wrong, although this was my suspect, too. But caching makes the artifacts worse as it internally uses drawImage which also contains wrong assumptions about webgl coordinates.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Yes, agree. Your line test proves that the line of one pixel width should be drawn using triangles
[...]
But it seems p2j.stroke.webgl.js builds the same triangles extending lines symmetrically for lines having their widths more than one pixel. Did you mean that clipping regions are calculated incorrectly?

So far I don't have any indication that clipping is wrong, although this was my suspect, too. But caching makes the artifacts worse as it internally uses drawImage which also contains wrong assumptions about webgl coordinates.

Sorry, in this case I have no idea.

Sergey, please review 3821c revision 11852. This resolves the artifacts in the rendered output. It also resolves the trimmed texts and trimmed images.

Please review all the places where you call pushTriangles and make sure the coordinates are correct in respect to how webgl handles the coordinates. Look in strokeRect how it is done there.

Is there anything left in this task in regard to making the webgl mode work properly without any drawing problems?

Are there opportunities to further optimize the webgl code to ensure that it always outperformance 2d canvas?

Is it feasible to measure which drawing approach is better when the client loads and automatically use that drawing mode?

Hynek Cihlar wrote:

Sergey, please review 3821c revision 11852. This resolves the artifacts in the rendered output. It also resolves the trimmed texts and trimmed images.

Please review all the places where you call pushTriangles and make sure the coordinates are correct in respect to how webgl handles the coordinates. Look in strokeRect how it is done there.

I tested with Hotel GUI and observed drawing artifacts. It can be that you changes are correct. I will recheck them with the clean setup. Did you test Hotel GUI and Check In Date dialog on the first tab, Available Rooms and the last tab, Rooms?

The results of Batch Fill Test (has overlapping rectangles) and WebGL Fill Test (has no overlapping rectangles) gives me another idea that WebGL actually draws overlapping rectangles in 3D space. I did the following experiments with the fragment shader and found that

if (abs(gl_FragCoord.z) > 0.5) { discard; }

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please review 3821c revision 11852. This resolves the artifacts in the rendered output. It also resolves the trimmed texts and trimmed images.

Please review all the places where you call pushTriangles and make sure the coordinates are correct in respect to how webgl handles the coordinates. Look in strokeRect how it is done there.

I tested with Hotel GUI and observed drawing artifacts. It can be that you changes are correct. I will recheck them with the clean setup. Did you test Hotel GUI and Check In Date dialog on the first tab, Available Rooms and the last tab, Rooms?

I tested Hotel GUI, demo_widgets.p and a large customer GUI app.

There are some remaining issues, which are caused by different source than the one I fixed in revision 11852.

• When graphicsCached is enabled the images alpha channel is rendered wrong.
• Combo box selection list when hovered quickly (causing rapid selection changes) sometimes leaves strange artifacts. It looks like two items are partly selected at the same time.
• Login dialog in the large customer GUI app has a white "hole" in the login dialog.
• Some lines are not connected fully, like button's border.

I'm looking at the first two items. Sergey, can you take the lines?

Greg Shah wrote:

Is there anything left in this task in regard to making the webgl mode work properly without any drawing problems?

See #4955-358.

Are there opportunities to further optimize the webgl code

Most likely, I haven't (and I don't think Sergey either) had chance to trace and measure in more detail the performance.

to ensure that it always outperformance 2d canvas?

I'm keen to see the results from Sergey's latest perf test changes. Especially with test sets closer to our real large GUI apps.

Is it feasible to measure which drawing approach is better when the client loads and automatically use that drawing mode?

I think it is feasible.

Sergey Ivanovskiy wrote:

Hynek, Greg, some artifacts are present. The empty or degraded triangles couldn't be root causes of these artifacts. Please look at this video.

The artifacts in the calendar and in the widget title seem to be the same issue as "Combo box selection list when hovered quickly (causing rapid selection changes) sometimes leaves strange artifacts."

The wrong text background seems to be the same issue as "When graphicsCached is enabled the images alpha channel is rendered wrong."

What is the status on resolving the remaining artifacts?

I didn't succeed in fixing issues from #4955-360. Hynek, please share your findings or write more thoroughly #4955-361.

Greg Shah wrote:

What is the status on resolving the remaining artifacts?

I'm still looking at the first two items in the list of tasks mentioned in #4955-358.

Sergey Ivanovskiy wrote:

I didn't succeed in fixing issues from #4955-360.

So please look at the item "Some lines are not connected fully, like button's border."

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I didn't succeed in fixing issues from #4955-360.

So please look at the item "Some lines are not connected fully, like button's border."

We didn't use special rules for joining lines for 2D Canvas renderer. I think it doesn't need to use it for WebGL Canvas renderer. My be it is required to fix input vertices for primitive drawings. Did you mean 3D rectangle?

Hynek Cihlar wrote:

Greg Shah wrote:

What is the status on resolving the remaining artifacts?

I'm still looking at the first two items in the list of tasks mentioned in #4955-358.

Please correct if my understandings are not correct. We use 2D offscreen canvas to draw text, then we get its 2D image and render this text image on WebGL canvas, and then we put WebGL canvas renderer on 2D screen canvas and on 2D offscreen canvas. Could we just use WebGL for lines and background drawings and then draw it on the 2D offscreen canvas using special composition with 2D offscreen canvas but text will be rendered only on 2D offscreen canvas?

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Greg Shah wrote:

What is the status on resolving the remaining artifacts?

I'm still looking at the first two items in the list of tasks mentioned in #4955-358.

Please correct if my understandings are not correct. We use 2D offscreen canvas to draw text, then we get its 2D image and render this text image on WebGL canvas, and then we put WebGL canvas renderer on 2D screen canvas and on 2D offscreen canvas. Could we just use WebGL for lines and background drawings and then draw it on the 2D offscreen canvas using special composition with 2D offscreen canvas but text will be rendered only on 2D offscreen canvas?

Yes, I think this is doable. Text could be rendered directly on the 2D canvas. It just requires an extra layer of logic to handle depth, to handle text covered by other widgets.

In any case we should optimize the hot-spots. From the profiling results I have done so far, text performance is not that bad, there are other more severe candidates. The reason for this is the use of offscreen canvas, which doesn't require screen buffer sync when image data is read from it (and perhaps other attributes of the offscreen canvas).

Sergey, FYI 3821c revision 11857 resolves the image drawing issue where a specific color was used in the triangle data to direct fragment shader to go fetch the final color from a texture.

Committed revision 11859 (3821c) added cancel batch test functionality and one new test for drawing rectangles with 4 different solid line styles and this test showed that there is some issue in strokeRect, its bottom right corner is drawn incorrectly. Looking on this issue.

3821c revision 11860 resolves all issues mentioned in #4955-358 except "Some lines are not connected fully, like button's border.". This also resolves the related issues mentioned by Sergey.

Sergey, please retest the revision.

The remaining issues I'm now aware of are:

• Some lines are not connected fully, like button's border. (assigned to Sergey)
• Window system menu background has wrong color. This is new, I found this today during testing. I will look at this.

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c). I think drawImage implementation is not correct now. Please explain why coordinates are calculated by

         var clip = clipped[i];
         x1 = clip.left;
         y1 = clip.top;
         x2 = clip.right;
         y2 = clip.bottom;

         var startOffset = y1 * cwidth + x1;

Sergey Ivanovskiy wrote:

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c).

Which video do you mean?

Sergey Ivanovskiy wrote:

I think drawImage implementation is not correct now.

Please give details why do you think it is not correct.

Please explain why coordinates are calculated by

Sorry, I'm not sure what you mean.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c).

Which video do you mean?

I tried hotel_gui_disabled_graphics_cache_1.mkv and I can replicate them, too. I will look at these, too.

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c).

Which video do you mean?

I tried hotel_gui_disabled_graphics_cache_1.mkv and I can replicate them, too. I will look at these, too.

You are not alone, I am working on these issues too.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c).

Which video do you mean?

I tried hotel_gui_disabled_graphics_cache_1.mkv and I can replicate them, too. I will look at these, too.

You are not alone, I am working on these issues too.

I haven't started on these, so feel free to work on them.

Sergey Ivanovskiy wrote:

Committed revision 11859 (3821c) added cancel batch test functionality and one new test for drawing rectangles with 4 different solid line styles and this test showed that there is some issue in strokeRect, its bottom right corner is drawn incorrectly. Looking on this issue.

These issues of strokeRect and draw3DRect should be fixed in the committed rev 11864 (3821c).

Sergey Ivanovskiy wrote:

Hynek, all issues in my video can be reproduced with this rev 11860 (3821c). I think drawImage implementation is not correct now. Please explain why coordinates are calculated by
[...]

Let me explain my question.

WebGLCanvasRenderer.prototype.drawImage = function(x, y, width, height, imgData, noTranslate, invert, noClip, cached)
{
   var originX = this.origin.x;
   var originY = this.origin.y;

   if (noTranslate === true)
   {
      originX = 0;
      originY = 0;
   }

   var x1 = Math.round(x + originX);
   var x2 = x1 + width - 1;
   var y1 = Math.round(y + originY);
   var y2 = y1 + height - 1;

   var clipped = null;
   if (noClip === true)
   {
      clipped = [{left: x1, right: x2, top: y1, bottom: y2}];
   }
   else
   {
      clipped = this.clipRegion(x1, y1, x2, y2);
   }

   if (clipped)
   {
      var textureData = cached ? this.cacheTextureData : this.imageTextureData;
      var cwidth = p2j.screen.webgl.gl.drawingBufferWidth;
      var tlen = textureData.length;

      for (var i = 0; i < clipped.length; i++)
      {
         var clip = clipped[i];
         x1 = clip.left;
         y1 = clip.top;
         x2 = clip.right;
         y2 = clip.bottom;

         var startOffset = y1 * cwidth + x1;
         for (var j = 0; j < height; j++)
         {
            var srcLine = invert === true ? width * (height - j - 1) : width * j;
            var toffset = (startOffset + j * cwidth) * 4;
            if (toffset >= tlen)
            {
               break;
            }

            var srcLineEnd = srcLine * 4 + Math.min(width * 4, tlen - toffset);
            textureData.set(imgData.subarray(srcLine * 4, srcLineEnd), toffset);
         }

         var txid = cached ? 0 : 1;
         this.pushTriangles([[x1, y1, x2 + 1, y1, x1, y2 + 1, txid, 0, 0, 0],
                             [x1, y2 + 1, x2 + 1, y1, x2 + 1, y2 + 1, txid, 0, 0, 0]]);
      }
   }
};

y1 * cwidth + x1

         var clip = clipped[i];
         x1 = clip.left;
         y1 = clip.top;
         x2 = clip.right;
         y2 = clip.bottom;

         var startOffset = y1 * cwidth + x1;

var srcLineEnd = srcLine * 4 + Math.min(width * 4, tlen - toffset);
textureData.set(imgData.subarray(srcLine * 4, srcLineEnd), toffset);

Sergey Ivanovskiy wrote:

It seems that we place the top part of the target image that should be cut off iff we suppose that clip intersects only the bottom part of our target image.

You are right, j and height must be adjusted accordingly, when the source image is clipped. Can you go ahead and fix this?

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

It seems that we place the top part of the target image that should be cut off iff we suppose that clip intersects only the bottom part of our target image.

You are right, j and height must be adjusted accordingly, when the source image is clipped. Can you go ahead and fix this?

Btw, this seems to be behind the many draw artifacts. Like those related to tooltips.

Sergey, also note that similar logic must be applied to x-axis, when the image is clipped from left or right.

Hynek, I tried to fix drawImage rev 11866(3821c) but the case of invert == true was not changed. Please review the code. It seems that invert parameter is not used now. Tooltips have the same issue now, they leave toolptip's ghosts after disappearing.

Sergey Ivanovskiy wrote:

Hynek, I tried to fix drawImage rev 11866(3821c) but the case of invert true was not changed. Please review the code. It seems that invert parameter is not used now. Tooltips have the same issue now, they leave toolptip's ghosts after disappearing.

When I disable clipping in drawImage completely, the ghosting goes away, so it must be related or somehow connected to clipping. Can you look at this? I modified the clipping condition to be if (noClip = true || true).

Indeed, invert is no longer used. Can you remove it?

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I tried to fix drawImage rev 11866(3821c) but the case of invert true was not changed. Please review the code. It seems that invert parameter is not used now. Tooltips have the same issue now, they leave toolptip's ghosts after disappearing.

When I disable clipping in drawImage completely, the ghosting goes away, so it must be related or somehow connected to clipping. Can you look at this? I modified the clipping condition to be if (noClip = true || true).

I added drawImage test that shows that drawImage doesn't work with clip regions. If clip region is from left top corner of the image, then it cuts correctly by the given clip area. But adding another clip area doesn't work as expected.

According to Draw Image Test committed revision 11868 (3821c) should fix drawImage. The idea was to copy full image on the texture array and then clip it with help of triangles mesh as it was done previously but was overcommitted by your code,

Committed revision 11869 (3821c) cleaned the code.

Sergey Ivanovskiy wrote:

According to Draw Image Test committed revision 11868 (3821c) should fix drawImage.

The idea was to copy full image on the texture array and then clip it with help of triangles mesh

Sergey, note that this may cause incorrect image data fetched from the texture when unclipped coordinates of two images rendered in close proximity will overlap. Also note that copying unclipped image data introduces unneeded overhead.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

According to Draw Image Test committed revision 11868 (3821c) should fix drawImage.

The idea was to copy full image on the texture array and then clip it with help of triangles mesh

Sergey, note that this may cause incorrect image data fetched from the texture when unclipped coordinates of two images rendered in close proximity will overlap. Also note that copying unclipped image data introduces unneeded overhead.

I didn't succeed when following this way because clipping parts of the image were not displayed but using triangles mesh the result of Draw Image Test was correct.

What is the open list of drawing artifacts?

Greg Shah wrote:

What is the open list of drawing artifacts?

The remaining issues:

• Window system menu background has wrong color.
• Panel size drawing differences. This can be seen in Hotel GUI, in the login form (size of borders and panels around fillins), in the calendar and in the main window in the frames inside the tabs.
• Wrong Chrome smoothing when system scaling changed to non-default.

Sergey, do you see anything else? Also, can you look at the panel sizes?

In what cases do we expect the current webgl implementation to out-perform 2d canvas?

Do we have a reasonable expectation that we can improve the current approach to make webgl much faster?

I can add these particular issues: Check-in date calendar (Hotel GUI) displays only left and top sides of the border of the selected cell, a background of a selected cell is displayed incorrectly after a selection cell has been changed, on the Rooms tab the Floor combo box label isn't displayed, the currently focused button's background is displayed with grid patten after the input focus is moved out.

Sergey, Hynek,

There are the UI artifacts with big customer's application too. The frame around message or dialog is not correct in right and bottom parts. Also I see distortions on the right side of the child window (white vertical band over application window ~10 pixels width). Do you see these artifacts?

Eugenie Lyzenko wrote:

Also I see distortions on the right side of the child window (white vertical band over application window ~10 pixels width). Do you see these artifacts?

Eugenie, please post a screen shot.

Eugenie Lyzenko wrote:

Note I have erased/colorized the application specific data here. Do not be surprised, this is not an GUI issue. Only marked in red.

This looks similar to #4955-400.

Sergey Ivanovskiy wrote:

LoginFillins-2Dx400.png

Sergey, is this from your system? Does it mean, you don't see it?

Yes, it is from my system if renderer is 2D canvas. The picture is named LoginFillins-2Dx400.png. I think that these differences between 2D and webgl renderers were due to the extra pixels added to the triangles mesh. I am still working to fix these differences but the idea is still the same one, extra pixels.

Sergey Ivanovskiy wrote:

I think that these differences between 2D and webgl renderers were due to the extra pixels added to the triangles mesh.

This should be easy to confirm, no? Render a rect of a known size and just measure the outcome.

Sergey, FYI, I modified text rendering according to your idea. Instead of drawing each individual as an image put in webgl texture, the texts are rendered directly on the offscreen canvas during flash. I needed this to resolve the "wrong menu background" (and related issues). Besides, it has positive effect on performance. I expect this to be checked in either later tonight or sometime tomorrow, I have to resolve a regression in the implementation first.

Sergey, did you find the cause of the wrong rectangle size?

Hynek Cihlar wrote:

Sergey, did you find the cause of the wrong rectangle size?

I fixed strokeRect and draw3DRect code in rev 11872 to draw the same rectangle size as 2D canvas renderer draws. These changes are respectively

-      this.drawLineSegment(x, y, x + width - 1, y, renderer);
-      this.drawLineSegment(x + width - 1, y, x + width - 1, y + height - 1, renderer);
-      this.drawLineSegment(x + width - 1, y + height - 1, x, y + height - 1, renderer);
-      this.drawLineSegment(x, y + height - 1, x, y, renderer);// close the path
+      this.drawLineSegment(x, y, x + width, y, renderer);
+      this.drawLineSegment(x + width, y, x + width, y + height, renderer);
+      this.drawLineSegment(x + width, y + height, x, y + height, renderer);
+      this.drawLineSegment(x, y + height, x, y, renderer);// close the path

    // the Java implementation draws 1 pixel larger in each dimension
-//   width  = width + 1;
-//   height = height + 1;
+   width  = width + 1;
+   height = height + 1;

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, can you look at the following issue?

The window message area sometimes doesn't fit the window bounds. This depends on the actual window size. To replicate this issue, resize the window multiple times. See the attached screenshots, the first shows the message area before resize, the second after resize.

Tested on revision 11871.

I can reproduce this issue with demo/simple_windows.p and 2D renderer. Resizing (squeezing) window leaves the message area unchanged.

Did you mean that this issue could be due to WindowGuiImpl and MessageAreaGuiImpl

Actually I was assuming this was caused by webgl renderer. Even if the improper message area is caused by Java driver there seems to be a clipping issue, the area reaches out of the window area. Compare your image and the images I attached.

Can you create new Redmine issue for the wrong size as this looks to be out of the scope of webgl?

Yes, #5045 is for fixing this issue.

On 12/21/20 11:11 AM, Serg Ivanovskiy wrote:

Hynek,

Are there sub tasks within #4955 that can be shared with me? Line primitives are complete.

I am planning to switch to another tasks. #5045 seems a high priority task too.
Thanks,
Sergey

Sergey, please look at the multiple selected combo box items. When you open up a compo box and quickly run through the items with mouse cursor, to cause rapid selection changes, sometimes multiple items get selected.

Ok. I committed the current revision 11884 (3821c) for the case of disabled pixel manipulations and tested these changes with Hotel GUI.

I found that this program ./selection_list/sel_list_test9.p (from the old testcases project) doesn't work as expected by 4GL. The target selection list is displayed collapsed by the swing and web clients. Hynek, it seems that 4GL doesn't support MULTIPLE attribute for combo-box widget, but selection-list widget has this attribute, correct?

Sergey Ivanovskiy wrote:

Hynek, it seems that 4GL doesn't support MULTIPLE attribute for combo-box widget, but selection-list widget has this attribute, correct?

Yes, you can always select only single item from plain combo-box.

Hynek Cihlar wrote:

Sergey, please look at the multiple selected combo box items. When you open up a compo box and quickly run through the items with mouse cursor, to cause rapid selection changes, sometimes multiple items get selected.

I observed only that two items were colored with more pale background for a moment. Did you mean this issue?

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please look at the multiple selected combo box items. When you open up a compo box and quickly run through the items with mouse cursor, to cause rapid selection changes, sometimes multiple items get selected.

I observed only that two items were colored with more pale background for a moment. Did you mean this issue?

Yes. The selection should be exclusive to only single item.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please look at the multiple selected combo box items. When you open up a compo box and quickly run through the items with mouse cursor, to cause rapid selection changes, sometimes multiple items get selected.

I observed only that two items were colored with more pale background for a moment. Did you mean this issue?

Yes. The selection should be exclusive to only single item.

Hynek, could you record and post a short film that would show how this issue reproduced on your system with graphics card? I would like to estimate how it worked on different systems.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, could you record and post a short film that would show how this issue reproduced on your system with graphics card? I would like to estimate how it worked on different systems.

Sergey, please see the attachment.

Thank you. You probably have a rapid graphics processor because in my tests these artifacts are cleared. I didn't reproduce them so they were present constantly after the mouse pointer was out of the drop down area.

Sergey Ivanovskiy wrote:

Thank you. You probably have a rapid graphics processor because in my tests these artifacts are cleared. I didn't reproduce them so they were present constantly after the mouse pointer was out of the drop down area.

Sergey, OK, I will look at this. I'm not aware of anything else assignable to you, besides those issues I'm currently working on now.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Thank you. You probably have a rapid graphics processor because in my tests these artifacts are cleared. I didn't reproduce them so they were present constantly after the mouse pointer was out of the drop down area.

Sergey, OK, I will look at this. I'm not aware of anything else assignable to you, besides those issues I'm currently working on now.

It seems that if dash line logic is thrown out from the fragment shader, then this issue is not reproduced on my system. Please test if it is true on your system.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Thank you. You probably have a rapid graphics processor because in my tests these artifacts are cleared. I didn't reproduce them so they were present constantly after the mouse pointer was out of the drop down area.

Sergey, OK, I will look at this. I'm not aware of anything else assignable to you, besides those issues I'm currently working on now.

It seems that if dash line logic is thrown out from the fragment shader, then this issue is not reproduced on my system. Please test if it is true on your system.

Yes, same here. In fact it is this expression m <= dash_pattern.x. dash_pattern is a varying. Varying values assigned in vertex shader are interpolated. Perhaps the interpolation oscillates in some circumstances.

Status report.

There are three open issues:

• Window system menu background has wrong color, and related set of issues.
• Combo box item selection artifacts.
• Performance testing and possible improvements.

I'm currently working on the first item. In order to fix this I had to rewrite text rendering. The implementation is in place, but there are some remaining problems with cached mode. I also included some performance improvements to limit the number of screen buffer reads. Estimated time to completion 1 MD.

I will be on vacation starting tomorrow returning on January 4.

Planning to rework dash lines and to fix Combo box item selection artifacts. It seems that we get these issues when dash line code is running with texture logic fragment code, because dash line variables have varying type. It makes sense to try special processing code for dash lines following DRAW_CACHED that breaks the drawing circle by invoking flash().

There are repeated triangles when flash() is invoked. These repeated triangles can have different vertex attributes. It can be the root cause of these issues. It seems that this mesh of triangles should cover a given point only by a one triangle, otherwise we can get unpredicted effects.

Hynek Cihlar wrote:

Status report.

There are three open issues:

• Combo box item selection artifacts.

This issue was related to repeated triangles (#4955-431 and #4955-432) and should be fixed in the committed rev. 11890 (3821c). Please review.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Status report.

There are three open issues:

• Combo box item selection artifacts.

This issue was related to repeated triangles (#4955-431 and #4955-432) and should be fixed in the committed rev. 11890 (3821c). Please review.

What is the source of the duplicated triangles? What are the drawing primitives that cause these?

Is the added call to viewport needed? Viewport is already called each time the canvas is resized.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Status report.

There are three open issues:

• Combo box item selection artifacts.

This issue was related to repeated triangles (#4955-431 and #4955-432) and should be fixed in the committed rev. 11890 (3821c). Please review.

What is the source of the duplicated triangles? What are the drawing primitives that cause these?

Why do you think that it could be drawing primitives? It can be repeated drawing primitives with different line styles, different foreground and background colors

Is the added call to viewport needed? Viewport is already called each time the canvas is resized.

Yes. If this method is already invoked, then it can be omitted (eliminated).

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Status report.

There are three open issues:

• Combo box item selection artifacts.

This issue was related to repeated triangles (#4955-431 and #4955-432) and should be fixed in the committed rev. 11890 (3821c). Please review.

What is the source of the duplicated triangles? What are the drawing primitives that cause these?

Why do you think that it could be drawing primitives? It can be repeated drawing primitives with different line styles, different foreground and background colors

The reason I ask is so that we can identify the source of the problem. First we can hit this with a different use case and second, we want to eliminate the calls to flash as much as possible. Ideally there should be just a single flash for every draw cycle.

Is the added call to viewport needed? Viewport is already called each time the canvas is resized.

Yes. If this method is already invoked, then it can be omitted (eliminated).

OK, please remove it.

Hynek, did you change DefaultPathRenderer.strokeLine when you committed extra pixels to coordinates?

            if (width == 1)
            {
               bx1 = x1;
               by1 = y1 + 1;
               cx1 = x1;
               cy1 = y1;
               bx2 = x2 + 1; //adding an extra pixel
               by2 = y2 + 1;
               cx2 = x2 + 1; // looks incorrect because (bx2 + cx2)/2 should be ax2
               cy2 = y2;
               ax1 = x1;
               ay1 = y1 + 0.5;
               ax2 = x2 + 0.5; // middle point
               ay2 = y2 + 0.5;
            }
            else if (width == 0.5)
            {
               bx1 = x1;
               by1 = y1 + 0.5;
               cx1 = x1;
               cy1 = y1;
               bx2 = x2;
               by2 = y2 + 0.5;
               cx2 = x2;
               cy2 = y2;
               ax1 = x1;
               ay1 = y1 + 0.25;
               ax2 = x2;
               ay2 = y2 + 0.25;
            }
            else
            {
               bx1 = x1;
               by1 = y1 + w;
               cx1 = x1;
               cy1 = y1 - w;
               bx2 = x2 + w;
               by2 = y2 + w;
               cx2 = x2 + w;
               cy2 = y2 - w;
               ax1 = x1;
               ay1 = y1;
               ax2 = x2;
               ay2 = y2;
            }
         }
         else if (a == 0)
         {
            l = Math.abs(b);
            lx = 0.0;
            ly = 1.0;
            if (width == 1)
            {
               bx1 = x1;
               by1 = y1;
               cx1 = x1 + 1;
               cy1 = y1;
               bx2 = x2;
               by2 = y2 + 1;
               cx2 = x2 + 1;
               cy2 = y2 + 1;
               ax1 = x1 + 0.5;
               ay1 = y1;
               ax2 = x2 + 0.5;
               ay2 = y2 + 0.5;
            }
            else if (width == 0.5)
            {
               bx1 = x1;
               by1 = y1;
               cx1 = x1 + 0.5;
               cy1 = y1;
               bx2 = x2;
               by2 = y2;
               cx2 = x2 + 0.5;
               cy2 = y2;
               ax1 = x1 + 0.25;
               ay1 = y1;
               ax2 = x2 + 0.25;
               ay2 = y2;
            }
            else
            {
               bx1 = x1 - w;
               by1 = y1 - w;
               cx1 = x1 + w;
               cy1 = y1 - w;
               bx2 = x2 - w;
               by2 = y2 + w;
               cx2 = x2 + w;
               cy2 = y2 + w;
               ax1 = x1;
               ay1 = y1;
               ax2 = x2;
               ay2 = y2;
            }
         }
         else
         {
            l = Math.sqrt(a*a + b*b);
            lx = a/l;
            ly = b/l;
            //normal vector n = 1/l * [-b, a];
            bx1 = x1 - w*b/l;
            by1 = y1 + w*a/l;
            cx1 = x1 + w*b/l;
            cy1 = y1 - w*a/l;
            bx2 = x2 - w*b/l;
            by2 = y2 + w*a/l;
            cx2 = x2 + w*b/l;
            cy2 = y2 - w*a/l;
            ax1 = x1;
            ay1 = y1;
            ax2 = x2;
            ay2 = y2;
         }

(bx1 + cx1) / 2 = ax1
(bx2 + cx2) / 2 = ax2

(by1 + cy1) / 2 = ay1
(by2 + cy2) / 2 = ay2

Sergey Ivanovskiy wrote:

Hynek, did you change DefaultPathRenderer.strokeLine when you committed extra pixels to coordinates?

No, I didn't. But I think I asked you to review these places and modify accordingly.

OK. Fixing this issue.

Sergey, I just found that LineStrokesWebGL.DefaultPathRenderer.strokeLine sometimes returns NaN as one of the coordinates in the returned triangles. This is with revision 11871, tested on uast/demo/demo_widgets.p. For now I'm skipping these triangles in the output. Can you look at this?

Also the function sometimes return fractional coordinate values. The returned coordinates should always be whole numbers as they represent (logical) pixels.

Yes, the calculated coordinates in #4955-437 for horizontal and vertical lines were incorrect because they should represent coordinates of vertices that were results of translations of original line segment along its normal vector. I will check if there is NaN coordinates in my version and then commit these changes.

Hynek, please look at the committed revision 11899 - 11890 (3821c). Nan coordinates issue is not reproduced with these changes. My concern is only that I corrected triangles by probing and testing. I have no a systematic approach how to correct coordinates of triangles produced for a line segment.

Sergey Ivanovskiy wrote:

Nan coordinates issue is not reproduced with these changes.

If you haven't, please try uast/demo/demo_widgets.p.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Nan coordinates issue is not reproduced with these changes.

If you haven't, please try uast/demo/demo_widgets.p.

Yes, I tested this example uast/demo/demo_widgets.p and logged triangles in this method WebGLCanvasRenderer.prototype.pushTriangles = function(triangles). Please describe this issue more thoroughly (how to reproduce, to test) so we didn't waste the working time.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Nan coordinates issue is not reproduced with these changes.

If you haven't, please try uast/demo/demo_widgets.p.

Yes, I tested this example uast/demo/demo_widgets.p and logged triangles in this method WebGLCanvasRenderer.prototype.pushTriangles = function(triangles). Please describe this issue more thoroughly (how to reproduce, to test) so we didn't waste the working time.

There were no special steps to see this. Perhaps this was resolved in newer revisions, I will retest when I finish merge with latest changes.

OK, please retest. The root cause of NaN value could be the protected method addJointBetweenSegments of DefaultPathRenderer. The last method is based on the rectangle constructed symmetrically around the line segment and it extends its outer and internal bounds to meet the similar neighbour rectangle constructed around the adjacent line segment.

3821c revision 11906 improves text rendering, provides several fixes and performance improvements.

Sergey, please review the changes. Note I had to modify your changes in, please review those changes, too.

Hynek Cihlar wrote:

3821c revision 11906 improves text rendering, provides several fixes and performance improvements.

I tested the changes with several uast test cases, Hotel GUI and some very limited number of screens of a large GUI app. I need to clear some runtime issues (not related to UI) before I can test the large GUI app fully.

Sergey Ivanovskiy wrote:

Hynek, I am still studying your code changes but the test page shows the performance degradation. You can use this standalone test page. It must be extracted into the FWD project root and it contains dojo sources. For an example, for my local environment this page is accessed by ~/projects/3821c/perf_test.html.

Sergey, did you mean perf_page.html from the archive you attached? When I launch it, I just get a blank page. In Firefox console I see a couple of harmless warnings, in Chrome console I see the following:

3perf_page.html:1 GET file:///home/hc/gcd/p2j_repo/3821c/$%7BwebRoot%7D/common/beep.wav net::ERR_FILE_NOT_FOUND
DataGrid.js:15 Uncaught DOMException: Blocked a frame with origin "null" from accessing a cross-origin frame.
    at HTMLIFrameElement.f.onload (file:///home/hc/gcd/p2j_repo/3821c/dojo-1.10.0/dojox/grid/DataGrid.js:15:110409)
f.onload @ DataGrid.js:15
load (async)
... some useless entries ...
2p2j.keyboard.js:383 Uncaught TypeError: p2j.socket.sendKeyCode is not a function
    at sendKeyCode (p2j.keyboard.js:383)
    at sendKey (p2j.keyboard.js:349)
    at handleTypedKey (p2j.keyboard.js:1166)
    at handleKeyPress (p2j.keyboard.js:1259)
    at HTMLDocument.onkeyup (p2j.keyboard.js:1307)
sendKeyCode @ p2j.keyboard.js:383
sendKey @ p2j.keyboard.js:349
handleTypedKey @ p2j.keyboard.js:1166
handleKeyPress @ p2j.keyboard.js:1259
onkeyup @ p2j.keyboard.js:1307

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek, I am still studying your code changes but the test page shows the performance degradation. You can use this standalone test page. It must be extracted into the FWD project root and it contains dojo sources. For an example, for my local environment this page is accessed by ~/projects/3821c/perf_test.html.

Sergey, did you mean perf_page.html from the archive you attached? When I launch it, I just get a blank page. In Firefox console I see a couple of harmless warnings, in Chrome console I see the following:

[...]

Yes, the right mouse popup menu can be opened. It exports the same functionality as the FWD application index.html page. Exceptions shouldn't prevent from opening p2j.perf_tests.js

But the performance tests became showing the worst results, at least for my environment. It needs to debug what can be a bottleneck now.

With my latest changes in revision 11906, besides the regressions Sergey reports, I noticed missing borders in one of the large app frames, shifted button labels in a message box and area covered by tooltip window sometimes not redrawn.

On a brighter side. I measured screen open times of one of the large app complex screens. The times went down from around 2.4 s to around 1.8 s on my system. Also I didn't spot any of the previously reported issues.

Hynek Cihlar wrote:

With my latest changes in revision 11906, besides the regressions Sergey reports, I noticed missing borders in one of the large app frames, shifted button labels in a message box and area covered by tooltip window sometimes not redrawn.

The tooltip problem is already present with 2D canvas renderer.

The performance regression mentioned in #4955-449 is resolved in 3821c revision 11910.

I think the ChUI web client is broken with the recent changes:

Cookie “Auth-5d5f506a-15c7-4448-888c-9a2626decc3b” will be soon rejected because it has the “SameSite” attribute set to “None” or an invalid value, without the “secure” attribute. To know more about the “SameSite“ attribute, read https://developer.mozilla.org/docs/Web/HTTP/Headers/Set-Cookie/SameSite index.html
This page uses the non standard property “zoom”. Consider using calc() in the relevant property values, or using “transform” along with “transform-origin: 0 0”. index.html
Loading failed for the <script> with source “https://localhost:36201/client/p2j.strokes.webgl.js”. index.html:119:1
Loading failed for the <script> with source “https://localhost:36201/client/p2j.canvas_renderer_webgl.js”. index.html:120:1
Loading failed for the <script> with source “https://localhost:36201/client/p2j.perf_tests.js”. index.html:131:1
Uncaught SyntaxError: missing } after property list index.html:174:45note: { opened at line 140, column 21index.html:140:21
Cannot play media. No decoders for requested formats: audio/wav, audio/x-wav index.html

This is in 3821c rev 11912. I think Roger was able to use the web client last week so it is probably a recent change.

Greg Shah wrote:

I think the ChUI web client is broken with the recent changes:

Fixed in 3821c revision 11915.

Hynek Cihlar wrote:

Sergey, please check the rounded rect in uast/rectangle/rectangle-tests.p, they don't come up as expected. See the attached screen shots of webgl and swing renderers.

OK. There are another issues too. I checked Miscellaneous Tests>Draw Primitive Test and Miscellaneous Tests>Draw Primitive Test 2 and found that they were broken now. The broken primitives are related to drawings of ellipse, rectangle and polygon.

Hynek Cihlar wrote:

With my latest changes in revision 11906, besides the regressions Sergey reports, I noticed missing borders in one of the large app frames, shifted button labels in a message box and area covered by tooltip window sometimes not redrawn.

These issues are resolved as of 3821c revision 11919.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please check the rounded rect in uast/rectangle/rectangle-tests.p, they don't come up as expected. See the attached screen shots of webgl and swing renderers.

OK. There are another issues too. I checked Miscellaneous Tests>Draw Primitive Test and Miscellaneous Tests>Draw Primitive Test 2 and found that they were broken now. The broken primitives are related to drawings of ellipse, rectangle and polygon.

Sergey, do you need a hand with these issues?

Here are screen show times of a complex screen of a large customer application. All the results were gathered on bare metal with 3821c revision 11921.

                                             2d cached      2d noncached     webgl cached      webgl noncached
Linux Chrome Ryzen 5 3600X Radeon RX 590     1.80 s         1.80 s           1.75 s            1.60 s
Linux Chrome i5-6440HQ Intel HD 530          1.80 s         1.85 s           1.80 s            1.70 s
Chrome Google Pixel 1                        4.00 s         4.70 s           2.50 s            3.00 s

Eugenie, Sergey, can you both test the screen show times on your systems?

Hynek Cihlar wrote:

Here are screen show times of a complex screen of a large customer application. All the results were gathered on bare metal with 3821c revision 11921.

[...]

Eugenie, Sergey, can you both test the screen show times on your systems?

Hynek,

What was your hardware acceleration setting in Chrome browser during tests? On or Off?

Eugenie Lyzenko wrote:

Hynek,

What was your hardware acceleration setting in Chrome browser during tests? On or Off?

I left it on the default ON.

Hynek Cihlar wrote:

Here are screen show times of a complex screen of a large customer application. All the results were gathered on bare metal with 3821c revision 11921.

[...]

Eugenie, Sergey, can you both test the screen show times on your systems?

I have for Linux Chrome accelerated on i9-9980HK (max perf. mode) + Radeon Pro WX3200:

2d cached      2d noncached     webgl cached      webgl noncached
1.30 s         1.35 s           1.33 s            1.45 s

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Here are screen show times of a complex screen of a large customer application. All the results were gathered on bare metal with 3821c revision 11921.

[...]

Eugenie, Sergey, can you both test the screen show times on your systems?

I have for Linux Chrome accelerated on i9-9980HK (max perf. mode) + Radeon Pro WX3200:

How do you measure the times?

Hynek Cihlar wrote:

Eugenie Lyzenko wrote:

Hynek Cihlar wrote:

Here are screen show times of a complex screen of a large customer application. All the results were gathered on bare metal with 3821c revision 11921.

[...]

Eugenie, Sergey, can you both test the screen show times on your systems?

I have for Linux Chrome accelerated on i9-9980HK (max perf. mode) + Radeon Pro WX3200:

How do you measure the times?

https://sekundomer.net/ running on another machine.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please check the rounded rect in uast/rectangle/rectangle-tests.p, they don't come up as expected. See the attached screen shots of webgl and swing renderers.

OK. There are another issues too. I checked Miscellaneous Tests>Draw Primitive Test and Miscellaneous Tests>Draw Primitive Test 2 and found that they were broken now. The broken primitives are related to drawings of ellipse, rectangle and polygon.

I cleaned the code rev 11927 related to the recent changes in rev 11906(3821c), the combo box issues were not reproduced now. But flash(true) doesn't work and can't be used now.

Did you observe this warning message java.lang.NullPointerException when starting the FWD server?

212 [main] INFO org.eclipse.jetty.server.Server - jetty-9.4.22.v20191022; built: 2019-10-22T13:37:13.455Z; git: b1e6b55512e008f7fbdf1cbea4ff8a6446d1073b; jvm 1.8.0_265-8u265-b01-0ubuntu2~16.04-b01
246 [main] WARN org.eclipse.jetty.webapp.WebInfConfiguration - Can't generate resourceBase as part of webapp tmp dir name: java.lang.NullPointerException
300 [main] INFO org.eclipse.jetty.webapp.StandardDescriptorProcessor - NO JSP Support for /admin, did not find org.eclipse.jetty.jsp.JettyJspServlet

Sergey Ivanovskiy wrote:

Did you observe this warning message java.lang.NullPointerException when starting the FWD server?

Yes, I think I've seen this. But didn't have chance to look at it closer.

Eugenie Lyzenko wrote:

https://sekundomer.net/ running on another machine.

To get more precise times, uncomment the line console.log("Window draw time... in p2j.screen.js. This will output the ms time since last mouse down at the end of each draw cycle.

Sergey Ivanovskiy wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, please check the rounded rect in uast/rectangle/rectangle-tests.p, they don't come up as expected. See the attached screen shots of webgl and swing renderers.

OK. There are another issues too. I checked Miscellaneous Tests>Draw Primitive Test and Miscellaneous Tests>Draw Primitive Test 2 and found that they were broken now. The broken primitives are related to drawings of ellipse, rectangle and polygon.

I cleaned the code rev 11927 related to the recent changes in rev 11906(3821c), the combo box issues were not reproduced now. But flash(true) doesn't work and can't be used now.

Anyway, I'm afraid the extra call to flash was not addressing the original combo box issue. It was just masking it by forcing an extra redraw.

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I cleaned the code rev 11927 related to the recent changes in rev 11906(3821c), the combo box issues were not reproduced now. But flash(true) doesn't work and can't be used now.

Anyway, I'm afraid the extra call to flash was not addressing the original combo box issue. It was just masking it by forcing an extra redraw.

The root cause of combo box issue was repeated triangles (rectangles) with different vertex attributes in the input for webGLContext.drawArrays. These invocations split those triangles into separate invocations of webGLContext.drawArrays. I think this issue can appear in another unknown scenarios.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

I cleaned the code rev 11927 related to the recent changes in rev 11906(3821c), the combo box issues were not reproduced now. But flash(true) doesn't work and can't be used now.

Anyway, I'm afraid the extra call to flash was not addressing the original combo box issue. It was just masking it by forcing an extra redraw.

The root cause of combo box issue was repeated triangles (rectangles) with different vertex attributes in the input for webGLContext.drawArrays. These invocations split those triangles into separate invocations of webGLContext.drawArrays. I think this issue can appear in another unknown scenarios.

Shouldn't be the repeated triangles be prevented in the first place or was it a valid case?

It seems it can be a valid case when something is painted over exactly the same rectangle. There were also issues in strokeLine that produced repeated triangles.

Sergey Ivanovskiy wrote:

It seems it can be a valid case when something is painted over exactly the same rectangle. There were also issues in strokeLine that produced repeated triangles.

What draw primitives cause these duplicated triangles? When you keep only the last repeated triangle in the triangles array does the issue go away?

I did more performance testing with other system configurations, in particular bare metal Ubuntu, bare metal Windows, VirtualBox Ubuntu, VirtualBox Windows, with Chrome, Edge and Firefox. All tests done with browse defaults, i.e. use GPU acceleration where available was always on. All tests performed with latest stable browsers. Below are some notable points from the tests.

• In all tests webgl non-cached performs better than webgl cached.
• In all tests 2d canvas cached performs better than 2d canvas non-cached.
• Non-cached mode increases draw message size by about 5% (140000 bytes vs 120000 bytes on a complex screen).
• In general Chrome performs better than Firefox. The gap is bigger on Windows.
• Edge gives similar performance as Chrome.
• VirtualBox Linux Chrome performs about the same as bare metal (with or without VirtualBox 3D acceleration and with or without guest additions).
• VirtualBox Linux Firefox performs worse than Chrome. The performance worsens with increasing canvas size (i.e. higher resolution, larger browse window). I noticed performance differences as much as 30 %.
• Bare metal Windows Firefox, webgl non-cached outperforms 2d canvas cached by about 30%.
• Bare metal Windows Chrome, webgl non-cached outperforms 2d canvas cached by about 5%.
• Bare metal Windows Edge, webgl non-cached outperforms 2d canvas cached by about 5%.
• Bare metal Linux Cinnamon SW renderer fails to initialize webgl context on Firefox.

According to the above and my previous tests webgl non-cahed gives a comparable or better performance to 2d canvas, except for the VirtualBox Linux Firefox case. Webgl renderer should be always configured in non-cached mode.

The default renderer should be webgl in non-cached mode while the user should be allowed to select the renderer at will (the selection should be applied without restarting the client). I'm not certain we can easily do the automated renderer selection as the overall performance is influenced not just by the performance of the individual draw primitives, but also by other parts of the renderer logic, particular states of the canvases, actual window size, instant system load. For illustration, for the VirtualBox Linux Firefox case the GUI performance tests show slightly better performance of the webgl primitives.

In the case where webgl context fails to initialize the init logic must fail over to 2d renderer.

Hynek Cihlar wrote:

• VirtualBox Linux Firefox performs worse than Chrome. The performance worsens with increasing canvas size (i.e. higher resolution, larger browse window). I noticed performance differences as much as 30 %.

This is caused by the call to drawImage when the webgl rendered image is copied over to the offscreen canvas just before the canvases get resized. In this case drawImage took about 500 ms to complete on my system, while typical times are around 10 ms. In the test scenario the resize is triggered multiple times yet before the legacy window is made visible, so perhaps we could eliminate these calls when the window is not displayed.

The default renderer should be webgl in non-cached mode while the user should be allowed to select the renderer at will (the selection should be applied without restarting the client). I'm not certain we can easily do the automated renderer selection as the overall performance is influenced not just by the performance of the individual draw primitives, but also by other parts of the renderer logic, particular states of the canvases, actual window size, instant system load. For illustration, for the VirtualBox Linux Firefox case the GUI performance tests show slightly better performance of the webgl primitives.

In the case where webgl context fails to initialize the init logic must fail over to 2d renderer.

I'm OK with this plan. We should retain the ability to set the default renderer in the directory, but absent that value the default should be webgl non-cached.

Also, we should consider that webgl cached mode might be faster in cases where the network is the bottleneck. This means we should retain the ability to set it in caching mode.

How quickly can you implement this?

What is the status of the rounded rectangles issue? That is the last bug, right?

Greg Shah wrote:

Also, we should consider that webgl cached mode might be faster in cases where the network is the bottleneck. This means we should retain the ability to set it in caching mode.

Yes, providing the value in the directory will override the default, i.e. cached for 2d canvas, non-cached for webgl. Also the UI should provide controls to allow the user to select both the renderer and the caching mode.

How quickly can you implement this?

About 2 MD.

Greg Shah wrote:

What is the status of the rounded rectangles issue?

Sergey?

That is the last bug, right?

We still have the combo box selection issue. In #4955-470 Sergey mentions that he can't reproduce it anymore, but I still see it. It seems to be less pronounced after revision 11906 though. I'm looking at this.

Btw. I found a new problem with paragraph text rendering, but this was a relatively simple fix and it has been resolved.

Hynek Cihlar wrote:

Greg Shah wrote:

What is the status of the rounded rectangles issue?

Sergey?

That is the last bug, right?

Sorry, I missed this question. Yes, the case looks odd for me because my tests p2j.perf_tests.js work properly iff they are running standalone, but they are failed iff they are running from this rectangle/rectangle-tests.p test. It can be related to the cache mode or clipping issues that aren't reproduced by these tests.

I just found an issue when switching renderer using the new taskbar popup menu. In Firefox when renderer is switched mouse enter messages don't get delivered to the app and widgets stop responding to mouse enter. This doesn't happen in Chrome.

Sergey, I'll need help with the combo box selection issue. I got out of ideas and Greg wants me to move to another task.

I don't think I was able to get any much further than you did when you were working on this. The problem is somehow related to webgl DefaultPathRenderer and its strokeLine method. The problematic part seems to be the code that outputs triangles below the code comment "corrected the output by adding extra pixels". When these triangles are not output in the webgl buffer, the issue goes away. Perhaps this is a combination of specific triangle coordinates (or the other line metadata) and the shaders implementations. I think it would be useful to identify these specific inputs and work from there.

Hynek Cihlar wrote:

I just found an issue when switching renderer using the new taskbar popup menu. In Firefox when renderer is switched mouse enter messages don't get delivered to the app and widgets stop responding to mouse enter. This doesn't happen in Chrome.

I couldn't replicate this since I initially observed this. Either this was caused by some of my local changes or it requires some other steps to appear.

Please watch for any suspicious mouse input behavior and let me know.

Hynek Cihlar wrote:

Sergey, I'll need help with the combo box selection issue. I got out of ideas and Greg wants me to move to another task.

I don't think I was able to get any much further than you did when you were working on this. The problem is somehow related to webgl DefaultPathRenderer and its strokeLine method. The problematic part seems to be the code that outputs triangles below the code comment "corrected the output by adding extra pixels". When these triangles are not output in the webgl buffer, the issue goes away.

It is just a line itself. This comment explains that this code takes into account different cases and it approximates the target line and is got by testing.

Perhaps this is a combination of specific triangle coordinates (or the other line metadata) and the shaders implementations. I think it would be useful to identify these specific inputs and work from there.

The root causes can be the same vertices with different attributes. I should reproduce this issue on my environment.

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey, I'll need help with the combo box selection issue. I got out of ideas and Greg wants me to move to another task.

I don't think I was able to get any much further than you did when you were working on this. The problem is somehow related to webgl DefaultPathRenderer and its strokeLine method. The problematic part seems to be the code that outputs triangles below the code comment "corrected the output by adding extra pixels". When these triangles are not output in the webgl buffer, the issue goes away.

It is just a line itself. This comment explains that this code takes into account different cases and it approximates the target line and is got by testing.

Yes, I'm not saying the implemented logic is incorrect. It just causes unexpected results. It will be a combination of the specific inputs and some other WebGL feature.

Sergey Ivanovskiy wrote:

Committed revision 11973 (3821c) should fix #4955-490. Please review.

Sergey, can you give more details about the mechanism of the fix?

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Committed revision 11973 (3821c) should fix #4955-490. Please review.

Sergey, can you give more details about the mechanism of the fix?

#4955-475

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Committed revision 11973 (3821c) should fix #4955-490. Please review.

Sergey, can you give more details about the mechanism of the fix?

#4955-475

There are other cases where "repeated" triangles occur. Do you know why those don't cause this issue?

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Hynek Cihlar wrote:

Sergey Ivanovskiy wrote:

Committed revision 11973 (3821c) should fix #4955-490. Please review.

Sergey, can you give more details about the mechanism of the fix?

#4955-475

There are other cases where "repeated" triangles occur. Do you know why those don't cause this issue?

Also in the commit message you mention "parallel webgl processing". Which processes did you have in mind?

I'm asking these questions to get better understanding of the real cause of this issue and to come up with a fix that doesn't require the additional expensive flashes.

Other than #5390, #5532 and #5582, is there any additional work or known issues related to webgl support? I want to create tasks for any such work and close this task.

Greg Shah wrote:

Other than #5390, #5532 and #5582, is there any additional work or known issues related to webgl support? I want to create tasks for any such work and close this task.

I don't know any new issue if the issue #4955-475 will be fixed in #5582.

Sergey Ivanovskiy wrote:

Greg Shah wrote:

Other than #5390, #5532 and #5582, is there any additional work or known issues related to webgl support? I want to create tasks for any such work and close this task.

I don't know any new issue if the issue #4955-475 will be fixed in #5582.

Neither do I, these three issues should be the remaining work.