Another good demonstration of how incredibly slow software is. Not 2x or even 10x, but rather 100x-1000x. Courtesy of our best practices. Zero regards for the user, CO2 or anything like that. Use libraries, frameworks and abstractions for everything. Not even understanding that sometimes (many?) this actually decreases maintainability, extensibility and performance.
The cherry on top is the level of ignorance on display in the github comments.
> Setting the technical merits of your suggestion aside though: peppering your comments with clauses like “it’s that simple” or “extremely simple” and, somewhat unexpectedly “am I missing something?” can be read as impugning the reader. Some folks may be a little put off by your style here. I certainly am, but I am still trying to process exactly why that is.
They really are missing several important things. For example, DirectWrite and similar APIs are optimised for all possible scenarios, including really obscure ones such as text with arbitrary transformations applied to it. Similarly, it can handle smoothly animating the size or position of the text, and can do so to subpixel accuracy. This is useful for making web browsers or flashy GUIs, and is unnecessary complexity for a terminal emulator, which in all practical cases renders fixed-width text aligned to a uniform grid. This really is trivial, and the Microsoft developers lost sight of this. It's easy to just assume that optimisation is impossible if you believe that the problem is too hard to optimise. They're wrong.
> it lays down glyphs in your "texture", by using a backing glyph atlas internally already. Basically the thing you suggest us to do, is already part of the framework we use.
Almost certainly it'll be sending one or more commands to the GPU per "glyph run". This is a well-known issue in game programming circles, and the solution is to bend over backwards to batch your calls. E.g.: sorting the characters into identical colours and drawing those all together in a single call can be faster than the naive left-to-right order. Ideally, you want create a texture (or two) with the colours, and use that as an input into a shader when drawing all of the characters in one go.
> DirectWrite doesn't necessarily cache glyphs between renderings.
That's silly.
> when the problem you have is caused by the number of calls and not the complexity to layout a couple ASCII letters.
So then they are aware of the problem with the number of draw calls. So why not optimise it?
> Also ClearType can't be trivially alpha blended making it impossible to render into a separate glyph atlas.
Not exactly true, and definitely not true if you rethink your rendering approach. E.g.: store horizontally stretched glyphs and then scale them down, finalising the ClearType in a pixel shader as the final step. As a bonus, this would make some special cases render more accurately. Also note that unlike the general-purpose case, terminals don't need every possible subpixel alignment, since the glyphs are always aligned to a regular grid.
> this is something WT will probably do as well in the future
No, it won't. Windows Terminal has been in development for years already, and clearly they haven't thought about performance at all, or even bothered to eliminate unnecessary std::string operations in their inner graphics loops, which is clownshoes.
This is what it boils down to: a typical terminal shows 130x100 = 13000 characters in its viewport, which is a uniform two dimensional rectangular grid. Updating this is simply a matter of drawing different content in those cells. That's it. In terms of graphics complexity, this is hilariously trivial. You can easily get hundreds of frames a second in pure CPU code. This is a task that doesn't even require advanced instructions like division.
Do not forget the part at 44:04 (for which the "epilepsy warning" is for really... or actually a few minutes later, but he starts around there to explain what follows).
The sad part here is that the Windows Terminal is a pretty old codebase originally written in C.
Most people would consider it fast enough as it is and not worth the effort to optimize/rewrite the rendering part because it would probably lead to marginal gains.
This is something I've heard way too many times, and it always turns wrong by several orders of magnitude.
From the sound of this is sounds like Microsoft should just clean slate and fire the entire Windows terminal team and management, and hire casey to rebuild a new team.
he sounds like he might not consider himself an expert on all of the elements, but he definitely has enough understanding and vision to know who to hire in, and get rid of the whole “acceptance of horrible performance” culture plaguing the current team.
Microsoft's culture simply doesn't include performance as something to aspire to. Having read through a decent amount of their code, I'm certain that performance issues don't get flagged in their internal code reviews. I say that I'm certain because the kind of code they ship to production is so fantastically, gloriously inefficient, that in my limited imagination I could not produce something so verbose and unnecessary even if I tried.
As a random example, I recently had to explain to an Azure tech that maybe, just maybe, it's a bad idea to force all PaaS networking to criss-cross back and forth between their availability zones (data centres) like a shoelace. He incredulously asked me why I even thought that was bad, and wondered if I was working on some sort of "high frequency trading app" or something.
No, not trading apps. Just apps. Ordinary, boring, LoB apps that I simply did not want to see run like cold treacle...
The lack of performance as an aspiration is certainly part of the problem, but it also feels like there is a certain level of lack of intuition for the subject matter. If they believe that they are doing great and you’d need a full PhD dissertation to improve, and they 1000x below the base expectation of what someone can do in 2 days without even doing basic profiling and optimization. Then the problem is deeper than not valuing performance, it also speaks to a team that severely lacks an awareness of what they are working with.
I feel like this is similar to situations I have seen where people “explain” why database queries take several minutes to return with excuses like “that makes sense because we have years worth of data stored in the database” when in reality it’s because someone doesn’t understand how a composite index works. Someone who has a bit of intuition for db’s instantly knows something is wrong, but if your team lacks anyone with that intuitions, your team will spend their time finding “explanations” for the bad performance instead of fixing it.
I made a comment here recently that one cause of this problem is that computer performance crosses something like 13 orders of magnitude: Everything from functions taking mere nanoseconds all the way up to multi-minute processing.
In the "middle" of that enormous range, it's easy to make an incorrect judgement of how long something ought to take. This came up with the Grand Theft Auto game loading issue that was discussed here. Many people made statements like: "JSON parsing is slow", which isn't even wrong, it's just that it's not that slow.
A significant part of "experience" is having a good feel for what's normal. But even there, experience can lead people astray: the more years of experience you have, the more your expectations are weighted by out-dated information! Again, this crosses many orders of magnitude. Rules of thumb from the 1990s are incorrect by a factor of a thousand, and averaging them into an overall skillset will lead to enormous mistakes.
I noticed this the other day; a program I was writing that produced a lot of output and with a lot of ANSI colour was much slower (like 10x) on the new Windows Terminal vs mintty.
Another good demonstration of how incredibly slow software is. Not 2x or even 10x, but rather 100x-1000x. Courtesy of our best practices. Zero regards for the user, CO2 or anything like that. Use libraries, frameworks and abstractions for everything. Not even understanding that sometimes (many?) this actually decreases maintainability, extensibility and performance.
The cherry on top is the level of ignorance on display in the github comments.
The mentioned GitHub issue: https://github.com/microsoft/terminal/issues/10362
So can anyone knowledgeable regarding text rendering comment on the last comments there?
Sure.
> Setting the technical merits of your suggestion aside though: peppering your comments with clauses like “it’s that simple” or “extremely simple” and, somewhat unexpectedly “am I missing something?” can be read as impugning the reader. Some folks may be a little put off by your style here. I certainly am, but I am still trying to process exactly why that is.
They really are missing several important things. For example, DirectWrite and similar APIs are optimised for all possible scenarios, including really obscure ones such as text with arbitrary transformations applied to it. Similarly, it can handle smoothly animating the size or position of the text, and can do so to subpixel accuracy. This is useful for making web browsers or flashy GUIs, and is unnecessary complexity for a terminal emulator, which in all practical cases renders fixed-width text aligned to a uniform grid. This really is trivial, and the Microsoft developers lost sight of this. It's easy to just assume that optimisation is impossible if you believe that the problem is too hard to optimise. They're wrong.
> it lays down glyphs in your "texture", by using a backing glyph atlas internally already. Basically the thing you suggest us to do, is already part of the framework we use.
Almost certainly it'll be sending one or more commands to the GPU per "glyph run". This is a well-known issue in game programming circles, and the solution is to bend over backwards to batch your calls. E.g.: sorting the characters into identical colours and drawing those all together in a single call can be faster than the naive left-to-right order. Ideally, you want create a texture (or two) with the colours, and use that as an input into a shader when drawing all of the characters in one go.
> DirectWrite doesn't necessarily cache glyphs between renderings.
That's silly.
> when the problem you have is caused by the number of calls and not the complexity to layout a couple ASCII letters.
So then they are aware of the problem with the number of draw calls. So why not optimise it?
> Also ClearType can't be trivially alpha blended making it impossible to render into a separate glyph atlas.
Not exactly true, and definitely not true if you rethink your rendering approach. E.g.: store horizontally stretched glyphs and then scale them down, finalising the ClearType in a pixel shader as the final step. As a bonus, this would make some special cases render more accurately. Also note that unlike the general-purpose case, terminals don't need every possible subpixel alignment, since the glyphs are always aligned to a regular grid.
> this is something WT will probably do as well in the future
No, it won't. Windows Terminal has been in development for years already, and clearly they haven't thought about performance at all, or even bothered to eliminate unnecessary std::string operations in their inner graphics loops, which is clownshoes.
This is what it boils down to: a typical terminal shows 130x100 = 13000 characters in its viewport, which is a uniform two dimensional rectangular grid. Updating this is simply a matter of drawing different content in those cells. That's it. In terms of graphics complexity, this is hilariously trivial. You can easily get hundreds of frames a second in pure CPU code. This is a task that doesn't even require advanced instructions like division.
Do not forget the part at 44:04 (for which the "epilepsy warning" is for really... or actually a few minutes later, but he starts around there to explain what follows).
The sad part here is that the Windows Terminal is a pretty old codebase originally written in C.
Most people would consider it fast enough as it is and not worth the effort to optimize/rewrite the rendering part because it would probably lead to marginal gains.
This is something I've heard way too many times, and it always turns wrong by several orders of magnitude.
Windows Terminal is a new replacement for cmd.exe
From the sound of this is sounds like Microsoft should just clean slate and fire the entire Windows terminal team and management, and hire casey to rebuild a new team.
he sounds like he might not consider himself an expert on all of the elements, but he definitely has enough understanding and vision to know who to hire in, and get rid of the whole “acceptance of horrible performance” culture plaguing the current team.
Microsoft's culture simply doesn't include performance as something to aspire to. Having read through a decent amount of their code, I'm certain that performance issues don't get flagged in their internal code reviews. I say that I'm certain because the kind of code they ship to production is so fantastically, gloriously inefficient, that in my limited imagination I could not produce something so verbose and unnecessary even if I tried.
As a random example, I recently had to explain to an Azure tech that maybe, just maybe, it's a bad idea to force all PaaS networking to criss-cross back and forth between their availability zones (data centres) like a shoelace. He incredulously asked me why I even thought that was bad, and wondered if I was working on some sort of "high frequency trading app" or something.
No, not trading apps. Just apps. Ordinary, boring, LoB apps that I simply did not want to see run like cold treacle...
The lack of performance as an aspiration is certainly part of the problem, but it also feels like there is a certain level of lack of intuition for the subject matter. If they believe that they are doing great and you’d need a full PhD dissertation to improve, and they 1000x below the base expectation of what someone can do in 2 days without even doing basic profiling and optimization. Then the problem is deeper than not valuing performance, it also speaks to a team that severely lacks an awareness of what they are working with.
I feel like this is similar to situations I have seen where people “explain” why database queries take several minutes to return with excuses like “that makes sense because we have years worth of data stored in the database” when in reality it’s because someone doesn’t understand how a composite index works. Someone who has a bit of intuition for db’s instantly knows something is wrong, but if your team lacks anyone with that intuitions, your team will spend their time finding “explanations” for the bad performance instead of fixing it.
I made a comment here recently that one cause of this problem is that computer performance crosses something like 13 orders of magnitude: Everything from functions taking mere nanoseconds all the way up to multi-minute processing.
In the "middle" of that enormous range, it's easy to make an incorrect judgement of how long something ought to take. This came up with the Grand Theft Auto game loading issue that was discussed here. Many people made statements like: "JSON parsing is slow", which isn't even wrong, it's just that it's not that slow.
A significant part of "experience" is having a good feel for what's normal. But even there, experience can lead people astray: the more years of experience you have, the more your expectations are weighted by out-dated information! Again, this crosses many orders of magnitude. Rules of thumb from the 1990s are incorrect by a factor of a thousand, and averaging them into an overall skillset will lead to enormous mistakes.
I noticed this the other day; a program I was writing that produced a lot of output and with a lot of ANSI colour was much slower (like 10x) on the new Windows Terminal vs mintty.
Note that this is a terminal for Windows.
it's the basis of a terminal rendering engine targeted at Windows but using concepts that can map to any operating system or environment
Not the same hardware (i7-8565U) but it should give some ballpark estimates.
xterm: $ time cat enwik9
real 4m32,750s user 0m0,032s sys 0m21,799s
gnome-terminal: $ time cat enwik9
real 1m37,299s user 0m0,030s sys 0m13,300s
Curiously, gnome-terminal actually seems much slower. It appears to only renders the text in chunks(?).
Try in st, for me it's much faster than xterm.