I have been building a Canvas diagramming library since 2010 (https://gojs.net), if anyone has any questions about 2D Canvas use in the real-world I'd be happy to answer them.
roundRect is great. Though you don't need 4 arcTo in order to make a rounded rect, you can use bezier instead (we do). Their example is also 1% amusing because they set the `fillStyle` but then call `stroke` (and not `fill`). I'll have to do some performance comparisons, since that's the operative thing for my use case (and any library author).
text modifiers are very welcome. It's crazy how annoying measuring still is, especially if you want thinks to look perfectly consistent across browsers. Though the chrome dominance is making things easier in one way, I guess.
context.reset is kinda funny. Most high-performance canvas apps will never want to use it. For that matter you want to set all properties as little as possible, especially setting things like context.font, which are slow even if you're setting it to the same value. (Or it was, I haven't tested that in several years).
I'm sure most users know this by now, but generally for performance the fewer calls you make to the canvas and the context, the beter. This is even true of transforms: It's faster to make your own Matrix class, do all your own matrix translation, rotation, multiplication, etc, and then make a single call to `context.setTransform`, than it is to call the other context methods.
One of the greatest things to happen to 2D canvas in all these years was hardware acceleration. It used to be significantly slower. (You could switch to WebGL canvas, but at a cost of complexity and all kinds of annoyances rendering text, and if you had a lot of text it often wasn't worth it.) Thinking back, I'm also a little disappointed that after the death of Flash, it felt like Canvas never really took its place. The playful web that came before it in some ways simply closed up.
Sounds like you're involved in this effort, so I'm taking the opportunity to ask...what about the TextMetrics API? It's listed as supported by Chrome, but I remember trying it last year and it's definitely not supported. Has there been progress on that front?
>Thinking back, I'm also a little disappointed that after the death of Flash, it felt like Canvas never really took its place. The playful web that came before it in some ways simply closed up.
I've been wondering if this is cultural or technical. The things that are easy to do get done more, and thus the tech (and how well it's designed) shapes mass behavior and culture. On the other hand, where there's a will to do something, a way will be found, or created...
A great example is sending Flash holiday greeting cards in the email. Watching them with my mom is one of my favorite computer-related childhood memories. Yet they seemed to go out of style long before Flash did.
Some of this has got to be the fact that macromedia flash originally was more targeted to artists and it was dead simple to create nice 2d animations. AKA it wasn't a programming environment so much as an artistic tool. The current version (adobe animate) doesn't seem nearly as popular despite the fact that it can apparently target html/canvas. Part of that just might be fragmentation, with wick editor, opentoonz, there isn't a single tool everyone talks about. Although I'm not sure if those tools can legitimately "grow" into programming tools the way flash and now animate can. (aka the artist draws up a bunch of stuff and then needs to learn some programming or have a programmer to come along and fix the donate button so it actually takes money like was possible with flash).
OTOH, I used to know a bunch of artistic types that had pirated copies and would sit around and create flash animations for their bands/etc. Now those people just do video editing and post things to youtube/etc, or spend time editing photos for facebook/etc. There isn't a hypercard/programming aspect anymore buried under their tooling.
edit: actually it looks like wick can do the hypercard/flash programming too, which is sorta cool.
> Distributing a HTML file and its assets? God damn complex
There's always Web Bundles[0], but I have no idea what the UX is like. I haven't heard about Web Bundles for a while, but they seem like such a good idea.
I think this is largely (entirely?) a failure of marketing on Adobe's part. I spoke to several people who created flash animations and games in the 2000s and they thought Flash-the-tool had died along with Flashplayer, they didn't know it had been rebranded Animate.
My recollection of Flash was it was just an annoyance. I remember avoiding flash websites with the exception being games. And I’m not a huge gamer but I recall a handful of flash games that stole a few days of attention from me. I always find it a bit odd when I hear people have positive nostalgic memories of flash because I had thought everyone equally agreed it was annoying: similar to Java applets, the Real Player, and actually many JavaScript things at the time.
However much Flash was abused on the web, it was an absolute joy to create games and animations with. There aren't any tools I know of today that allow for the same rapid art+animation+code workflows. I've tried using Canvas2D+JS as well as SVG+JS to do some of the same things, and they don't even come close to Flash!
Interesting account. I was diving into tech around turn off the millennia and I remember spending all of a day or so in action script and went back to the server side stuff. There’s so much game content around I have to believe the void is being met somewhere? Was this just an easy on ramp for low/no experienced folks? I believe that’s how the dreamweaver and other connected adobe products were positioned.
Yeah, it's really not. That entire segment that Flash games owned doesn't exist any more in the same way that it did.
The closest thing would be the mobile games like Candy Crush and Garden Saga, but even there the tide has passed, and it's too difficult to make anything - King was bought by Activision for 6 billion dollars.
There were several major indie games that started out in Flash and ended up on consoles. Alien Homind, N++, and Castle Crashers come to mind, in particular. They were pretty great games. Castle Crashers on Xbox 360 was a favorite of my roommates.
> That entire segment that Flash games owned doesn't exist any more in the same way that it did.
Really? Or did move to a platform like App Store/Steam. I’ve seen it quantified and there’s a absurd amount of new games posted daily. I won’t argue that the tools in use are easier or better than flash, but developers certainly pivoted or so it seems. From a laymen/consumer perspective, Browsing the App Store doesn’t feel much different that I remember Kongregate or some of the other big flash game sites of the past.
The functionality isn't really the same. Yes, you can export animations, but it's much more difficult to provide the same level of interactive game that Flash did.
Optional typing, e4x, normal inheritance (non-prototypal), etc...
The web would be a better place if AS3 took over instead of js (and I love js).
I imagine wasm would have shown up and been a ton easier to implement as well. AOT bits that are full static, JIT bits that aren't. No need for TypeScript, Flow, etc...
Especially sad since iirc Adobe donated the entire thing and was pushing for it to become standard in HTML4.
I think Flash died 2 years too early, or WebAssembly came 2 years too late. Because it always seemed to me that Flash was a good idea for the web, if it existed as a layer that was not "native" could be linked as a library (on top of Wasm and canvas). Alas, there were plans to emulate Flash using Wasm, but it was on its deathbed by the time we were starting the open design process. Adobe's VM seems to have had some design decisions that made it difficult to just "hit the button" and make it run decent on Wasm.
I think, fundamentally, delight and whimsy don't scale.
They are based on some element of surprise. Once the surprise is gone because you've seen it a few times, the joy evaporates and then it's just annoying.
I can understand where that sentiment comes from, but I think what really killed the playful flash web was accessibility. It's difficult enough to be fully accessible for a normal website; anything out of the ordinary is going to be significantly more difficult.
> context.reset is kinda funny. Most high-performance canvas apps will never want to use it. For that matter you want to set all properties as little as possible, especially setting things like context.font, which are slow even if you're setting it to the same value. (Or it was, I haven't tested that in several years).
yes, ctx.font mutation still sucks very badly. and you cannot avoid it during drag-resizing a canvas, which is miserable. (i maintain a high perf charting lib).
99% the problem is one of pixel density. That is, you have a 400x400 canvas, and a 400x400 CSS pixel space you are drawing it to, but your devices pixel ratio is higher than 1, so it looks kinda blurry. This is because devices squish more than one "hardware pixel" per CSS pixel. Often at a ratio of 1.5 or 2, or higher.
The solution is to make a larger canvas, say, 800x800 and put it into that 400x400 space.
Here is an example, using that MDN code, with a 400x400 canvas (red) next to a 800x800 canvas (blue). CSS is forcing them both to appear the same 400x400 size. The blue one should look sharper on most devices.
Note how the 800x800 canvas needs to be scaled double with ctx2.scale(2,2) so that it appears correct.
Dude!! Freaking awesome!! Thanks a bunch!! That's exactly what I was looking for! If you have some sort of ko-fi account or whatever I'll buy you a coffee!
One thing to keep in mind with the window.devicePixelRatio is that it is not a static value as it can change over time. Some common reasons it could change are if the user drags the browser window from a low density screen to a high density one. Additionally, it will change if the user zooms the page in and out.
Zooming also fires the Window's `resize` event, which will also catch a user rotating their smartphone/tablet, or, erh, resizing the window on desktop.
No, it's the same concern. The concern is not "how do I keep track of just devicePixelRatio?" It's "how do I fill this window with a canvas at native resolution?" Resize handles all of that. Device changes orientation? Resize fires. User zooms on content (this changing devicePixelRatio)? Resize fires. Anything that changes the page's clientWidth/clientHeight causes the Resize event to fire. You need only the one event.
All the ways in which you will care about how to buffer dimensions of your canvas to avoid aliasing artifacts can be handled with just the window.onresize event, whether your canvas is fixed or proportionally sized.
Throw a check in there to make sure the native dimensions actually did change so you don't needlessly redraw. Debounce it with setTimeout if you're super worried about people waggling their windows around or whatever.
Yeah I was struggling with this one a while, seems there is no way to know the actual devicePixelRatio of the display inside a browser, as the browser zoom level directly affects that.
DevicePixelRatio and clientWidth/clientHeight change together. You need only the Resize event on the Canvas. Your canvas.clientWidth * window.devicePixelRatio will always be the native width of the canvas.
To add a small caveat here for others, assuming 2x as the necessary scaling multiplier is not necessarily a safe shortcut when choosing your virtual canvas size; iPhones use 3x resources to look crisp, and 5K macOS users can have their 5120x2880 displays set to 2048x1152 or 1600x900 which are 2.5x and 3.2x respectively.
It still helps when you do your canvas at 2x rather than just at 1x. But if you're looking for best-fidelity rather than better-fidelity, duly noted.
I recently embarked on a months-long just-for-fun effort of porting an old AS3/Flash game to Haxe and make sure it worked on every imaginable Haxe target I could find. If there was something that bit me was how ugly everything looked inside the browser, even when using a WebGL backend.
It turns out it was very obviously exactly what you described. You basically have to double the resolution to get it to look decent on retina and 4k screens. Once I found this out I was over the moon!
(On a related note, colouring looks way off on the HTML5 target compared to native ones, and I spent an embarassing number of hours manually brute-forcing a shader to make everything look similar.
Oh, and sound sucks in the browser.
Oh, and Safari sucks. Seriously, it just sucks. The amount of hacks I put in place to workaround its egregious limitations, particularly regarding canvas operations is insane.)
What would your strategy for working with raw image data and implementing a digital “zoom” that doesn’t do any anti aliasing or smoothing? My current thought is to just have an original copy of the data and then the current canvas is just a smaller subset of the original data when zoomed. So a 100x100 pixel image when zoomed 1x would take a 50x50 subset based on the center of the viewport and copy each pixel into a 2x2 to get back to a 100x100 image.
Unless I misunderstand you, you should be able to `ctx.drawImage` from one canvas to another, with `imageSmoothingEnabled` set to false, and it will do what you want.
"One of the greatest things to happen to 2D canvas in all these years was hardware acceleration."
I loved this one, too. Except for the one drawback, getImageData is now very, very slow. And no asynchronous method in sight, that could make it fast again.
How slow is it these days? Even readPixels() in OpenGL and hw buffer copies in C++ are “slow” because you’re transferring the image data over your PCI bus.
Also curious how & whether an async method could help? Is the problem that getImageData() has to wait to flush all pending draw calls before reading back? Are you thinking of some kind of callback that returns the previous frame’s data, rather than demanding it ‘now’? There probably is room for perf improvement, totally, just curious how you’re imagining it would look and if you have data on where the current bottlenecks are.
PBOs allow for asynchronous readbacks from opengl. It's not slow because of the pcie bus (heck for most devices this is unified memory in the first place, it doesn't hit any bus other than dram). Rather it's slow for the same reason glFinish() is. You're blocking waiting for the results of the asynchronous rendering which you probably also just kicked off right at that point.
Also it's not like the opengl is happening on the same thread as the canvas2d (or even the same process), so that's yet more synchronization that a blocking getImageData needs to wait on.
Yeah, exactly, that’s what I’m curious about - if the browser’s renderer is adding significant amounts of delay due to additional synchronization, or if the problem with canvas.getImageData() is the same problem we’ve always had and if the delays are similar. This does make it seem like a callback to receive last frame’s image could be much faster than a synchronous read call, and I’d certainly use that if it existed…
canvas.getImageData() would have gotten a lot slower as GPU acceleration of Canvas2D became increasingly more common. The initial versions of Canvas2D were all software rasterized, which means getImageData() likely didn't involve any IPCs and definitely didn't involve any GPU readbacks. It was just a memcpy if that.
Interesting to hear about that doing the matrix translations on the js side are faster. Have you benchmarked this recently ? Will have to test it out myself, was under the naive impression that the underlying implementation would be optimal.
var transform = ...your own data structure
transform.scale(zzz);
transform.rotate(zzz);
transform.translate(zzz);
transform.scale(zzz);
// perhaps as many as 50 as you go down the visual tree
ctx.setTransform(use your data structure to set these values);
This is a little faster than calling all the corresponding methods on ctx. Of course you gotta implement a transform class.
And of course it becomes meaningfully faster if you are drawing a large number of objects in a visual tree, and their locations do not all change, so you can save these transforms that you've created, and then the only thing you are doing in the draw loop is a single call to `ctx.setTransform` instead of all the calls you'd normally need. Again this really depends on what you're doing, but you can imagine for drawing the visual tree of a diagram, it can be quite an improvement.
The problem is: browser implementors are never/rarely website implementors. So you always end up with half-baked tangentially useful standards and APIs like this. They just don't know what is actually needed.
Well, there's this and the fact that browser implementers all need to come to consensus over what features to add and that's an extremely difficult process to move forward. There also is one big browser implementer who doesn't really want the "playful web" to exist and would prefer that everyone live inside of apps.
Edit: To be clear, we all work together and mostly get along. It's a long and arduous process to reach consensus and not everyone's incentives are aligned and this can be frustrating.
Your post implies that Google cares about browser consensus. For example, "This feature has been in Firefox for a while and we're finally making it part of the canvas spec." Google DOES NOT decide what the spec is. "You" don't make something part of the spec.
> There also is one big browser implementer who doesn't really _want_ the "playful web" to exist and would prefer that everyone live inside of apps.
No. There's one humongous browser implementer who couldn't care less about consensus or what other browser vendors think. This vendors ships literally hundreds of new APIs every year and pretends they are now standards that everyone else must implement.
There are a bunch of implementation concerns here (and mentioned on that issue) with regards to availability on various native 2d backends... Needs more investigation
SVG filter interface:
We would really rather people use WebGL if you want fast/efficient filters. (I made a number of comments on that issue) As is, we're generally against this one for the time being
--- end quote ---
There's literally zero response on that from @mysteryDate who is now gaslighting Safari in the comment above, and presenting these API additions as fait accompli.
Honestly, at this point any time I see any public Chrome person write anything I immediately assume it's a distortion of reality at best and a blatant lie at worst. And this is always the case.
FWIW Mozilla and Safari signed off on all of these changes at some point in time somewhere, hence why it's allowed to be part of spec. There were some changes that were not allowed to be part of the new API because one of those two said no (like perspective transforms, conic curves).
For jdashg's concerns on that thread, 4x4 matrices were cancelled, and you can follow up with much more debate from all parties on roundRect and filters:
> We would really rather people use WebGL if you want fast/efficient filters.
This one made me laugh. Yes, WebGL excels at pixel manipulations but it is possible to write fast and efficient filters to work in the 2D canvas environment.
For a case-in-point, I struggled for a long time to find a decent, fast implementation of a gaussian blur filter for my canvas library. Then I stumbled upon a JS implementation[1] based on some very clever work done by Intel devs which blew all my previous attempts out of the water - so of course I stole it (even though I still don't understand the approach they take)[2].
> "Safari is the bad guy"
As much as Safari often brings me to despair, I do like the work they've recently done to add color space support in CSS. They haven't yet pushed the functionality over to the canvas element, but I live in hope. For now, I have to emulate the calculations to get them working for my library[3].
It's not just about using SIMD instructions (they help), and laying out memory to optimize cache performance (which also helps), but most importantly that Gaussian blur is a "separable filter" that you can break up into a horizontal and vertical pass, each of which require a lot fewer memory references (on the order of just two times the number of pixels times the kernel size, instead of the number of pixels times the kernel size squared):
IIR Gaussian Blur Filter Implementation using Intel® Advanced Vector Extensions
>This white paper proposes an implementation for the Infinite Impulse Response (IIR) Gaussian blur filter [1] [2] [3] using Intel® Advanced Vector Extensions (Intel® AVX) instructions. [...]
>The IIR Gaussian blur filter applies equation (1) on each pixel through two sequential passes:
The horizontal pass: This pass processes the input image left-to-right (row-wise), then right-to-left. The output of the left-to-right pass is added to the right-to-left pass.
>The vertical pass: Usually, the vertical pass processes the output from the horizontal pass top-to-bottom (column-wise), and then bottom-to-top. Accessing the input column-wise leads to a lot of cache blocks and impacts the performance of the filter. To avoid this, the horizontal pass transposes the output before writing to the output buffer. It makes the vertical pass similar to the horizontal pass and processes the intermediate output left-to-right, then right-to-left. The vertical pass again transposes the final output before writing the blurred image.
>Separate your filters! Separability, SVD and low-rank approximation of 2D image processing filters
Posted on February 3, 2020 by bartwronski
>In this blog post, I explore concepts around separable convolutional image filters: how can we check if a 2D filter (like convolution, blur, sharpening, feature detector) is separable, and how to compute separable approximations to any arbitrary 2D filter represented in a numerical / matrix form. I’m not covering any genuinely new research, but think it’s a really cool, fun, visual, interesting, and very practical topic, while being mostly unknown in the computer graphics community.
>In addition to being circularly symmetric, the Gaussian blur can be applied to a two-dimensional image as two independent one-dimensional calculations, and so is termed a separable filter. That is, the effect of applying the two-dimensional matrix can also be achieved by applying a series of single-dimensional Gaussian matrices in the horizontal direction, then repeating the process in the vertical direction. In computational terms, this is a useful property, since the calculation can be performed in O(w_kernel w_image h_image) + O(h_kernel w_image h_image) time (where h is height and w is width; see Big O notation), as opposed to O(w_kernel h_kernel w_image h_image) for a non-separable kernel.
Your library seems absolutely great. I can't believe I've never stumbled upon it before since I've literally gone looking for stuff exactly like this several times and never found anything very good.
Excellent advice! Your setmatrix advice is unintuitive, but I believe you. The JavaScript/C barrier is expensive, while JavaScript calling JavaScript is optimized up the yin-yang.
I love canvas because I cut my teeth on PostScript, and it's basically the PostScript imaging model grown up.
What you said about minimizing graphics state changes is an important point, that also applies to PostScript. And a lot of PostScript optimization advice also applies to canvas/JavaScript too, because they're similar in many ways.
Glenn Reid (who worked for Adobe, and was the author of the "Green Book" on PostScript language program design) wrote "The Distillery", which I've written about before on HN (link and excerpt below), that was a PostScript program that loaded in and partially evaluated another PostScript drawing program, and output another program that drew the exact same thing, only (usually) much faster and usually smaller. Unless you had any loops, which it would unroll.
One of the most important optimizations it did was to transform all the graphics into the same default coordinate system, and optimize out not only graphics state changes but also importantly calls to gsave/grestore, which could be quite common.
Glenn Reid described his classic "Grandfather Clock" metaphor to comp.lang.postscript that really opened my eyes about writing code in interpreted languages like PostScript (and JavaScript, while JIT'ed at runtime, still makes calling native code expensive), and balancing optimization with readability. With modern JavaScript/canvas, JavaScript code that calls other JavaScript code runs really fast, but calling between JavaScript and built-in code is still slow, so it's good to have the built-in code do as much as possible when you do (like rendering a long string of text at once, instead of rendering it word by word like many PostScript drivers would):
Glenn's post in a comp.lang.postscript discussion about PostScript programming style and optimization:
From: Glenn Reid (Abode Systems)
Newsgroup: comp.lang.postscript
Subject: Re: An Idea to Help Make Postscript Easier to Read (and Write)
Date: 10 Sep 88 17:26:24 GMT
You people tend to forget that the PostScript language is interpreted.
It is well and good to use tools to convert to and from PostScript,
but it is not quite as "transparent" as we all might think.
I like to think of a big grandfather clock, with the pendulum swinging.
Each time pendulum swings, the PostScript interpreter gets to do one
operation. The "granularity" of the clock is nowhere near the speed
of a microprocessor instruction set, and any comparison with assembly
languages doesn't make sense.
The difference between:
0 0 moveto
and
0 0 /arg2 exch def /arg1 exch def arg1 arg2 moveto
can sort of be measured in "ticks" of the interpreter's clock. It's
not quite this simple, since simply pushing a literal is faster than
executing a real PostScript operator, but it is a rough rule of thumb.
It will take about three times as long to execute the second of these
in a tight loop, and about five times as long if it is transmitted and
scanned each time. My rule of thumb is that if you have roughly the
same number of tokens in your stack approach as you do with your 'exch
def' approach, the 'exch def' is likely to be much more readable and
better. Otherwise, I usually go with the stack approach.
One other thing of note is that if you have too much stack manipulation
going on, it may well be symptomatic of a problem in the original program
design.
Also, most procedures don't do any stack manipulation at all, they
simply use their arguments directly from the stack. In this situation,
it is especially wasteful (and confusing, I think) to declare
intermediate variables.
Compare:
% sample procedure call:
(Text) 100 100 12 /Times-Roman SETTEXT
% approach 1:
/SETTEXT { %def
findfont exch scalefont setfont moveto show
} def
% approach 2:
/SETTEXT { %def
/arg5 exch def
/arg4 exch def
/arg3 exch def
/arg2 exch def
/arg1 exch def
arg5 findfont arg4 scalefont setfont
arg2 arg3 moveto arg1 show
} def
Which of these is easier for you to understand?
Anyway, I think the discussion is a good one, but let's not forget
that PostScript it is an interpreted language. And I don't think
it is terribly hard to use and understand, if it is written well.
Glenn Reid
Adobe Systems
Here's Glenn's "Green Book", which was like a bible to me, and still is quite relevant to canvas 2d context programming -- see especially page 9, section 1.5, Program Design Guidelines, page 72, section 4.6, Optimizing Translator Output, and page 99, chapter 7, The Mechanics of Setting Text:
>There are a few items that may be kept in mind while implementing a driver for a PostScript device. As with most software development, the most difficult part of writing programs in the PostScript language is the design of the program. If the design is good, implementing it is easy. If the design is poor, it may not even be possible to correctly implement it. Below are some helpful items to keep in mind when writing your software. All of them are explained more fully within the text of this book; this is only an attempt to prime the pump before you start reading:
>• Use the operand stack efficiently. Pay particular attention to the order of the elements on the stack and how they are used.
(Using the stack efficiently by designing fluent words that chain and dovetail together elegantly in pipelines (and systematically writing line-by-line stack comments) instead of using named variables in dictionaries is good idiomatic PostScript and Forth, aka tacit programming or point-free style, the stack-based equivalent of fluent interfaces.)
>• Avoid unnecessary redundancy. When a program is produced, check for many repetitive steps that perhaps could be condensed. Keep identifiers short if they are to be transmitted many times.
(Only create paths once!)
>• Use the PostScript imaging model effectively. When printing, a document must be translated into the language of the printer. This includes a philosophical adjustment to the nature of the PostScript imaging model.
(Use the graphics state stack!)
>• It is better to translate into the PostScript imaging model than to maintain another set of graphics primitives using the PostScript language for emulation.
The hardest problem I ever tried (and failed) to solve properly with PostScript was making a printer driver for rendering user interfaces drawn with X11 using a combination of bitmaps and lines, that looked perfect on the screen at 1:1 scale, but didn't look terrible when you zoomed into them or printed them at high resolution on paper. Because of X11 "half open" pixel rounding rules versus PostScript's "stencil paint" model, they just don't line up right when you zoom into them, and there's no fudge or compromise that works in all cases. Here is my commented-out failed attempt:)
line-color line-cap line-join dash-pattern
thickness
% dup -1 ne { .5 add } if % fudge outline width thicker
StrokeShape
>page 100: Note: There is one principle to keep in mind when deciding upon an algorithm for setting text. The longer the string presented to one of the show operators, the more efficient the system is likely to be. This is because the PostScript language built-in operators, such as show, widthshow, and ashow, operate essentially at compiled speed once they have been invoked. Each moveto or div operation performed must first be interpreted, which is significantly slower.
Here's a description of Glenn's PostScript Distillery, which foreshadowed Acrobat Distiller.
>Glenn Reid wrote a PostScript partial evaluator in PostScript that optimized other PostScript drawing programs, called "The Distillery". You would send still.ps to your PostScript printer, and then send another PostScript file that drew something to the printer. The first PostScript Distillery program would then partially evaluate the second PostScript drawing program, and send back a third PostScript program, an optimized drawing program, with all the loops and conditionals unrolled, calculations and transformations pre-computed, all in the same coordinate system.
>It was originally John Warnock's idea, that Glenn implemented. And it led to Adobe Acrobat's "Distiller". Acrobat is basically PostScript without the programming language.
>No, you could not make it optimize itself by sending it to a PostScript printer two times in a row. It was not magic: all it did was intercept and capture the side-effects of the PostScript drawing commands (fill, stroke, show), read out the path, and optimize it in a uniform coordinate system. Since it didn't do any drawing, so it would just output an empty program if run on itself. (Take that, halting problem!)
>From: greid@adobe.com (Glenn Reid) Newsgroups: comp.lang.postscript Subject: release 10 of the Distillery Date: 10 Mar 89 10:21:52 GMT
>Here is another release of the PostScript Language Distillery. I know it's not terribly long after the last release, but there are some significant enhancements, and I thought it would be worthwhile.
>I finally took a closer look at user-defined fonts, which now seem to be working fairly well. In particular, it seems to handle the Macintosh screen bitmap fonts that get used if the native font is unavailable when the print file is genreated. The entire user-defined font is reverse-engineered to the output file as it stands, and is used exactly like the original file used it. I also fixed some rotate text bugs, rotated charpath, and a few other things.
>I want to emphasize that probably the two best uses of this program, currently, are to do speed comparisons of various PostScript language drivers and to convert "non-conforming" files into "conforming" files. It is not particularly well suited to carefully hand-written programs, especially not those which use looping constructs. It works (usually), but it unrolls the loops and makes the files much bigger.
You can use cubic Bézier curves to approximate a circular arc: https://pomax.github.io/bezierinfo/#circles_cubic. In many cases that's even how circular arcs are rendered, even when they are represented with a different data structure. You'll often see the the constant 0.5522... which is for quarter circles. E.g. both WPF and Java's Swing render arcs with that approximation, perhaps others, but those where the places where I checked the source code recently.
The benefit of doing this for quarter-circle arcs directly may be that you save a bunch of trigonometry that otherwise has to happen (cf. the Pomax link).
i think you'd still need 4 bezierCurveTo commands, so not sure there's any advantage over 4 arcTo:
function roundedRect(ctx, x, y, width, height, radius) {
ctx.beginPath();
ctx.moveTo(x, y + radius);
ctx.arcTo(x, y + height, x + radius, y + height, radius);
ctx.arcTo(x + width, y + height, x + width, y + height - radius, radius);
ctx.arcTo(x + width, y, x + width - radius, y, radius);
ctx.arcTo(x, y, x, y + radius, radius);
ctx.stroke();
}
You laugh, but actually in the 90s rounded rectangles were a must to portray your site/app as "modern" (and signify you didn't bother to give it basic usability like desktop apps have so you could spare that part for your MVP).
Same thing with gradients in forum design. The moment we could use CSS instead of a gradient handcrafted with a pirated version of Photoshop - gradients stopped being commonly used in forum skins.
I would love to have a CanvasFilter that makes your smooth 24 bit procedurally generated gradients look like they've been error-diffusion dithered to an 8 bit web safe color palette. And with a FatBits option to compensate for retina displays. Now THAT would be fashionable!
> makes your smooth 24 bit procedurally generated gradients look like they've been error-diffusion dithered to an 8 bit web safe color palette
Do you mean the gradient's output should look like it's been put through a risograph printer? I created a CodePen demo a while back which attempts to recreate that sort of effect - https://codepen.io/kaliedarik/pen/RwgwpyG
When it comes to reducing a palette from (potentially) millions of colors down to a set of 8 bit web safe colors - I'm not convinced such a process can generate decent output for a wide range of different inputs. I get much better results by generating a "commonest colors" palette with a minimal distance (in LAB color space) between each selected color to generate the most pleasing dithered output, whereas restricting the palette to web-safe colors limits the output to just a handful of those commonest colors. See an interactive example of the filter here: https://scrawl-v8.rikweb.org.uk/demo/filters-027.html
Oh I wasn't looking for decent output, I LOVE the pixelated error-diffusion dither effect, especially with an ugly garish color map that brings out all the jiggly artifacts. ;)
Oh, nicely done! Trying to code up cellular automata simulations are pretty much guaranteed to push my brains through my nostrils - I've never progressed far beyond classic Conway. Your CAM6 library[1] may be about to steal my weekend from me!
I'll add some of my thoughts about this article's take on latest Canvas2D developments.
1. ctx.roundRect() - is a nice addition to the 2D stable of shapes, fitting alongside other recent additions like ctx.ellipse(). I was a little disappointed that the example of how to achieve this using existing functionality relied on a set of .moveTo, .lineTo and .arcTo instructions; the article misses a trick by not mentioning that we can also use an SVG path.d string to define a Path2D object and use that to fill/stroke the shape. Note: I can't find mention of .roundRect() in the MDN documentation.
2. Canvas Conic gradients have been supported in both Firefox and Safari for a while now. Good to see it arriving in Chrome. To complete the set of gradients, it would also be nice to see support for ports of the CSS repeating-linear-gradient() and repeating-radial-gradient() into the Canvas API.
3. Text modifiers - text has always been an over-simplistic mess in the Canvas2D API. In theory a user should be able to use ctx.font to load a valid CSS font string (including all the supported font size lengths, etc) and have the text rendered as expected; sadly the implementations between various browsers are all over the place. In my dreams I hope that one day we will be able to set a width/justification and have the text string broken into lines while respecting those requirements. Browsers do it already for DOM/CSS layout and rendering - I wonder how difficult it would be to tap into that functionality rather than having to reinvent it for 2D canvases?
4. context.reset() - nice!
5. Filters - SVG filters are massively powerful, yet also difficult to master. We can already build some really complex filters in SVG and reference them in the Canvas2D context; I also like the way we can add CSS filter strings to the context (eg for a nice, easy-to-understand and performant blur effect). The article mentions a `ctx.filter = new CanvasFilter([])` construction, but I can't find CanvasFilter in the MDN documentation.
The "we" in the statement is the maintainers of the spec, which includes engineers from Chromium, Firefox, Safari, Edge, Opera and many other stakeholders.
This is great to hear! I'm an artist and much of my work builds on Canvas2D, it's a simple API and provides a nice balance of features and constraints.
I'd love to see more development on Canvas2D API and standards. There's a few things that are missing and/or not yet standardized:
- Advanced text metrics and more reliable text loading
- More robust high-resolution rendering, for example using Canvas2D to generate large 300 DPI prints
- More control over line rendering, such as placement on polygons ("inside" / "center" / "outside"), stippling, variable thickness
So, these functions are are now currently part of the whatwg spec, so other browsers have stated their intention to implement them and this is underway.
> Let's make sad tabs happy again! In the event that a client runs out of GPU memory or some other disaster befalls your canvas, you can now receive a callback and redraw as needed:
No... this isn't how it should be. The browser should hide such details from the javascript. The canvas contents either shouldn't be lost by the GPU at all, or if they are, the browser should keep enough information to be able to recreate it (for example, a backup of all the canvas pixels plus recent draw commands executed).
Lets be honest - most GPU context losses are GPU driver bugs and should just be fixed, not require every web page to try to redraw to work around them.
A very annoying part of the Canvas API is that colors are represented as (CSS) strings, except for the pixel array, obviously. This forces to write code like this:
:) Good to see web news that appears to be at first glance just straightforwardly good news and uncontroversial. In particular, I'm really excited to see `willReadFrequently` on this list. Particularly for games, canvas caching is a huge deal.
And holy crud, rounded rectangles! It doesn't fundamentally change anything, it just means I don't need to write a helper method for literally every project I make that uses them. It's a tiny thing, but I'm grateful for the people working on that.
A legitimate point! Unfortunately from our end it's also impossible to tell what canvases are being used for. Somewhat reassuringly, the vast majority canvases that are being created are much larger than would be necessary for fingerprinting, so we have some reason to believe that most uses are legitimate. There's also the fact that the vast majority of canvases never perform `getImageData`, which is an essential part of every fingerprinting technique I've ever seen.
So to say that canvas is an element "for fingerprinting" I think is an outdated notion that's the result of some bad PR the element got like 10 years ago.
So, yes, fingerprinting is an issue and we all wish it would go away. Unfortunately there is a ton of work to do. A lot of very smart people are working on this and I wish them the best of luck.
Fingerprinting isn't a boolean state. The more fingerprinting vectors you have the more reliably you can identify users. Therefore having CSS fingerprinting (or any other fingerprinting vector) doesn't really make canvas fingerprinting less bad.
True. Any data the user leaks is data the user leaks and we should work to prevent this. However I don't think people realize just how easy it is to reliably fingerprint users who are blocking javascript and canvas and other web features.
>how easy it is to reliably fingerprint users who are blocking javascript and canvas and other web features.
The key difference is that most of the fingerprinting attributes listed in the article, most are very innocuous and/or easy to change (ie. unreliable for fingerprinting). That includes:
1. pointer type
2. prefers-color-scheme
3. window size
4. @supports
The only other one (local fonts) is interesting, but is still easily changeable and conceivably be mitigated by removing all non-default fonts. This is in contrast to canvas/webgl fingerprinting which fingerprints both the software and hardware parts of your rendering stack, and is nearly impossible to change.
In the future, everyone is using a 640x480 window into a browser that's completely software stack in a VM that runs at a regulated frequency with a specifically allocated amount of RAM and hard drive space, doesn't accept cookies, and clears cache after every page load.
I guess it didn't occur to you that retrieving and setting image data with getImageData()/putImageData() actually turn out to be indispensable features when generating complex multi-layer canvases. Your trivial heuristic would flag a lot of false positives.
I flat-out don’t believe feature usage statistics on the web for this kind of reason, because too many numbers are patently absurdly wrong as far as any kind of legitimate use is concerned, and so any time I see an element or attribute occurring more often than I’d expect, I just assume some widely-deployed script like Google Analytics that I almost certainly don’t want is using it, probably for fingerprinting.
I came here precisely to say 30-40% of websites seems remarkably high.
The number of actual canvas elements used as canvases I've noted in the wild is pretty small. I've been playing with the canvas element since it was a Safari exclusive they added to make building dashboard widgets more powerful, and I can probably say the number of times I've found them actually useful in building a site is on one hand.
I think you're right that most of them are probably for fingerprinting.
Google, Facebook, Reddit, Amazon, TikTok. I entered main pages of those websites, and they tried reading pixels of an invisible canvas for whatever reason.
The canvas element is great for basic fingerprinting because it's practically impossible to stop it without breaking legitimate use cases. Shortly explained, there's differences between text rendering in different operating systems, browsers, and between browser versions as well. A simple fingerprint would be then to render some text, convert the canvas to an image and then hash it. Modern fingerprinting approaches combine this with a number of other sources of data (such as WebGL, available fonts, etc) to uniquely identify users.
So stupid. You know what would be nice? If my processor didn't attempt to do a bunch of rendering and extra computation, costing more energy and wear, to spy on me. Without eliminating legitimate use cases.
For now, eliminating legitimate use cases will have to do.
This problem could also be solved by making the specs so rigid and implementations so constrained that none of these techniques could be used to differentiate one user agent from another.
For many reasons, the lesser of two evils is probably to make user agent differentiation possible. The number of apis one has to cut out to make it impossible basically rolls HTML back to the 1.0 with no JavaScript era.
Implementation details of the machine rendering a document leak through all over the place. The fact that they are allowed to vary while confirming to spec has historically been considered a feature of HTML rendering, not a bug.
Now hang on for a bit, I can vaguely understand what is going wrong with the ClearRect (though why it draws a line to the bottom right when the code just moves the pencil and draws a horizontal line is beyond me) but what on earth is with:
canvas.width = canvas.width;
That's worse than a code smell, that reeks. Adding a new function to clear the state is nice, but what on earth is going on?
I was curious also, so went looking. Here are some relevant stackoverflow questions: [1], [2]. Basically, setting the width of the canvas to any value different than the current width clears the canvas, because it forces the browser to re-allocate the relevant bitmap. Current implementations don't check if the width value is different, just re-allocate the bitmap anyways. Hence the trick.
> Whenever the width and height content attributes are set, removed, changed, or redundantly set to the value they already have, then the user agent must perform the action from the row of the following table that corresponds to the canvas element's context mode.
> | 2d | Follow the steps to set bitmap dimensions to the numeric values of the width and height content attributes. |
> When the user agent is to set bitmap dimensions to width and height, it must run these steps:
> 1. Reset the rendering context to its default state.
> 2. Resize the output bitmap to the new width and height.
There's opportunity to reuse the previous surface and just clear the contents, but I'm not sure if this is done. (But the reset of state and clear are both mandatory for `.width=.width`)
Things like this are to be expected if you truly value "never break anyone, ever, even if everyone unanimously agrees something is bad" as an immutable goal of your software. Programming languages are especially important for this sort of philosophy... If you make a mistake and people rely on it, your mistake is now part of the way the language works, always and forever.
My hot take: Software that doesn't take this approach isn't truly serious in the same way as software that does. Personally I'm glad for this level of discipline because at some point, there has to be a "bottom"... software that we have to rely on in order to make our own code work. Not everything gets to enforce deprecation/removal cycles, sometimes it just has to Always, Always Work.
Totally agree. I think it's way too late to change out the behavior of `typeof`, but Canvas2D is young enough that it might be great if one or more browsers removed that optimization and didn't cause width = width to blank the canvas contents, so that people didn't come to rely on a bad API to solve a problem that has nothing to do with canvas width.
... of course, that would require extending the API with an actual blanking function...
There are some getters and setters in the web APIs with cheeky side effects. Presumably this one causes the canvas to be re-created, or something. Here's another good one, try it in your browser console:
Oh, it works, it's just cheekier than you think - it won't refresh the page if the URL includes a hash/fragment (such as after you clicked one of the "next" links while browsing comments here).
It's not really. The code smell is people using it to clear rather than using clearRect but the fact that setting canvas.width or canvas.height resets the canvas and even does so if the size is the same is good engineering IMO.
It's consistent and so your code can know if it sets the size things will reset. It doesn't have to special case when you happen to set it to the same size.
I would guess that when a dimension is "changed" (in this case it's not changed, but it is assigned), the canvas is recreated, along with all its internal state.
One canvas limitation I ran into was obtaining the height of a given piece of text. Width is available, but height is not, so you can't center some text vertically. Stack overflow tells you to use an "M" character as it has roughly the same width and height...
This is actually a hard problem even for most font libraries as it requires 'execution' of the font and kerning to actually know the parameters. I'm not saying this is a good answer but this is a tricky problem outside just Canvas2D. E.g. sour, soup, slip, and shin are four words with different heights, how do each in turn play out? Most of the time this is external to the font library and is in the layout engine which is talking to the font rendering context the same way you are in a raw font library/canvas.
To be honest this is one of the things that made web browsers and html so good is because it solves this problem really well compared to having to figure this out in native apps across languages the hard way.
Depends on your use case, in my case I'm using the font size as the height just fine, m.actualBoundingBoxAscent + m.actualBoundingBoxAscent or m.fontBoundingBoxAscent + m.fontBoundingBoxDescent also works for some cases.
This number is a little bit of a guess. It is the number of calls to `<canvas>` divided by the total number of page loads. Obviously some pages are using more than one canvas, hence the large range.
As for fingerprinting, most canvases are 1. not their default size, 2. not 1x1 (like fingerprintjs uses) 3. never call `getImageData`. All these things together point to the idea that these are legitimate uses. Yes, fingerprinting on the web platform is a problem. It's also a problem with canvas. I think it's disingenuous to say "canvas is mostly used for fingerprinting" without ways to support that statement.
Perhaps a better way to word that is “30-40% of page loads in last {n} {unitOfTime}”. This also jumped out at me as wildly inaccurate to the point where I felt compelled to be this guy: https://xkcd.com/386/ rather than enjoying the rest of the article ;)
If they want to improve performance, it would be good to be able to specify a colour without invoking the CSS parser. If you animate 1000 coloured shapes at 60fps, the CSS parser is invoked 60,000 times per second. Seriously.
I assume these are all currently non-standard APIs that Firefox will nonetheless be expected to implement? Does Firefox ever force work on the Blink team, or it is it only ever the other way around?
Getting the features into the spec is the vast majority of the work. Though some features are not yet implemented in Firefox or Safari, they have both indicated their intention to do so.
The article should make that clear right at the start. I read the whole thing and came away with the strong impression that these things were Chrome-only (apart from the one mention of Firefox).
Is there an easy way to make canvas always fill parent while retaining 1:1 pixel size (no blur). Currently I need to use various hacks.
Basically just like normal div, it currently cannot be done in canvas because once you set c.width=c.clientWidth it starts interfering, so the only solution is extra div (relative, canvas absolute) and on resize update width and height). Also it should work in flexbox.
In chart.js we have a very complex solution for dealing with this. We've gone through a number of iterations, but settled on a using ResizeObserver. You also need to listen to `window.resize` to get notified about DPR changes which can happen when a browser window is dragged from one screen to another.
This is my experience as well, to do it correctly, it needs to get many minute details exactly right.
They still developing canvas so my proposal is to forget about these extremely complex solutions and add new property to canvas that would make width/height same as client width/height (perhaps with one event to let canvas know it was resized).
I am not entirely sure if this is what you mean, on resize/orientationchange you set canvas.width and canvas.height to its clientWidth/clientHeight and redraw. Is that what you mean by hacks?
The CSS Painting API[1] shows some promise in this area - it adds a (restricted) canvas to elements which always maintains the same size as the div element it's attached to.
I'd like if there was a "mode 7" style affine transformation available, but I understand my needs are niche. Still, nice to see some of the performance improvements and that reset function.
Seems like you could implement your own in code. It would not be fast though — having to walk each pixel in the canvas buffer, apply the transform to grab a pixel from another buffer, then assign the transformed buffer back to the canvas.
I, too, had to look it up. This is about adding a (pseudo) perspective effect to the 2D canvas yes? There is an open issue in the whatwg/html repo[1] asking for this functionality - I have no idea whether the request will be taken forward by any browsers.
The current solution is to "roll your own" functionality. It is doable - for instance I've added such functionality to my canvas library[2]. The results work, but are slower than ideal.
>"Surprisingly, SpaceX Senior Software Engineer Sofian Hnaide revealed that all of the user interfaces on the Dragon Capsule run using Chromium, the open source backbone of the ever-popular Google Chrome internet browser."
Wow, I would never expect something like this. But I guess they know better about reliability of their particular solution.
These changes look really nice! Out of interest does anyone know what the most common use cases for canvas are? The article says "30-40% of web pages have a <canvas> element" which is way higher than I'd guess. Is it used by ad networks or something like that that could increase the percentage a lot?
Probably the majority of these would use Canvas2D for a mix of: fingerprinting, tracking/ad tech, feature detection, font metrics, CPU rasterization & pixel manipulation. Occasionally it will be used for specific elements, games, artworks, animations, font rendering, video effects, image resizing, and sometimes entire applications (Figma, Google Docs, Google Maps, although they likely all use WebGL Canvas for performance & control).
one [more] thing i want is to be able to retain canvas context settings when resizing the canvas. e.g. it's unreasonably very expensive to re-sync the lost ctx.font on every dragged pixel - this operation alone dominates my perf profiles. i have no idea why it should be so expensive once the font is already loaded / cached in mem.
ctx.fillText() is the next perf killer :(
maybe i can sacrifice some quality for speed during resize with the new ctx.textRendering, though.
...and one more thing. why is there no easy inverse of ctx.clip(), e.g. ctx.mask()? currently i think it requires resorting to awkward Path2D composition/cloning + evenodd ctx.fill() hacks:
It might be off-topic but it's worth to mention, there's a Path2d API which allows you to create a path and apply it to a canvas or check whether a specified point inside that path or not. It's useful when you want to do a collision detection. Point in polygon is a well-known problem and it might be a good idea to move it to the browser's context and trust.
These are nice additions, but what I've always wanted is proper a proper font measurement API. Right now, in order to measure the width of a line of text (for wrapping purposes), you have to render it offscreen and measure the pixels.
yes! I went through this a few years back. My god, what a pain in the ass. According to MDN, the TextMetrics API is fully supported by all modern browsers. Chrome added the necessary height support in version 87 it looks like. But the entire API is still marked as experimental. Maybe them moving Google Docs to canvas helped that out.
Chrome also introduced the Local Font Access API. Which sounds like a privacy nightmare. You can get access to the font's actual byte data and render it yourself, bypassing the browser entirely. Sounds insane to me. Imagine trying to beat your browser or OS at font rendering, but doing it in JavaScript. I guess Adobe or Figma have deep enough pockets to try though.
The TextMetrics thing is so confusing because the docs clearly say that it's supported, but I tried to use it as recently as last year, and it's definitely not working.
edit: getting access to the font bits sounds enticing to me actually, because I'm kind of a masochist :)
Are there any easy ways to try out drawing on a canvas in TypeScript, with full type annotations for the Canvas2D API so that I can get nice autocompletion and an easy ability to browse the API?
I like the idea of creating drawings directly in javascript even for my own purposes, but it'd be great if I had a type system to help me get my code right the first time...
This is as pointless as adding MathML to a browser.
Don't further bloat browsers. If developers need those features (which 99.9% do not), let them create and use libraries based on WebGL/WebGPU.
Or rather: go ahead!
The more stuff you add, the quicker a lite web (no JS/HTML, just webASM/webGPU) will emerge.
Just because you don’t use the feature doesn’t mean others don’t. I use Canvas2D regularly and enjoy the simplicity of it, it is definitely one of the best features of the Web APIs.
You might notice that their claim of 98% of canvas contexts being 2d rather than 3d is completely unsourced.
The reason is that the bulk is these most likely come from browser fingerprinting and a big chunk of the remainder is ads. Not something they want to highlight in this blog post.
What I want to know (but am too lazy to look up (-: ) is do the text metrics return the width and height of the text's bounding box, or is still just one of them? Can't believe this oversight has existed for so long, but that's the way the web works.
That said, I'm happy that canvas is getting some improvements. It's one of the more accessible ways to do fun things in a web browser.
Canvas2D is awesome, but generally speaking, when I built interactive stuff(in 2D), I either used SVG, or some library that used WebGL as a backed (even for 2D stuff), like PixiJS. That was the most reliable way to get the best performance across the largest amount of devices.
Nice to see these updates! I maintain a free/open-source Adobe Illustrator plug-in for generating Canvas drawings: https://blog.mikeswanson.com/ai2canvas
How about a native DOM Diff algorithm in the browser instead? I mean something actually useful for most front-end developers? This canvas2D stuff isn't, in the era of WebGL.
Have you tried ctx.imageSmoothingEnabled (boolean) and ctx.imageSmoothingQuality (string: 'low' | 'medium' | 'high')? From my experiments the iSE value seems to be respected across modern browsers, but the iSQ appears to be ignored.
There's also the ctx.smoothFont (boolean) value but I've not yet discovered a browser that cares about implementing it.
These aren't my needs particularly. They are just canvas' main good reason for existing, as every other use case is better covered by webgl and/or CSS. Canvas is the Direct2D of the web and development of the API should focus on its unique strength - namely, on being a straightforward high level API for performant rendering of 2D primitives. No one needs garbage API clutter like rounded rectangles. What we need is performance improvements because performance improvements are not a helper function anyone can write themselves in ten minutes.
roundRect is great. Though you don't need 4 arcTo in order to make a rounded rect, you can use bezier instead (we do). Their example is also 1% amusing because they set the `fillStyle` but then call `stroke` (and not `fill`). I'll have to do some performance comparisons, since that's the operative thing for my use case (and any library author).
text modifiers are very welcome. It's crazy how annoying measuring still is, especially if you want thinks to look perfectly consistent across browsers. Though the chrome dominance is making things easier in one way, I guess.
context.reset is kinda funny. Most high-performance canvas apps will never want to use it. For that matter you want to set all properties as little as possible, especially setting things like context.font, which are slow even if you're setting it to the same value. (Or it was, I haven't tested that in several years).
I'm sure most users know this by now, but generally for performance the fewer calls you make to the canvas and the context, the beter. This is even true of transforms: It's faster to make your own Matrix class, do all your own matrix translation, rotation, multiplication, etc, and then make a single call to `context.setTransform`, than it is to call the other context methods.
One of the greatest things to happen to 2D canvas in all these years was hardware acceleration. It used to be significantly slower. (You could switch to WebGL canvas, but at a cost of complexity and all kinds of annoyances rendering text, and if you had a lot of text it often wasn't worth it.) Thinking back, I'm also a little disappointed that after the death of Flash, it felt like Canvas never really took its place. The playful web that came before it in some ways simply closed up.