Big fan of this... suspect a lot of the modern SaaS BI / analytics players would benefit. Just end up hitting limits trying to use Javascript to render large datasets, having tried to build some of this myself. Happy to share some potential use cases / thoughts on where you could apply this if helpful! Feel free to reach out.
I know right? Hilariously bad title, especially considering the I/l ambiguity - a Boys Love novel? Bi-sexual computing? Bachine Learning?
Obviously jargon is a thing, we all deal with it, but when you put something up like this to a more general audience, you’ve got to give some thought to presentation, otherwise you end up with apparent word salad.
People inside a company prefer to have financial and operational information to make informed operational decisions or projections.
Sometimes that data is stored in a variety of places (excel files, a smattering of disconnected databases, sometimes enriching the data with more data pulled from an API...).
Extracting that data to a common location (historically called a data warehouse) tends to be the work of a data pipeline, and the tool used to
join, display, filter, dynamically aggregate, and visualize the data has been historically categorized as a "Business Intelligence" tool. Normally these BI tools provide data caching and allow temporary integration of multiple data sources.
The intent is to make it easy for business users to explore, analyze, visualize, share, and present datasets or results.
The biggest examples off the top of my head would be PowerBI, Tableau, and Apache Superset, but "data reporting tool" is a competitive market with many entrants.
A data warehouse is a specific use-case for a database.
> Sounds like essentially consolidating data in a database and visualizing it
Yes, and the database in which you consolidate the data is called a "data warehouse" In many cases the sources of data are ... not optimal for querying. (And I mean like "Spread across a dozen excel spreadsheets that people e-mail out new versions of when they make a change" by "not optimal")
Consolidating data makes sense and taking it from sources you can't query makes sense, but once you put it into a source you can query, that sounds like a database. I'm not sure why there is a different term here.
The database is often arranged and optimized differently than one used to support operations, and a lot of things are often done to make what would otherwise be long-running, slow queries faster -- e.g., aggregating or otherwise summarizing data likely to be queried in reports. But sure, it's still backed by a database.
What is the different architecture and what is the different use case? Databases are already general tools. You put data in and query it. At what point does it become a "data warehouse" ?
People argue over the definition of a data warehouse, but usually a data warehouse
(1) optimized for bulk inserts and bulk read operations, not updates or deletes, nor for single row updates
(2) stores data from multiple sources
(3) often a "columnstore" columnar table format is used. This format usually compresses columns in a format that is trivial to decompress (so if a column only contains a handful of similar flags, the database will run-length-encode them to optimize a full column scan), and additionally if you limit the number of columns accessed you can substantially reduce the cost of the query
(4) the data warehouse can easily shard or partition data
If you want to technical file format examples, I suggest looking up how the Parquet file format compresses data, or how Microsoft SQL Server columnstore tables work under-the-hood.
they're both general terms, but serve different purposes. Data Warehouses tend to focus on read-specific operations, more of an emphasis on historical and/or analytical problems than transactional ones. Subsequently they are likely to optimize data ingress, distributed queries and downplay referential integrity and consistency. The different workloads promote different compute models, but you're right that the line is blurry. Similarly you could ask "Excel has multiple tables (sheets) that you can join and query; is it a database"?
If there is ever a project that needed an electron app it’s this one. I’d love to have a Perspective.app that efficiently loads local data style files.
Electron is going to kill the good performance gains of Perspective. Even if you have a fully beefed-up workstation, Electron is going to trigger the CPU fans.
Electron is discouraged nowadays in favor of lightweight solutions like Sciter[0], Tauri[1] or even WebView2[2].
I don't see why rendering this in Electron (which is essentially just a bundled Chromium browser) would perform differently than in a standalone browser. Neither do I see how projects like Tauri or WebView2 which just embed the system webview would give better rendering performance. Microsofts webview is even using the same rendering engine as Electron - Blink. What you'd get is significantly smaller binary size but most people don't care much about that.
> I don't see why rendering this in Electron (which is essentially just a bundled Chromium browser) would perform differently than in a standalone browser.
WebView2 has further optimizations as claimed by Microsoft in their recent blog post about Microsoft Teams[0]: "The key benefits observed from the transition from Electron to WebView2 include reduced memory usage and a lowered disk footprint as resources are shared with Edge. Additionally, we have been able to take greater advantage of the native capabilities provided by WebView2 and ensure support for more up-to-date versions of Chromium (latest performance and security updates)."
So, I'd assume WebView2 is better than Electron.
> What you'd get is significantly smaller binary size but most people don't care much about that.
In the case of Sciter (quoted in my parent comment), Sciter is a bit different: The frontend is HTML, CSS and JS, but the app's local backend is whatever you want it to be: C#, C++, Python, Assembly, etc. That's why it's popular among performance-critical software such as malware scanners, where you need native performance but as well as a you know customers that like a nice UI.
Any kind of software could benefit from Sciter's approach, if just companies liked to drop a bit more of budget at their shipped products.
You said Electron would kill the performance gains of Perspective. You also said it would make a workstation fans spin somehow.
The performance gains in rendering of Perspective are due to using canvas + Webassembly. This would be exactly the same in Tauri or Webview2 because it would run in the same browser rendering engines which execute the Canvas renderer and Webassembly like in a browser or in Electron.
> The key benefits observed from the transition from Electron to WebView2 include reduced memory usage and a lowered disk footprint as resources are shared with Edge.
That's exactly pretty much where the benefits end. The memory usage reduction is even debatable if you are not running Edge as your browser. See WebView2 as an installed Electron core that's pre-installed on Windows. Doesn't change rendering performance.
> In the case of Sciter (quoted in my parent comment), Sciter is a bit different: The frontend is HTML, CSS and JS, but the app's local backend is whatever you want it to be: C#, C++, Python, Assembly, etc. That's why it's popular among performance-critical software such as malware scanners, where you need native performance but as well as a you know customers that like a nice UI.
Yea that's why I didn't list Sciter in my reply. With Sciter you don't get the system webview - you get a custom minimalistic web html/css rendering engine that supports a limited subset of the HTML5/CSS3 spec. There's no WebGL and there's no Webassembly. So you'd have to bring your own libraries for that. Easy enough with Webassembly and they could even skip that part and go native but with Canvas and trying to run on all major platforms... it's just a lot of added work for minor gains.
I'm not against libraries like Tauri. Perspective could totally use that to make a desktop app. It would not have any advantage when it comes to rendering performance though.
Their demos performed fine for me on both desktop and mobile. Not sure what the issue is here, you're allowed to require some baseline performance level for something like this.
I had no performance problems, but right on the first demo each scrollbar is broken in it's own unique way.
The main area with the sparkgrid has good horizontal and vertical scrollbars, but they are nearly invisible, being dark grey on slightly lighter dark grey, and only visible if you hover in the area they scroll. Also the vertical scroll bar doesn't take you to the very bottom.
Then there's the top right area, which has a normal looking scrollbar (good), but once you scroll it changes the size of the entire right panel (weird), and if you drag it past the point where it has reached the bottom (really easy to do accidentally) it starts spasming uncontrollably. If you instead use the scroll wheel, you can scroll to the bottom, but if you scroll a bit further you are back on the top? None of the other scrollbars has either of those issues.
Then there's the bottom right, where the scrollbar is visible, takes you all the way to the bottom, and neither spasms nor resets, but just to be special in its own way it starts out as a big bar as if there was little content below the fold, but the further you scroll the smaller the bar gets. As you scroll up, it grows again. It's also stuttering a bit while making these size changes.
Another example besides scroll bars is the lack of tool tips on the UI (they do exist on data though!), which makes for a weird toolbar that moves its content all over the place on hover. And I guess you're supposed to guess that check boxes are delete buttons.
This doesn't take away from the app, it's still cool and useful, but little annoyances like that add up to make it feel weird and un-native (I guess foreign is the word).
It sounds like you have some browser/OS combination with aggressive scroll inertia enabled. A bug report on our GitHub with some of system details would be appreciated!
As a Firefox user I definitely want to see better FF support here!
The first step is to get a good test suite. Currently our tests only run on Chrome. So after we get tests running on Firefox, we can move to better support it.
Only the "treemap" chart is broken for me, everything else seems to work ok (Firefox Developer Edition 111.0b6 64-bit Linux. Enhanced tracking protection turned on).
I'm having issues with it loading in Safari (14.1.2) on OS X, but it seems to be loading fine with Firefox (111.0.1) for me. Its pretty zippy too, and this computer is no spring chicken...
No doubt I have poor a11y on my own projects - but if you are going to launch into critique of the landing page, at least do it from a position of authority :)
TBH, I am not sure how a charting lib could effectively be accessible to any great extent. Its not covered in the resources you linked, and in my experience with other charting libs this seems to be commonplace. I would be interested if you have any insights or suggestions on that, as I use charts extensively on one of my projects.
If we are going to go this way: You previously commented that something doesn't work on Firefox, do all your project superior browser support? That doesn't make sense does it? What does my website which I updated 6 years ago (and created more than a decade ago) have anything to do with the validity of my comment? It just has a contrast issue (let me fix that BTW [edit: done]), a criteria that was set way later than I built that site. Also this is a component library aimed at businesses, not a personal website that I'm commenting about. Please stop trying to poison the well (going as far as hunting a11y issues on my personal site) and argue on merits.
I do it from a position of support, which I had the chance of developing by observing the users of the apps I created, and I made a giant software suit (more than 1000 views) pass an accessibility audit in my current company.
> If we are going to go this way: You previously commented that something doesn't work on Firefox, do all your project superior browser support?
It was in a thread specifically discussing browser support for the guys project where he stated he only has the ability to test on chrome/firefox on fedora: https://news.ycombinator.com/item?id=35448674
> a criteria that was set way later than I built that site
It is highcharts that I use extensively, and I completely disagree that a11y is a solved problem there. Ironically on the referenced page they make the same mistake as you regarding color shades.
Apparently 2001 (and I stand corrected). But well, I'm learning and improving am I not? I was a worse web developer a decade ago than I am now, and I am proud of that.
> Stop being so hostile.
You are the one who analyzed my personal website and reported a single color contrast issue to dismiss my complaint, why am I hostile? Regardless of that discussion, isn't my point coming across: Do we need to be authority on issues we report?
I tried to help, linked some resources, why is that wrong? I feel rather you being hostile.
> Ironically on the referenced page they make the same mistake as you regarding color shades.
Okay now you get to criticize open source libraries?
Anyway, few years ago I created an SVG based charting library for internal use, and we did (and still have) users with sight impairments, and yes it is hard to make it right. NVDA is a pain to bring under control, at least :) And a colleague of mine said something I really like: You cannot make software completely accessible, even for a perfectly capable person with above-human IQ, no, but you can make it always more accessible. It's great that they are trying!
> I can do this all day
What you are doing is something I sincerely don't understand. Teaching me a lesson? Proving I was over the line? Reading my first comment, I did see it's a bit whiny, and in another branch I tried to apologize. Whatever it is, maybe we are misunderstanding each other and perhaps everything is coming across more and more hostile. I do apologize from you if it came across to you like that too.
Its exactly what I think it is: "Ensure that foreground and background color combinations provide sufficient contrast" (with reference to lighthouse.org for specifics about contrast).
They didn't specifically define that ratio in the doc until WCAG 2, but thats irrelevant really.
> They didn't specifically define "sufficient" until WCAG 2, but thats irrelevant really.
It is relevant. I didn't have any means to test it, and that's what I saw as sufficient back then. You never admit you were wrong, I assume?
edit: you edited your comment, so here is the lighthouse.org page: https://web.archive.org/web/20080211110529/http://www.lighth... There was no ratio, so it's just my gut feeling which WCAG 2 had apparently decided was not enough and I didn't know it back then - not that there were enough tooling.
They explain how to test it on the WCAG link you gave. I never said they mentioned a ratio on the lighthouse page. Pretty sure i have not been wrong about anything in this discussion so far... Shall we continue?
Oh you were (like your very last points even), you just keep changing focus. But I'll let it be because I have no more time for these. good luck with your further discussions :)
My focus has solely been on the contrast aspect, as per my original post. But at least we have cleared up your misunderstandings, so I guess we can leave it there. Have a good day!
Perhaps you need to re-read, it was never just the contrast, see my first reply to you, and try to understand my point, which you yourself proved later.
You were right from the start, I had created a site 9 years ago that gets only 65 points in an accessibility check today! So I can't suggest anyone to focus on accessibility?!
What are your eyes going to do with such a table? People pause right? So it would be nice to have some help to navigate. aria-sort in that example would help a lot, also some menus usable with a keyboard, and not disabling outlines.
Not sure what specifically you are looking at - the data grid element e.g. is a well-formed, native HTML `<table>`, as recommended by the w3 [1] and renders fine AFAICT with the screen readers in Safari and Chrome (the browsers I have installed).
A specific report on the GitHub for issues you find would be much appreciated!
I was looking at the datagrid. I can't use the menus with the keyboard. I can't select columns. I can't see the currently focused element. My screen reader does not inform me of the menus, nor can I interact with them. Inputs have outline: none. Labels are not connected to the inputs.
I can't use the tree with the keyboard. From the patterns page I linked, you can see an implementation example for an accessible tree element.
I mean, I can continue. I'm not saying this to devalue your work, and I wish I had the time to go in depth and analyze all the components and give you more feedback, but I don't. So I humbly suggest you do a bit research about the topic and improve the product some.
Perhaps my original comment sounded too dismissive and therefore got many negative feedback, but I'm sincerely concerned. I wanted to hint at this as this is a tool that may be used by businesses and some people cannot do their job because of inaccessible tooling.
I do like what you are doing and the components do look cool - nothing to take away from that. Please take my comment as constructive as you can take it to be because I was trying to go for that, and perhaps my frustration in the last few years with this topic made me also whine a little bit :)
Like I said - GitHub is the preferred method of reporting issues:
* Some of these (keyboard nav, focus) we test for and sounds like a specific issue with your browser/OS setup, which we'd very much like to fix. GitHub Issues allow us to ask these followup context questions.
* Some of these are vague, I'm not aware of any label+input in the component, e.g., nor how column selection is related to accessibility. The venue for this discovery is GitHub Issues - just because they are obvious to you does not make them obvious to us such that RTFM is a suitable correction, humbly or otherwise.
The project has relatively wide financial industry usage (and some of the code was originally developed at JPMC), and we're very interested in fixing real issues!
Well, I'm sorry, I can only help so much. I also try to open source many things as I can, and I'm active on Github in many projects but I have only so much time.
> Some of these (keyboard nav, focus) we test for and sounds like a specific issue with your browser/OS setup
It's probably not (tried on another computer and browser). Please open this page: https://perspective.finos.org/blocks/superstore/index.html Try to navigate with your keyboard. There are no outlines and the tree view is not possible to interact with.
> I'm not aware of any label+input in the component
The column headers aren't possible to interact with a keyboard anyway, and the screen reader doesn't announce anything to indicate that the table can be sorted.
In my current job, we paid a lot of money for an army of auditors to hunt these down for us, and I don't expect that you do something like that for an open-source product, but if you have corporate use (I mean if people are paying for this directly or indirectly, and I hope they do!), maybe it makes sense to go for one?
I just commented about something I found lacking, it's not a hostile comment, and suggested improvements to the authors who seem to be hanging out here. I think if they have interest, they can add these as an issue, or choose not to.
To the downvoters: Do you really think I should open issues for every critic I place to open source projects here and otherwise not mention anything? I also love and contribute to open-source but this feels like too demanding.
