First, nice work. I'm certainly glad to see such a tool in this space right now. Besides a UI, what does this provide that something like Browserless doesn't?
Thanks! Wasn't familiar with Browserless but took a quick look. It seems they're very focused on the scraping use case. We're more focused on the agent use case. One of our first customers turned us on to this - they wanted to build an RPA automation to push data to a cloud EHR. The problem was it ran as a single page application with no URL routing, and had an extremely complex API for their backend that was difficult to reverse engineer. So automating the browser was the best way to integrate.
If you're trying to build an agent for a long-running job like that, you run into different problems:
- Failures are magnified as a workflow has multiple upstream dependencies and most scraping jobs don't.
- You have to account for different auth schemes (Oauth, password, magic link, etc)
- You have to implement token refresh logic for when sessions expire, unless you want to manually login several times per day
We don't have most of these features yet, but it's where we plan to focus.
And finally, we've licensed Finic under Apache 2.0 whereas Browserless is only available under a commercial license.
Sounds like a prooblem that can be solved with a Playwright script with a bit of error checking in it.
I think this needs more elaboration on what the Finic wrapper is adding to stock Playwright that can't just be achieved through more effective use of stock Playwright.
I recently implemented something for a use case similar to what they described. To make something like that work robustly is actually quite a bit more effort than playwright script with a bit of error checking. I have not tried the product, but if it does what it claims on back of the box it would be quite valuable if for nothing more than the time savings of figuring it all out on your own.
Prefect has more polish and is easier to get started than any of the existing options. We've been running their self-hosted for over three years and it basically stays out of the way.
We looked at Dagster as well as Airflow. I really, really liked Dagster but the BI team didn't.
I cannot imagine using Airflow for anything meaningful and respecting myself at the end of a work day. The local development experience was abysmal. Deployments sucked.
That being said, if you're not using anything except maybe cron right now, and if you don't care about the solution being a proper data pipelines orchestration platform trademark symbol, I'd recommend starting with Windmill.
Thanks! We're similar to Twingate in how we model resources, but our policy system has optional conditions you can apply to restrict access further, which works a bit differently. We expect to continue building in that area over time. We also use WireGuard as a transport while I believe Twingate operates over QUIC.
My understanding is that Twingate uses a service-based access model, rather than host/IP/ACL-based, as Wireguard defines the world.
As you are based on WG, have you somehow paperer over that to move away from network trust and lack of scalability (that I read across HN/Reddit/YT) inherent to other WG based solutions?
I may not be fully understanding the question, but I think you may be referring to DNS-based resources? Those will allow you to manage access to an app or service by its DNS name (wildcards supported). You can also use IP or CIDR resources as well of course.
In terms of scalability, are you referring to throughput or simply the complexity of policy management as the number of resources grows?
I refer to doing service based connections, abstracted away from whether its DNS, IP or something else. To do this you really need a private DNS function and to operate with attribute based access controls.
Complexity of policy mngt. I read that ACLs are fine at small scale but become a nightmare at larger enterprise scale.
Firezone's DNS-based routing is able to manage access to multiple services independently, even if they share the same IP address. So you could for example allow access to gitlab.company.com but not jira.company.com even if they were on the same webserver / loadbalancer.
It took a couple iterations to get it right - lots of fun edge cases involved. We ended up having to build automatic NAT64 and 46 for DNS resources to handle some of them. We wrote a post on how this works: https://www.firezone.dev/blog/how-dns-works-in-firezone
Yeah but none of this music slaps as hard as it did in my 20s. Don't get me wrong I appreciate it the same I'm sure, there's just such a thing as youth and emotion and I wouldn't trade that exploratory musical phase for anything.
There's quite a bit of new tooling in this space, selecting the right one is going to depend on your needs then you can spike from there. Check out Prefect, Dagster, Windmill, Airbyte (although the latter is more ELT than ETL).
…and, frankly, they are in most cases. Most “cloud native” apps are designed to run on cloud services like VendorHere cloud storage, cloud functions, cloud containers, etc.
A vanishingly few of them are actually self hostable.
