Ditto uses your personal information in the following instances: ...
Deliver direct marketing communications to you regarding our products and services that we may think are of interest to you. ... We share Personal Information with third parties ... Depending on how you use the Site or Services, the following categories of third parties collect data on our behalf or receive Personal Information ... Advertising and marketing partners.[1]
Adam (Ditto cofounder) - thanks for the feedback! I can see how this can create the wrong impression and will follow up with our counsel to improve the language.
To be clear, we are not in the business of selling personal information. We are in the "picks and shovels" business - building great tools for other developers to use. Both myself and cofounder Max are app developers by background and we got into infrastructure software by wanting to simplify app development for ourselves and others. P2P is highly motivating because it enables use-cases that have greater data control and privacy - both of which are deeply important to us.
You wrote your terms, which officially say you are in the business of selling personal information. Informal comments to the contrary have little contractual value.
This interpretation of the terms is straining it almost to the breaking point. The section you quoted says explicitly:
"We do not allow our third-party service providers to use your Personal Information for their own purposes and only permit them to process your Personal Information for specified purposes and in accordance with our instructions, unless the data is rendered fully anonymous."
You didn't quote this part.
To me, the most straightforward reading is that Ditto does their own advertising, and targets it to specific users by using third party advertisers (Ditto paying the advertisers to place an ad, not the advertisers paying Ditto for the information), who are contractually bound to only use that information to deliver the ad that Ditto placed and not to use the information for their own future benefit.
The only possible reading I can see where this can be interpreted as selling personal information is the fact that the "Advertising and marketing partners" bullet doesn't use the phrase "service providers", and that thus, the contractual restriction on "service providers" could thereby exempt that bullet. Is this what you are hanging your interpretation on?
(Reason for my interest: I regularly read privacy policies, looking for loopholes; there's a huge range not just in privacy practices but in how clearly they are described. Ditto's could be clearer - perhaps by including an explicit statement that they don't sell personal information, as some do - but Ditto's privacy policy is not what that of a company who sells personal information as part of their business model looks like.)
I'm mostly a software developer but I've long been interested in legal stuff on the side as long as I can remember. Not for career reasons, I just find it interesting. It's not always boring! For a nonprofit I've volunteered for for a long time, I ended up helping a lot with bylaws and policy writing. Reading the terms and policy documents of a company, in addition to learning the actual rules, gives you a glimpse into their culture (granted, a noisy glimpse - may mean nothing more than "founder doesn't give the terms any mindshare so just had their lawyer do everything bog-standard"). Terms/privacy policies are often a factor in products I choose to use (or not use).
This is even worse than google/facebook... with them, you're pretty sure they won't sell your data, but only sell your eyes to advertisers... here, they sell bulk, raw data to possibly many third parties.
>Ditto is designed for "peer-to-peer" synchronization where it can directly communicate with other devices even without an internet connection. In addition, Ditto automatically manages the complexity of using multiple network transports, like Bluetooth, P2P Wi-Fi, and Local Area Network, to find and connect to other devices and then synchronize any changes.[1]
This is cool! I have no idea what's the use-case behind it, but it is really awesome if properly implemented (which seems like a gigantic challenge).
Super cool to see us on HN! The use cases are for projects, products, enterprises to build collaborative apps that can sync with the internet but fall back to local P2P connections when internet connectivity fails. Furthermore we added offline first as an additional guard for some resiliency.
We help solve a class of apps like:
1. Desk-less worker apps for the enterprise: think people who use mobile apps for the vast majority of their work: pilots, flight attendants, construction, maintenance
2. Retail, inventory warehousing, point of sale systems where being able to communicate between kiosks, service tablets, and display systems quickly even if there are interruptions in service
3. Education: it’s very often parents give their kids phones without a data plan, so their school or learning apps need to be able to collaborate with other students or teachers regardless of connectivity
4. Robotics, agriculture technology, IoT - often these applications have hard requirements to communicate with other local devices without any dependency of the internet
We are in the process of buying it for retail applications (just signed the paperwork) and so far so good. I would ask that you make the documentation a little easier to find from the home page. Engineers want things to be easy ya know :).
EDIT: is there ever a plan to open this as a spec?
One thing I don't fully understand is how KV stores like this fit into the overall architecture of an app. Is Ditto supposed to completely replace SQL or is there some Ditto DB <-> SQL DB syncing mechanism happening?
Adam (Ditto cofounder) - Ditto can be used either as the primary datastore in the client app (we call these "Small Peers"), or alongside others local DBs (existing apps adopting Ditto tend to pick off key P2P features and run Ditto with others like Couchbase, SQLite, or Core Data - and we have built connectors to help with this). With regards to the backend, our server product, or "Big Peer" provides webhooks or a Kafka topic of all changes to pipe them into other systems, so it similarly can be a primary datastore or middleware.
All-in-all, the design of Ditto is to be flexible so users can add Ditto to their architecture as easy as possible.
My vision is that all the current use of IT should work in this mode. Depending on a central server for synchronization should be the exception, not the norm.
Interestingly, most of the Ditto folks originally worked on Realm, an object database that eventually monetized into the "sync via a central server" space.
Of course, Realm was later acquired by MongoDB, and currently exists as some kind of Frankenstein's monster product in an attempt to drum up Atlas sales.
Interesting how the acquisition pushed some of the devs to create better tech, but they had to give up the original product along the way.
Agreed, and I expect it to set new standards for user expectations. Users are tired of slow and fragile server interactions on every click, or managing connectivity during use.
I'm not sure about latency since I haven't tried Ditto, but there's a class of in-person multiplayer games that would benefit from something like this. See Spaceteam[0] for an example.
Another use case that comes to mind is US Air Force bomber drones having navigation and peer to peer capability through intermittent connectivity with central control
edit: turns out my guess was essentially correct! that general use case is indeed what the USAF is using Ditto for https://www.sbir.gov/node/1629457
I’ve thought about building a messaging app the doesn’t depend on cell networks, it could be useful for example if you are on a snowboarding trip or mountain biking and need to communicate with a group of friends, but in a place with spotty cell service. I’ve played around with Apples P2P networking framework but it was pretty clunky, this looks like it would be much easier to use.
So happy to see the support from the Hacker News community.
I should mention Ditto is a massively ambitious company, we are doing fantastic with traction across major industries.
Our entire core code base is written in Rust and I know HN goes crazy for Rust. If you’re really interested in working on very hard problems like CRDTs, partial replication, query based replication, peer to peer adhoc sync, mesh network or distributed security: please definitely take a look at our openings.
www.ditto.live/jobs
You’ll work on problems that are incredibly difficult, unique and rewarding that you won’t find anywhere else!
Adam (cofounder) - Yes we evaluate applicants holistically. Rust is still newish and we have a mix of team members that came with lots of experience to just some familiarity. I have a strong philosophy that Ditto's success starts with a great team (build the inputs to get outputs concept), so aptitude is a major factor - please apply if interested!
Hi Max, this is a really cool technology and touches a lot of areas I'm personally interested in. If I wasn't happy with my current job I'd even consider applying.
I had a poke around your documentation but didn't find a high level system architecture. Without having to dig into the open source code could you provide a link to somewhere I can read about the underlying data model and how it works? Do you have a paper or patent describing this technology anywhere? Thanks!
I did some digging as well and it looks like their docs have a pretty good overview [1]. (And to be perfectly honest, it looks like they've done a very thorough job with them! I'm impressed!)
Being always on the road, I often have my emails/contacts/todolist/pictures/whatever scatered accross all my devices, with no easy way to synchronize them without getting access to wifi.
This means in transport, in the country side, or in another country, my system is in shamble. Also, it means everything is slow to sync.
Yesterday I had to send a AI model to my colleague next to me. I was on linux, he was on windows. The fastest way was to get a usb stick. That should be a click in 2022. The closest to something reliable to do this was duckto, but it's not maintained anymore.
Syncthing is painless, with available local network detection of other clients. If either device has a webcam, the QR codes in the web UI make it quite quick to have a synchronized folder between two systems.
Yeah it's really cool. I tried to introduce it years ago, but something you must pip install is a deal breaker for most people, even python devs. Too many ways to fail. Besides, casual drag and drop is always going to be easier than cli for windows users.
For a lot of simple use cases that don't need automatic conflict resolution you could just share files using Syncthing. Then the only dependency on external infrastructure is the Syncthing STUN server if one or both peers is behind a NAT.
Why does "everyone lose" if a well-run open source initiative forks it and runs it better? That seems like like an improvement for everyone except maybe the original maintainer of the project, if they don't want to join up with the rest of the people.
Might seem like odd questions, but is there a plan to handle the "too much data" case? and how protocol agnostic can this be? This seems really cool, and I can think of some strait forward uses for it ... but working on a satellite project where communications are 75% of the hassle, this sort of thing seems like it could be very useful...
If it can deal with all the "weird" (its very normal for space, just not used anywhere else) batched radio communication standards from the Consultative Committee for Space Data Systems (CCSDS) then intermittent communications aren't as big of a problem.
However it will also need to handle "old data that can be deleted client side". A satellite will generate a lot of data that it doesn't need to keep but will want to transfer to the ground and keep on the ground, telemetry and tracking metrics, system statuses etc. One way is to keep all this buffered on the satellite, and transmit to the ground, then you can throw it away, etc.. But having a synchronised store is a much simpler programming model, you push important data into the store and let it look after itself, and simple code is easier to make more reliable and reliability is key for anything in aerospace. However it can't result in the remote device being "full", it's a 100% non-starter. The client/remote end has to be able to throw away old/expired data while the central store is able to keep that data, without requiring awkward secondary systems polling the latest state on the ground side to save current data into a separate system to store historical data, that would just be shifting the complexity around.
Adam (cofounder) - great question! Yes, this situation is very common for our use-cases. Ditto supports the concept of `eviction` [1]. For example, imagine a restaurant that wants to store all of today's orders in their devices, but then evict the data when they close out at end of day (and data is synced into Cloud backend). Eviction is a core concept because we consider devices using Ditto's SDK a "Small peer" that should only have the data it wants - queries define which data is synced and you can always evict anything unneeded. Conversely, the server version of Ditto is a "Big Peer" that is a cluster designed for scalability and fault tolerance, so it wants to capture everything. Another way to put it is selfish vs. greedy replication.
To help users with eviction, we are adding support for dynamic queries, for example, using "now" for a date in the query to aid in creating TTL logic. In addition, to built-in TTL as well.
Satellite comms are definitely a great fit for another reason, since Ditto's replication is "delta-based" it will only sync differences. Our protocol is designed to overlay on top of custom arbitrary links, including unreliable ones. Not familiar with CCSDS, but we have explored Link16, and even built-in we replicate over BLE GATT.
Hope that helps and perhaps we can get Ditto into space!
Thanks for the reply! I had a very quick look at the documentation and didn’t see the eviction information. That’s excellent to know it’s already a built in capability. As for space, don’t worry if you’ve never heard of CCSDS, it’s just yet another standards group for various things, the key thing is that it’s delta based and already designed to handle non-TCP-IP based data transfer mechanisms. One of the big challenges I’ve found with bringing Linux and other modern software into an environment where “normal” (you can get a 24/7 IP link, but it’s unusual for satellite to maintain an open link like that) space communication is that a lot of useful modern open source tools is just not designed for a non IP based networking environment, so much so that you dig into things like delay tolerant networks and you’ll find mechanisms for trying to fake TCP-IP over the top of them just to help accomodate this software. Having a mechanism to replicate important data on the basis of just a transmission mechanism agnostic “packet” of delta update data, would be a huge boon for low cost satellite systems.
Im definitely going to have to dig in deeper and check out the rust stack! I’ll be sure to get in touch about the various CCSDS data transmission related stuff once I get there, since if you’ve looked at Link16 I’m sure you’re probably interested in adding more aerospace communications link standards/mechanisms.
For a split second I was wondering about the lesser known ditto command on macOS, and whether it had some new superpower features I did not know about...
Looks interesting but a few questions I don’t see answered in the docs:
How do permissions work? Ie if I had a notes app, and wanted users to only be able to upset and query _their_ notes, how do I make that happen?
Why no LAN for web browser mode? A pwa with offline support should be able to sync with other resolvable ips on the same pan, no? Hopefully WebBluetooth will be available soon too?
Regardless you definitely wouldn't be able to do browser-to-browser discovery and connection via HTTP or WebSockets, since you can't create a web server in browser, but I imagine you could do P2P via WebRTC if you can just find a way to discover the other peer's details in the first place.
Webrtc allows getting the local IP address. maybe these could be distributed manually or when a discovery service is available, and then tried when it's not?
Great concept! @mbalex99 any plans to add a GraphQL interface to it? This would enable many existing platforms to "easily" migrate to a platform like this.
No unfortunately not in our core offering. We’d like to stick to what we do best and then leave it up to the community next year to build a graphql implementation.
Good thing that it supports any authentication provider. Less good that it still needs a roundtrip to Ditto's servers to enroll each device after authentication.
Alternatively, for "truly" offline sync, a "license" token must still be obtained from Ditto's servers at some initial deployment phase.
Really cool library, but the dependency on Ditto's infrastructure is an itch to scratch.
You hit the nail on the head. We are working on this as a major initiative to solve the problem you detailed. As you can guess the truly decentralized authentication is quite new and we are releasing some features to get us less reliant on online services for connectivity.
Thank you! After feedback from a lot of our customers, they want more access to explicit types so you'll see a lot more APIs to control how you want conflict resolution.
Do you think it'd be possible to use your mesh network layer as a generalized network transport? e.g., I have ethernet, wifi, and BLE devices, and the BLE device is able to access the network and/or internet via one of the other two devices?
(Ditto developer here) Yes most definitely! We are rolling out a feature called Ditto Bus which lets you open reliable or lossy streams for arbitrary data flows between between your device and another in the mesh. Each stream flows over any transport available and will multihop via the most efficient path if needed. An early version of this is already available in the iOS SDK.
It's a powerful building block that can be used for proxying any kind of other traffic, either yourself as an app developer or if we end up providing proxy/VPN/other functionality as part of the Ditto SDK.
slightly o/t: do you have any stats for end to end latency between n number of nodes, and between the different connectivity types? I'm curious how workable this would be for a game with real time logic.
How do they get peer-to-peer discovery in a browser working without server in an intranet ? As far as I know websockets do not support IP finding (though there where some leaks once)
PS: They don't. Was wondering whether I missed something
That’s correct the browser limits us to do automatic discovery sadly.
WebBLE support is on our roadmap though!
However, if you use Ditto in an electron (and soon Tauri) you’ll be able to take advantage of multicast, BLE, P2P WiFi etc… while “feeling browser”-like with your development.
Very Clever. From a compliance perspective - How much back-end infrastructure is required to bootstrap the sync process for the devices? Can this be deployed on customer / private hardware?
Ben (product engineer) - Ditto can be used purely P2P with only users' devices or in combination with our managed Ditto Cloud. Alternatively, you could use this with your own infrastructure with as little as one box. It really depends on how much data your application has and creates as to the size of server you would need.
A) PouchDB, Couchbase, Firebase, Realm are all databases that can sync to a "master" node in the cloud.
B) AODV and BATMAN are really an ad-hoc mesh networking and routing protocol.
Ditto is both A and B. You work with Ditto as a database on your mobile, IoT, web app with common database functions (querying, updating, deleting etc...) and we will sync the changes between edge and cloud devices. Most developers cannot sensibly use AODV and BATMAN to build robust collaborative applications, it's too hard. We abstract all of the routing, network resiliency, and replication away from you; just work with the database
What is the proudly-displayed use case from the US Airforce?
Something with its new unguided bombs they’re testing, or the new bomber it’s rolling out this year? The hundreds of Phoenix Ghost “Kamikaze” drone bombs they just revealed?
Are programs like this funding this project? I will pass.
USAF just happens to be a customer, and I really don't believe Ditto has any strong stance one way or the other on this. This statement falls into the "Charlie Manson openly liked McDonalds so as a McDonalds franchisee you openly support serial killers" bucket.
The stance they’re advertising evidently is the contractual partnership they chose to enter - that they agreed to take their money and provide them support on their integrations.
Why are you simping for them without basis? Your absurd comparison would only make sense if the franchise the killer shopped at continued to provide them service and used them in marketing materials as a marker of trust or prestige.
But how do you know exactly in what capacity they are using this technology? The USAF flies in millions of tons of humanitarian aid and personnel every year - far beyond what any other country, military, or organization does solely.
It's not a "simp" as you say, only pointing out that they are a rather large customer who is free to implement the technology in any way they wish. You're letting your political bias undermine any sort of credible argument.
What political bias exactly do you think I’m exhibiting? An anti military bias is valid regardless, there are successful companies that choose not to engage
Btw my post was asking what the air force is using it for. I know they do a variety of things that’s why I asked.
edit: Thank you, below, for the details - so my guesses were more or less "spot on" (communication systems connectivity for things they put in the air) or less charitably, within the bounds of the details in that link.
The rest of this silly argument aside: the USAF flies millions of tons of humanitarian aid as a PR move for itself (helping justifying the existence of a standing "professional" military, a relatively new thing for the US) and to further the foreign policy interests of the United States.
Ditto uses your personal information in the following instances: ... Deliver direct marketing communications to you regarding our products and services that we may think are of interest to you. ... We share Personal Information with third parties ... Depending on how you use the Site or Services, the following categories of third parties collect data on our behalf or receive Personal Information ... Advertising and marketing partners.[1]
That pretty much says what they're really doing.
[1] https://www.ditto.live/legal/privacy-policy