My take is that one of Datomic's original value props was that it ran in your Java app's process. That plus immutability plus cache-foo allowed you to write your Java app as-if your entire db was not only "in-memory", but "in-your-app's-process-memory." That is, um, "pretty dope".
Then they introduced Datomic Client, which is more of the "normal" db paradigm. Your app includes the client lib, which connects to the running db process "somewhere else". I don't know why they did this (easier to include?), but it meant that the "db-in-your-app-process" value prop went away.
Initially when they deployed Datamic Cloud (a sort of managed Datamic service deployed in AWS), they only supported the client model.
That is context for your question. The problem Ions solves is how to get that "db-in-your-app-process" value prop while using Datamic Cloud. It seems to me that it effectively reverses the original method. Original method was add the DB as a lib into your app, Ions method is push your app into the db (which is running in a configured-for-you cluster on AWS).
(It also allows you to not have to provision/configure servers to actually run your app code in addition to the db servers, but that seems secondary to me.)
Note, that I'm not affiliated with Cognitect, and I've never used Datamic, or Clojure in a production system. So have a salt shaker.
> I don't know why they did this (easier to include?)
I wanted to use Datomic, but none of the languages I work in run on the JVM. Datomic Client is really the only way I could. (Though I also considered, briefly, reimplementing the entire "transactor" library for this other platform, since the transactor living "inside" your process really is a selling point.)
An "in memory" DB is just a DB that can answer queries quickly (e.g. in O(1) time.) You still speak to it over a network protocol (even if it's going over the loopback interface), where everything gets serialized into long binary strings and then deserialized again. You can maybe achieve zero-copy access into the packets the DB is sending you if the DB is communicating using something like Capn Proto, but that's still two copies of the data: one in your app process, and one in the DB process.
A database whose data is in your process (like, say, BDB, or Erlang's ETS, and not like SQLite†) allows you to hold pointers to data (usually either "tuples" or "objects") that are in the DB, from your process, and treat them as if they were just regular data sitting on the stack/heap. Depending on the DB's persistence architecture, this can mean that you can even mutate the data in object form, and that mutation will be automatically persisted—because what you're mutating is, after all, the DB's copy of the data. Unlike e.g. Firebase, there's no secondary synchronization between "your local" copy of an object and "the database's copy" of the object; your process is holding the database's copy of the object.
Or, if you like, you can think of an "in-process-memory" database like a server in a distributed-RPC stack, ala CORBA or Windows COM. You ask the DB for an object, it gives you an RPC handle to the object. Except, since you and the COM server share memory, this handle doesn't need to do any marshalling or IPC or locking; it's an optimized-path "direct" handle.
Another potential path to enlightenment: if you create a "flat-file database" by just mmap(2)ing a file—and then you write structs into that file-backed memory, and keep pointers to those structs—then you've got an "in-process-memory database."
† SQLite the library resides in your process's memory, and it can even allocate tables that reside only in-memory (which is also "your process's memory") but SQLite still communicates with your process through the SQL protocol, as if it was on the other side of a socket to your process, with no way to "get at" SQLite's internal representation of its data.
> An old work of philosophy does not describe the thing you are meant to be learning about. It was created by the thing you are meant to be learning about, much like watching a video from skater-Aristotle’s GoPro. And the value proposition is that with this high resolution Aristotle’s-eye-view, you can infer the motions.
tldr; read old philosophy to learn how to reason, read new physics to learn the results of reasoning.
So, in a metaphorical way, reading old philosophy isn't about reading a legacy code base. It's about reading the git history and project documentation.
Hmm, I'd say more like reading the (original) source code of git itself to learn about Linus and his coding"philosophy" [0] vs reading a modern tutorial about how to use git.
[0] I haven't done this so not putting any value prop on this specific example.
> We wanted to know: What would it look like if you built a technology startup which could not make anyone rich. If you eliminated all the promises of wealth from the roadmap up front, and tried to build a good company, how would that affect the product, business, customers, and every little decision in between?
...
> We're currently looking at testing an affiliate program whereby we'll pay out a 30% commission every single month on the lifetime revenue of anyone referred to Ghost(Pro).
...
I believe this is called colocating your money and your mouth. I'm inspired.
I agree, I don't like the magic string approach (even if it is mostly just dot-notation attribute lookup). However, there is some good stuff here, and nested data lookup when value existence is unknown is a pain point for me.
In addition to the string based lookup, it looks like there is an attempt at a pythonic approach:
from glom import T
spec = T['system']['planets'][-1].values()
glom(target, spec)
# ['jupiter', 69]
For me though, while I can understand what is going on, it doesn't feel pythonic.
Here's what I would love to see:
from glom import nested
nested(target)['system']['planets'][-1].values()
And I would love (perhaps debatably) for that to be effectively equivalent to:
nested(target).system.planets[-1].values()
Possible?
---
edit: Ignore the above idea. I thought about this a bit more and the issue that your T object solves is that in my version:
nested(target)['system']
the result is ambiguous. Is this the end of the path query and should return original non-defaulting dict, or the middle of the path query and should return a defaulting dict? Unknown.
Objects are overkill. Use an iterable of getitem lookups. They compose easily and don’t have much overhead (cpu or cognitive). From the library I use at work, inspired by clojure.core/get-in, it would look like this:
def get_in(obj, lookup, default=None):
""" Walk obj via __getitem__ for each lookup,
returning the final value of the lookup or default.
"""
tmp = obj
for l in lookup:
try:
tmp = tmp[l]
except (KeyError, IndexError, TypeError):
return default
return tmp
data = {“foo”: {“bar”: [“spam”, “eggs”]}}
# find eggs
get_in(data, [“foo”, “bar”, 1])
By using __getitem__ you naturally work with anything in the python ecosystem.
I prefer your nested() version. I don't see why your final example would be ambiguous either, if it always returns a defaulting dict which you always have to extract with .values() there is no ambiguity. This is similar to how Javas Optional<T> work. The problem could be that the type checker isn't good enough and writing code like if nested(target)['system'] == "lazer" could pass the type checker.
The other problem that T solves and nested doesn't is being able to reuse a spec. Once you have a spec, whether created with T or directly, you can call glom multiple times with the same spec, pass the spec to another function, tc.
ncurses seems powerful. Any good suggestions on a tutorial or introduction to ncurses (which it appears is what was used for the ui here)? Ideally for python, or generalized... Or is there a better tool to learn for generic command line UIs?
Kilo is a great way to learn how to manipulate the terminal directly (rather than using a wrapper library like ncurses): https://github.com/antirez/kilo (A text editor in less than 1000 LOC with syntax highlight and search.)
My own version adds Lua support, multiple buffers, and similar features. It is by no means the best of the editors but I had fun playing with it - to the extent I was almost tempted to write an editor. (But then I realized I already have vim for writing email, and emacs for everything else. The world really doesn't need another editor!)
If you want to build interactive terminal interfaces, check out blessed [0]. Even better, you can use react-blessed [1] to write declarative UI code! You can also find additional widgets in blessed-contrib [2].
For actually shipping the code around, you could use pkg [3] to generate a single binary. That lets you avoid having to install node or any supporting libraries.
Then they introduced Datomic Client, which is more of the "normal" db paradigm. Your app includes the client lib, which connects to the running db process "somewhere else". I don't know why they did this (easier to include?), but it meant that the "db-in-your-app-process" value prop went away.
Initially when they deployed Datamic Cloud (a sort of managed Datamic service deployed in AWS), they only supported the client model.
That is context for your question. The problem Ions solves is how to get that "db-in-your-app-process" value prop while using Datamic Cloud. It seems to me that it effectively reverses the original method. Original method was add the DB as a lib into your app, Ions method is push your app into the db (which is running in a configured-for-you cluster on AWS).
(It also allows you to not have to provision/configure servers to actually run your app code in addition to the db servers, but that seems secondary to me.)
Note, that I'm not affiliated with Cognitect, and I've never used Datamic, or Clojure in a production system. So have a salt shaker.