Anyone with experience with both this and Oracle Coherence (formerly Tangosol Coherence)? I'd dearly love for this to be a reasonable alternative to Coherence, which I love but which is fabulously expensive.
Terracotta isn't peer-to-peer; it requires one or more management-ish servers. However, it does offer persistence on those servers, so one could use Terracotta in place of a database. Also, Terracotta works in the (historically) Spring-like way of not requiring any modification to an existing codebase. One declares which objects ought to be distributed, and magic just happens (supposedly, I've only been reading up lately). Terracotta does bytecode manipulation on those object references and on Java synchronization blocks to achieve what is explicitly specified with this system.
Another project to look into is JBoss's recently re-branded and improved cache:
I've always wanted to use such a technology to build a write-behind persistence mechanism. Let's say that you want your data in a relational DB (for reporting, company policy, integration, whatever) but can stand a very small risk of losing a few seconds worth in the case of a huge failure. You'd gain by avoiding the latency of network-to-db and disk forcing.
So, you write DAOs that "persist" to a queue and to a shared map(s). When you read, you look in the map first and go to the DB upon failure. Queue consumers persist messages (inserts/updates) to the db. After your application reaches a steady state, you shouldn't have to read from the DB much at all. And, you could build an eager loading mechanism for efficient start-up.
Even without async persistence, I'd prefer one of these solutions over memcached/MySQL replication hackery.