May 11, 2018
8 min. read
I’ve been at Swarm Orange Summit 2018 in Ljubljana for 2 days (May 8 and 9). It was a relatively intimate (maybe 30–40 people on average) setting, but many of those attending are deeply involved in Swarm or a related project and many (most?) of the attendants also presented something themselves, giving the event a more conversational and symmetric character.
First of all, it became clear to me that Swarm is being developed quite actively and my intuition of Swarm being a pretty ambitious project with huge potential — an impression I already had when first test-riding it in December 2016 — has been confirmed.
However it remains unclear how it will perform in real-world scenarios and to what degree a Swarm user can control and monitor her data. For example:
Swarm is designed to in a way auto-scale. Content chunks requested by a node (either directly or indirectly for relaying) remain cached on that node. According to the devs, nodes by default allow up to 20GB being stored that way. I don’t know what policy governs the removal of cached chunks once that limit is hit, but could be something as simple as least recently requested.
In any way, thus mechanism results in popular content remaining highly replicated and thus available.
But what about my personal, encrypted backup file? It will not be requested by anybody, yet is very important to me. As I understand it, the incentive layer would need to allow me to request higher availability guarantees for such a file — that is, high replication is ensured by higher payment.
I couldn’t yet get a satisfying answer about this — but admittedly didn’t yet read the respective paper.
I’ve caught the opportunity to setup an up-to-date Swarm node on my PC — instructed by Aron.
It’s pretty easy, the problem is lack of (polished) documentation.
Currently, most up-to-date docs are on https://swarm-docs.s3-eu-west-1.amazonaws.com and most up-to-date code is on https://github.com/ethersphere/go-ethereum/tree/swarm-network-rewrite.
I’ve also realized that it’s time for me to get hands-on with ENS since Swarm is closely tied to it. It can be used without, but won’t then have support for human readable names or mutable resources.
There’s efforts dedicated to better support data streaming via Swarm.
Two members of the Status team (one arriving from Singapore, one from Moscow) gave an update about Status, with focus on the Whisper protocol.
I have a special relation to this project, because it was a participant (and winner) of the Blockchain Startup Contest in fall 2016 — an event organized by Thomas where I helped pre-selecting projects and where I was responsible for the behind-the-scenes communication and live presentation dispatching during the final event (most participants were connected remotely).
Back than, the price of 5.000€ (in Ether — which back than was somewhere around 10$) was incentive enough to get a Status founder to stay up until late into the morning hours (4 AM or so Singapure time).
According to Jacek, Status is out for a long run — willed to do a lot of basic research and work in order to get where they want.
The team has grown to about 70 people and is highly decentralized across the globe.
Part of that research effort is Nimbus, a new Ethereum client focused on light clients, written in the exotic Nim programming language — according to Jacek a main reason for doing that project is to better learn the Ethereum protocol — makes sense to me (I’d love to re-implement it in Clojure if I could clone myself or pause time or delegate everything else).
Boris talked about the status of the Whisper integration (the Mailbox construction was new to me) and pointed me to this code as a reference for basic workflows in Whisper.
I’m still looking for a guide of basic Whisper workflows based on its RPC API.
Status is also trying to connect with the wider community, part of that effort is their incubator program.
I always wondered why Swarm implemented another messaging protocol — there already existing Whisper.
Louis (or was it Aron?) explained to me that PSS made sense for use cases with reduced privacy requirements — delivering better performance than Whisper. This is confirmed by this explanation of Victor (which I first found just now). A bit more can be found in this wiki.
PSST is a layer above PSS which makes it connection oriented, similar to what TCP is to IP.
Elad explained how to use Swarm through APIs.
It’s essential to understand the various URL schemes. He stressed that in general the HTTP API should be preferred.
Ralph Pichler presented experimental Solidity contracts implementing Swap, Swear and Swindle, something Victor was especially excited about.
I’m not yet much into those, but it‘s basically about deposits, payments and conflict resolution.
This was again presented by Louis — on a PC running i3 🙂
My understanding: Swarm supports a way to have dynamic content / mutable resources which is more lightweight than updating ENS records.
Swarm has to tackle the same challenge like IPFS: content addressing is great for many use cases, but not great for content which is supposed to change while retaining the same identifier — e.g. a news website. IPFS handles this with a built-in name service (IPNS) — something I briefly experimented with a while ago, but now can’t find good docs for.
So, while Swarm has ENS, that’s not suited for frequently changing content (a blockchain transaction for every change is expensive).
The presented resource updates work by initially registering an id on ENS, marked as mutable. Then, every n blocks (the presentation had n=4) a resource may be updated. My intuitive (half-guess) understanding for how that works is that there’s a global hash (of a merkle tree?) for all such updates which points to a Swarm location where the actual pointers to the updated data lie. But I think I’m missing something.
In any way, in order to get the latest version, a node needs to to check blocks after the one creating the ENS entry for updates. Usage is documented here.
They have a very cool domain: https://mainframe.com/ and I was surprised not to have heard about them before.
First, they invited everybody to deploy the Onyx Dapp to a personal AWS instance — with a walk-through especially for the summit prepared at http://sos18.mainframe.com/.
I don’t have an AWS account (or maybe I have one, but last used it 10 years ago and didn’t have credentials at hand) and aborted the registration process when reaching the “accept terms and condition” checkbox (not because I never check those without reading, but because at this point I perceived the effort as too high for what my goal was).
I was not very focused during the rest of the presentation. To be honest, it didn’t became clear to me what mainframe is about.
They obviously are pretty involved in base technology development (at least showed deep understanding) and the name mainframe suggested to me that it was about a kind of decentralized SDK building.
The website however tells a different story (mainframe is a network with Onyx as a reference Dapp for it). I may find out one day.
Is one of those projects promising to out-compete centralized service providers by coordinating idling resources in a decentralized network.
Eric seems to know what he’s doing and was pleasant to talk with.
I’m intrigued by the pragmatic approaches they’re taking e.g. regarding the issue of handling illegal content.
Being censorship resistant vs not attracting all kinds of abuse is a delicate task which a lot of decentralization projects have yet to find a convincing answer to.
Eric’s vision is to develop tooling for allowing workflows similar to YouTube. There is something like a content id — which is a kind of probabilistic fingerprint (in a way a radically scaled down version of the content).
Interested entities can compile blacklists of such id’s.
It’s then up to node operators to decide about which blacklists to consider. This makes it easy (can be automated) for nodes to process only content which is legal in their jurisdiction, at the same time doesn’t force the whole network to comply to the same ruleset. I like that approach because it avoids bad alternatives:
Livepeer also uses such a content_id for verifying the work of nodes (since they could also stream out something else not related to the input data, avoiding the effort of transcoding).
For that they’re looking into TrueBit and also already have a PoC for it.
A guy recently joining Livepeer who’s name I forgot (but who’s cool Rasta’s I didn’t) and who previously worked for Wowza explained that GPUs can do video transcoding alongside crypto mining because that involves different HW circuits (there is a small performance penalty due to shared memory).
Which means that livepeer can tap not only into the resource of gamer’s GPUs when they’re not playing, but also into GPU based mining farms.
He estimates a cost advantage compared to Amazon’s dedicated offering of about factor 20.
PS: They performed proper dogfooding by live streaming their talk on their platform.
In talks and conversations I learned about 2 things I didn’t know before:
Anton presented his work on a node simulation framework, which is part of go-ethereum (as is Swarm itself for now).
Has a lot of useful features and may be useful for ARTIS too.
He asked for support for extending such simulations to multiple machines in order to get realistic network behaviour (especially delays), but also because a single machine can simulate only so many nodes (performance bounds).
If I remember well, the framework also contains a visualization component, something I find especially intriguing.
Javier of epic labs presented Ethergit which is git modules/plugins (?) pushing to Swarm and using Ethereum contracts for governance (e.g. restricting push permissions).
Sounds like a good idea to get less reliant on the central entity github.
Not sure if there’s anything released yet.
I’m not at all surprised about the difficulties mentioned. P2P on end-user devices (especially mobile ones) has always been and still is a tricky business — especially if high bandwidth, low latency and/or high reliability are required.
There’s so many potential obstacles… many are known and to be expected, e.g. NAT/firewalls, filtered ports, ISP throttling.
But I’ve also seen more subtle pitfalls, e.g. company proxies intercepting HTTP requests of arbitrary sizes in order to run a Virus scan on the full file before forwarding ANY data to the client or MVNOs interfering with the HTTP Content-Type header.
The original vision of the Internet was a network of nodes which could easily connect to each other. Some still hope we could get back from an Internet divided into servers and clients behind NATs to that ideal network — IPv6 would allow that on a technical level. I don’t see that happening, but total elimination of distinctive roles of Internet nodes is not needed for considerably stronger decentralization.
Finally, I also presented something: “Streaming Money” (video of the talk should be released by the Swarm team soon).
I’ve uploaded the presentation material to the Swarm testnet: http://open.swarm-gateways.net/bzz-list:/cbf610efa8ddbcc61441a67e009f71d5a13a11e7aa00efa52da1772a98916afa/
Note however that clicking a file doesn’t show or download it (at the time of writing). For that you need to manually edit the URL from “bzz-list” to “bzz”. E.g. http://open.swarm-gateways.net/bzz:/cbf610efa8ddbcc61441a67e009f71d5a13a11e7aa00efa52da1772a98916afa/SWARM_Summit_Streem.odp
I had the impression that my (first public) explanation of Streems was met with a lot of interest and curiosity.
There was a great question (I think it was by Carl): what about automated, micro-payment based pay-per-use instead of subscriptions? Wouldn’t that make even more sense?
I agree and disagree at the same time. In theory, automated pay-per-use looks like a superior model. I have extensively brainstormed that model in the past, e.g. here. I have however come to the conclusion that in the B2C business flatrates/subscriptions tend to win because
I mentioned that we see Swarm as a potential part of ARTIS. But it’s now our turn to bootstrap ARTIS and build a core community.
Swarm announced a public alpha for Q1 2019, that could align nicely with the go-live of the ARTIS mainnet.