Hiding Ethereum and IPFS under the hood

While I fought claims of the more marketing oriented team mates to describe Play4Privacy (P4P) as “running on a Blockchain”, it is true that the Blockchain — even though only a fraction of the computation took place on it — played a central role.

In this article I’ll explain the role and evolution of the Ethereum contracts, point you to the full game state data published on IPFS and explain how those technologies were combined to enable verification of correct behaviour of the game server.

Contracts

Lets start with a brief overview of the Ethereum contracts.
Their source code can be found in the linked Etherscan views (in Tab Contract Source) and will also be published on GitHub soon.

This section gives a brief overview of the contracts in order to understand the basic structure. The next section contains more details.

PlayToken(Token view) implements the ERC-20 standard and is based on the code of a ConsenSys implementation.
The only significant changes are the added mint() function (needed because the contract was deployed with 0 tokens issued) and the possibility to switch controller (that is, the only address allowed to execute mint()).

P4PPool is the contract managing the Ether donations.
It pools the extra PLAY tokens reserved for donors, holds the donated Ether and finally allows donors to withdraw their share of PLAY tokens.
It’s kept configurable (e.g. the switch from one phase to the next is triggered from the outside) because several decisions were not yet made at the time of deployment.

P4PGameis the main contract connecting Token and Pool.
It contains the function distributeTokens() for batched token payout to players with corresponding payout to the pool.
Supply of Play tokens can be fixed for all times by locking the controller (through the proxy call setTokenController()) and invoking shutdown() on the Game contract.
The function addGame() was supposed to be used for persisting game states, but in the end never used.
Two prior versions of the Game contract were deployed and briefly used.

P4PStateis a minimal contract, not connected to the other ones, for persisting the game state hashes.

The evolution

P4P was intended as an education project for both the team and the audience.
It was the first lab10 project involving contracts on Ethereum mainnet.
The choice of the Go game and of the Kunsthaus Facade display were made quite early in the project.

Low entry barrier

Initially, we thought about building a kind of remote Slot machine — not in terms of the actual game, but of the entry mechanism — were one would need to pay some Ether in order to participate (inspired by BTC-Fish).
This idea was abandoned as we realized that it would essentially exclude everybody not yet familiar with crypto-currency — a contradiction to the idea of getting outsiders familiar with Blockchain technology in a playful way.
This is also the reason why we chose not to make P4P a Dapp — normal people don’t have Dapp capable browsers (or browser extensions (or mobile browsers)) installed.

MELBDRHB002 - Image 1
Early project notes assuming players would have to pay in order to participate.

Instead the P4P App was implemented as a web application working in any modern browser.
When first initialized, the App created an Ethereum wallet with one account. This account was used to cryptographically sign all moves a player votes for.
If the player finally chose to claim the earned tokens, they would be transferred to that same account.

This was implemented by using Ethereum’s web3 wallet, without interacting with the Blockchain — since interacting with the Blockchain would have required the player to have Ether for transaction fees.
We opted for the 1.0 beta of web3 because it contained additions which considerably simplified the implementation.

The PLAY token

After discharging the idea of having players pay for participating, we needed another mechanism to generate donations for organisations caring about privacy — which was also a main goal of the project.

At some point the idea to issue a token based on playing activity evolved.
After some discussion about the exact mechanics we settled for the scheme: 1 vote for a move earns 1 token.

About 2 weeks before the start of P4P, we had a visit of the then Austrian Federal Minister of Science, Research and EconomyWe thought it’d be nice for him to participate in a first, preliminary game of P4P and thus deployed the not-yet-finished P4P application.

The first game producing tokens on Ethereum mainnet is being played
The first game producing tokens on Ethereum mainnet is being played

The Token was final. But we weren’t yet sure about all the issuance details, thus deployed the Token and Game contract such that the Game contract could be easily swapped out by another one.

Only hours after deploying those contracts, the first PLAY tokens were created with this transaction.
Here you can see that the sender doesn’t invoke the Token contract directly, but version 1 of the Game contract which then internally invokes mint() on the Token contract.
Here you can see the transactions involving that first version of the Game contract. After the first transaction (we didn’t expect other people in the room to take out their phones and join the game, thus also generating tokens) there’s only one more transaction about 10 days later which tells the Token contract that the controller changed (that is, the contract allowed to mint tokens changes) to version 2 of the Game contract.

The Pool

The reason for updating the Game contract was another idea we developed at a late stage:
Every token created by a player voting for a move is matched with a token put into a Pool contract. That pool contract is the receiver of Ether donations. While the Ether then goes to privacy supporting organisations, the PLAY tokens go to the donors (proportional to the donation).

Token share of the pool contract visualized in the token holders chart. It’s not exactly 50% because the Pool contract was not yet in place when the tokens for the preliminary game were created.
Token share of the pool contract visualized in the token holders chart. It’s not exactly 50% because the Pool contract was not yet in place when the tokens for the preliminary game were created.

Now we had a mechanism to collect donations without creating an entry barrier for the game itself.
At the same time this mechanism allowed donors to be rewarded.
Lastly, since a share of the tokens submitted to the pool goes to us (the P4P project team) it also creates an incentive for ourselves to support community efforts towards finding and implementing use cases for the PLAY token.

This pool mechanism was implemented by adding a Pool contract and by updating the Game contract to generate tokens not only for the Player, but also for the Pool.

At this point, the contract code became less trivial, because it required the math to calculate token shares based on Ether donations.
An example is this line of the Pool contract:

var virtualEthBalance = (((totalPhase1Donations*2 + totalPhase2Donations) * 100) / (100 - ownerTokenSharePct) + 1);

Here, the final “+1” makes sure the pool can’t run out of tokens because of small rounding artefacts, leaving the last donors to call withdrawTokenShare() without tokens because playToken.transfer() would then fail.

In smart contract land, small mistakes can lead to bad outcomes like locked contracts, stolen or frozen funds etc. Thus, the pool contract got not only extensively tested, but was also provided with a fallback recovery mechanism:

function destroy() onlyOwner {
require(currentState == STATE_PAYOUT);
require(now > 1519862400);
selfdestruct(owner);
}

This function would allow us (the owner) to recover all remaining Ether (a side effect of selfdestruct()) in case payout for some unforeseen reason failed despite all testing. However this could be done only after March 1 2018 (a somewhat arbitrary date far enough in the future), also all PLAY tokens still owned by the contract would thereby be lost.

Fraction of test cases for the Pool contract.
Fraction of test cases for the Pool contract.

Batching and transaction cost

The simplest possible way to pay out tokens would have been to trigger a transfer transaction for every single token earned.
While this would have been possible in theory, it would have been overkill in practice.
In some game sessions (typically 9 games per evening session were played) more than 10 000 tokens were created. This would have meant an average of around 1 transaction per second over the 3 hour period.
Since a token transfer transaction consumes about 30 000 gas (a bit more for new receivers, because those require allocation of additional state storage), the cumulated transaction costs of a single evening session could have exceeded 1 000 USD worth of Ether (assuming a gas price of 10 GWei).
Not only would that have quickly bankrupted the P4P project, it would also have put a pretty tight upper limit on the possible number of players, because 1 transaction per second is already approximately 5% of the overall transaction throughput Ethereum is currently able to handle.

So, the first obvious transaction fee optimization was quite obviously to not process every single token, but keep track of the tokens earned throughout a game and transfer them in a single transaction.

The next optimization is related to a specific property of Ethereum’s gas cost model: Every transaction has a base cost of 21 000 gas, on top of which goes the cost of the actual processing the transaction involves.
This means that it’s cheaper to batch operations into fewer transactions even if they are not dependent on each other.
Because of that, already version 1 of the Game contract was designed such that the cumulated token transfers for multiple players could be batched into a single or a few transactions.

function gamePlayed(bytes32 gameHash, address[] players, uint8[] amounts)

The idea here was to persist the game state (represented by a hash) and transfer the earned tokens in a single transaction by using arrays for the token receivers and respective token amounts.

There was an important detail to consider:
The arrays can’t contain an arbitrary number of elements, because there’s an upper limit to the amount of work which can be handled by a transaction.
This limit is determined by the block gas limit — that is, the maximum amount of gas allowed (by the protocol) in a block.

Example block gas limit (of Block 4546571).
Example block gas limit (of Block 4546571).

Since a transaction cannot span multiple blocks, that poses a hard upper limit to the amount of gas (and thus computation) a transaction can involve.

It’s by the way important to keep the gas limit for transactions set as close as possible to the actual gas used on execution.
That’s because the block gas limit is calculated by summing up the gas limit of the included transactions while the transaction fees are calculated based on the gas actually consumed. From a miner’s perspective, gas that’s not consumed by a transaction metaphorically speaking occupies block space without paying for it.

Another reason for keeping the gas limit of a transaction low is: if a transaction fails, it consumes all the gas up to the given limit — which can be much higher than the cost which would have occurred had the transaction succeeded. Here is an example of such a transaction (consumed gas more than 10x higher than when successfully executed).

The web3 library offers a method estimateGas which allows automation of gas limit adjustment per transaction (in P4P, we always added a safety margin of 10%).

contract.methods.addGames(stateHashes, boardHashes).estimateGas().then((gasEstimate) => {
const gasLimit = gasEstimate + Math.round(gasEstimate * 0.1);contract.methods.addGames(stateHashes, boardHashes).send({ gas: gasLimit })
...

For the Game contract, the upper bound dictated by the block gas limit meant that the batching needed to be implemented such that it could be split up into multiple transactions.

The contract itself didn’t enforce any limit on the size of the argument list.
Since this functionality was accessible only to the backend system of P4P — which is under our control — it was deemed ok to make the contract caller responsible for not exceeding the block gas limit (we actually set the threshold for splitting up at ~30% of the block gas limit).

Version 2 of the Game contract was deployed a few days before the official start of P4P. As can be seen here, it didn’t last long. After the transaction creating the contract, only 3 other transactions followed:
* two of them for setting pool address (first attempt failed due to gas limit set too low)
the last one for setting version 3 of the Game contract as new token controller

The reason why version 2 was never actually used was that the way of distributing tokens turned out not to be well suited for the way the final P4P application worked:
When writing the Game contract, it was assumed that all players (identified by their account address) would always receive the earned tokens and that the game server would automatically create a transaction containing a hash of the game state and a list of token receivers after every game.
This assumptions turned out to not hold.

First, we decided to not automatically pay out all tokens to the participants of a game. Instead, tokens were paid out only if players chose to redeem them.
The main motivations for this decision were:
* Avoid wasted transaction fees for players not interested in the tokens
* Give players the possibility to donate tokens to the dev team
* If necessary, have more options to prevent bots from earning a lot of tokens

The redeem process contained a step for locking the wallet with a password. Only after doing so was it persisted to the browser’s local storage and tokens cleared for payout.
The redeem process contained a step for locking the wallet with a password. Only after doing so was it persisted to the browser’s local storage and tokens cleared for payout.

Since we didn’t want to force players to redeem tokens after every single game, the bundling of persisting game state and token payouts on contract level, as implemented by gamePlayed(), didn’t make sense anymore.
In the new model, as implemented in version 3 of the Game contract, tokens earned throughout different games could be summed up and processed as a single state change per player account.
Thus the final version of the function for distributing tokens was:

function distributeTokens(address[] receivers, uint16[] amounts) onlyOwner onlyIfActive {
require(receivers.length == amounts.length);
var totalAmount = distributeTokensImpl(receivers, amounts);
payoutPool(totalAmount);
}

The optimization changes turned out to be useful, because during most of P4P’s runtime the Ethereum Blockchain was quite saturated and thus market mechanisms for transaction prioritization fully kicked in, driving the minimum gas price required to get a transaction mined up.
While it’s currently possible to get transactions mined with the gas price set as low as 0.1 Gwei (example), there were periods where even a 90 times higher gas price (other example) left the pending transaction stuck in the mempool (that is the list of transactions waiting to be included in a block) forever.
In order to easily get a picture of the current situation regarding gas price, ETH gas station is a good place to go. Adjusting the gas price of transactions to current market conditions can save a lot of Ether for huge transactions consuming elevated amounts of gas.

A main reason why required gas price fluctuated so strongly during October was the difficulty bomb (aka Ice Age) kicking in, effectively halving the processing capacity because of the doubled block time.
This bomb was defused with the Byzantium Hardfork (part of the Metropolis release).

Ramping up and defusing of the difficulty bomb (source).
Ramping up and defusing of the difficulty bomb (source).

Special permissions and locks

One of the most interesting aspects of Blockchain contracts is the permission handling.
By default, all accounts interfacing with a contract have the same permissions (as long as the transaction fees are paid).
Most importantly, the contract creator gets no special permissions whatsoever. Any special permissions need to be explicitly added.
In case of P4P, several special permissions exist, e.g.
* Mint and distribute new PLAY tokens
* Switch phase of the Pool (donation rounds, playing phase, payout phase)
* Set donation receiver of the Pool’s Ether funds

The usual pattern is to store the address of the contract creator in a field named owner and allow some functions to be executed only by this owner.
This is the case in both the Game and Pool contract, implemented with a function modifier:

modifier onlyOwner() {
require(msg.sender == owner);
_;
}

In P4P, two additional permission mechanisms were used:
1. set / lock semantics: allow specific fields to be updated by the owner after contract creation, but only as long as they were not locked.
2. timelock: allow specific functions to be executed only during specific timeframes, implemented by using Unix timestamps which are compared to the current block time.

With those mechanisms combined, the Pool contract could be designed such that the Ether funds it holds can never be accessed by the contract owner.
The timelock donationUnlockTs forbids Ether funds to be withdrawn before its timestamp is reached. It can be updated, but only into the future (it’s basically locked towards the past).
The set/lock field donationReceiver allows setting and locking the address which has exclusive permission to withdraw the Ether funds. By setting and locking it before donationUnlockTs is reached, the contract owner can ensure that at no time it has access to the funds while still keeping the flexibility to decide about timeframes and donation receiver after contract creation.

In this specific case we wanted that flexibility because at project start the decision about which entities would get the donations was not yet made.
Deferring that decision even left the option to later deploy a donation distribution contract which would split up the donations to more than one entity. Or a contract which would implement some kind of voting for donation allocation. Or … you name it!
Also, we wanted to never actually hold donated funds because of the possible legal implications that may have.

(Cheap) data authenticity

A principal property of crypto systems is the possibility to have authenticated data.
A transaction on the Blockchain proves that the action was triggered by the owner of a private key corresponding to the address from which the transaction originated (or from a contract, governed by its code).

In P4P, individual players didn’t interact with the Blockchain.
Despite of that, their actions (voted moves) can be authenticated. How so?

Whenever a player clicked a field on the Go board, the following piece of Javascript was executed:

const sigData = `${this.gameId}_${this.roundNr}_${move}`;
const sig = ethUtils.sign(sigData);

In sigData, a string containing the unique gameId (just the Unix timestamp of when the game started), the roundNr (an Integer incremented with every round, that is, every move executed on the board) and move (the coordinate of the gameboard field the player voted for) was created.
This string was than signed using the locally created Ethereum account (ethUtils is just a tiny wrapper around the web3 library).

The so created signature, represented as hex string, was then transmitted to the game server together with the actual vote.
By including the signature into the game state, the server can now proof that specific votes were actually submitted by the claimed player (represented by the Ethereum address).
By enumerating all such votes, any interested 3rd party can verify that the move finally executed on the gameboard is correct according to the consensus rules (that is: a stone is placed on the most voted field).

Since voting for moves is also the mechanism by which PLAY tokens were earned, verifying the state also verifies that PLAY tokens were assigned correctly.

At this point it’s worth mentioning that during the first 3 days of P4P, that mechanism contained a bug: the aforementioned sigData was by mistake calculated by using a timestamp created on client side. Since the clocks of Computers aren’t perfectly synchronized, that resulted in vote signatures to become useless, because the server couldn’t know the exact timestamp used by the client.
That’s why the state data doesn’t include signatures for the sessions which took place during that timeframe.

Since the logic for processing (extracting, re-formatting, hashing) the game state was also not ready initially, we decided to focus on improving the playing experience and make sure Token distribution works as intended, and care about game state publishing afterwards.

With the playing phase over, this has now happened.
All the data needed to fully reconstruct the games played was extracted from a MongoDB instance and converted into JSON files (one per game played).
Surprisingly, the recovery of player addresses from the move signatures turned out to be pretty slow. On my modestly powered Notebook the whole process took several hours for the whole dataset (containing approx. half a million signatures). I’ve yet to figure out if ECDSA signature checking is just as computationally expensive or if the used Javascript libs are slow.

function getPlayerByMove(gameId, move) {
const sigData = `${gameId}_${move.round}_${move.move}`;
return web3.eth.accounts.recover(sigData, move.sig);
}

IPFS

In order to live up to lab10’s goal of promoting decentralization, we decided to publish the raw game state data to IPFS:

Here you can find an overview HTML file containing just a hyperlinked list of game states.

Here you can find an overview JSON file which for every game contains:
* the name of the originating file (unix timestamp = game id)
* the IPFS hash
* the state hash as written to the Blockchain (see notes below)
* the final state of the gameboard encoded such that it fits into 32 bytes

The single game state files contain:
* id: Unix timestamp of the start data
* startDate: start date in a more human readable format (more precisely: ISO 8601)
* moves: array of the moves executed on the gameboard. This information is missing for the first 3 days. We expected the moves to be reconstructible from player’s votes, overlooking that there was a tie breaking rule involving randomness
* finalBoard: array representing the final state of the gameboard, with 1 representing a black stone, -1 a white stone and 0 an empty field
* votes: an array of all votes submitted for that game, including the round number, field the player voted to put a stone on, player’s Ethereum address and signature (constructed as described above)

IPFS hashes are base58 (popularized by Bitcoin) encoded multihashes.
While those could be stored as is (using String type) in the Ethereum Blockchain, that would be a waste of storage space (and thus transaction fees).
Of course those hashes can be decoded to bytes. But there’s a catch: The multihash format has a 2 byte header which specifies the hash function used and the digest length. Since Ethereum aligns to 32 bytes, that extra 2 bytes would have wasted an extra 32 byte per hash.
Thus after base58 decoding the hash, I removed the header and stored only the actual SHA256 hash in the Blockchain.
Since the State contract is not gonna be used again, possible future hash function changes of IPFS aren’t relevant for this application.

For storing the game state, we used a technique labelled Stateless Smart Contracts. Since the only purpose here was to have a bunch of hashes immutably persisted in the Blockchain, there was no need to create (expensive) state data.
The only purpose of the contract was to act as receiver address and document the data structure on the Blockchain:

contract P4PState {function addGames(bytes32[] states, bytes32[] boards) {}
}

We didn’t even bother to restrict access to it (anybody could call it) since the data persisted is the transaction object itself.

Verification

This is the transaction containing the state and final board hash for all 358 games played: 0x9f341944eb0fe23e70819c470a9a7bb96ee643abd95f887067085bef53e5bdfc

In order to verify the state of a game, the above described process needs to be reversed.
For a quick verification test, this should work:
* Copy a hex formatted IPFS hash from Input Data
* Paste it into the input field below Bitcoin Address Base58 Encoder of this online base58 converter
* Prefix it with “1220” (that’s the 2 bytes of multihash header previously cut away)
* Do Encode address from hex
Now copy the resulting String and retrieve it from IPFS (e.g. by appending it to https://ipfs.io/ipfs/ and opening that URL in a browser)

A note on IPFS:
For convenience, URLs pointing to the IPFS gateway ipfs.io are used here.
The very idea of a central IPFS gateway contradicts the whole point of IPFS, but it is of course not necessary to use it.
If you have IPFS running, you can directly operate with the hashes.
You may also use the browser plugin IPFS companion in order to intercept requests to the ipfs.io gateway and have them handled by your local (or whatever you configure) IPFS node.

Retrieving an IPFS file without gateway.
Retrieving an IPFS file without gateway.

The game state data was pushed to a lab10 IPFS node.
You’re of course welcome to pin that data to your IPFS node in order to make sure it remains available even if the lab10 IPFS node disappeared.
(Contrary to what some believe, uploading something to IPFS doesn’t mean it will magically remain available)

Overall, this design allowed us to authenticate the whole game state data (including about half a million votes) by just saving a bunch of hashes in the Blockchain.
Of course it could even have been reduced to a single hash, but we didn’t want to be too stingy 😉

Fun fact: When executing the transaction for persisting the game state, I forgot to adjust the gas price, resulting in the most expensive transaction done over the whole P4P runtime.

Censorship

The taken approach has one serious flaw:
There was no guarantee against the game server censoring player’s votes.
In fact we did censor a few of them. At times the behaviour of individual “players”made it so obvious that there was some kind of automation behind it that we decided to enforce some basic decency (or — alternatively — more sophistication) by temporarily kicking them.

After the games were over, we even got an anonymous confession in the form of this screenshot:

MELBDRHB002 - Image 9

But with this design of barrier-less entry and nothing at stake from a player’s perspective, it was obvious that something like that could and probably would happen.

It’s also obvious that creating new Ethereum wallets is cheap (it basically boils down to choosing a password and do some hashing).
Thus despite all authenticity and verifiability we could just have created a lot of accounts ourselves, pretending to distribute tokens to a lot of players.

The reason why we made the extra effort of implementing it like this anyway was to learn by doing and to show the variety of ways cryptography based technologies like can be used to replace blind trust (or resigning hope) in remote systems.

In order to avoid the issue of potential invisible censorship, players would have to directly interact with the Blockchain — something currently made difficult not only by its lack of scalability, but also by the high entry barrier of getting hold of some crypto-currency for transaction fees.
Also, some kind of authenticated identity to avoid sybil attacks would be required in order to make the concept of signed data useful in such a context.

With our upcoming flagship project, we’ll tackle both of that challenges.


Conclusion

Play4Privacy gave us the opportunity to get hands-on with the Ethereum Blockchain and to experiment with various concepts in a playful way.
Much of what we could learn along that way will be useful on the road ahead.

We thank all players (even those participating in other ways then envisioned by us) for giving life to this project.
Special thanks go to the donors. Thank you for supporting a good and important cause!

Subscribe to get the latest
news and articles first

We follow privacy by design principles and do not tolerate spam.