~kirkland/pollen/trunk

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
ABSTRACT
--------

Pollen is a scalable, high performance, free software (AGPL) web server, that provides small strings of entropy, over TLS-encrypted HTTPS or clear text HTTP connections.  You might think of this as 'Entropy-as-a-Service'.

Pollinate is a free software (GPLv3) script that retrieves entropy from one or more Pollen servers and seeds the local Pseudo Random Number Generator (PRNG).  You might think of this as PRNG-seeding via Entropy-as-a-Service.

Please understand...neither Pollen nor Pollinate increase the amount of entropy available on the system!  Rather, Pollinate adequately and securely seeds the PRNG in cloud virtual machines through communications with a Pollen server.


DESCRIPTION
-----------

The Linux kernel provides two special character devices interfaces to high quality entropy -- /dev/random and /dev/urandom.  Both are pseudo random number generators (PRNGs), but the former conservatively guarantees quality entropy, and userspace processes reading from /dev/random will block until sufficient bits are available to fulfill the request.  The latter, /dev/urandom, provides a non-blocking, limitless stream of pseudo random numbers.

The manpage random(4) has far more complete description of /dev/random and /dev/urandom.  See: http://manpg.es/random.4

For most practical purposes, /dev/urandom is a perfectly adequate source of entropy, as long as it is seeded properly at each boot.

Most Linux distributions (including Debian and Ubuntu) carry over a random seed from one boot to another, typically in an init script, such as /etc/init.d/urandom.  In Ubuntu, that init script does the following:

  ...
  SAVEDFILE=/var/lib/urandom/random-seed
  POOLBYTES=512
  dd if=/dev/urandom of=$SAVEDFILE bs=$POOLBYTES count=1 >/dev/null 2>&1
  ...

There is, you may notice, a bootstrapping problem... How does one seed /dev/urandom on a system's very first boot?

On laptops, desktops, tablets, phones, and other physical systems, input devices, such as a keyboard, mouse, touch screen, or microphone can provide sufficient entropy to seed the PRNG through the kernel's collection of timers and interrupts.

However, virtual machines typically have no access to real hardware and few, if any, sufficient entropy sources.  Several real attacks have been demonstrated recently against SSH and SSL, with certificates generated with poor entropy, such as: https://factorable.net/weakkeys12.extended.pdf

The cryptographic security of virtual machines and cloud instances can be significantly improved by fetching a sufficient amount of entropy at first boot (and periodically thereafter) to seed the PRNG with external sources of entropy.


IMPLEMENTATION
----------

Pollen is a fast and efficient web service, implemented in Golang.  It provides small random strings to its clients over network connections.  Pollen utilizes TLS (SSL) to ensure privacy, security, and non-repudiation of connections among its clients.

Pollinate is a client utility, implement in Shell, which wraps curl(1) and communicates securely with one or more Pollen servers.

The default protocol for all connections is HTTPS, however HTTP is available for debug, testing, or other purposes.  To ensure the privacy and security of connections, the Pollen server should ideally have a CA-signed certificate, or pre-arrange the distribution of certificates to its clients.

An entropy request should optimally contain a POST argument:
 - challenge - a randomly generated checksum to ensure unique communication with the Pollen server

The challenge POST argument is a hex-encoded sha512sum(1) value, which is 128 ASCII characters of [0-9a-f].  Regardless of the value of the 'challenge', the Pollen server will treat the input as a string and calculate the sha512sum.  This ensures that any malicious input from a deviant client is whitened to a simple hash before the server operates upon it.

The server then responds to the client with the sha512 checksum of the client's challenge on the first line, and the second line of the response will contain a sha512sum of 64 bytes of entropy.  This second line is what the client can use as a random seed.

The client verifies the challenge/response, which is intended to help ensure that this communication between the client and server is a custom response, and that the server actually "did some work", and thus affected the entropy state on the server.

The client uses a special option to curl(1), which details all of the communication to the server, and includes high resolution, local timestamps.  This information, which is not easily detectable or reproducible by an attacker (or the Pollen server administrator), is combined with the server's responses, and written into the Linux PRNG, /dev/urandom, which is folded into the local system entropy.


POLLEN AND POLLINATE IN UBUNTU
------------------------------

Canonical provides a Pollen server as a service to the Ubuntu community at https://entropy.ubuntu.com.  Beginning with Ubuntu 14.04, Ubuntu cloud images include the Pollinate client, which will try (for up to 3 seconds at first boot) to seed the PRNG with input from https://entropy.ubuntu.com.  This service is highly available, with multiple physical servers deployed in a cluster using Juju service orchestration.  Each of these Pollen servers have at least two hardware random number generators, ensuring high quality entropy as a service, and diversified against hardware failure.  Moreover, a busy Pollen server, handling many challenge/response calculations and serving numerous concurrent connections, will have a computationally complex and impossible to reproduce entropy state.

Ubuntu cloud users are welcome to add other Pollen servers to their pool, or just run their own internally, behind their firewall.  Simply edit the configuration file in /etc/default/pollinate.  Ubuntu users and other distributions are certainly welcome to install and run their own Pollen server, with 'sudo apt-get install pollen' or 'bzr branch lp:pollen' and compile from source.

Be safe, and secure out there!
:-Dustin