Politicians should only use the power they’re given by the people in the interests of the people. If donors influence politicians when they vote on laws or nominate peers to the House of Lords, then they are doing their jobs badly.

Thankfully, the Labour Government’s 2000 reforms forced political parties to disclose their significant donors to the Electoral Commission. The Commission publishes these donation records online. This gives journalists the chance to investigate which donors might be paying politicians for influence, rather than supporting a cause they believe in as most do.

But even with these sensible measures in place, the system is let down by poor implementation. The most obvious problem: the Electoral Commission simply publishes whatever donor name is reported by the political party. If parties are inconsistent about a donor’s name, either internally or amongst themselves, that donor will be listed under different names. For instance, the Conservative Party and Reform UK donor Christopher Harborne is listed under seven.¹ The Electoral Commission’s system hides how much a donor has given, making it harder for the public to learn when they have undue influence.²

A related problem: it is hard to understand the data at a glance. You could download the donation records to build Excel (or similar) charts whenever the data change, but that’s a lot of work. And the Electoral Commission publishes some graphs on this page, but they’re hard to get to and uninformative. They’re hard to get to because they’re not directly linked from the donation register page or the homepage (it seems to take five clicks). They’re also uninformative. By default they include Public Funds (government cash distributed to opposition parties), which is very confusing to anyone who doesn’t know what short money is. They only split out donors by category (individuals, companies, trade unions etc) - you cannot see any donor’s total giving. They don’t compare trends over time - there is no way to see which year parties raised the most since records began.³

This causes real problems. For example, journalists, even at respected institutions like the Times, understand the data so poorly that they just repeat whatever they’re told. In September, the Times described John Caudwell, the Phones 4U founder, as “The biggest donor to the Conservative Party before the last election”. This is a completely empty boast. If you count ‘before’ as one month, he’s the fourth largest, or if you count it as three months, he’s the eighth largest.⁴ If the Electoral Commission is meant to be providing a window into our democracy, it’s so fogged up so as to be useless.

One solution: Westminster Accounts

Others have improved public understanding of political finance through solid data journalism. Tortoise and Sky News should be commended for Westminster Accounts, a project which combined the Electoral Commission’s donations database with the House of Commons’ Register of Financial Interests and its Register of All-Party Parliamentary Groups (APPGs).

This wide scope allows the Westminster Accounts to tell a compelling, well-visualised story about individual lawmakers. However, it also creates three problems.

First, it only covers the most recent Parliament. I guess this is because converting the terribly-formatted House of Commons records into usable data is far too much work. Covering 25 years rather than four would be impractical.

Second, the reporting thresholds are higher for donations to MPs (£1,500) than to central political parties (£7,500). In fairness, the Westminster Accounts do not directly compare or aggregate the two, so this is a stylistic choice. However, in my view it is potentially confusing.

Third, some donations are made both to parties and to individual members (eg where a donor donates to a party to cover an MP’s member of staff’s salary). This leads the Westminster Accounts to list some duplicate donations. This can’t be solved under the current rules, because they don’t require political parties to share the purpose of donations. As far as I can tell, it is impossible to accurately combine the MP and APPG with the Electoral Commission data.

Another challenge with the Westminster Accounts is open access. To know how much money each donor is giving, we need to connect donations listed under different donor names (as discussed above). This work has to be done manually (and with clever “AI machine learning techniques” in the Westminster Accounts’ case). But, while the Westminster Accounts’ authors share the way they have interpreted the MP and APPG Registers’ data in a Google Sheets document, they have not published the way they have combined the names of political party donors. As the authors rightly point out at the end of each MP’s Westminster Accounts page:

the amount of money flowing into the political parties is on a completely different scale to the sums MPs and APPGs are collecting. In particular, the Labour and Conservative parties have each – alone – taken in two and four times more money than all MPs and all APPGs put together.

This fact is either not reflected in their bubble chart, or the chart is not to scale, as the MPs’ purple bubble is larger than Labour’s. But it is true that party donations dwarf parliamentary donations. If anything, it’s much more important that we know and can iterate on this aggregation work.

My solution: Donation Whistle

I have made Donation Whistle, a webapp based on the Python web microframework Flask. It is completely open source, so anyone can audit the source code on my GitHub. Here’s a video tour of its features:

Automatic import

The app starts by requesting all of the Electoral Commission’s donation records from its website. It then throws most of them away. Why?

First, it removes donations to local parties like Conservative Associations or Constituency Labour Parties. The reporting limits for these are lower than for central parties (£500 rather than £7,500), so counting them as part of of how much parties receive would be unfair. This brings the number of donations from around 80,000 to 27,000 (at the time of writing).

Next, it removes duplicate records which exist because parties double-report donations made in the run-up to an election.⁵ There are only about 1000 of these.

Finally, it removes donations which parties forfeited (because they couldn’t work out who the donor was) or returned (eg because the donor turned out not to be eligible to donate). This removes a further 200 or so donations.

Having cleaned the data, Donation Whistle puts the records into a SQLite database. It uses the date the donation was received by the political party or, if that’s missing, the date that the donation was accepted by the political party. This choice explains some minor differences between Donation Whistle’s figures and the Westminster Accounts’.⁶

Using Donation Whistle

We then see the list of donations on Donation Whistle’s homepage. The most important information about each donation is presented in a table. The table can be searched, filtered and sorted by donor name, donation date and amount. The data can be exported in CSV format.

By default, Donation Whistle filters out two sorts of payments received by political parties. It excludes short money (public cash distributed to opposition parties) because it is an legally obligatory payment as opposed to a donation and because it isn’t paid to the party in government, so including it would be an unfair comparison. It also removes donations which are returned to the donor, as these do not benefit the party.

Clicking ‘Recipients’, you can see an interactive, birds-eye view of how political donations have changed since records began in 2001. Neither the Electoral Commission nor the Westminster Accounts show this. You can zoom in to see smaller timescales, you can add parties to or remove them from the graph and you can downloaded any view you have generated as an image.

Donation Whistle allows admin users to add aliases to group together donations which have been recorded under different donor names. You can also export your alias settings or import them from other users. This way, you can always audit the choices made by anybody running a copy of Donation Whistle.

Whether you choose to make your own or import somebody else’s sample aliases (such as the ones included in the GitHub repository, Donation Whistle needs these aliases to generate the graphs on the ‘Donors’ page (biggest donors to all parties) and on each individual ‘Recipient’ page (biggest donors to each party). As with the Recipients graph, these graphs are also interactive, so you can zoom and export however you like.

Under the hood

I’ll now explain some technical choices I made when building Donation Whistle.

I chose the SQLite database because, at 26,000 records in its largest table, the dataset is relatively small. It is likely to grow only at a roughly linear rate. I therefore valued the simplicity and small resource footprint of SQLite.

Similarly, I chose flask_caching to provide a lightweight filesystem cache. Visualisation and API calls are much faster thanks to the cache. I didn’t feel I needed anything more complex than the filesystem cache, though, because (as mentioned above) the database size is so small. A filesystem cache which is mostly being read rather than written to allows for data persistence, therefore speedy access, but without the constant memory overhead of a more fully-featured cache. That said, I ended up needing a Redis container anyway for background tasks, so I may switch to a Redis cache in a later version.

I needed to build a very basic background task system. All I needed was to offload the data import process so the user would not be stuck staring at a loading bar for the five minutes it might take to import all the data. I chose RQ (Redis Queue), again for its simplicity.

The core technologies I built off were Flask (therefore Python), HTML, JavaScript and, for deployment, Docker. Being less familiar with JavaScript than I am with Python, the most challenging part of the project was ensuring that the filters selected by the user in the Flask form on the homepage are synced to the UI so as to avoid user confusion.

Next steps

Donation Whistle is not finished! As an open source project, of course I’d be delighted if anyone wanted to contribute code. In particular, I’d love to add these features:

• A fully featured donation data API - currently, only GET access is supported but I’d like to add PUT, POST and DELETE methods for records, mostly as a technical exercise, but also so that third party developers could build other interfaces if they wanted to
• Improving the alias generation feature by suggesting similar donor names suitable to be aliased
• Simpler alias sharing: some means of auto-importing the suggested aliases from an online source
• Recording the date that individuals were nominated to the House of Lords and including it in the context of their donations