Hayek’s Bastards and the House of Lords

What connections exist between the House of Lords and some of the institutions pivotal in cultivating today’s alt‑right politics from strands of neoliberalism?

We set about identifying meaningful connections between 858 House of Lords peers and a selection of institutions from Quinn Slobodian’s recent book, Hayek’s Bastards. Why would we do this and who are these bastards?

Hayek’s Bastards offers a particular history of today’s alt‑right that suggests it did not spring up in opposition to neoliberalism but rather mutated from its core, evolving into a strain obsessed with racial hierarchy and hereditarian ideas of intelligence. Slobodian charts this evolution through a web of neoliberal think tanks, lobby groups and billionaire benefactors, chiefly in the United States, who re‑engineered the market absolutism of Friedrich Hayek and Ludwig von Mises into a politics of exclusion. Except for outliers such as Britain’s Institute of Economic Affairs, most of these actors are American, prompting us to ask how far their influence and networks extend within the UK’s political elite.

The following institutions from Hayek’s Bastards were chosen to cross reference against the set of peers from the House of Lords:

Methodology

Our pipeline implements the following steps.

8. Archive Results in Obsidian

All curated data was archived to an Obsidian vault, providing an interactive workspace where you can browse search hits and navigate the entity graph.

Data Release

To explore the dataset, you must first download Obsidian, download and unzip the dataset below and then follow the instructions to create a vault from an existing folder (in this case the folder you just unzipped).

📂 Download corkboard_vault.zip (~1MB)

Every hit comes with an LLM‑generated synopsis and a summary receipt that records the model used to generate the synopsis, the fraction of the article included within the prompt, and whether this fraction of text was supplied as a single continuous block or split into windows. This metadata gives researchers an additional measure to judge the reliability of each synopsis before deciding to open the original page. Each synopsis should be interpreted only as a guide to what may exist on the webpage and absolutely not as fact as it was generated by an LLM. Similarly the content of each webpage should be carefully judged as to whether of not the information is reliable:

After manually removing links that appeared to refer to other persons or organisations, we identified potentially meaningful connections between 20 organisations from Hayek’s Bastards and 152 peers from the House of Lords.

Most Cited Organisations	Connections
Institute of Economic Affairs	94
Heritage Foundation	42
American Enterprise Institute	22
Atlas Network	22
Cato Institute	18

Most Cited Peers	Connections
Lord Hannan of Kingsclere	11
Lord Elliott of Mickle Fell	10
Lord Moynihan of Chelsea	9
Lord Gove	8
Lord Hintze	7

Overview

By comparing the latest “Risk Factors” sections in S&P 500 companies’ 10-K filings (submitted by May 1st, 2025) with their previous filings, we found:

• 3 in 4 companies (380 total) have added or expanded upon risk concerning AI.
• Over 1 in 3 companies (193 total) have added or expanded upon risk concerning malicious actors using AI.
• The number of companies citing deepfakes as a risk has doubled, from 16 to 40.
• 1 in 5 companies (95 total) have added or expanded upon the risks concerning protection of their proprietary data or IP through interacting with AI systems.
• 1 in 10 companies (56 total) have added or expanded upon risk concerning third-party providers of AI models and software and their vulnerabilities.
• 1 in 3 utilities firms (10 total) have added references to AI’s increasing energy requirements.
• The number of companies citing EU AI Act – the European Union’s primary legislation on artificial intelligence – related risk has tripled, from 21 to 67.
• 1 in 3 companies (168 total) have added or expanded upon competitive risk relating to AI.
• The number of companies citing AI bias risk has doubled, from 70 to 146.
• 1 in 10 companies (57 total) have added or expanded upon the risk of AI failing to deliver intended benefits, success or return on investment.
• Risks to jobs rarely feature among reported risks, despite being a prominent public concern.

Dataset

Each of the above findings was derived from the following curated selection of 10-K filing extracts, where green-highlighted sections indicate newly added text, red strikethrough sections indicate removed text, and regular text remains unchanged between filings:

Charts

Medicine

More pharmacies opened in Ukraine throughout 2024 than any other type of store. Data provided by the Retail Association of Ukraine shows that pharmacies are the most common store type, making up 36% of the 22,073 retail premises tracked by the organisation. Food retail makes up 21% of trade for comparison. This growth in pharmaceutical retail amid war creates opportunities to exploit a supply chain historically vulnerable to corruption.

In 2014, government auditors estimated that Ukrainians were overpaying for medicines by up to 40%, largely due to a procurement system dominated by a few oligarchic intermediaries who sourced cheap drugs through offshore shell companies and sold them at inflated prices. Ukraine’s Ministry of Health attempted to reform the system by employing international agencies such as the British development organisation Crown Agents alongside UNICEF and the UN Development Programme. Despite this attempt at reform saving the Ministry of Health tens of millions of dollars, it is still possible to find offshore intermediaries selling medicine into Ukraine with connections to the old system.

Mili Healthcare is a UK-registered company that operates solely in Ukraine, supplying hospitals and retail pharmacies with drugs, ranging from cold medicines to cancer treatments. From 2016-2019, the company was led by Borys Lytovsky, a businessman active in the pharmaceuticals sector since the 90s. Though largely unknown outside Ukrainian and Russian media, investigative journalists from the Ukrainian anti-corruption project Nashi Groshi (Our Money) have called Lytovsky the most underestimated pharmaceutical oligarch.

Nashi Groshi alleges Lytovsky’s involvement in Ministry of Health procurement schemes as far back as the early 2010s. In 2014, the outlet wrote of his connections to the Farmex Group that hiked the price of vaccines by 60% over the course of a year. As recently as 2020, journalists have connected Lytovsky to Norton-Ukraine LLC, which sold coronavirus testing machines to hospitals at inflated prices. A number of scientists who used the machines found them to be ineffective.

In 2015, before Lytovsky became director of Mili Healthcare, international organizations supporting Ukraine’s pharmaceutical procurement reforms purchased oncology drugs from the company. When the list of purchased medicines was made public, concerns about the quality of Mili Healthcare’s drugs surfaced on social media, drawing criticism from prominent figures, including a well-known charity leader and investigative journalist. At the same time, some argued that these concerns were orchestrated through a covert campaign by the pharmaceutical oligarchy to undermine public confidence in the reforms and discredit the role of international agencies. Regardless of the controversy, Lytovsky’s appointment as director of Mili Healthcare in 2016 is notable for how closely it followed the company’s emergence as a supplier within the reformed system.

In 2019 Lytovsky ceded control of Mili Healthcare to Olga Lytovska, the current director of the company. According to Companies House filings she is, like Lytovsky, a Ukrainian national based in Monaco. Furthermore, according to investigative journalists from Bihus, Olga Lytovska is the name of Borys Lytovsky’s wife.

Mili Healthcare recently won Ministry of Health tenders to deliver oncology drugs from 2024-2025 for around $7 million. The company occupies 0.4% of the market and grew its sales by 28% in 2023. Whilst British companies have previously attempted to aid Ukraine in reforming its procurement system, it is unclear to what extent the UK’s own corporate registry has and perhaps continues to facilitate some of this graft.

Defence

Global military expenditure has reached a record $2.4 trillion, with Europe experiencing its sharpest increase in three decades. Against this backdrop, it’s no surprise that many suppliers to the UK’s armed forces anticipate rapid growth in demand, with some already reporting strong order books and increased turnover.

Large defence contractors are the main beneficiaries of rising military spending, relying on governments as their primary clients. SAAB UK, a subsidiary of the Swedish defence firm SAAB AB, describes its relationship with the Ministry of Defence (MoD) and the Department for Business & Trade as very close. This close proximity allows them to better anticipate government needs and navigate changes such as those that may follow the upcoming Defence Spending Review. SAAB’s newly opened advanced manufacturing site in Hampshire was inaugurated earlier this month by the Secretary of State for Business and Trade.

Rising demand for UK defence products is being driven not only by domestic needs but also through increased European investment. Merlin PCB Group, the UK’s largest printed circuit board (PCB) manufacturer, reports record-high order volumes, supplying electronics to replenish EU and NATO stockpiles while also supporting Ukraine. Elsewhere, Marine Specialised Technology, a manufacturer of rigid hulled inflatable mission boats, has secured significant orders through NATO contracts, including a landmark deal with the German Navy under its F126 Frigate Programme; the largest procurement in the navy’s history.

The transformation of recent conflicts through drone warfare and AI is stimulating product development in the UK across legacy players and smaller SMEs. Over the past year, SAAB has focussed its UK operations towards autonomous systems, acquiring a start-up specializing in AI-driven drone swarms and relocating production of its underwater robotics portfolio to the UK. Meanwhile, Callen-Lenz, a company developing in-house uncrewed aerial systems (UAS) that counts former Thales CEO Victor Chavez as a non-executive director, quadrupled its turnover. Notably, 97% of its turnover was generated in the UK, where it supports the Ministry of Defence (MoD) across R&D and UAS supply. Callen-Lenz observe that the current geopolitical situation has accelerated uncrewed air systems to become a central pillar of Defence Procurement, giving the company what appears to be its most exceptional year of growth across 15 years of operation.

Many companies driving the defence sector’s growth develop dual-use technologies, serving both military and civilian markets. Such firms balance defence contracts with projects in other highly regulated industries such as marine science, aerospace and energy. These domains require extreme reliability in some of the world’s most hostile environments. Salt Separation Services, for instance, supplies reverse osmosis plants that provide drinking water to Royal Navy submarine crews submerged for months, as well as to scientists stationed at Antarctica’s remote Rothera Research Station. Likewise, Alcon Components’ bespoke braking systems are deployed in Supacat Jackal vehicles, a number of which are reportedly in use on Ukrainian battlefields, while also servicing Aston Martin hypercars competing in prestigious 24-hour endurance races in Le Mans.

One of the key challenges of assembling highly sophisticated and expensive equipment in close proximity is ensuring its effective operation, maintenance and long-term viability. This places emphasis on training, military simulation, engineering education and technical documentation with providers such as Allan Webb reporting increased opportunities and Mabway securing recent contracts with the British Army.

While investments in new hardware and training are essential for long-term military readiness, they must be balanced against the immediate demands of ongoing operations. With wars raging on multiple continents and new hardware taking years to develop, existing military equipment has been highly sought after. As a result, portions of defence budgets are being directed toward extending the lifespan of aging assets. The MoD recently funded vehicle manufacturer Supacat to repurpose a number of its High Mobility Transporter (HMT) off-road vehicles for use in Ukraine.

It is striking that amongst all companies reporting financial uplift from ongoing conflict, the UK’s largest supplier of ex-military hardware, L Jackson & Co, saw one of the steepest relative increases in turnover. The company, based at The Rocket Site in Doncaster (an old missile testing facility) reported its highest-ever sales growth, with turnover tripling to £33 million last year. The majority of its £18.4 million in revenue was generated domestically, but European sales reached £11.2 million, surpassing the company’s total global turnover from the previous year. L Jackson & Co attributes its record sales growth primarily to ongoing global insecurity. Data from Import Genius suggests the company has recently shipped a number of vehicles to Ukraine.

L Jackson & Co may export its equipment, that is largely if not solely derived from the MoD, to any foreign army for whom the UK government grants an export licence. Failure to comply with these regulations has led to legal trouble in the past. In 2009, the company’s owner was jailed for bypassing government restrictions and selling former military equipment to Sudan. Despite strong performance in 2023, the company has cautioned that procurement opportunities have declined over the past 12 to 18 months, suggesting that high demand may be catching up with the market for ex-military equipment.

Energy

Wholesale electricity and gas prices rocketed as the war in Ukraine stoked supply fears. These cost increases were largely passed on to households as oil and gas majors reported record earnings. ExxonMobil, North America’s largest producer, posted the highest profit in its 152-year history. This marked yet another extraordinary year in a series of extraordinary years for the energy sector, following the whiplash of lockdowns and economic recovery. The impacts of high prices extended across the broader industry, lifting companies throughout the supply chain.

Smaller independent businesses distributing oil products from major suppliers like ExxonMobil also reported increased turnover as a result of the oil price. Companies including Country Oil Group, Kinch Fuel Oils, and Estuary Oils operate fleets of fuel trucks that serve a diverse customer base, including commercial, agricultural and domestic sectors. 5% of British households, primarily in rural areas, still rely on oil for heating, making these distributors essential. Many of those living in oil heated homes may join oil clubs to secure lower prices by purchasing in bulk for their village or collective. These buying groups may closely watch geopolitical developments to time their purchases strategically.

Beyond responding to the immediate shock of high energy prices, the conflict refocused attention on long-term energy security, making it an urgent strategic priority. European countries began the painful process of reducing their reliance on Russian oil and gas, thereby minimizing their exposure to future supply disruptions and sanctions. This shift naturally opened up opportunities for those in the business of constructing and maintaining critical energy infrastructure. LFF (Scotland), an Aberdeen-based manufacturer of pipes, fittings, and flanges, exceeded expectations, reporting a substantial increase in turnover—rising to £54 million from £34 million, partially as a result of energy security concerns. Similarly the offshore engineering consultancy London Marine Group, anticipated growing demand for projects that reduced reliance on Russian oil and gas, from offshore windfarms to the construction of liquefied natural gas (LNG) terminals. However, recent events have reinforced the fact that energy security extends beyond shielding consumers from price volatility and reducing reliance on geopolitical adversaries.

Covert attacks on energy infrastructure are becoming more frequent and environmentally devastating. Covelya Group, a consortium of companies specializing in bespoke sensor systems and software across defence and engineering, reported a record-high number of orders, with turnover jumping from £70.4 million to £101 million. They primarily attribute their growth in the oil and gas sector to the shift away from Russian energy but observed a significant transition in demand from oil and gas exploration technologies to those focused more on inspection, repair, and monitoring. While this trend may be partly attributed to declining exploration in regions with depleted reserves like the North Sea, it also raises questions about the extent to which geopolitical concerns are driving increased investment in these technologies, particularly among companies operating at the intersection of energy and defence.

Protecting critical energy infrastructure from coordinated attacks also requires getting the basics right. High-performance security doors and locking systems will need to withstand everything from gunshots and explosions to fires. Such systems are likely necessary across a wide range of locations, including power plants, natural gas processing plants, LNG terminals, oil refineries, offshore oil and gas platforms and energy storage facilities. These measures may also be required to protect the many smaller, often unmanned remote sites across wider energy distribution networks. Surelock McGill, a provider of advanced locking systems, reported increased demand and revenue last year as customers across all its core markets attempt to secure themselves against terrorism and similar threats.

Broader energy security concerns have led some to invest in renewable infrastructure for greater energy independence. Harris & Bailey, one of the UK’s largest building material suppliers, described the energy crisis as an event that fundamentally undermined their confidence in future pricing and supply. This prompted them to install 500 solar panels on their warehouse in Croydon, generating up to 198kW per hour at peak output. On sunny days, the company produces excess energy, which is fed back into the grid under a favourable export tariff with E.On Energy, providing additional passive income. During the energy crisis, those generating renewable energy, such as solar, and selling it on the market benefited from higher electricity prices, boosting their turnover.

The shift toward energy security has also driven demand for the critical metals essential for developing the batteries that support renewable energy technologies. Revert Alloys and Metals, a company specializing in recycling nickel, cobalt, and complex superalloys, sees the conflict in Ukraine further amplifying an already exceptionally strong forecasted demand for these critical metals.

The severity of energy security concerns in the UK pale in comparison to those experienced in Ukraine, where relentless attacks on infrastructure heightened the need for portable backup power as winter set in. In Germany, the government urged citizens to purchase generators in anticipation of potential power shortages as Russia threatened to cut off its energy supplies to the country. This urgent requirement for back-up power was met by businesses across Europe, including UK-based Bimson Power; the exclusive European distributor of Champion power generators. By servicing this brief spike in demand, the company doubled its sales between August 2022 and January 2023.

Mobility

UK net migration hit record levels in recent years, driven in part by government schemes for displaced populations from Hong Kong and Ukraine. This increased the need for a range of essential support functions to welcome newcomers and help them to integrate. A number of organisations providing language and education services reported significant increases in participation.

Community centres and religious groups offering free English language courses have seen increased uptake from recent Ukrainian arrivals. As community-led initiatives, these organizations often offer a broader range of support services beyond language instruction, including assistance with paperwork and skills training in employability, first aid, and basic IT. Such spaces can serve as a bridge for newcomers, helping them to foster social connections before attempting to find work in the UK.

Increased participation in education also drives demand for testing and qualification services. A commercial entity that provides language tests and qualification recognition for UK visa applicants, reported a 50% year-on-year increase in turnover to £38.3 million, fuelled by rising transaction volumes partially due to displaced people from Hong Kong and Ukraine. However, any optimism surrounding these results was likely short-lived. It was recently revealed that the company had amassed £13 million in profits over the years while operating government visa services under a not-for-profit contract. The company has since repaid these unlawful gains back into the government language test service and a number of executives resigned.

Ukrainian translation and interpretation skills have also been highly sought after, particularly within the MoD. Worldwide Language Resources, the government’s preferred linguistic service provider, saw revenue nearly triple to £31.1 million in 2023. This significant boost in revenue is likely due to the company’s involvement in recruiting a large number of linguists to assist the British Army train Ukrainian soldiers in the UK.

Whilst conflict can open up special routes between countries, it more often disrupts and destroys existing mobility channels. Following the October 7th attack on Israel, international airlines suspended flights due to security concerns, leaving national airline El Al as one of the few still operating. To meet the reported shortages in medical and military supplies, the airline swiftly retrofitted several Boeing passenger planes as exclusive cargo carriers.

British cargo provider Airbridge International Agencies, which worked with El Al, reported increased revenue margins tied to the processing of this increase in cargo being transported by the airline. Some of the transported supplies were donated by charities with El Al claiming to have delivered first aid kits, medical supplies, thermal clothing for soldiers and equipment purchased for families near Gaza.

Dataset

Noteworthy Items

The largest cost associated with financial irregularities were reported by 144 Midland Road Limited, a development run by the well-known property mogul Nicole Bremner. The company claims to have lost £11M through material misappropriation of assets over a number of years through the activities of former director and property developer Abraham “Avi” Dodi. The company is conducting a forensic accounting review and yet to make a claim against Dodi. £11M is a significant sum of money for a development with net liabilities of £4.4M, most of which is owed to one of Dodi’s companies.

In the largest reported bank hack, £3.8M was stolen from Nutri-paw Ltd, a company that develops health supplements for dogs. Unfortunately, the company was only able to recover £800K of the funds with the rest being written off. The most expensive cyber-attack was targeted at Capita Pensions Solutions Limited costing the organisation £3.6M. These costs went towards specialist professional fees, recovery and remediation costs and investment to reinforce Capita’s cyber security environment. Although the company’s report provides limited details on the incident, external sources indicate that the broader Capita Group, of which Capita Pension Solutions Limited is a subsidiary, estimates the total financial impact of the March 2023 cyber-attacks to be around £25M. The attack was reportedly linked to an unsecured Amazon S3 bucket, which allowed the Black Basta ransomware group to gain access to sensitive data, including the personal information of half a million pension holders.

The most expensive arson attack was the incident at SWX Bristol, a music venue owned by Electric Group Holdings Limited, that warranted a £2.95M insurance payout. The venue was attacked by Owen Marshall, who dowsed a towel in petrol before setting it alight and posting it through the venue’s letterbox in the early hours of July 13th 2021. Marshall’s motive stemmed from an unhealthy obsession with lockdown restrictions, leading him to target entertainment venues in Bristol based on the false belief that they would be introducing COVID-19 vaccine passport requirements imminently.

It’s rare for a business to have all its assets seized by the police and still live to tell the tale. However Eidolon-UK Chemical Services Ltd operates within a heavily scrutinised corner of the market. The company manufactures and distributes an organic chemical called trans-β-methyl-β-nitrostyrene, otherwise known as 1-phenyl-2-nitropropene. Beyond research applications, the chemical is used as a precursor in the synthesis of amphetamines. Eidolon clarify that the chemical is not on any postal service’s banned list, is not yet illegal anywhere in the world nor included on the USA’s TCSA list of toxic chemicals. Despite having to refund all customers and offer discounts following the police raid the company remains undeterred, having relocated to a new commercial lab. Eidolon faces continued obstacles marketing the chemical through established channels and has been subject to shadow bans on eBay and had its Etsy store suspended. The company maintains its own website with independent payment processing, offering discounts for cryptocurrency transactions.

Perhaps the most unsettling disclosure comes from Lehram Capital Investments Ltd who reveal that since 2019 they have come under sustained attack from an organised Russian crime syndicate determined to annihilate the business and prevent it from exposing the extent of the group’s criminal activities throughout the continent. A former company director is reported to have suffered repeat accidents causing them serious harm and permanent damage resulting in confidential settlements. Directors report facing threats of murder, kidnapping, extortion and persecution towards themselves and family members. Although Lehram’s report does not name those it accuses of harassment, legal proceedings and media reports suggest the company’s troubles date back to its 2013 acquisition of the Gramoteinskaya coal mine in Siberia. Lehram’s director Igor Rudyk was supposedly detained on immigration charges, coerced under duress, and forced to sign over the mine to a criminal group allegedly linked to Siberian industrial magnate Alexander Shchukin, with the involvement of regional officials. Lehram continues to pursue legal action internationally.

Loss of stock due to theft is a common occurrence with stolen items across the reporting period including construction tools, jewellery, consumer electronics, pharmaceuticals, C18th kitchenware and a salt bin. Primo Drinks, the UK’s largest family run drinks wholesaler, writes of a large theft issue in their North East branch in 2023 involving all but one employee. The company fired and replaced the entire team.

Affordances and constraints of the index

Indexes map the most important terms within a text, helping readers navigate large documents with greater ease. They reveal underlying structure through page references. Listing the most important terms has a levelling effect that facilitates chance encounters with subjects the reader may have overlooked.

Indexers face multiple challenges in crafting a useful index. The level of detail, known as depth, is an important consideration. We want to capture all the relevant subjects from the text without inducing cognitive overload in the reader. The process of selecting terms to include in the index requires skill and is more subjective than we might initially think. The indexer is not exonerated from the potential charges of bias that journalists and memers are more accustomed to facing. There is a well-documented history of partisan indexers subtly undermining the content of political texts through their choices. The president of the UK’s professional body for indexers describes the role as a ‘difficult, vital, and underappreciated art’, reminding us that ‘a computer can’t compile it for you, and that “just doing it yourself” isn’t nearly as easy as it sounds’.

Selecting terms

Selecting terms is a challenging task and we do not claim to have expertise on par with that of a professional indexer. To achieve an acceptable level of performance, we leaned on the existing knowledge taxonomy of Wikipedia; most subjects encountered within Project 2025 are likely to have a Wikipedia page with a well chosen title and useful metadata. For cases in which significant terms did not have a suitable Wikipedia page, we reverted to indexing the term as it appeared in the text or generated a term that seemed to be the most accurate.

AI-augmented features

Language models can enhance reader interaction with indexes, without compromising the simplicity and familiarity of the format. We applied language models to implement sort, summarisation and search functionality to encourage wider exploration and more efficient navigation of the document.

Sort

Indexes are typically sorted alphabetically. By generating embedding vectors for each term, it’s possible to instead sort the index by semantic similarity, where proximity correlates with similarity in meaning. This kind of sort results in the natural formation of thematic regions. In the image below I have attempted to hand label some of the regions within this conceptual gradient. Semantic sort could prove even more useful when comparing multiple texts; aligning indexes could highlight conceptual gaps and overlaps.

Summary

Indexes provide a tradeoff between convenience and context. While they quickly direct readers to relevant pages, they lack the ability to convey how a subject is discussed on the page— requiring the reader to navigate through the source document for details. In an attempt to retain more context, we incorporated hidden summaries generated by language models, which offer a brief overview of the subject’s treatment on the page. These summaries, accessible via tooltips that appear when hovering over page numbers, allow readers to grasp context without leaving the index. Optimised for conciseness, neutrality, and accuracy, each summary gives a sense of context, encouraging the reader to navigate to the source if the context is relevant to their line of enquiry.

Search

Given that the index contains thousands of terms and summaries, a search engine is a very necessary convenience for readers who know what they’re looking for. By generating embedding vectors for both terms and page summaries, we enable the reader to search by subject and document content. This distinction allows the reader to search with greater precision because of the separation between independent subjects and document context. As an example, searching for the closest term to ‘enemies of the USA’ returns ‘United States foreign adversaries’. However, when searching for page-level discussion with the same query, the top result is the reference to ‘Nazism’ on page 768 where the author discusses America’s war efforts against Nazi Germany.

Network

The following network illustrates the connections between Reform and a selection of companies registered within the past decade (2014-2024) at Companies House. Each time a company from the network is mentioned within this article, we use it’s full title in Companies House and highlight in grey: Reform is currently registered under the name Reform UK Party Limited.

All companies at the edge of the network were formed within the past decade. Nodes within the network represent officers or persons of significant control 👤, active companies 💰, companies no longer active ❌ and addresses 🏢. A selection of important nodes are numbered, corresponding to each of the six sections within this blog post:

• 1️⃣ Richard Tice
• 2️⃣ Bathurst Estate
• 3️⃣ Leave.EU
• 4️⃣ Nigel Farage
• 5️⃣ Reform UK
• 6️⃣ UKIP

To understand how any two nodes are connected, hover over the arrow that joins them to view the tooltip description (e.g. the arrow connecting Richard James Sunley TICE with QUIDNET REIT LIMITED reads ‘Appointment’ because QUIDNET REIT LIMITED is a current appointment of Richard James Sunley TICE in Companies House). Red arrows indicate either an officer’s past appointments or a company’s former officers.

Six degrees of Reform

1. Richard Tice

Richard Tice has spent decades working in real estate and continues to be active in this sector. There are several profiles in Companies House attributed to Tice with one listing roles in 155 active and historic companies. He is currently the CEO of Quidnet REIT Limited a real estate investment trust (REIT) setup in 2015 and listed on the International Stock Exchange in the Channel Islands. The company acquires properties, refurbishes them, and leases them to businesses. Quidnet REIT is a subsidiary of the real estate asset manager Quidnet Capital.

Quidnet REIT Limited manages a portfolio of four properties that range from manufacturing sites to commercial office space. In 2022, the REIT reported leasing one of its properties in Northampton on a long-term basis to a kidney dialysis facility. Based on streetview imagery and planning documents, this facility is likely operated by DaVita, an international provider outsourced by the University Hospitals of Leicester NHS Trust. Healthcare providers can be lucrative tenants for REITs due to their long-term leases, inflation-adjusted rents and limited availability of alternative properties.

The REIT’s refurbishment efforts also include installing green technologies at various sites, such as electric car charging points and solar panels on roofs. A 2023 announcement from the REIT noted that, besides reducing emissions, the technologies substantially increase rental income and provide further asset management opportunities. The company has embraced an emerging trend amongst REITs to recognise their rooftops and parking lots as embedded assets ripe for profitable green development. Reform have opposed the use of farmland for solar development in their manifesto with Tice echoing this sentiment on social media in favour of solar development on roofs.

Coincidently, the UK office of RwandAir is registered at the REIT’s Gatwick House property. Rwanda’s national airline notably refused the Conservative proposal to run removal flights of migrants from the UK to Rwanda, fearing reputational risk. Tice has publicly criticised the Rwanda policy, advocating instead for something closer to Australia’s ‘Stop the Boats’ policy.

Tice has loaned Reform over £1.4M between 2020 and 2023 via his company Tisun Investments Ltd, according to Electoral Commission data. In recent years, Tice has established a range of businesses under the ‘Tisun’ moniker, including Tisun One Ltd, Tisun Two Ltd, Tisun Three Ltd and Tisun Four Ltd all of which list Tisun Investments Ltd as a person of significant control. Reform also recieved ~£150k in donations from Britain Means Business Ltd between 2021 and 2022, where Tice is a director. Additionally, Reform’s events have been supported by Sunley Events Limited, a company connected to Tice through his relatives.

2. Bathurst Estate

Richard Tice was a director at Bathurst Development Limited, a subsidiary of the Bathurst Estate in Cirencester, from 2014 to 2019. The company was established to build homes on the 15,000-acre estate, owned by the Bathurst aristocracy since the 17th century. The housing development came under public scrutiny for using a land trust registered in Bermuda to avoid paying capital gains tax. A number of present day persons of significant control for the development are Bermudian or based in Bermuda alongside the company’s initial shareholder Bathurst Trust (PTC) Limited.

In recent years, the estate has faced public backlash for implementing a paywall for Cirencester Park, a 3,000-acre area that had been free to access for 326 years. In early 2024 the estate was targeted by Right to Roam protesters, prompting local councillors to call on the Bathurst family to reconsider the paywall.

On the subject of the Bathurst Estate, it would be remiss not to highlight the eccentric business activities of Roger Tempest, who was a director of Bathurst Development Limited in the year before Tice joined. Tempest is the custodian of Broughton Hall Estate and one of the UK’s largest land owners. He is also a director of Wyrd Technologies Limited, an organisation developing “consciousness technology”. Their latest invention is the Wyrdoscope, a physical “scientific” device (available for pre-order) that claims to measure the extent to which events like human intentions and feelings can impact the physical world. This phenomena bares a striking resemblance to telekinesis, a word you will not find on Wyrd’s website.

Wyrd Experience Limited purports to be “the world’s only interactive consiousness lab” and invites groups of participants to undergo parapsychological experiments building upon the research of the now defunct Princeton Engineering Anomalies Research Lab (PEAR). Operational for nearly three decades at Princeton, the lab drew heavy criticism from the university and scientific community; however, it seems to have found a more secluded home in the tranquil grounds of the Broughton Hall Estate.

Roger Tempest’s esoteric directorships continue with The Youth City Foundation which appears to be a subsidiary of Rural Concepts, an investment and international development organisation also directed by Tempest. The Youth City Foundation was dissolved in 2018 however the website for Rural Concepts describes the company’s vision for a project called International Youth City, a smart city in the Middle East to showcase youth development potential. Although International Youth City never graduated beyond the drawing board, Rural Concepts seem to be actively developing Lebdah City, a smart eco-city in Libya through the subsidiary Rural Concepts (Libya) Limited.

3. Leave.EU

Richard Tice co-directed Leave.EU Group Limited, one of the main political organisations that campaigned for the UK to leave the EU between 2016 and 2017. Tice co-founded the group with the insurance millionaire and UKIP donor Arron Banks. Leave.EU’s campaign was run by Better for the Country Limited with both companies receiving £8M from Banks in what amounts to one of the UK’s largest ever political donations.

Banks controls a number of insurance companies including Somerset Bridge which is represented by several subsidiaries: Somerset Bridge Group Limited, Somerset Bridge Shared Services Limited, Somerset Bridge Premium Finance Limited, Somerset Bridge Ltd and Somerset Bridge Capital Limited. Formerly known as Eldon Insurance, Somerset Bridge rebranded at the end of 2019. The company operated under various tradenames, including Business Choice Direct Limited and Vavista Life Limited. ICS Risk Solutions Limited based in the Isle of Man, was Eldon Insurance’s parent company. The involvement of Banks’ insurance companies within the Brexit campaign attracted scrutiny and investigations from the Information Commissioner’s Office, resulting in Eldon Insurance and Leave.EU receiving a £135k fine for data misuse.

Banks was inspired to emulate some of the successful tech and media organisations on the populist right. In 2016 he founded Big Data Dolphins Ltd, a start-up for collecting and analysing big data for political purposes, inspired by Cambridge Analytica. Despite concerns that UK citizen data may have been transferred to the US for analysis, an investigation by the Information Commissioners Office concluded that there was no evidence to suggest Big Data Dolphins was ever operational. Westmonster Ltd was a media website launched by Arron Banks, that aspired to be the British counterpart to the American far-right websites Breitbart and Drudge Report. Directed by Michael Heaver, former Brexit Party MEP, and Liz Bilney, chief executive of Leave.EU, the organisation behind the site dissolved in 2020.

Following the referendum, some key players in Leave.EU continued to influence politics, while others returned to business. Arron Banks persisted in his efforts to shape Brexit by founding a Tea Party-inspired movement called The Blue Wave Movement Limited which aimed to flood the Conservative Party with new members holding a more hard-line Eurosceptic stance. Despite Westmonster Ltd going offline, the website now links to Michael Heaver’s YouTube channel, which pushes pro-Reform content, with some videos achieving hundreds of thousands of views.

After her unsuccessful attempt to establish Britain’s Breitbart, Liz Bilney continues to collaborate with Leave.EU’s finance director Alison Marshall on Fuzzy Insurance Limited which provides commercial motor and business insurance. Caroline Drewett, the former Leave.EU Ambassador for Small Businesses now directs Anglo Belgravia UK Limited, a company offering guardianship for UK boarding school pupils.

4. Nigel Farage

Nigel Farage is the co-founder and current leader of Reform. He previously collaborated with Richard Tice on the Grassroots Out (GO) movement, represented by the Go Movement Ltd and Grassroots Out Limited companies. The GO movement was affiliated with Leave.EU which unsuccessfully competed against Vote Leave to lead the Brexit campaign.

Following the referendum Farage and collaborators have launched a range of media and PR companies. Farage Media Limited is a video production company incorporated in late 2020, however it remains dormant. Farage Media Limited’s former co-director and Reform’s deputy leader David Bull is a director at Oak Hall Studios Ltd, video production service that frequently films Reform events. In recent years Bull has presented on TalkTV and is a former-medical doctor and MEP.

Another frequently-cited media collaborator of Farage is Dan Jukes, previously head of communications at UKIP who continues to advise Farage on digital strategy, including social media during Farage’s time on I’m a Celebrity… Get Me Out of Here! Jukes recently registered Robert Wood Media Ltd, a firm offering digital strategy, PR and crisis management. The company provided PR consultancy to Michelle Mone and Douglas Barrowman to manage the reputational fallout of their involvement in the PPE scandal.

5. Reform UK

Reform Party UK Limited was founded as The Brexit Party Limited by Nigel Farage and Catherine Blaiklock in late 2018. Blaiklock, a former UKIP politician, financial trader, and hotelier, registered The Brexit Party Limited to one of her properties- a B&B in Norwich named Annapurna. The property’s profile on TripAdvisor was inundated with negative reviews and branded the ‘Brexit Party B&B’. In recent years Blaiklock has established several other real estate and holiday companies including Oneika Limited, Kamarina 2 Limited, Penguin House Limited, Azur Sea Limited and possibly a pest control company: Bye Bye Bed Bugs Limited.

Blaiklock resigned from the Brexit party following relevations of Islamophobic tweets posted through her public account. Despite being ousted, in 2023 she registered another company under the name The Brexit Party Limited in which she is the sole director and person with significant control. Other companies setup by Blaiklock and dissolved within 2 years include Magna Carta Campaign Ltd (FKA Big Brother Is Watching You Ltd), Our Country Limited and Our Land, Our People Limited.

One final Reform-related thread worth exploring is that of Michael McGough, the former party treasurer, who was booted out of the party following a series of offensive Facebook posts. McGough was once the treasurer of Freedom Association Limited(The) (the Freedom Association), a libertarian pressure group founded in the 1970s that continues to campaign for a range of causes, including scrapping the TV licence fee. Some more recently incorporated projects its members are involved in include the defunct Conservative Democratic Organisation (CDO Ltd), a grassroots movement within the Conservative Party that opposed the removal of Boris Johnson and Liz Truss. Ewen Stewart, who was a director at the Freedom Association until 2019 was also a director at Centre Point Strategies Limited which runs the conservative website Think Scotland.

6. UKIP

It may come as little surprise that many of the more colorful companies spun up in Reform’s orbit are directed by past and present members of Nigel Farage’s former party, UKIP (United Kingdom Independence Party Limited).

Neil Hamilton, a disgraced MP and current UKIP director, backed Slim-Be Limited, a manufacturer of weight loss drinks which he boasted could “transform the health and waistlines of the nation”. Patricia Bryant, a member of UKIP’s National Executive Committee held a directorship at Fortis est Veritas Media Company Limited which may be connected to the populist right blog and events company of the same name. The company dissolved last year with Fortis est Veritas’ Facebook page promoting an event sponsored by a hypnotist that invited former-MP Andrew Bridgen to discuss the negative impacts of the COVID vaccine.

Perhaps the most jarring company name in UKIP’s orbit is White Indigenous Rights Alliance CIC (WIRA) set up by former party director Katie Fanning. The social enterprise outlines its mission statement as building ‘strong and supportive White communities’ in the face of a ‘globo-Marxist White-eradicationist agenda’ that neglects so-called ‘indigenous white communities’.

Of all UKIP’s former directors, the zanniest resume belongs to Henry Bolton, who led UKIP from 2017 to 2018 and previously served in the armed forces and police. Over the past decade alone, Bolton has tried his hand at setting up a think tank (The Boudicca Centre for (Post Brexit) Policy), TV channel (GB TV) and a political party (One Nation Party Limited) whilst also penning a book entitled What a State! Boltonomics and the Appalling State of British Politics released through his own publishing house Wise Walrus Ltd. All of these companies have since dissolved. At the time of writing Bolton appears to be juggling multiple engagements including directing a private investigations firm (Vedette Resources Ltd) and starting-up the British counterpart to the US Conservative Political Action Conference (CPAC) that Bolton calls BritPac. As of last year he has also ressurected the Primrose League, a conservative movement founded in the 19th century by Churchill’s father. Bolton’s Primrose League aims to counter the rising threat of ‘new socialism’.

Methodology

We processed data from Arts Council England, Arts Council of Wales, Arts Council of Northern Ireland and the Scottish Arts Council, each with its own time span and geographical boundaries. We then proceeded to harmonise these datasets to produce aggregated arts funding time-series for each UK Westminster constituency (2024 boundaries).

The Scottish dataset, based on postcodes, allowed us to map geographical coordinates to the constituency boundaries, covering 2011 to 2023. The English dataset, which ranged from 2014/15 to 2022/23 and included data beyond England, was updated from 2015 constituencies to 2024 boundaries using geographical weighting.

For Northern Ireland, we handled data spanning from 2010 to 2023 using 2010 boundaries, also updating it to 2024 boundaries through geographical weighting. The Welsh dataset was already aligned with the 2024 parliamentary constituencies and covered the period from 2010 to 2023.

After harmonizing the geographical data, we integrated the datasets, accounting for cross-country constituency values and labeling non-specific extra-regional allocations as ‘Outside Scotland’ and ‘Outside Wales’. This resulted in a comprehensive dataset spanning from 2010 to 2023, with varying start and end years for each country.

To make the funding values comparable across different years, we converted them in real terms to 2023 price levels using the Office for National Statistics Consumer Price Index (ONS CPI). The user has the option of viewing both the nominal or the adjusted real term figures.

We then two measures for the change in arts council funding for each constituency. The absolute measure gives the difference in funding between two given years, while the trend-line measure, based on linear regression, considers the change in funding over time, reducing the impact of multi-year fluctuations.

All data processing, including cleaning, harmonization, integration, and analysis, was done in Python, ensuring robust handling and accurate computations. We developed the dashboard using Svelte and Apache ECharts.

Methodology

Risk Quantification

Whilst developing GERM, there were several examples of software for visualising risk across large corpuses of unstructured text that informed our thinking.

US Federal Reserve economists Dario Caldara and Matteo Iacoviello developed a text-based geopolitical risk (GPR) index that mines news articles for combinations of well-chosen keywords correlating to geopolitical risk. The sources of news that feed the index, including the Financial Times, The New York Times (Figure 2) and The Wall Street Journal, have extensive digital archives allowing the researchers to test how well their index captures historic events. The index tends to spike in times of war and peaked on 9/11.

Blackrock developed a comparable geopolitical risk indicator for measuring ‘market attention’ towards 10 of the top geopolitical risks as they see it. As of April 2024 they are closely tracking the potential for a Russia-NATO conflict, gulf tensions and major terror attack(s). The index combines the outputs of machine learning models fine-tuned to detect relevance to each risk topic and sentiment.

Compared to text-mining, the use of more advanced language models allows for a greater level of context to be factored into any classification of unstructured text. This expanded capability invites the need for some constraints to guide the development of novel risk monitors. After all what risks are we interested in tracking? When designing prompts to guide an LLM, how can we keep the language sufficiently adaptive so as to account for emerging risks? LLMs may be able to highlight sections of text discussing risk but in aggregate we still require some degree of classification to understand if war is being reported on more than supply chain disruption or drought. This suggests the need for a good taxonomy.

Taxonomy

Taxonomies form the corner stone of corporate risk strategy. They establish a common language for describing risk. Consultants might say “if it is in the risk taxonomy, it gets managed.” Some criticisms levelled at these frameworks include the failure to sufficiently incorporate emerging risks or black swans. For our project we are looking to identify geopolitical and environmental risks but don’t actually know what subsets of these broad genres of risk might be found in Companies House. How many references to the ongoing war in Palestine might we encounter daily? Would any companies be impacted by earthquakes abroad? We are really looking to use a taxonomy as a tool for discovery. Something to draw upon when contructing a series of flexible prompts to guide an LLM in filtering data. Therefore we sought out the most extensive taxonomy possible. This led us straight to the Cambridge Taxonomy of Business Risks (Figure 3).

Prototype

Building upon the preceding research, we developed the first prototype for GERM to visualise the geopolitical and environmental risks flagged within annual reports filed electronically with Companies House across the month of March 2024 (Figure 4).

GERM classifies each risk using a selection of categories adapted from the Cambridge Taxonomy of Business Risks. Our adapted taxonomy takes the following shape:

GERM’s Risk Monitoring Pipeline

GERM processes annual reports through a series of steps. Documents are downloaded and the text within them is extracted and separated into chunks. Each chunk of text is searched for keywords associated with each category of risk within the taxonomy (Figure 5).

Any chunks containing keywords relevant to a category of risk within our taxonomy proceeds to be processed by an LLM that classifies the chunk as to whether or not it contains discussion relevant to the flagged category of risk.

If the LLM flags the chunk as containing relevant discussion pertaining to the category of risk, the chunk proceeds to be processed by a sequence of LLMs that summarise the risk discussion, extract and summarise stated impacts and extract the names of any countries mentioned. This is the data that ultimately populates the risk database where it can be searched for keywords, sorted and downloaded (Figure 6). This pipeline is biased towards higher precision at the expense of recall so there are likely to be instances of risk reporting that are not flagged as such.

Risk Impact Embeddings

Each risk impact, such as loss of staff, increase in costs, or damage to building, is processed by an embedding model that generates a semantic embedding vector for each description. Within GERM these vectors are represented in a semantic space—a kind of map where each point (vector) represents the meaning of a risk impact (Figure 7). The closer two vectors are on this map, the more similar they are in meaning. For instance, in two-dimensions, the vector for loss of staff might be positioned closer to the company struggles to retain employees than to crops were damaged by drought. This mapping allows us to visually identify and analyse:

1. Different types of risks leading to similar impacts on different companies. This is shown as clusters of mixed colour points on the map. For example, extreme weather, interstate conflict, and food security might all cause similar increases in farming costs and prices.
2. The most frequent impacts associated with each type of risk on different companies. This is shown as clusters of points of the same colour. For example, interstate conflict often results in supply chain disruptions, rising energy costs, and inflation.

The extraction of named entities in the form of countries can serve as input to a global risk heatmap visualising the countries most frequently appearing within risk disclosures (Figure 8).

March 2024 Observations

Of the 266,989 annual reports processed by GERM throughout March 2024, only 621 companies were flagged as reporting any of the risks from our taxonomy. As suggested within our methodology, the majority of companies filing electronically will be smaller in size and are therefore less likely to include detailed risk disclosure. Some of the largest companies reporting relevant risks within the month included Rapiscan Systems (airport security hardware specialists), FP Mccann Group (supplier and manufacturer of precast concrete) and Enerveo (contractors).

Discussion of interstate conflict and climate change appeared more frequently that other types of risk, with the wars in Ukraine and Palestine continuing to focus attention (Figure 9). Climate change risks were often flagged by GERM when companies share emissions data under the Streamlined Energy and Carbon Reporting guidelines. Some categories of risk were not detected in any reports throughout the month including privatisation, nationalisation and space risks. Some risk types including modern slavery and corruption deterioration returned similar boilerplate compliance statements and were therefore omitted as they provided minimal information.

Trends in risk reporting across each category of risk could be summarised as follows:

Risk Type	Risk Trends	Impact Trends
Change in Government	UK & US upcoming elections in 2024	Economic uncertainty; Changes in government funding for specific sectors; Reduced consumer confidence
Climate Change	GHG emissions (Scope 1 & 2); High energy consumption; SECR compliance	Financial commitments to Net Zero; Increased costs of optimising processes, sustainable materials and carbon reducing technologies; Challenges in meeting the changing preferences of carbon conscious consumers
Criminal Activity	Young people getting involved in crime and anti-social behaviour; Fraud	Implementation of new policy; Repayment of illegal dividends
Emerging Regulations	Brexit	Increased costs and financial pressure; Increasing trade friction and complexity; Reduced business with the EU; Lobbying efforts advocating for legislative change
Environmental Degradation	Challenges in sourcing of sustainable materials; Use of landfills for waste management; Risk of dangerous leakage into the environment	Increased costs due to recycling and sustainable materials; Reputational risk
Extreme Weather	Flooding; Heavy rain; Drought; Extreme heat	Crop damage; Delayed planting; Increased importation; Investment in new technologies; Damage to infrastructure
Food Security	Avian influenza	Operations impacted; Loss of food and lifestock; Rising operational costs
Industrial Action	Entertainment industry strikes; UK rail strikes	Challenging business conditions; Challenges in travelling by rail; Uncertainty regarding planned events
Interstate Conflict	Wars in Ukraine and the Middle East	Inflation; Energy price rises; Supply chain disruptions; Market volatility; Raw material costs increased
Logistics Restrictions	Global conflicts; COVID-19 government restrictions; Suez Canal; Houthi strikes in the Red Sea	Increased shipping and transportation costs; Extended lead times; Redesign of supply chains
Minimum Wage Hike	10% Increase to the National Minimum Wage; Commitment to Real Living Wage; Wage inflation	Significant rise in payroll and labour costs; Decrease in profit; Pressure on employee retention and recruitment efforts
Natural Resource Deficiency	Volatile raw material prices (fossil fuels and metals); Energy crisis; Brexit and global conflict impacting raw material availability and prices; Increased costs of sustainable materials	Increased or unpredictable production costs; Investment in renewables (solar) and recycling
Sanctions	EU, US and UK sanctions against Russia and Belarus; Potential future sanctions against China; Ban on Western airlines flying within Russian airspace	Challenges in accessing funds and customers; Revenue loss from the Russian market; Potential difficulties for companies to continue operating; Future uncertainty around inclusion on sanctions list
Social Unrest	London protests; Instability in Ethiopia and Syria	Projects and travel abroad delayed or suspended
Subnational Conflict	Civil war in Sudan	Negative impact on Exotic Debt Fund; Negative impact on existing projects and company performance
Talent Availability	Brexit-induced labour challenges; Skill shortages in construction, nursing and technology	Wage inflation and increased labour costs; Opening of Tier 2 visa sponsorship schemes to attract talent from the EU and elsewhere; Investment in recruitment and in-house training; Continued support for apprenticeship schemes
Terrorism	Increasing alertness towards terrorist threats in public spaces such as sports stadiums	Financial, care and social support required for victims of terrorism; Potential for a reduction in international travel following terrorist attacks

A closer reading of GERM’s dataset reveals some eye-catching risks worth digging into further:

Geopolitical

Environmental

GERM also flagged instances of risk that could work out to be advantageous for certain companies. Kabina own the patent for an amphibious flood-adaptive home and write in their annual report that “the UK’s relentless and exponential increase in population, combined with increased areas of flood land, bode well” for their business (Figure 12). Mercian Limited, the UK’s largest supplier of crisping potatoes, recorded their best financial year yet as a consequence of drought and the war in Ukraine leading to a hike in the price of potatoes.

AI-Augmented Risk Research

As we continue to experiment with GERM within our broader practice of prototyping AI-augmented research tools, there are several language model specific research directions on the horizon that we look forward to developing.

Companies House for Interactive Planning Simulations

Most companies don’t write about risk. Most risk management professionals will have considered using ChatGPT to fill in the blanks of a risk report. What remains less explored is how far we could push the obvious potential for LLMs to generate risk profiles for companies. Annual reports could be processed to infer the level of exposure each company has to every single category of risk within the Cambridge Taxonomy (even the most Emmerichian risks). Coupled with external data from news, reports, social media and fiction, LLMs could generate plausible narrative-based scenarios detailing how companies might respond to each flavour of crisis. These narratives could form their own labyrinthian genre of grey-literature; a house of generative risk narratives for each company in Companies House. Elements of this data might serve as input to an agent-based-model for simulating and studying the emergent competition and cooperation within fragments of the economy under stress. Could Companies House serve as the partial backend database to simulations for incubating and archiving novel plans and strategies?

1. Crawl

We begin by providing ‘Pubcrawler’ with the url of the organisation we want to investigate. For testing purposes we decided to embark on the meta project of investigating our own organisation and so https://autonomy.work/ is the input. Using Selenium the tool will recursively crawl through every subpage that eventually links back to the Autonomy landing page, whilst saving the location of each PDF publication it spots along the way:

Crawling through the nearly 1000 subpages on the Autonomy site completed in less than an hour:

2. Publications

After navigating through the entire site, 190 PDF files were found. Before opening the files we assume that most of these PDFs were written by Autonomy whilst some will come from other organisations. The next stage requires downloading as many of the found files as possible using Selenium. Due to the presence of some links that no longer exist, only 170 publications could be successfully downloaded.

3. Metadata

With all the publications in one place we can begin the task of extracting useful metadata that will allow various characteristics of the organisation to be visualised. It is worth noting that PDF files often contain some metadata although it is often sparse and low quality. Here’s an example of some of the metadata we can grab from the Autonomy publications with PyPDF2:

To extract higher quality metadata from the actual content within a PDF we require an approach that is flexible towards the vast differences in presentation format and structure across documents.

Thankfully language models like GPT-3.5 and GPT-4 demonstrate strong capabilities in entity recognition across a broad range of text formats. It is even possible to pass an entire PDF to GPT-4 within a single API call due to recent extensions in the model’s context window, however this becomes costly when processing multiple documents (potentially several $ per document), incentivising us to minimise the number of tokens transmitted via API.

To demonstrate the kinds of metadata that could be extracted reliably from PDF text, we attempt to get some basic bibliographic information on each document, including authors, dates and institutional affiliations. As a general rule, much of this information is likely to exist in the first few pages of each document, therefore passing the first 5 pages of each PDF to GPT-3.5/GPT-4 should allow us to save costs. Using the following function call, we instruct the model to return the desired metadata in a structured format:

{'name': 'generate_citation',
 'description': 'Generate a citation for report',
 'parameters': {'type': 'object',
  'properties': {'title': {'type': 'string',
    'description': 'Report title. If unknown leave empty'},
   'authors': {'type': 'array',
    'description': "Array of the author's full names. If unknown leave empty",
    'items': {'description': "Author's full name", 'type': 'string'}},
   'organisation': {'type': 'array',
    'description': 'Array containing the names of the research organisations that produced the report. If unknown leave empty',
    'items': {'description': "Research organisation's name",
     'type': 'string'}},
   'date': {'type': 'string',
    'description': 'Date the report was published on. Try to find the day, month and year. If unknown leave empty'},
   'keywords': {'type': 'array',
    'description': 'Array of keywords that indicate the content of the report. If unknown leave empty',
    'items': {'description': "Keyword title ie 'feminism'", 'type': 'string'}},
   'funders': {'type': 'array',
    'description': 'Array of organisations that provided funding or financial support. If unknown leave empty',
    'items': {'description': 'Name of funding organisation',
     'type': 'string'}}},
  'required': ['title',
   'authors',
   'organisation',
   'date',
   'keywords',
   'funders']}}

By passing the first 5 pages from the following report to GPT-4 with the above function call, we get the proceeding structured output in return:

{'title': 'The Shorter Working Week: A Radical And Pragmatic Proposal',
 'authors': ['Will Stronge',
  'Aidan Harper',
  'Danielle Guizzo',
  'Kyle Lewis',
  'Madeleine Ellis-Petersen',
  'Nic Murray',
  'Helen Hester',
  'Matt Cole'],
 'organisation': ['Autonomy', 'Autonomy Research Ltd', '4 Day Week Campaign'],
 'date': '2019',
 'keywords': ['Shorter Working Week',
  'Current Model of Work',
  'Future Model of Work-Time'],
 'funders': []}

When applied across all of Autonomy’s documents we observe that both GPT-3.5 and GPT-4 are able to extract entities with minimal errors in spelling/formatting, lifting most entities correctly from the test. The main sources of inaccuracy stem from false positive and false negative classifications of persons and organisations as authors or funders. GPT-3.5 seemed to be much less capable in recognising funders than GPT-4. However whilst GPT-4 demonstrates superior classification quality, GPT-4 cost $9.79 to classify 850 pages of text whilst GPT-3.5 cost $0.41.

4. Visualisation

With GPT-4 we have generated structured metadata for each downloaded PDF that can be plugged into data visualisations. GPT-4 was prompted to return a list of authors for each publication and so we can easily visualise the most common occurances. Its unsurprising to find Autonomy’s founders and researchers make up the top 6 most cited authors:

The same approach can be followed to visualise Autonomy’s most frequent funders and institutional collaborators (usually in the form of co-authors). GPT-4 has been broadly successful in differentiating between these two fuzzy classes of organisation. This metadata could be prove useful in connecting research organisations together and sketching how wider political projects are funded:

The presence of keywords within text can indicate which policy areas are of greatest interest to an organisation. Whilst the following results are quite general and unsurprising in Autonomy’s case given that the organisation is focussed on the future of work, it can be useful to compare the most common keywords with the stated aims of an organisation for misalignment:

Temporal data can help us chart the peaks and troughs in research output over time:

Where the month of publication is mentioned we can generate a more granular map:

It is also possible to explore how collaborative an organisation is through investigating instances of co-authorship. The following plot shows which authors found in Autonomy’s reports have collaborated together, represented by an edge connecting them. Everytime two authors are found in the same citation we count this as a single collaboration. Edge weight represents the number of times the authors have collaborated:

5. Conclusion

In this brief post we’ve demonstrated the automated search and retrieval of an organisation’s publication archive from ~1000 webpages in less than hour. LLMs can process these large collections of documents producing rich metadata to power useful visualisations. We continue to find the possibilities of combining web scraping and LLMs exciting as they allow us to build highly versatile information retrieval systems that easily overcome the nuances of heterogeneous web design and file formats. We look forward to testing ‘Pubcrawling’ on a much wider set of organisations and expanding the kinds of metadata we retrieve.

Intro

The process of discovering datasets through manual Google searches and individual downloads presents a significant challenge. This approach often results in datasets being sourced individually, leading to difficulties regarding cross-comparison of data . As a result, the endeavour of finding relevant datasets and conducting meaningful comparisons becomes both arduous and time-consuming. Of course, even through such searches, there remains the possibility that some datasets simply haven’t been unearthed and thus remain unknown to the researcher.

To address this issue, I’ve been automating the gathering of datasets from key sources via APIs and web scraping, and creating a single metadata dataset - what we’ve been calling the ‘database of databases’. This dataset becomes the core of a sort of recommendation system: within this system, finding datasets with relevant content or resembling ones already in the metadata collection becomes much easier. This approach is all about making dataset discovery faster and more effective, using clear visualisations and organised management.

This blog walks through the process of achieving this.

Data Collection

Autonomy is a research organisation that focuses on the future of work and economic planning, and therefore the vast majority of the datasets come from the ONS (Office for National Statistics). The ONS is the UK’s largest independent producer of official statistics with data relating to the economy, population, and society at various levels. This data is collected either through their API or the Nomis API. As well as the ONS, the Monthly Insolvency Statistics, NHS Quality and Outcomes Framework, and police data are also included. These datasets provide insights into bankruptcy trends, healthcare data used to assess the NHS services, and crime statistics.

The metadata being collected for each dataset includes the title, a short description, a long description, the column titles, unique non-numeric column values / parameters, and the release date / date the dataset was last updated. This combination of data should provide a solid overview of the dataset and enough information to make decisions on their similarity with other datasets.

Comparing Datasets

Utilising the gathered metadata, several methods emerge for dataset comparison. The first and most straightforward approach involves column matching. When two or more columns align, datasets can be merged, creating one larger combined dataset. However, a challenge arises due to the variability in column names, including differences in terminology, abbreviations, and spelling. This can result in missing out on potentially valuable column matches. To address this, we not only consider exact matches but also explore similar column names using fuzzy string matching, which helps us find comparable columns.

However, even with these methods, some situations remain unaccounted for. Some datasets might share similarities or relevant information without having any common columns. For such cases, a more inventive approach comes into play. We can use embeddings to compare dataset descriptions, in-depth explanations, and titles. This method enables us to uncover connections that might not be immediately apparent.

Network Diagrams

Using the techniques above, I am able to compile datasets that provide insights into the similarity of datasets through their metadata. These can then contribute significantly in the exploration and discovery of available datasets, helping to uncover more data on particular topics and bring to light interesting connections that may never have been observed otherwise. While this could be done by simply creating a search function to query the datasets, the integration of network diagrams introduces additional functionality allowing a user to interact with the datasets in new ways.

The network diagrams were created using D3 in JavaScript as it offers much more customizability than python which I used for the data collection and preparation. It also works with web standards and therefore can be integrated within a website easily.

1. Token Classification

Skills extraction can be approached as a named entity recognition task where models are trained to label sequences of words that correspond to skills. This approach does require quality labelled datasets that are usually time-consuming to build and difficult to source. The highest quality open-source dataset we could find is the ‘Skillspan’ dataset from Zhang et al. (2022) consisting of 265 annotated English language job profiles (14.5K sentences and over 12.5K annotated spans) for hard and soft skills. This excellent resource is released alongside a codebase and paper ‘SkillSpan: Hard and Soft Skill Extraction from Job Postings’ in which the researchers benchmark several transformer models fine-tuned on SkillSpan. Furthermore, the paper includes the extensive annotation guidelines used by the researchers to label the SkillSpan dataset (Appendix B. of SkillSpan paper). The guidelines are thorough and indicate the dataset took significant time and care to produce.

The data in SkillSpan is seperated into sentences labelled in the CoNNL format which looks like this:

Words are labelled ‘O’ if they are not part of a skill. The first word in a skill sequence is labelled B-Skill (b for beginning) and all proceeding words labelled I-Skill.

{
'tokens': ['You', 'will', 'do', 'this', 'by', 'setting', 'up', 'partnerships', 'to, 'improve', 'supply', 'building', 'partnerships', 'to', 'acquire', 'new', 'customers', 'and', 'set', 'up', 'global', 'partnerships', 'with' 'leading', 'wineries'],
'ner_tags': [0, 0, 0, 0, 0, 1, 2, 2, 0, 1, 2, 1, 2, 0, 1, 2, 2, 0, 1, 2, 2, 2, 0, 0, 0]
}

{ "messages":
    [{'role': 'system', 'content': 'You are a helpful skills extractor. You are to extract skills from each of the sentences provided.'},
    {'role': 'user', 'content': 'You will do this by setting up partnerships to improve supply building partnerships to acquire new customers and set up global partnerships with leading wineries.'},
    {'role': 'assistant', 'content': '["setting up partnerships", "improve supply building partnerships", "acquire new customers", "set up global partnerships"]'}]
}

2. Zero-shot skills extraction with GPT-4

The majority of ChatGPT’s users implement prompting strategies that include a limited number of examples (few-shot) or none at all (zero-shot). Rather than fine-tuning an LLM on a curated set of examples as we have demonstrated, they are instead leveraging the model’s vast knowledge aquired through pretraining, which obviously includes an extensive understanding of employment skills.

Interacting with GPT-4 programmatically opens up some creative possibilities for skills extraction. The model’s context-window is continuously being expanded such that we can now pass relatively large passages of text (~100,000 words) within a single API call. Rather than extracting skills from another job ad, lets examine a much larger document in the form of a video transcript (generated by Youtube’s speech-to-text AI) from a controversial 2022 panel at Southern California Institute of Architecture (SCI ARC). The discussion consists of several faculty members sharing their experiences and insights with an audience of students, unintentionally spotlighting some of the toxic beliefs prevalent within industry.

The video transcript includes ~14,000 words and ~1500 lines text, littered with transcription errors and no reference to who is speaking when. A key strength of GPT-4 compared to other LLMs is the model’s flexibility in processing raw data without being overly sensitive to inconsistencies or unusual formatting :

We can easily pass the entire transcript to gpt-4-1106 in a single prompt and steer the model to return structured data using OpenAI’s function calling capabilities. Function calling requires us to define the schema for a function we would like to pass data from our chat with GPT-4 to. By sharing this function schema with GPT-4, if the model “feels” it should “call” the function based on the content of the conversation, it will return the function’s arguments in whatever format we specified in the schema (array, boolean etc.) rather than its defacto unstructured text response. The name ‘function calling’ is perhaps slightly misleading for two reasons: firstly we aren’t sharing a function with the LLM but instead the function’s schema and secondly rather than a rogue LLM running some code because it feels like it, the model is just presenting us with correctly formatted arguments for our own functions that we can run if we choose.

For this use-case, we define the following schema for a function called list_skills that we can use to get a Python list of skills from GPT-4 rather than some other list (bullet points, numbers) sitting within a body of text:

tools = [{
            "type": "function",
            "function": {
            "name": "list_skills",
            "description": "Print an array of skills.",
            "parameters": {
                "type": "object",
                "properties": {
                    "skills_list": {
                        "type": "array",
                        "description": "array of strings corresponding to each skill",
                        "items": {"type": "string"}
                        },
                    },
                },
                "required": ["skills_list"]
            }
}]

We can instruct GPT-4 to ‘use’ the function we defined:

messages = []
messages.append({"role": "system", "content": "You are a helpful skills extractor able to identify skills mentioned in text. Provide as much detail and specificity for each skill as possible and avoid being to generic or vague."})
messages.append({"role": "user", "content": f"Print a list of the unique skills mentioned in the following video transcript using 'list_skills': {transcript}"})

skills = openai.chat.completions.create(**{"model": 'gpt-4-1106-preview',"messages": messages, "tools": tools})

GPT-4 returns the arguments list_skills as an array as we hoped:

ChatCompletion(id=‘chatcmpl-8KlPRDI1a6avNgmN6gufjSfnLmewp’, choices=[Choice(finish_reason=‘tool_calls’, index=0, message=ChatCompletionMessage(content=None, role=‘assistant’, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id=‘call_xN0YXqz02nIeXYBmzAc9WdFH’, function=Function(arguments=‘{“skills”:[“Cultural knowledge of architectural education and professional transitions”,“Understanding of evolving expectations in architectural practice”,“In-depth experience in hiring and firing processes within an architecture firm”,“Awareness of the differences between various architectural office cultures”,“Ability to compare design and corporate office environments”,“Strategic thinking towards one's architectural education and career trajectory”,“Familiarity with the concept of an architectural firm as an extension of education”,“Comprehension of the relationship between portfolio, education, and pedigree in architecture”,“Teaching experience in architecture, specifically history and theory”,“Awareness of the impact of personal attitude and optimism on career progression”,“Understanding of principles like mutual investment in career building”,“Knowledge of the importance of reputation and relationships in architecture”,“Ability to assess and respond to job expectations and office demands”,“Understanding of the impacts of architectural decisions on real projects”,“Knowledge of remuneration scales in corporate vs. boutique architecture firms”,“Ability to balance work, study, and personal commitments”,“Recognition of the importance of side hustles for financial independence”,“Insight into financial considerations when pursuing graduate architectural education”,“Acknowledgment of the challenges international architecture students face”,“Understanding of career flexibility with an architectural education”,“Awareness of efficient working habits and productivity”,“Understanding of problem-solving and proactive strategies within a firm”,“Insight into work-life balance and wages in different architectural firm sizes”,“Knowledge of personal financial planning and investment for architects”]}’, name=‘list_skills’), type=‘function’)]))], created=1699959493, model=‘gpt-4-1106-preview’, object=‘chat.completion’, system_fingerprint=‘fp_a24b4d720c’, usage=CompletionUsage(completion_tokens=287, prompt_tokens=18416, total_tokens=18703))

We can now easily parse the list from the response with ast:

ast.literal_eval(skills.choices[0].message.tool_calls[0].function.arguments)['skills_list']

['Understanding and practicing effective office culture',
 'Recognizing the importance of ongoing learning and professional development',
 'Historical and theoretical knowledge in architecture',
 'Entrepreneurial skills for running an architecture firm',
 'Awareness of architectural firm structures (boutique vs. corporate)',
 'Team management and delegation',
 'Hiring and firing within a firm',
 'Prioritizing efficiency and accuracy in architectural work',
 'Contract negotiation and client communication',
 'Understanding of construction and building processes',
 'Implementing time management and project management strategies',
 'Ability to adapt and be resilient in the face of challenges',
 'Developing skills for schematic design',
 'Detailing and drafting proficiency',
 'Understanding architectural contracts and specs',
 'Client relations and business development',
 'Networking within the architecture industry',
 'Professional licensure and certification',
 'Academic research and teaching in architecture',
 'Financial planning and management',
 'Investment strategies']

Its interesting to see what skills have been extracted from such a long document although they’re a bit generic and perhaps not as toxic as we might have expected. Unfortunately with this approach we are left unclear as to where in the document each skill resides. The number of skills identified also seems kindof abritrary.

To alleviate these shortcomings we could revert to passing chunks of indexed text to the model so we know where any extracted skills come from. Secondly we can introduce an initial filtering step to avoid leading the model to predict skills in sentences where there aren’t any. The described prompt chain might look like this:

We can implement this prompt chain by adding another function to tools that will check if each sentence contains skills returning either true or false:

tools.append({
            "type": "function",
            "function": {
            "name": "skills_check",
            "description": "Report if text contains mention of skills.",
            "parameters": {
                "type": "object",
                "properties": {
                    "skills_check": {
                        "type": "boolean",
                        "description": "True or False",
                    },
                },
                "required": ["skills_check"],
            },
    },
})

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 150/150 [00:50<00:00,  2.96it/s]

The above prompt chain generates the following much smaller set of skills that appear to be lifted directly from the text rather than paraphrased. However these skills perhaps more accurately track the tone of the discussion:

References

[1] Alpaca: A Strong Open-Source Instruction-Following Model. Link: https://crfm.stanford.edu/2023/03/13/alpaca.html

Locating Editions

Deliveroo Editions launched in May 2017 as a ‘concept’ to end ‘postcode food envy’. The first kitchens were set-up in Camberwell, Battersea, Dulwich and Canary Wharf with the intention to scale to 30 locations across the UK within the first year. The Internet Archive’s Wayback Machine maintains a capture of the webpage announcing the launch.

As of July 2023 it was possible to identify nearby Editions restaurants from the Deliveroo website by appending tags=deliveroo+editions to any delivery search url. For example to find all the Editions partners delivering to Stepney Green, we could try: https://deliveroo.co.uk/restaurants/london/stepney-green?fulfillment_method=DELIVERY&tags=deliveroo+editions:

At the time of publication (October 2023) this filter tag still exists on Deliveroo’s website, however it no longer filters Editions restaurants. An alternative filtering method would need to be implemented to replicate our approach to data collection, perhaps by identifying the restaurants with ‘editions’ included in their href attributes:

If we navigate to an Editions partner webpage (for example, Popeyes Louisiana Kitchen), we can tell the restaurant is based at “Whitechapel Editions” because the location name is contained within the restaurant’s url: https://deliveroo.co.uk/menu/London/whitechapel-editions/popeyes-editions-wme/?day=today&geohash=gcpvngtckn3c&time=ASAP. We therefore assume the address for Popeyes Louisiana Kitchen is also the address for Whitechapel Editions which happens to be 10 Assembly Passage, a 2-storey B-class office space (with disco ball included) built in 1906, next to the busy A11 road into central London.

To find all the other partners based at Whitechapel Editions (london/whitechapel-editions), we can search for Editions partners (tags=deliveroo+editions) nearby with the following url: https://deliveroo.co.uk/restaurants/london/whitechapel-editions?fulfillment_method=DELIVERY&tags=deliveroo+editions. We infer all restaurants located 0.0 miles away are based out of the Editions facility and can validate this by checking for whitechapel-editions in their urls.

Using this approach, we searched the Internet Archive for all historic data on Deliveroo Editions to see if we could chart their growth from 2017.

Historic Editions Data

Deliveroo’s website has been archived many times by the Wayback Machine. Some of the captured webpages include search results specific to UK locations with Deliveroo Editions filters. Only 46 webpages with Editions search results could be retrieved, each of which was captured between 1 and 5 times over a period of 2 years. Unfortunately this data is likely too sparse to allow for detailed analysis of the growth of Editions in the UK. However the data has some use in providing an incomplete historic list of Editions partners between 2020 and 2022.

By iterating through every capture extracting the restaurant names, urls and timestamps, we can build a dataset of ~260 Editions partners:

Whilst it may not be possible to map the growth of Deliveroo’s dark kitchen footprint with great detail, we can use more recent data to build a snapshot in time of Deliveroo Editions in the UK.

Present-Day Editions Data

Many of Deliveroo’s well known Editions locations can be found by searching the Food Standards Agency (FSA) for businesses containing “Deliveroo” in their title. Deliveroo requires its restaurants to have an FSA rating of at least 2 to use the platform. Editions are advertised as going above and beyond these minimal requirements, maintaining the highest possible hygiene ratings across all sites. This is a far cry from the poor hygiene practices and lack of regulation associated with the original ‘ghost kitchens’ of New York that advertised through Grubhub and attracted negative media attention in 2015.

We can cross reference 16 of these businesses with Editions locations encountered from searching Deliveroo’s site:

The above list captures some of Deliveroo’s most well-known and established dark kitchens, although to get a complete picture we turn to Deliveroo’s website to identify all of the restuarants tagged as operating from an Editions site.

Mapping Deliveroo Editions

To get all of the Editions partners in the UK at any one time we can search Deliveroo’s website with a postcode from each of the ~2900 postcode districts. This brute force search strategy is perhaps inefficient given that Deliveroo doesn’t operate within most UK postcodes outside of cities and large towns, although it does likely offer a near-complete picture of Deliveroo’s operations in the UK.

By searching all of the Deliveroo restaurants delivering to postcodes within each of the UK’s ~2900 postcode districts in July 2023, 238 editions partners were identified across 53 addresses and 12 regions:

Wingstop (2021 Deliveroo Restaurant of the Year winner), Dishoom (2021 Deliveroo Best Indian Restaurant winner) and the Athenian appear to be the most popular partners appearing in 6+ Editions across the UK. Editions sites also include partners that don’t offer cooked food like the Californian CBD drinks vendors ‘Trip CBD Store’ and Ben & Jerry’s.

London is the region containing the most dark kitchens with 32 Editions sites compared to the 21 locations spread across all other regions of the UK:

London also offers the greatest choice of Editions partners at 127 compared to the 112 vendors spread across the other Editions sites.

The Editions site with the most partners is in Leeds with 22 partners under one roof at 1 Scott Hall Street: