Thursday, November 10, 2016

Steering The Enterprise of Brexit

Two contrasting approaches to Brexit from architectural thought leaders.

Dan Onions offers an eleven-step decision plan based on his DASH method, showing the interrelated decisions to be taken on Brexit as a DASH output map.

A decision plan for Brexit (Dan Onions)

A stakeholder map for Brexit (Dan Onions)

Let me now contrast Dan's approach with Simon Wardley's. Simon had been making a general point about strategy and execution on Twitter.
Knowing Simon's views on Brexit, I asked whether he would apply the same principle to the UK Government's project to exit the European Union.

Simon's diagram revolves around purpose. OODA is a single loop, and the purpose is typically unproblematic. This reflects the UK government's perspective on Brexit, in which the purpose is assumed to be simply realising the Will of the People. The Prime Minister regards all interpretation, choice, decision and direction as falling under her control as leader. And according to the Prime Minister's doctrine, attempts by other stakeholders (such as Parliament or the Judiciary) to exert any governance over the process is tantamount to frustrating the Will of the People.

Whereas Dan's notion is explicitly pluralist - trying to negotiate something acceptable to a broad range of stakeholders with different concerns. He characterizes the challenge as complex and nebulous. Even this characterization would be regarded as subversive by orthodox Brexiteers. It is depressing to compare Dan's careful planning with Government insouciance.

Elsewhere, Simon has acknowledged that "acting upon your strategic choices (the why of movement) can also ultimately change your goal (the why of purpose)". Many years ago, I wrote something on what I called Third-Order Requirements Engineering, which suggested that changing the requirements goal led to a change in identity - if your beliefs and desires have changed, then in a sense you also have changed. This is a subtlety that is lost on most conventional stakeholder management approaches. It will be fascinating to see how the Brexit constituency (or for that matter the Trump constituency) evolves over time, especially as they discover the truth of George Bernard Shaw's remark.
"There are two tragedies in life. One is to lose your heart's desire. The other is to gain it."

Dan Onions, An 11 step Decision Plan for Brexit (6 November 2016)

Richard Veryard, Third Order Requirements Engineering (SlideShare)

Based on R.A. Veryard and J.E. Dobson, 'Third Order Requirements Engineering: Vision and Identity', in Proceedings of REFSQ 95, Second International Workshop on Requirements Engineering, (Jyvaskyla, Finland: June 12-13, 1995)

Simon Wardley, On Being Lost (August 2016)

Related Posts: VPEC-T and Pluralism (June 2010)

Tuesday, November 01, 2016

Uber Mathematics

UK Court News. Uber has lost a test case in the UK courts, in which it argued that its drivers were self-employed and therefore not entitled to the minimum wage or any benefits. Why is this ruling not quite as straightforward as it seems? To answer this question, we have to look at the mathematics of two-sided or multi-sided platforms.

Platforms exist in two states - growth and steady-state. A mature steady-state platform maintains a stable and sustainable balance between supply and demand. But to create a platform, you have to build both supply and demand at the same time. Innovative platforms such as Uber are oriented towards expansion and growth - recruiting new passengers and new drivers, and launching in new cities.

New Passengers "Every week in London, 30,000 people download Uber to their phones and order a car for the first time. The technology company, which is worth $60bn, calls this moment “conversion”. It sets great store on the first time you use its service ... With Uber, the feeling should be of plenty, and of assurance: there will always be a driver when you need one." (Knight)
New Drivers "They make it sound so simple: Sign up to drive with Uber and soon you’ll be earning an excellent supplementary income! That’s the central message in Uber’s ongoing multi-platform marketing campaign to recruit new drivers." (McDermott)
New Cities "Uber has deployed its ride-hailing platform in 400 cities around the world since its launch in San Francisco on 31 May 2010, which means that it enters a new market every five days and eight hours. ... To take over a city, Uber flies in a small team, known as “launchers” and hires its first local employee, whose job it is to find drivers and recruit riders." (Knight)

But here's the problem. In order to encourage passengers to rely on the service, Uber needs a surfeit of drivers. If passengers want instant availability of drivers (plenty, assurance, there will always be a driver when you need one), then Uber has to maintain a pool of under-utilized drivers. (Knowles)

Simple mathematics tells us that if Uber takes on far more drivers than it really needs, some of them won't earn very much. Furthermore, people with little experience of this kind of work may underestimate the true costs involved, and may have an unrealistic idea of the amounts they can earn: Uber has no obvious incentive to disillusion them. (This is an example of Asymmetric Information.) Even if the average earnings of Uber drivers are well above the minimum wage, as Uber claims, it is not the average that matters here but the distribution.

The myth is that these are drivers who can choose whether to provide a service or not, so they are free agents. Libertarians wax lyrical about the "gig economy" and the benefits to passengers. However, the UK courts have judged that Uber drivers work under a series of constraints, and are therefore to be classified as "workers" for the purposes of various regulations, including minimum wage and other benefits.

Uber has announced its intention to appeal the UK judgement. But if the judgement stands, what are the implications for Uber? Firstly, Uber's overall costs are likely to increase, and Uber will undoubtedly find a way either to pass these costs onto the passengers or to pass them back to the drivers in some other form. But more interestingly, Uber now has a financial incentive to balance supply and demand more fairly, and to avoid taking on too many drivers.

Uber sometimes argues it is merely a technology company, and is not in the transportation business. Dismissing this argument, the UK courts quoted a previous judgement from the North California District Court:
"Uber does not simply sell software; it sells rides. Uber is no more a 'technology company' than Yellow Cab is a 'technology company' because it uses CB radios to dispatch taxi cabs."
However, Uber's undoubted technological know-how should enable it to develop (and monetize) appropriate technologies and algorithms to manage a two-sided platform in a more balanced way.

Update: similar concerns have been raised about Amazon delivery drivers. I have previously praised Amazon on this blog for its pioneering understanding of platforms, so let's hope that both Amazon and Uber can create platforms that are fair to drivers as well as its customers.

Mr Y Aslam, Mr J Farrar and Others -V- Uber (Courts and Tribunals Judiciary, 28 October 2016)

Tom Espiner and Daniel Thomas, What does Uber employment ruling mean? (BBC News, 28 October 2016)

David S. Evans, The Antitrust Economics of Multi-Sided Platform Markets (Yale Journal on Regulation, Vol 20 Issue 2, 2003). Multisided Platforms, Dynamic Competition and the Assessment of Market Power for Internet-Based Firms (CPI Antitrust Chronicle, May 2016)

Sam Knight, How Uber Conquered London (Guardian, 27 April 2016)

Kitty Knowles, 10 of the biggest complaints about Uber – from Uber drivers (The Memo, 5 November 2015)

Barry Levine, Uber opens up its API – and creates a new platform (VentureBeat, 20 August 2014)

John McDermott, I've done the (real) math: No way an Uber driver makes minimum wage (We Are Mel, 17 May 2016)

Hilary Osborne, Uber loses right to classify UK drivers as self-employed (Guardian, 28 October 2016)

Aaron Smith, Gig Work, Online Selling and Home Sharing (Pew Research Center, 17 November 2016)

Ciro Spedaliere, How to start a multi-sided platform (30 June 2015)

Amazon drivers 'work illegal hours' (BBC News, 11 November 2016)

See further discussion with @wimrampen and others on Storify: Uber Mathematics - A Discussion

Updated 23 November 2016

Wednesday, October 26, 2016

The Shelf-Life of Algorithms

@mrkwpalmer (TIBCO) invites us to take what he calls a Hyper-Darwinian approach to analytics. He observes that "many algorithms, once discovered, have a remarkably short shelf-life" and argues that one must be as good at "killing off weak or vanquished algorithms" as creating new ones.

As I've pointed out elsewhere (Arguments from Nature, December 2010), the non-survival of the unfit (as implied by his phrase) is not logically equivalent to the survival of the fittest, and Darwinian analogies always need to be taken with a pinch of salt. However, Mark raises an important point about the limitations of algorithms, and the need for constant review and adaptation, to maintain what he calls algorithmic efficacy.

His examples fall into three types. Firstly there are algorithms designed to anticipate and outwit human and social processes, from financial trading to fraud. Clearly these need to be constantly modified, otherwise the humans will learn to outwit the algorithms. And secondly there are algorithms designed to compete with other algorithms. In both cases, these algorithms need to keep ahead of the competition and to avoid themselves becoming predictable. Following an evolutionary analogy, the mutual adaptation of fraud and anti-fraud tactics resembles the co-evolution of predator and prey.

Mark also mentions a third type of algorithm, where the element of competition and the need for constant change is less obvious. His main example of this type is in the area of predictive maintenance, where the algorithm is trying to predict the behaviour of devices and networks that may fail in surprising and often inconvenient ways. It is a common human tendency to imagine that these devices are inhabited by demons -- as if a printer or photocopier deliberately jams or runs out of toner because it somehow knows when one is in a real hurry -- but most of us don't take this idea too seriously.

Where does surprise come from? Bateson suggests that it comes from an interaction between two contrary variables: probability and stability --
"There would be no surprises in a universe governed either by probability alone or by stability alone."
--  and points out that because adaptations in Nature are always based on a finite range of circumstances (data points), Nature can always present new circumstances (data) which undermine these adaptations. He calls this the caprice of Nature.
"This is, in a sense, most unfair. ... But in another sense, or looked at in a wider perspective, this unfairness is the recurrent condition for evolutionary creativity."

The problem with adaptation being based solely on past experience also arises with machine learning, which generally uses a large but finite dataset to perform inductive reasoning, in a way that is non-transparent to the human. This probably works okay for preventative maintenance on relatively simple and isolated devices, but as devices and their interconnections get more complex, we shouldn't be too surprised if algorithms, whether based on human mathematics or machine learning, sometimes get caught out by the caprice of Nature. Or by so-called Black Swans.

This potential unreliability is particularly problematic in two cases. Firstly, when the algorithms are used to make critical decisions affecting human lives - as in justice or recruitment systems. (See for example, Zeynap Tufekci's recent TED talk.) And secondly, when preventative maintenance has safety implications - from aeroengineering to medical implants.

One way of mitigating this risk might be to maintain multiple algorithms, developed by different teams using different datasets, in order to detect additional weak signals and generate "second opinions". And get human experts to look at the cases where the algorithms strongly disagree.

This would suggest that we maybe shouldn't be too hasty to kill off algorithms with poor efficacy, but sometimes keep them in the interests of algorithmic biodiversity.  (There - now I'm using the evolutionary metaphor.)

Gregory Bateson, "The New Conceptual Frames for Behavioural Research". Proceedings of the Sixth Annual Psychiatric Institute (Princeton NJ: New Jersey Neuro-Psychiatric Institute, September 17, 1958). Reprinted in G. Bateson, A Sacred Unity: Further Steps to an Ecology of Mind (edited R.E. Donaldson, New York: Harper Collins, 1991) pp 93-110

Mark Palmer, The emerging Darwinian approach to analytics and augmented intelligence (TechCrunch, 4 September 2016)

Zeynap Tufekci, Machine intelligence makes human morals more important (TED Talks, Filmed June 2016)

Related Posts
The Transparency of Algorithms (October 2016)

Tuesday, October 25, 2016

85 Million Faces

It should be pretty obvious why Microsoft wants 85 million faces. According to its privacy policy
Microsoft uses the data we collect to provide you the products we offer, which includes using data to improve and personalize your experiences. We also may use the data to communicate with you, for example, informing you about your account, security updates and product information. And we use data to help show more relevant ads, whether in our own products like MSN and Bing, or in products offered by third parties. (retrieved 25 October 2016)
Facial recognition software is big business, and high quality image data is clearly a valuable asset.

But why would 85 million people go along with this? I guess they thought they were just playing a game, and didn't think of it in terms of donating their personal data to Microsoft. The bait was to persuade people to find out how old the software thought they were.

The Daily Mail persuaded a number of female celebrities to test the software, and printed the results in today's paper.

Talking of beards ...

Kyle Chayka, Face-recognition software: Is this the end of anonymity for all of us? (Independent, 23 April 2014)

Chris Frey, Revealed: how facial recognition has invaded shops – and your privacy (Guardian, 3 March 2016)

Rebecca Ley, Would YOU  dare ask a computer how old you look? Eight brave women try out the terrifyingly simple new internet craze (Daily Mail, 25 October 2016)

Friday, September 02, 2016

Single Point of Failure (Comms)

Large business-critical systems can be brought down by power failure. My previous post looked at Airlines. This time we turn our attention to Telecommunications.

Obviously a power cut is not the only possible cause of business problems. Another single-point of failure could be a single rogue employee.

Gavin Clarke, Telecity's engineers to spend SECOND night fixing web hub power outage (The Register, 18 November 2015)

Related Post: Single Point of Failure (Airlines) (August 2016)

Monday, August 08, 2016

Single Point of Failure (Airlines)

Large business-critical systems can be brought down by power failure. Who knew?

In July 2016, Southwest Airlines suffered a major disruption to service, which lasted several days. It blamed the failure on "lingering disruptions following performance issues across multiple technology systems", apparently triggered by a power outage.
In August 2016 it was Delta's turn.

Then there were major problems at British Airways (Sept 2016) and United (Oct 2016).

The concept of "single point of failure" is widely known and understood. And the airline industry is rightly obsessed by safety. They wouldn't fly a plane without backup power for all systems. So what idiot runs a whole company without backup power?

We might speculate what degree of complacency or technical debt can account for this pattern of adverse incidents. I haven't worked with any of these organizations myself. However, my guess is that some people within the organization were aware of the vulnerability, but this awareness didn't somehow didn't penetrate the management hierarchy. (In terms of orgintelligence, a short-sighted board of directors becomes the single point of failure!) I'm also guessing it's not quite as simple and straightforward as the press reports and public statements imply, but that's no excuse. Management is paid (among other things) to manage complexity. (Hopefully with the help of system architects.)

If you are the boss of one of the many airlines not mentioned in this post, you might want to schedule a conversation with a system architect. Just a suggestion.

American Airlines Gradually Restores Service After Yesterday's Power Outage (PR Newswire, 15 August 2003)

British Airways computer outage causes flight delays (Guardian, 6 Sept 2016)

Delta: ‘Large-scale cancellations’ after crippling power outage (CNN Wire, 8 August 2016)

Gatwick Airport Christmas Eve chaos a 'wake-up call' (BBC News, 11 April 2014)

Simon Calder, Dozens of flights worldwide delayed by computer systems meltdown (Independent, 14 October 2016)

Jon Cox, Ask the Captain: Do vital functions on planes have backup power? (USA Today, 6 May 2013)

Jad Mouawad, American Airlines Resumes Flights After a Computer Problem (New York Times, 16 April 2013)

 Marni Pyke, Southwest Airlines apologizes for delays as it rebounds from outage (Daily Herald, 20 July 2016)

Alexandra Zaslow, Outdated Technology Likely Culprit in Southwest Airlines Outage (NBC News, Oct 12 2015)

Updated 14 October 2016.

Sunday, August 07, 2016

Why does my bank need more personal data?

I recently went into a High Street branch of my bank and moved a bit of money between accounts. I could have done more, but I didn't have any additional forms of identification with me.

At the end, the cashier asked me for my nationality. British, as it happens. Why do you want to know? The cashier explained that this enabled a security control: if I ever bring my passport into a branch as a form of identification, the system can check that my passport matches my declared nationality.

Really? Really? If this is really a security measure, it's a pretty feeble one. Does my bank imagine I'm going to say I'm British and then produce a North Korean passport? Like a James Bond film?

After she had explained how the bank would use my nationality data, she then asked for my National Insurance number. I declined, choosing not to quiz her any further, and left the branch planning to write a stiff letter to the head of data protection at the bank's head office.

As a data expert, I am always a little suspicious of corporate motives for data collection. So the thought did occur to me that my bank might be planning to use my personal data for some purpose other than that stated.

Of course, my bank is perfectly entitled to collect data for marketing purposes, with my consent. But in this case, I was explicitly told that the data were being collected for a very narrowly defined security purpose.

So there are two possibilities. Either my bank doesn't understand security, or it doesn't understand data protection. (Of course there will be individuals who understand these things, but the bank as an organization appears to have failed to embed this understanding into its systems and working practices.) I shall be happy to provide advice and guidance on these topics.

New White Paper - TotalData™

My latest white paper for @GlueReply has been posted on the Reply website.

It outlines four dimensions of TotalData - reach, richness, assurance and agility - and presents a Value Chain from Raw Data to the Data-Fueled Business.

TotalData™: Start making better use of Data (html) (pdf)

(Now I need to write some more detailed stuff, based on a few client projects.)

Saturday, June 04, 2016

As How You Drive

I have been discussing Pay As You Drive (PAYD) insurance schemes on this blog for nearly ten years.

The simplest version of the concept varies your insurance premium according to the quantity of driving - Pay As How Much You Drive. But for obvious reasons, insurance companies are also interested in the quality of driving - Pay As How Well You Drive - and several companies now offer a discount for "safe" driving, based on avoiding events such as hard braking, sudden swerves, and speed violations.

Researchers at the University of Washington argue that each driver has a unique style of driving, including steering, acceleration and braking, which they call a "driver fingerprint". They claim that drivers can be quickly and reliably identified from the braking event stream alone.

Bruce Schneier posted a brief summary of this research on his blog without further comment, but a range of comments were posted by his readers. Some expressed scepticism about the reliability of the algorithm, while others pointed out that driver behaviour varies according to context - people drive differently when they have their children in the car, or when they are driving home from the pub.

"Drunk me drives really differently too. Sober me doesn't expect trees to get out of the way when I honk."

Although the algorithm produced by the researchers may not allow for this kind of complexity, there is no reason in principle why a more sophisticated algorithm couldn't allow for it. I have long argued that JOHN-SOBER and JOHN-DRUNK should be understood as two different identities, with recognizably different patterns of behaviour and risk. (See my post on Identity Differentiation.)

However, the researchers are primarily interested in the opportunities and threats created by the possibility of using the "driver fingerprint" as a reliable identification mechanism.

  • Insurance companies and car rental companies could use "driver fingerprint" data to detect unauthorized drivers.
  • When a driver denies being involved in an incident, "driver fingerprint" data could provide relevant evidence.
  • The police could remotely identify the driver of a vehicle during an incident.
  • "Driver fingerprint" data could be used to enforce safety regulations, such as the maximum number of hours driven by any driver in a given period.

While some of these use cases might be justifiable, the researchers outline various scenarios where this kind of "fingerprinting" would represent an unjustified invasion of privacy, observe how easy it is for a third party to obtain and abuse driver-related data, and call for a permission-based system for controlling data access between multiple devices and applications connected to the CAN bus within a vehicle. (CAN is a low-level protocol, and does not support any security features intrinsically.)


Miro Enev, Alex Takakuwa, Karl Koscher, and Tadayoshi Kohno, Automobile Driver Fingerprinting Proceedings on Privacy Enhancing Technologies; 2016 (1):34–51

Andy Greenberg, A Car’s Computer Can ‘Fingerprint’ You in Minutes Based on How You Drive (Wired, 25 May 2016)

Bruce Schneier, Identifying People from their Driving Patterns (30 May 2016)

See also John H.L. Hansen, Pinar Boyraz, Kazuya Takeda, Hüseyin Abut, Digital Signal Processing for In-Vehicle Systems and Safety. Springer Science and Business Media, 21 Dec 2011

Wikipedia: CAN bus, Vehicle bus

Related Posts

Identity Differentiation (May 2006)

Pay As You Drive (October 2006) (June 2008) (June 2009)

Monday, May 30, 2016

Globally Integrated Enterprise 2

In my post on the Globally Integrated Enterprise (June 2006), I reported a comment by James Martin about the growth of the Indian CEO. A few years later, Megha Bahree reported Eight Indian CEOs At Big U.S. Companies (Forbes, December 2009). Now both Microsoft and Google CEOs were born in India. See India-born CEOs are taking the U.S. by storm (CNN MoneyInvest, August 2015).

@juliapowles notes that the relationship between Google and Microsoft started to improve in September 2015, shortly after Sundar Pichai became Google’s chief executive. Coincidence?

Meanwhile, the relationship between Google and Oracle remains tense. Google has just won the latest battle in the ongoing legal war over its use of Java code in the Android operating system.

In case you were wondering, Oracle does not have an Indian CEO. Unusually, it has two CEOs - an Israeli woman and an American man - as well as Larry Ellison remaining as CTO and executive chairman. (Not exactly normalized, eh Larry?)

Ms Catz has been leading the Oracle legal battle against Google. No doubt she has Mr Hurd's full support ...

Diane Brady, Oracle Hunger Games: Larry Ellison Creates Co-CEOs (Bloomberg, 19 September 2014)

Julia Powles, Google and Microsoft have made a pact to protect surveillance capitalism (Guardian, 2 May 2016)

Nicky Woolf, Google wins six-year legal battle with Oracle over Android code copyright (Guardian, 26 May 2016)

Claire Zillman, With co-CEOs, companies flirt with disaster (Fortune, 20 September 2016)