2018/11/29: we mapped a digital campaigning industry whose breadth and depth far surpassed our expectations. This overview is the start of the a series in which we unpack our findings.
There are two features of these organisations that are important to their impact on democratic processes. Firstly, most of them are for-profit companies, with the primary aim of generating, maintaining and growing revenue, a business model that inevitably guides their decisions rather than traditional political metrics such as voter participation. Secondly, the organisations are, for the most part, hired for their expertise in data technologies rather than their knowledge or engagement in politics. Political campaigning is now often largely mediated by data-driven technology organisations.
2018/11/17: The American investigations into foreign interference in Trump’s election, and British probes into Brexit, have increasingly become interwoven.
The possibility that both Brexit and the Trump campaign simultaneously relied upon the same social-media company and its transgressive tactics, as well as some of the same advisers, to further far-right nationalist campaigns, set off alarm bells on both sides of the Atlantic.
Damian Collins, a member of Parliament, and chair of its Digital, Culture, Media and Sport Committee, which held an inquiry into fake news, told the Observer, which has broken much of the news about Cambridge Analytica in the U.K., that the new e-mails “suggest that the role of Bannon and Mercer is far deeper and more complex than we realised.
There’s a big question about whether Mercer’s money was used in the Brexit campaign and it absolutely underscores why Britain needs a proper Mueller-style investigation. There are direct links between the political movements behind Brexit and Trump. We’ve got to recognise the bigger picture here. This is being coordinated across national borders by very wealthy people in a way we haven’t seen before."
2015/09/01: YouTuber Sir Mashalot has created a mix of popular country tunes that reveals just how formulaic the genre has become, confirming the suspicions of millions of urbanites who happened to turn on the radio during their rare trips outside the comforting womb of the major metropolitan area.
The mashup includes six songs, including ones by big acts like Blake Shelton and Luke Bryan, all of which performed strongly on the charts. There are indeed references aplenty to cars and beer and summertime, but what’s more striking is the nearly identical instrumentation and chord progressions. At one point, all six songs are overlaid on top of each other, and it’s a bit alarming how perfectly they fit together.
2015/01/07: A new study, surveying more than 500,000 albums, shows simplicity sells best across all music genres. As something becomes popular, it necessarily dumbs down and becomes more formulaic. So if you're wondering why the top 10 features two Meghan Trainor songs that sound exactly the same and two Taylor Swift songs that sound exactly the same, scientists think they finally have the answer.
in nearly every case, as genres increase in popularity, they also become more generic.
Record labels can use services like Shazam and HitPredictor to see which songs will break out next with surprising accuracy.
Top 40 stations last year played the 10 biggest songs almost twice as much as they did a decade ago.
2017/10/22: Parent company Alphabet would provide services in response to data harvested. a city “where buildings have no static use”. Like biomass
Alphabet’s long-term goal is to remove barriers to the accumulation and circulation of capital in urban settings – mostly by replacing formal rules and restrictions with softer, feedback-based floating targets. It claims that in the past “prescriptive measures were necessary to protect human health, ensure safe buildings, and manage negative externalities”. Today, however, everything has changed and “cities can achieve those same goals without the inefficiency that comes with inflexible zoning and static building codes”.
This is a remarkable statement. Even neoliberal luminaries such as Friedrich Hayek and Wilhelm Röpke allowed for some non-market forms of social organisation in the urban domain. They saw planning – as opposed to market signals – as a practical necessity imposed by the physical limitations of urban spaces: there was no other cheap way of operating infrastructure, building streets, avoiding congestion.
For Alphabet, these constraints are no more: ubiquitous and continuous data flows can finally replace government rules with market signals. Now, everything is permitted – unless somebody complains.
Google Urbanism means the end of politics, as it assumes the impossibility of wider systemic transformations, such as limits on capital mobility and foreign ownership of land and housing. Instead it wants to mobilise the power of technology to help residents “adjust” to seemingly immutable global trends such as rising inequality and constantly rising housing costs (Alphabet wants us to believe that they are driven by costs of production, not by the seemingly endless supply of cheap credit).
2018/11/02: A new free website spearheaded by the Library Innovation Lab at the Harvard Law School makes available nearly 6.5 million state and federal cases dating from the 1600s to earlier this year, in an initiative that could alter and inform the future availability of similar areas of public-sector big data.
Led by the Lab, which was founded in 2010 as an arena for experimentation and exploration into expanding the role of libraries in the online era, the Caselaw Access Project went live Oct. 29 after five years of discussions, planning and digitization of roughly 100,000 pages per day over two years.
The effort was inspired by the Google Books Project; the Free Law Project, a California 501(c)(3) that provides free, public online access to primary legal sources, including so-called “slip opinions,” or early but nearly final versions of legal opinions; and the Legal Information Institute, a nonprofit service of Cornell University that provides free online access to key legal materials.
The conversion, done in-house at the Harvard Law School Library to preserve the chain of custody of millions of cases it had collected, used a hydraulic cutter to trim the binding from thousands of volumes; and a machine similar to those employed in the meatpacking industry to vacuum-seal them after scanning. Scanning costs were in the millions of dollars. Scanned, resealed volumes were shipped out-of-state for long-term storage underground at a former limestone mine in Louisville, Ky. Pages were subsequently uploaded to an optical character recognition (OCR) vendor for extraction into text files.
The project, which was funded by venture capital-backed startup Ravel Law and the Harvard Law School, doesn’t aggregate every court battle. Its legal trove primarily focuses on supreme court and appellate decisions, but is limited, the Lab’s director said, by the extent to which bygone officials “cared enough at the time” to compile decisions. Director Adam Ziegler said the project has a high concentration of federal trial opinions and lots of trial opinions from the state of New York, an early legal center, but fewer from some other states.
In standing up the project website, Ziegler said the Lab hopes to provide “anyone and everyone” with easy access to the law via court opinions, but noted that concept will have different meanings to different groups and “definitely means things we don’t even envision ourselves.”
2018/10/10: Il sistema può scalare? (“Scalare” significa cambiare livello di scala, ovvero essere esteso da una porzione particolare a un intero ambito o a una generalità di ambiti). Ad esempio le mie valutazioni scolastiche nei test possono essere utilizzate come una variabile per valutare la mia affidabilità nella concessione di un mutuo? È proprio la scalabilità che rende l’ADM un’arma terribile. Sin tanto che una valutazione negativa, giusta o sbagliata, resta in un ambito ristretto, il danno è limitato, ma se “scala” a un contesto più ampio, il danno può essere terribile. Provate a ritrovarvi classificati dal sistema come “cattivi pagatori” e a vivere comunque una vita normale: questo capita già oggi, poiché per essere considerati tali basta aver saltato il pagamento delle rate di un debito. E che succederebbe se creassimo un modello che identifica un probabile “cattivo pagatore” mettendo questa variabile in relazione con altre caratteristiche personali rilevate statisticamente, fino a scoprire una correlazione tra “cattivi pagatori” e neri o meridionali? Chi pensa che sia futurologia o fantascienza alla Minority report non conosce i primi sistemi anti-cheating di Invalsi che prevedevano una correzione automatica dei risultati sulla base della provenienza regionale.
Il libro di Cathy O’Neill non è un invito a rinunciare al potere descrittivo e modellizzante della matematica, ma a riconoscere il suo enorme potere, i suoi usi nefasti se non addirittura fraudolenti, per poterne così chiedere un uso corretto e legittimo.
2018/09/19: DeAngelo was found using GEDMatch, a website that pools user-uploaded genetic profiles from other genealogy websites. GEDMatch exists “to provide DNA and genealogy tools for comparison and research services” and has an open-source database of 650,000 genetically connected profiles. DeAngelo, who evaded capture for decades, was meticulous in his crime scenes and certainly did not upload his DNA to a website in the hopes of uncovering his ancestry. But a third cousin of his did.
“The privacy concerns raised by the Golden State Killer investigation don’t disappear just because GEDmatch, the genealogical database investigators reportedly used, was a public site. In fact, investigators’ decision to upload a detailed genetic profile generated from crime-scene DNA to a public website likely violated the alleged perpetrator’s privacy rights,” Vera Eidelman of the American Civil Liberties Union (ACLU) wrote in an op-ed in the Washington Post. “Even if DeAngelo is found guilty of the crimes he is accused of, penalties for such crimes do not typically entail releasing a person’s entire genetic makeup. People may not be so troubled by such an intrusion when it comes to a serial killer, but imagine the implications of using this technique for shoplifters or trespassers.”
2018/10/05: When white respondents perceived the share of non-white residents in the nation and in their cities to be higher, they tended to feel that they themselves were being discriminated against more. While the actual size of the non-white population in their neighborhood also went hand in hand with this attitude, that association was less strong.
Having diverse neighbors move in seems to have two effects on white Americans. It can affirm that their overestimation of the extent of demographic change happening in the country is correct, and therefore increase the threat they perceive. Or, by giving them opportunities to interact with people who don’t look like them, it can mitigate some of their fears.
2018/10/03: If anything, rich countries are leapfrogging ahead of the poor, by benefiting from the expanded market and lower labour costs that they provide.
The latest technologies are almost always designed for advanced markets and the rich who live in them, and are well beyond the means of the poorest. Hence, if these technologies do indeed have benefits associated with them, these will accrue disproportionately to the rich. Poor countries and people are either left to pick up the scraps of remaining older technologies, or have to purchase inferior products at the lower end of the market. The Internet of Things and Artificial Intelligence are going to be used in the so-called Smart Cities of the developed world long before they are used at all widely in remote rural villages in Africa or Asia; big data are going to be used by large corporations with the expertise to analyse them, long before they are understood, let alone, used by people in the poorest countries of the world.
This is why terms such as “bridging the digital divide” or “digital leapfrogging”, although widely used, are so inappropriate. When the rich are designing and implementing technologies in their own interests, to move them further ahead of their competitors, the gap or divide between rich and poor becomes yet more difficult to reduce, or bridge; the horizon is always moving further and further into the distance… Moreover, the notion of a “divide” generally implies a binary divide, as in the gender divide, whereas in reality it is complex and multifaceted; it is not one divide, but many. The notion of leapfrogging is also problematic, since it implies benefiting from someone else; using a person’s back to lever an advantage ahead of them.
2018/10/02: Do airlines use these cost factors to calculate a rational price for my ticket? No. That is determined by Rudy the Fare Chicken, who decides the price of each ticket individually by pecking on a computer keyboard sprinkled with corn.
How might we best research this from our side—the one where humans use browsers and actually buy stuff? Is it possible to figure out how we're being profiled, if at all—and how might we do that? Are there shortcuts to finding the cheapest Amazon price for a given product, among all the different prices it presents at different times and ways on different browsers, to persons logged in or not? Is this whole thing so opaque that we'll never know much more than a damn thing, and we're simply at the mercy of machines probing and manipulating us constantly?
2018/09/30_ What would happen if we made all of our data public—everything from wearables monitoring our biometrics, all the way to smartphones monitoring our location, our social media activity, and even our internet search history?
Would such insights into our lives simply provide companies and politicians with greater power to invade our privacy and manipulate us by using our psychological profiles against us?
A burgeoning new philosophy called dataism doesn’t think so.
In fact, this trending ideology believes that liberating the flow of data is the supreme value of the universe, and that it could be the key to unleashing the greatest scientific revolution in the history of humanity.
2018/09/27: Insurance works because we are ignorant of our individual fates. It is the fact that any of us might turn out to be a bad risk that makes it sensible for everyone to insure against that remote chance. The pooling of individual risks that can only be known in aggregate underlies the whole system. But there is a subtle mismatch of aims between insurers and their customers. The customers want to avoid the consequences of misfortune; the insurers want customers who avoid misfortune. The two aims are reconciled because both sides are operating behind a veil of ignorance.
Insurers have an interest in knowing as much as possible about their customers. Customers have an interest in insurers underestimating their real risk. But both sides will benefit if ways are found to reduce the risk of the misfortune insured against. The balance between knowledge and ignorance of risk has traditionally been struck at the level of statistical knowledge about large groups.
But statistically significant groups are getting smaller in the age of big data.
2018/09/16: Nothing says more about someone than the music they listen to and their porn habits. This is certainly ingrained in the streaming service’s business model.
Over the past few years, Spotify has been ramping up its data analytic capabilities in a bid to help marketers target consumers with adverts tailored to the mood they’re in. They deduce this from the sort of music you’re listening to, coupled with where and when you’re listening to it, along with third-party data that might be available.
Spotify is far from the only platform helping brands target people according to their emotions; real-time mood-based marketing is a growing trend and one we all ought to be cognisant of. In 2016, eBay launched a mood marketing tool, for example. And last year, Facebook told advertisers that it could identify when teenagers felt “insecure” and “worthless” or needed “a confidence boost”. This was just a few years after Facebook faced a backlash for running experiments to see if it could manipulate the mood of its users.
You can see where this could go, can’t you? As ad targeting gets ever more sophisticated, marketers will have the ability to target our emotions in potentially exploitative ways. You are more likely to spend more on a product if you’re feeling sad.
12018/09/19: one of the oldest and largest North American life insurers, will stop underwriting traditional life insurance and instead sell only interactive policies that track fitness and health data through wearable devices and smartphones.
Privacy and consumer advocates have raised questions about whether insurers may eventually use data to select the most profitable customers, while hiking rates for those who do not participate. The insurance industry has said that it is heavily regulated and must justify, in actuarial terms, its reasons for any rate increases or policy changes.
2018/09/19: 15 employers in past year, including Uber, advertised jobs on Facebook exclusively to one gender.
In a statement, Facebook spokesman Joe Osborne said, “There is no place for discrimination on Facebook; it’s strictly prohibited in our policies. We look forward to defending our practices once we have an opportunity to review the complaint.”
The company has previously said that giving advertisers the ability to target employment ads by sex and age does not facilitate discrimination.
In response to other suits, Facebook has argued that it is not liable for the content its users—in this case, advertisers—post on its platform.
2018/9/18: For years, Facebook has publicly positioned its Messenger application as a way to connect with friends and as a way to help customers interact directly with businesses. But a new report from The Wall Street Journal today indicates that Facebook also saw its Messenger platform as a siphon for the sensitive financial data of its users, information it would not otherwise have access to unless a customer interacted with, say, a banking institution over chat. In this case, the WSJ report says not only did the banks find Facebook's methods obtrusive, but the companies also pushed back against the social network and, in some cases, moved conversations off Messenger to avoid handing Facebook any sensitive data. Among the financial firms Facebook is said to have argued with about customer data are American Express, Bank of America, and Wells Fargo.
The report says Facebook was interested in helping banks create bots for its Messenger platform, as part of a big push in 2016 to turn the chat app into an automated hub of digital life that could help you solve problems and avoid cumbersome customer service calls. But some of these bots, like the one American Express developed for Messenger last year, deliberately avoided sending transaction information over the platform after Facebook made clear it wanted to use customer spending habits as part of its ad targeting business. In some cases, companies like PayPal and Western Union negotiated special contracts that would let them offer many detailed and useful services like money transfers, the WSJ reports. But by and large, big banks in the U.S. have reportedly shied away from working with Facebook due to how aggressively it pushed for access to customer data.
2018/09/17: We can judge the moral fibre of a society by how it treats its least well-off members.
That sounds like a reasonably straightforward idea. But knowing who the poorest people are is not as easy as it sounds. The traditional measure of poverty in the UK has been to consider a household as being in relative poverty if its income is below 60 per cent of the median household income after housing costs.
by counting assets and assessing people’s real needs more effectively around two and a half million people who were in the old poverty measure - in particular many pensioners with assets - are no longer counted as poor. These are replaced by a different set of people - mostly people with disabilities and families with children.
While the overall figures have remained broadly the same, this paints a different picture for UK society.
Statisticians hope that their new measure can command a political consensus so we can at least agree who the poorest are.
2018/09/15: data from sensors located in vehicles have an important advantage over traditional data-gathering systems:
Currently, city managers and planners are faced with the challenge of relying on incomplete or out of date information.
A less obvious application of Geotab’s dataset is the ability to spot problems like potholes. Aggregated vertical axis accelerometer data from vehicles can be analyzed in near real-time to indicate areas in need of road maintenance. Other aspects of urban life that can be monitored in this way include areas where cars idle, thus wasting fuel and increasing air pollution, and roads where drivers are searching for parking places. Gathering this kind of data would be expensive using other approaches, but emerges naturally from aggregated traffic flows.
Huge datasets generated by sensors on connected vehicles offer interesting new opportunities for urban analytics. But there are naturally privacy concerns, too. Connected vehicles inevitably track the people who drive them. Analyzing the habits of drivers as revealed by their journeys can expose extremely sensitive information - think of repeated visits to a hospital, or unexpected overnight stays at private houses.
2018/09/13: Grocery data startup Basket is among the companies trying to help traditional brick-and-mortar grocery stores succeed against alternative players such as Amazon and Walmart.
The death of local grocery retail is greatly exaggerated.
For the past four years, Basket built up a database of grocery store product pricing by tasking shoppers with capturing that information.
For the past couple months, the company has been working with a dozen brands, to give them access to real-time and historical data for future planning purposes.