After the Christchurch mosque attacks, how are governments and social-media platforms responding to the spread of white-nationalist extremism?

Sydney, Australia Correspondent

The man who did this was ours. He grew up in Grafton, in northern New South Wales, worked at a gym, inherited some money, travelled, and spent two years in New Zealand preparing, and when he was done he wrote in his manifesto that he had chosen Christchurch because it would show that nowhere was safe, and he was right, and the country he came from has spent three weeks deciding what it owes the country he went to. I want to start there because the question asks how governments are responding, and the Australian response has been shaped by a guilt that is specific and, I think, useful: it has forced a government that was reluctant to regulate the platforms to do so in a week, and it has forced a political class that had tolerated a certain kind of talk about Muslims for two decades to look at where the talk ends.

Start with what has actually been done, because the Australian and New Zealand responses have been faster than anything in the United States or Europe and faster, frankly, than I expected. Prime Minister Ardern announced within six days that New Zealand would ban military-style semi-automatic weapons, and the legislation will pass next week with the support of every party but one member; she has refused to say the gunman’s name and has asked the country to do the same, and the country has largely complied. Her chief censor has made possession of the video and the manifesto a criminal offence. On our side, Parliament passed a law on Thursday, in a single day, that makes it a crime for a platform to fail to remove abhorrent violent material expeditiously, with penalties of up to ten percent of global turnover and prison for executives, and it did so over the objections of every technology company and most of the legal profession, who pointed out that the law was drafted in a weekend and does not define its central term. They were right about the drafting. The government did it anyway, because the public wanted it done, and because the alternative was to explain to the families in Christchurch why an Australian company could be fined for a workplace death and an American one could not be fined for hosting a massacre.

Now the platforms, where I think the response has been more interesting than either the companies or their critics admit. Facebook has said that fewer than two hundred people watched the stream live, that no one reported it during the broadcast, that the first report came twelve minutes after it ended, and that it removed one and a half million copies in the first day, most of them at upload. Those numbers are offered as a defence and they are, in fact, an indictment: a company whose systems can block 1.2 million uploads of a video in twenty-four hours is a company that could have built the system to block the first one, and did not, because live video was a growth product and moderation was a cost. Two weeks later Facebook announced that it would ban praise and support of white nationalism and white separatism, which it had previously distinguished from white supremacy on the theory that the former were legitimate political identities. That distinction was always indefensible and it took a massacre to end it. YouTube’s response was that it struggled to keep up with re-uploads, which is true and is also a description of a product designed to make re-uploading easy. The platforms are responding, in other words, the way they always respond, which is after the event, with a policy change that addresses the last event, under threat of regulation that they will spend the next year lobbying to narrow.

On the political response, I will be direct about my own country, because clarity requires it. The senator from Queensland who issued a statement within hours blaming Muslim immigration for the attack was censured by both houses of Parliament, and the teenager who broke an egg on his head became a national hero, and both of those are the easy part. The hard part is that the senator’s view did not come from nowhere. Australian politics has for twenty years permitted a register of talk about Muslims, about boats, about African gangs, that stops well short of what the gunman wrote and that shares its grammar, and the people who used that register are now discovering that it had an audience they did not intend. I do not say that to assign blame for the murders to anyone but the murderer. I say it because the question asks about the response to the spread of this extremism, and a response that regulates the platforms and ignores the register is a response that treats the pipe and not the water. The New Zealand prime minister understood that, which is why she refused to say his name and went to the mosque in a headscarf. Our prime minister understood it less well, and said so in a way that suggested he had not read the manifesto.

On the United States, I will say what the region says. The President, asked on the day of the attack whether he saw white nationalism as a rising threat, said he did not, and his own government’s data on domestic terrorism arrests says otherwise, and the manifesto named him as a symbol while disclaiming his policies, which is the kind of thing a leader should want to repudiate loudly and quickly and which he did not. The American response to Christchurch, at the level of government, has been nothing: no legislation, no regulatory proposal, no statement of the kind Ardern made. The platforms that hosted the attack are American, the imageboard where the manifesto was posted is American, and the regulatory pressure on them is coming from Canberra and Wellington and Brussels, because Washington has concluded that the First Amendment settles the question, and perhaps it does, but it settles it in a way that leaves the rest of the world to write the rules for American companies, and that is a consequence Americans should notice.

So my assessment is that governments are responding, in the countries that were struck, with speed and some clumsiness: a gun law in six days, a platform law in one, a chief censor’s order, a prime minister who will not say his name. The platforms are responding as they always do, after the fact and under duress, with policies that would have mattered before. The political class in Australia is being forced to confront a register of speech it had found convenient. And the United States, which built the pipes and hosts the water, is responding by saying the problem is not rising. Clarity over fashion: the fashionable view is that this is a problem of technology. The clear view is that it is a problem of what people have been permitted to say and to whom, and that the technology merely delivered it to Christchurch at the speed of light, from a man who grew up in a town six hours’ drive north of where I am writing.

Myrtle Beach, South Carolina Correspondent

The attack was designed for the internet, and I want to say that plainly before anything else, because I work in the medium it was designed for and I recognized the design within an hour of the news. The gunman posted his manifesto to a message board where he knew it would be copied before anyone could take it down. He announced the livestream on the same board. He wore a camera on his helmet so the video would look like a first-person shooter. He filled the manifesto with in-jokes and references that would send journalists chasing false leads and send the people who got the jokes into a frenzy of sharing. He told viewers to subscribe to a YouTube channel. Every one of those choices was a content strategy, and it worked: the video was copied, re-cut, and re-uploaded faster than the largest companies on earth could remove it, and the manifesto is now on a thousand servers that no government will ever reach. I do social media for small businesses, and I have never seen a campaign built with that much understanding of how the platforms reward content. The platforms built the reward system. He read the manual.

So the question of how the platforms are responding has to start with what they are responding to, which is a man who used their products exactly as designed. Facebook’s numbers are the clearest statement of this: fewer than two hundred people watched the stream live, nobody reported it during the seventeen minutes, the first report came twelve minutes after it ended, and the company removed one and a half million copies in the next twenty-four hours, 1.2 million of them at upload. Those numbers are offered as evidence that the system worked. Read them again. A system that can recognize and block 1.2 million copies of a video in a day is a system with extraordinary capability, and that capability was pointed at the problem only after the problem had already been distributed to the people who most wanted it. The three hundred thousand copies that got through were enough. The point of the video was never the two hundred people watching live. It was the file, and the file is out.

Now the responses, and I will take them as a marketer would, by what they actually change. Facebook announced two weeks after the attack that it would ban praise and support of white nationalism and white separatism, which it had previously permitted on the theory that they were political identities distinct from white supremacy. That is a real change in policy and it will be enforced the way every Facebook policy is enforced, which is imperfectly, by contractors, after a report, against content that has already been seen by the people it was meant for. YouTube said it struggled with re-uploads, which is true and is also a confession that its system is optimized for upload and not for removal. The imageboard where the manifesto was posted has defended itself as a free speech platform, which is a position its owner is entitled to take and which tells you that the smallest venues, the ones with no advertisers to lose, will not respond to pressure at all. The platform response, in short, is the one I would have predicted: the big companies tighten the written rules, the enforcement lags, and the content migrates to places that have no rules, where it was always going to end up.

The government responses are more interesting, because they are testing whether law can do what the companies will not. New Zealand’s chief censor has made it a crime to possess the video or the manifesto, which is a thing a small country with no First Amendment can do and which will have no effect on the file’s availability to anyone outside New Zealand. Australia passed a law on Thursday, in a day, that threatens platforms with fines of up to ten percent of revenue for failing to remove violent material fast enough, and the law does not define fast enough, and the companies will spend a year in court finding out. Those are both real and both are the responses of countries that cannot reach the servers and are reaching for the companies instead. The United States, where the servers are, has done nothing, and will do nothing, because the First Amendment protects the manifesto and Section 230 protects the platforms, and I will say as someone who makes a living in this medium that I am not sure I want the alternative. A government that can order Facebook to remove a video of a massacre can order it to remove the next thing, and the next thing is decided by whoever holds the government.

Here is what I think is actually happening, and it is the part the policy debate misses because the policy debate is about rules and this is about audiences. The audience for this content is small, young, male, and online, and it lives in places the platforms do not reach and the governments cannot touch. The platforms’ rules push it from the big sites to the small ones, where it concentrates and radicalizes faster, because there is nothing there but itself. The governments’ laws push the big sites to remove faster, which pushes more of it to the small sites. Every response so far has the effect of sorting the audience into rooms where no one else is, and the man who did this came out of one of those rooms and designed his attack to be shared back into them. I do not know what the answer to that is. I know that removing the file is not it, because the file is a souvenir and the room is the problem, and nobody has a policy for the room.

So my answer is that the platforms are responding the way platforms respond, with rules written after the event and enforced after the report, and that the governments that can act are acting against the companies they can reach rather than the content they cannot, and that the United States, which hosts the whole system, has decided that its constitution settles the question and is letting Canberra and Wellington write the rules for American companies. All of that is understandable. None of it touches the thing the gunman understood better than any regulator, which is that the platforms are a distribution system and he was a content creator, and that a content creator with a camera and a message board can reach his audience in seventeen minutes and no one can get the message back. I build campaigns for a living. His was the most effective one I saw this year. That sentence should frighten everyone who reads it, and it should frighten the companies most, because they built the thing he used and they have not yet admitted it was working as intended.

Long Island, New York Correspondent

I want to explain why the United States has done nothing, legally, in response to Christchurch, and why it will continue to do nothing, and why that is both a scandal and a constitutional necessity, because I think the debate about how governments should respond to the spread of this extremism has been conducted as if the American government had the same tools New Zealand and Australia have, and it does not, and the difference is the most important fact about the response and the least discussed. I spent ten years in a prosecutor’s office and I have read a great deal of hateful material that was, every word of it, protected. The manifesto is protected. The ideology is protected. The video, in the hands of anyone who is not the shooter, is almost certainly protected. That is the law, and a response that ignores it is a response that will be struck down by the first federal judge who sees it.

Let me lay out the three doctrines, because together they explain the American position. The first is the Supreme Court’s rule from 1969 that speech advocating violence may be punished only if it is directed to inciting imminent lawless action and is likely to produce it. The manifesto advocates violence in general terms against groups of people; it is not directed at an imminent act against a specific person, and under that rule it cannot be criminalized. New Zealand’s chief censor has made possession of it a crime; an American official who tried the same thing would lose in court within a week. The second is the federal statute from 1996 that shields platforms from liability for what their users post, which is the reason Facebook and YouTube cannot be sued for hosting the video and cannot be fined for failing to remove it. Australia passed a law on Thursday imposing exactly such fines; Congress could not pass it without repealing the statute, and the statute is the legal foundation of the entire American internet industry, and no Congress is going to repeal it over one event, however terrible. The third is the First Amendment itself, which forbids the government from compelling or forbidding speech by private actors, and which means that the platforms’ content rules, whatever they are, are the platforms’ to write, and the government may not write them.

So the American response to the spread of this extremism is, legally, confined to three channels. The first is prosecution of the acts, not the speech: the shooter, had he been American, would face murder charges and possibly terrorism and hate-crime charges, all of which turn on what he did rather than what he wrote. The second is the platforms’ own rules, which are private action and therefore unconstrained by the Constitution, and which is why Facebook’s announcement on March 27 that it will ban praise of white nationalism and white separatism is, in the United States, the only regulatory event of the past three weeks: a private company changing its terms of service, under public pressure, with no legal compulsion. The third is the federal government’s domestic terrorism apparatus, which can investigate and surveil and, where a plot exists, prosecute conspiracy, and which has been, by the account of its own agents, under-resourced for the white-nationalist threat relative to the Islamist one for fifteen years. The President said on the day of the attack that he did not see white nationalism as a rising threat; the FBI’s own arrest statistics say it is, and the gap between those two statements is the only place where American policy could actually move, because it is a question of resources and attention rather than of law.

Now the scandal, because I promised one. The platforms that hosted the attack are American companies operating under American law, and the law gives them a shield that no other industry in America has: a newspaper that printed the manifesto could be sued; a television station that broadcast the video could be fined; a bookstore that sold it could face obscenity or incitement charges in some jurisdictions. The platforms can do all three and face nothing, because in 1996 Congress decided that the internet was a fragile new thing that would die under ordinary liability, and the fragile new thing is now the largest industry in the country and still has the shield. That is a policy choice, not a constitutional requirement, and Congress could narrow it, as it did last year for sex trafficking, without touching the First Amendment. It will not, because the industry is powerful and because every narrowing produces unintended consequences, as last year’s did. But the American public should understand that the inaction is a choice, and that the rest of the world is now writing rules for American companies because Washington declined to.

I want to say something about the responses abroad, from a paralegal’s chair. New Zealand’s gun law will pass next week and it is a response to the weapon, not the speech, and it is the one response that is squarely within any government’s competence. Australia’s platform law was drafted in a weekend, does not define its central term, and will be litigated for years; it is the kind of law that gets passed when a public demands action and a legislature has no better idea. The chief censor’s possession order is a small country’s gesture against a file that lives on servers it cannot reach. All three are understandable and the first is wise, and none of them would survive an American court, and American commentators who praise them as models are praising things the American system was designed to prevent.

So my answer is that governments are responding according to what their constitutions permit: New Zealand and Australia with speech and possession laws that the United States cannot enact, and the United States with nothing, because the one tool it has, a narrowing of the 1996 shield, is politically unavailable. The platforms are responding with private rule changes that are, in the United States, the only regulation there is, and that are enforced as well as private companies enforce anything, which is to say unevenly. The extremism itself spreads in the one space American law has deliberately left unregulated, and the American response to that is a debate about whether the First Amendment should mean what it says. I think it should. I also think that the people who wrote the 1996 statute did not imagine Christchurch, and that a Congress with any nerve would say so. It has not. The response, in this country, is the silence of a legal system that has decided the question is too hard, and the silence is itself a policy, and the gunman, who read our law as carefully as he read our platforms, counted on it.

Gastonia, North Carolina Correspondent

We did not run his name. That was the first decision made at my desk on the Friday the story broke, and it was made before the New Zealand prime minister asked anyone to make it, by an editor who had read the manifesto and understood what it was for. The manifesto was written to be quoted. It was full of bait: references to video games and internet jokes and historical grievances and the names of politicians, arranged so that any journalist who pulled one thread would find himself amplifying the whole garment, and the gunman said as much in the text, which is the part that made it so hard to handle. He told us what he wanted us to do. He wanted the name, the face, the quotes, the debate about which of his references were sincere, and the longer the debate ran the more of his audience would find him. So we did not run the name, we did not quote the manifesto, we did not link to the video, and we did not run the still from the helmet camera, and I want to describe that decision because I think the response of the people in my profession is a part of the question the coverage has not examined, since the coverage is us.

Here is the problem a copy editor faces with a story like this, and it is a real problem and not a pose. Our job is to tell people what happened, and what happened includes who did it and why he said he did it. Withholding the name and the manifesto is a departure from that job, and the departure has a cost: readers who want to understand the thing are told they cannot, and they go find it somewhere that will tell them, and the somewhere is usually worse than we are. The argument for withholding is contagion, the finding, which is now well established, that coverage of a mass killer’s name and image and grievance produces imitators, and that the imitation is faster and more reliable when the coverage is heavy. The gunman cited two previous attackers by name in his manifesto. He had read their coverage. He was the contagion, and he knew it, and he wrote his document to spread it further. So the newsroom response, in the places that thought about it, was to break the chain at our link, and I think that was right, and I am aware that it was also a decision to know something and not print it, which is the thing we are supposed to never do.

Now the platforms, where the same problem exists at a scale no newsroom can imagine and where the response has been, as it always is, after the fact. Facebook removed one and a half million copies of the video in a day and still could not keep it off; YouTube could not keep up with the re-uploads; the message board where the manifesto was posted defended itself as a free speech venue and is still up. Two weeks later Facebook announced it would ban praise of white nationalism and separatism, a distinction it had drawn and defended until the week it could not. That is the pattern I have watched for a decade: a policy written to address the last event, enforced by contractors reading reports, against content that has already reached its audience. The platforms are a wire service with no editor, and the thing I decided at my desk on a Friday morning, not to run the name, is a decision no platform is built to make, because the platform does not know what it is running until someone reports it, and by then the running is done.

The government responses are the responses of countries that cannot edit the platforms and are trying to make the platforms edit themselves. New Zealand’s censor has criminalized possession of the video and manifesto. Australia passed a law in a day that threatens the companies with fines for failing to remove violent material fast enough, with a definition of fast enough that will be litigated for years. New Zealand’s gun law will pass next week. The United States has done nothing, because the First Amendment protects the manifesto and the 1996 statute protects the platforms, and I will say as a journalist that I would not trade the First Amendment for any of the responses abroad, and I will also say that the 1996 statute gives the platforms a shield my newspaper does not have, and that the asymmetry is the thing Congress could fix and will not.

I will say where I am, since this forum wants it and I am in two places. I think the President’s remark on the day of the attack, that he did not see white nationalism as a rising threat, was wrong on the facts and careless in the moment, and I think the manifesto’s reference to him as a symbol was bait of exactly the kind I described, designed to make the coverage about him, and the coverage took it. Both things are true. The man who did this did not need an American president to tell him what to think; he had a message board for that. But a president asked a direct question on the day fifty people were murdered in a mosque, by a man who had named him, should have had a better answer than that, and the better answer would have cost nothing.

So my answer is that the responses divide by what each institution can do. Newsrooms can break the chain at their own link, and the good ones did, at the cost of telling readers less than they know. Platforms can tighten rules they will enforce late and unevenly, which they have, and will. Small countries can pass laws against possession and against the companies, which they have, and which will not reach the servers. The United States can narrow the shield it gave the platforms in 1996, which it will not. And the extremism itself spreads in the rooms none of those institutions can enter, where the file is a trophy and the name we did not print is already known. I made the right call on Friday. I am not sure it mattered to anyone but us, and I am sure that the gunman, who wrote his manifesto knowing some of us would not print it, had a plan for that too. He planned for the ones who would. There were enough.

Orange County, California Correspondent

I follow technology-driven business models, and the business fact at the center of this question is one that neither the platforms nor their critics want to state plainly: content moderation is a cost center with no revenue attached, livestreaming is a growth product with revenue attached, and the companies have been making the trade between them for five years in exactly the direction you would expect. Facebook launched live video in 2016 to compete with Periscope and YouTube and promoted it aggressively, because live content holds attention longer than recorded content and attention is the product. It built the moderation for live video afterward, on a smaller budget, with a reporting system that depends on a viewer flagging the stream, which is to say with a system designed for a world in which the viewers are not the audience the broadcaster is seeking. On March 15 the broadcaster was seeking an audience that would never report him, and nobody did, and seventeen minutes of murder went out in high definition to be copied before the first flag arrived. That is not a failure of the system. It is the system, priced as the company chose to price it, and the response question is really the question of whether anything has changed the price.

Let me take the platforms’ own numbers, because they are the clearest evidence of the economics. Facebook says fewer than two hundred people watched live, the first report came twelve minutes after the stream ended, and it removed one and a half million copies in twenty-four hours, 1.2 million of them blocked at upload by matching against the original. Those are impressive figures and they describe a company with enormous detection capability that it deploys after it has a reference copy to match against. The capability to catch the first upload, before there is a reference, is a different and much more expensive problem, requiring either a delay on live video, which the product teams have resisted because delay kills engagement, or a human review of streams at a scale no company has staffed, or an automated classifier for violence that works in real time and does not generate so many false positives that it shuts down the product. The company has not built any of those because none of them pays, and the response to Christchurch, as of this week, is a policy change on white nationalist content and a promise to look at live video restrictions. The policy change costs nothing. The live video restriction, if it comes, is the first thing that would cost something, and I would watch whether it comes.

Now the governments, and here the business view is useful because it tells you which responses will actually change behavior. New Zealand’s gun law and its censor’s order on possession of the video are directed at New Zealand, which is a market of five million people, and the platforms will comply there because compliance is cheap. Australia’s law, passed Thursday, is different in kind: it threatens fines of up to ten percent of global revenue and prison for executives if a platform fails to remove abhorrent violent material expeditiously, and it applies to any platform Australians can reach, which is all of them. The companies’ lawyers have spent the past two days pointing out that the law does not define its central term and was drafted over a weekend, and they are right, and it does not matter, because the law’s function is not to be enforced; it is to change the price. A company facing even a small probability of a fine equal to ten percent of its revenue will spend real money on removal speed in Australia, and because the systems are global, the spending will improve removal everywhere. That is the mechanism by which a country of twenty-five million can regulate an industry headquartered in California, and it is the same mechanism Europe used with its privacy law last year: make the penalty large enough that compliance in one market is cheaper than segmentation, and the market sets the global standard.

The United States, where the companies live, will do nothing, for the reasons a lawyer would give better than I can: the manifesto is protected speech, the 1996 statute shields the platforms from liability, and no Congress will repeal it. I will add the business reason, which is that the platforms are the largest industry in my state and among the largest in the country, and their lobbying presence in Washington has grown to match, and the political will to impose costs on them, which was real after Cambridge Analytica, has produced exactly one law in a year, a narrow one on sex trafficking. The American response to Christchurch is therefore outsourced: the rules that change the platforms’ costs will be written in Canberra and Brussels, and the companies will comply globally because segmentation is expensive, and American users will get the benefit of regulation their own government declined to impose. That is a strange arrangement and nobody designed it.

I want to say something about the advertisers, because they are the one American actor that can change the price and the one the coverage ignores. After the video circulated, several large brands paused advertising on YouTube, as they did in 2017 when their ads ran beside extremist content, and the platforms responded then with policy changes that had no legal compulsion behind them. That is the American regulatory mechanism, such as it is: the companies that buy the attention deciding that certain attention is a brand risk. It works, intermittently, on the content advertisers see. It does not work on the message boards with no advertisers, where the manifesto lives, and that is the limit of market regulation, and nobody has an answer to it, because the places with no revenue cannot be reached by anything that threatens revenue.

So my answer is that the platforms are responding with the cheapest changes first, a written policy on white nationalism and a promise to review live video, and that the costly change, a delay or a review system for livestreams, will come only if a regulator or an advertiser makes it cheaper than not changing. Governments are responding according to their reach: New Zealand with laws that bind New Zealand, Australia with a law that is really a global price signal, and the United States with nothing, because the law forbids most responses and the industry forbids the rest. The extremism itself has moved to venues with no revenue, which is the one place no market mechanism and no large-platform rule can follow it. I analyze business models for a living. The platforms’ model worked exactly as designed on March 15, and the design is what needs to change, and the people who can change it are deciding this month whether a fine in Australia is larger than the engagement they would lose by delaying a live stream thirty seconds. That is the actual decision. Everything else is a press release.

Prescott Valley, Arizona Correspondent

I taught adolescents for twenty years, a great many of them boys who were angry and isolated and spent too much time alone with a screen, and I want to answer this question from that room rather than from the platforms or the legislatures, because the man who did this was in that room once. He was twenty-eight when he killed fifty people, but the manifesto he left is the document of someone who found a community online in his teens and early twenties, a community that gave him a grievance, a vocabulary, a set of heroes, and a plan, and the question of how governments and platforms are responding to the spread of this extremism is really the question of whether anyone is responding to the room, and the answer, as far as I can see from Arizona, is that almost no one is, because the room is not where the policy tools reach.

Let me describe what I saw as a teacher, because it is the thing the policy debate does not have. The boys who are vulnerable to this are not, mostly, the ones with a record of violence or a diagnosable condition. They are the ones who have been left out, who are bright enough to be bored and not confident enough to be heard, who have discovered that there is a place online where their anger is treated as insight and their isolation as a mark of seeing clearly, and where the people who have never spoken to them in school are reframed as the enemy. Special education taught me to look for the student who is quietly disappearing, and the students who disappear into these rooms are not disappearing from school; they are present, they do the work, they are polite, and they go home and log in. A teacher can see the first part. A teacher almost never sees the second. By the time the second produces something visible, it is a manifesto on a message board.

Now the responses, and I will say first what I think the governments that acted got right. New Zealand’s prime minister refused to say the gunman’s name and asked her country not to, and the country complied, and that is the single response of the past three weeks that addresses the room, because the room runs on fame. The men who do this cite each other. The manifesto named two previous attackers and was written to be quoted by the next one. A government that denies the name denies the room its trophy, and a teacher who has watched a troubled boy seek notoriety knows exactly how much that matters. New Zealand’s gun law, which will pass next week, addresses the weapon, and it is the response that is within every government’s competence and the one the United States will never make. Australia’s platform law, passed Thursday, addresses the companies, and it may change their behavior at the margin, and it will not touch the message boards where the room actually lives.

The platforms are responding as the platforms respond. Facebook will now ban praise of white nationalism and separatism, which is a real change and will be enforced imperfectly on a site that the boys in the room left years ago for places with no rules. YouTube will struggle with re-uploads, which is true, and YouTube is also the place where a boy who watches one video about immigration is shown a slightly angrier one next, and a slightly angrier one after that, and the mechanism that does that is the business model and no policy change announced this month touches it. I have watched that progression in a student’s browser history, with permission, and it is not a conspiracy; it is a recommendation engine doing its job, which is to hold attention, and anger holds attention. The platforms’ response to Christchurch is a set of rules about what can be said. The thing that matters is a set of rules about what is suggested next, and nobody has proposed one, because the suggestion is the product.

I want to say something about the American side with the care a teacher owes a subject this hard. The President was asked on the day of the attack whether he saw white nationalism as a rising threat and said he did not, and I think that was a wrong answer and a careless one, and I think the people who have spent three weeks saying he caused this are also wrong, and dangerously so. The man who did this was in his room long before 2016. He named the President as a symbol in a document designed to make the coverage about the President, and the coverage obliged, and every hour of that coverage was an hour the room spent feeling important. A teacher learns not to give the student who craves attention the attention he craves. The country has not learned that, on either side, and the argument about whether a politician’s words caused a massacre is itself a kind of attention the massacre was designed to produce.

So what would a response to the room look like? I will say what I think, knowing it does not fit in a statute. It would look like adults who notice. It would look like a school that treats the quiet, polite, disappearing boy as a student at risk rather than a student who is fine. It would look like a recommendation engine that does not escalate, which the companies could build and will not unless made to. It would look like a press that does not print the name and a government that does not debate it. And it would look like a country willing to say that the ideology in the manifesto is wrong, plainly, without first checking which party the saying helps. None of that is a platform policy or a gun law. All of it is harder than either.

So my answer is that governments are responding, where they can, to the weapon and the companies, and the platforms are responding to the content, and almost no one is responding to the room where the content is made, because the room is private, young, and online, and no tool in the policy kit reaches it. New Zealand came closest by refusing the gunman his name. The rest is rules about what may be said, which the room does not follow, and arguments about who is to blame, which the room enjoys. I taught in that room for twenty years without knowing it was there. I would like someone to walk into it before the next boy walks out, and I do not know who that is, and I am fairly sure it is not a regulator.

Dayton, Ohio Correspondent

I am going to push back on something that has become the consensus in the three weeks since Christchurch, which is that the platforms could have stopped the video and chose not to, and I am going to do it as a process engineer, because I think the claim is being made by people who have not looked at how the process actually works and because a response built on a wrong account of the failure will fix the wrong thing. The claim rests on Facebook’s own numbers: fewer than two hundred live viewers, no reports during the stream, a first report twelve minutes after it ended, 1.5 million copies removed in a day, 1.2 million of them blocked at upload. People read those numbers and conclude that a system which can block 1.2 million copies could have blocked the first one. It could not, and the reason is the difference between matching and recognizing, and the difference is the whole problem.

Here is how the removal works, at the level a plant manager would need to understand it. Once a company has a copy of a video it does not want, it computes a digital fingerprint of the file and compares every new upload against the fingerprint, and a match is blocked in milliseconds. That is the 1.2 million. The technique is fifteen years old; it was built to catch child abuse images and it is extremely good at catching exact and near-exact copies. It fails in two ways that both happened here. It cannot catch anything until it has the reference, which means the first copy always gets through and the first few thousand get through while the reference is being generated and distributed. And it is defeated by alteration: re-encode the file, crop it, add a border, mirror it, change the speed, play it on a screen and film the screen, and the fingerprint no longer matches, which is why three hundred thousand copies got through the matching system and why the video is still findable today. The people re-uploading it knew exactly how the matching worked and altered the file to beat it, in real time, in a distributed contest between one company’s engineers and thousands of volunteers who wanted the video to spread. That is not a company choosing not to try. It is a company losing a race it entered late with a tool built for a different problem.

The harder problem, recognizing a violent livestream as it happens with no reference to match against, is a problem no one has solved, and I want to be specific about why, because the regulators in Australia who passed a law on Thursday requiring expeditious removal have not been. A classifier that watches live video and decides whether it depicts a real shooting has to distinguish it from a film, a video game, a news broadcast, a police body camera, a hunting video, and a thousand other things that look nearly identical to a machine, and it has to do it in seconds, across millions of simultaneous streams, with a false positive rate low enough that it does not shut down the product. Nobody has built that. The best systems flag a stream for human review, and the human review takes minutes, and the gunman’s stream lasted seventeen. The only intervention that would have worked on March 15 is a delay on all live video, long enough for review, and a delay long enough for review is a delay that kills the product, because live video that is not live is recorded video, and the companies already have that. So the engineering choice is not between stopping the stream and letting it run. It is between live video with this risk and no live video, and the companies have chosen the first and have not said so, and the regulators have demanded the impossible middle and have not noticed.

So what are the actual responses, measured against what the process can do? Facebook’s policy change on white nationalist content is a rule about what may be posted, enforced by matching and reporting after the fact, and it will work on that content about as well as it works on anything, which is unevenly. YouTube’s admission that it could not keep up with re-uploads is an accurate description of the matching problem under adversarial conditions. Australia’s law, which threatens fines of up to ten percent of revenue for failing to remove abhorrent material expeditiously, will force the companies to spend more on the matching and the human review, which will shorten the window somewhat, and will not touch the first seventeen minutes, because nothing can. New Zealand’s censor has criminalized possession of the file in New Zealand, which the file does not care about. The response that would change the engineering is the one nobody has proposed, which is to decide, as a society, whether live video to the public without delay is a product that should exist, and I suspect the answer is yes, and I suspect nobody wants to say so out loud this month.

I will add one thing from the shop floor, because the announced-policy-versus-what-reaches-the-floor gap is my whole obsession. Every policy announced in the past three weeks, the gun law in New Zealand, the Australian removal law, the Facebook content rule, is aimed at the large platforms and the legal weapon. The manifesto was posted to a message board with a few dozen employees and no advertisers, which has defended itself and is still up, and the video lives now on file-sharing networks that have no owner at all. The policies reach the floor of Facebook and YouTube. They do not reach the floor where the thing was made and where it lives, because that floor is not a company. It is a protocol, and you cannot fine a protocol.

So my answer is that the platforms are responding with the tools they have, which are matching and reporting, and that those tools are being asked to do a thing they cannot do, which is recognize a massacre in progress with no reference copy. Governments are responding by threatening the companies with penalties for failing at that impossible thing, which will make them spend more on the possible things, which is useful and is not what the laws claim to do. And the actual engineering question, whether undelayed public livestreaming should exist, is not being asked, because the answer is uncomfortable for everyone: the companies because delay kills the product, the regulators because banning it is censorship, and the public because it likes the product. I run processes. When a process produces a catastrophic failure that no control can catch, the question is not how to catch it next time. It is whether to run the process. Nobody has asked. The man in Christchurch counted on that too.

Clermont, Florida Correspondent

I follow migration for a living, and the manifesto the gunman left is, among other things, a document about migration, and I want to address that part of it directly, because I think the response to the spread of this extremism depends on getting one distinction right and I have watched the public conversation get it wrong in both directions for three weeks. The gunman’s organizing idea is a theory he calls the great replacement: that the native populations of Western countries are being deliberately replaced by immigrants, that this is an act of war, and that the appropriate response is to kill. That is a murderous fantasy and it is also, in its structure, a distorted version of an argument about immigration that half the democratic world is having in good faith, and the distinction between the fantasy and the argument is the thing the response has to protect, because if the response collapses the two, it hands the fantasy every person who was merely making the argument.

Let me be concrete, because Florida is where the argument lives. This state has absorbed Cubans, Haitians, Puerto Ricans, Venezuelans, and Central Americans at rates that have changed its politics, its labor market, and its schools inside a single generation, and the people who have questions about that, about the pace, about the legal channels, about who pays for the hospitals and the classrooms, are not white nationalists. They are the roofers and the restaurant owners and the retirees I advise, and most of them employ immigrants and half of them are immigrants, and they would like a border that is managed and a visa system that works. The gunman would have regarded every one of them as a traitor, because his theory does not permit a managed border; it permits only purity or war. A response to Christchurch that treats their questions as the first step toward his manifesto is a response that teaches them the two are connected, and some of them, told often enough that they are on his side, will eventually go look at what his side says. That is the mechanism by which a fantasy recruits, and the mechanism is being operated this month by people who think they are fighting it.

Now the responses as they stand. New Zealand has acted with a speed and a dignity that I think everyone has noticed: a gun law in a week, a refusal to name the killer, a prime minister in a headscarf at the mosque. Australia has passed a law threatening the platforms with enormous fines for failing to remove violent material fast enough, drafted in a weekend and already being pulled apart by lawyers. The platforms have tightened rules on white nationalist content and are struggling, as they always do, with the re-uploads. The United States has done nothing, because the First Amendment protects the manifesto and the 1996 statute protects the companies and no Congress will touch either. Others will assess those more closely than I can. My interest is in the one response that is actually within reach of every government and that none of them is talking about, which is to take the legitimate argument back from the fantasy by making it in public, honestly, with numbers, and resolving it through policy.

Here is what I mean. The great replacement theory feeds on two things: the sense that migration is happening without the consent of the people it affects, and the sense that the people raising concerns are being told they may not. Both of those are partly true in most Western countries, and the fantasy converts the partial truth into a total one. A government that controls its border, sets its immigration levels through an open political process, enforces the levels it sets, and treats the people who question the levels as citizens rather than suspects, is a government that has removed the fantasy’s fuel. Australia, oddly, is the best example: it runs one of the harshest border regimes in the democratic world alongside one of the largest planned migration programs, with bipartisan support for both, and the public accepts the migration because it believes the government controls the door. The man who did this grew up there and found the room anyway, which tells you that no policy reaches everyone. But the room is smaller in a country where the argument is allowed than in one where it is forbidden, and the American argument is on its way to being forbidden by people who believe they are protecting the country from the manifesto.

On the President, I will say the measured thing. He was asked on the day whether he saw white nationalism as a rising threat and said he did not, and that was the wrong answer, and the people who have spent three weeks saying his rhetoric on immigration produced the manifesto are making the error I described above, in its most consequential form: they are telling the tens of millions of Americans who agree with him about the border that they are one step from a mosque with a rifle, and some of those Americans will conclude that the people saying so are the enemy the manifesto described. The gunman named the President as a symbol and disclaimed his policies in the same paragraph, because he understood that the symbol would be argued about and the argument would spread his name. It has. Every day of that argument is a day the fantasy spends feeling central.

So my answer is that governments are responding to the weapon and the companies, that the platforms are responding to the content, and that almost no one is responding to the thing the manifesto is actually about, which is a debate over migration that democratic societies are conducting badly and that the fantasy feeds on when it is conducted badly. The response that would work is the one that is hardest for everyone: a border that is controlled, a migration policy that is set openly and enforced, and a public conversation in which the people with questions are answered rather than accused. New Zealand did the parts of that it could. The United States has done none of it, in either direction, and the gap between the roofer in Clermont with a question about the visa system and the man in Christchurch with a camera on his helmet is the distance the response has to protect. This month it has been shrinking the distance. That is the one thing the manifesto asked for that it is getting.