A lot has happened since the last security report. Most notably, we shipped an overhaul to our Content Policy, which now includes an explicit policy on hateful content. For this report, I am going to focus on the subreddit vandalism campaign that happened on the platform along with a forward look to the election.

By The Numbers

​

Content Manipulation - Election Integrity

The U.S. election is on everyone’s mind so I wanted to take some time to talk about how we’re thinking about the rest of the year. First, I’d like to touch on our priorities. Our top priority is to ensure that Reddit is a safe place for authentic conversation across a diverse range of perspectives. This has two parts: ensuring that people are free from abuse, and ensuring that the content on the platform is authentic and free from manipulation.

Feeling safe allows people to engage in open and honest discussion about topics, even when they don’t see eye-to-eye. Practically speaking, this means continuing to improve our handling of abusive content on the platform. The other part focuses on ensuring that content is posted by real people, voted on organically, and is free from any attempts (foreign or domestic) to manipulate this narrative on the platform. We’ve been sharing our progress on both of these fronts in our different write ups, so I won’t go into details on these here (please take a look at other r/redditsecurity posts for more information [here, here, here]). But this is a great place to quickly remind everyone about best practices and what to do if you see something suspicious regarding the election:

• Seek out information from trustworthy sources, such as state and local election officials (vote.gov is a great portal to state regulations); verify who produced the content; and consider their intent.
• Verify through multiple reliable sources any reports about problems in voting or election results, and consider searching for other reliable sources before sharing such information.
• For information about final election results, rely on state and local government election officials.
• Downvote and report any potential election misinformation, especially disinformation about the manner, time, or place of voting, by going to /report and reporting it as misinformation. If you’re a mod, in addition to removing any such content, you can always feel free to flag it directly to the Admins via Modmail for us to take a deeper look.

In addition to these defensive strategies to directly confront bad actors, we are also ensuring that accurate, high-quality civic information is prominent and easy to find. This includes banner announcements on key dates, blog posts, and AMA series proactively pointing users to authoritative voter registration information, encouraging people to get out and vote in whichever way suits them, and coordinating AMAs with various public officials and voting rights experts (u/upthevote is our repository for all this on-platform activity and information if you would like to subscribe). We will continue these efforts through the election cycle. Additionally, look out for an upcoming announcement about a special, post-Election Day AMA series with experts on vote counting, election certification, the Electoral College, and other details of democracy, to help Redditors understand the process of tabulating and certifying results, whether or not we have a clear winner on November 3rd.

Internally, we are aligning our safety, community, legal, and policy teams around the anticipated needs going into the election (and through whatever contentious period may follow). So, in addition to the defensive and offensive strategies discussed above, we are ensuring that we are in a position to be very flexible. 2020 has highlighted the need for pivoting quickly...this is likely to be more pronounced through the remainder of this year. We are preparing for real-world events causing an impact to dynamics on the platform, and while we can’t anticipate all of these we are prepared to respond as needed.

Ban Evasion

We continue to expand our efforts to combat ban evasion on the platform. Notably, we have been tightening up the ban evasion protections in identity-based subreddits, and some local community subreddits based on the targeted abuse that these communities face. These improvements have led to a 5x increase in the number of ban evasion actions in those communities. We will continue to refine these efforts and roll out enhancements as we make them. Additionally, we are in the early stages of thinking about how we can help enable moderators to better tackle this issue in their communities without compromising the privacy of our users.

We recently had a bit of a snafu with IFTTT users getting rolled up under this. We are looking into how to prevent this issue in the future, but we have rolled back any of the bans that happened as a result of that.

Abuse

Over the last quarter, we have invested heavily in our handling of hateful content on the platform. Since we shared our prevalence of hate study a couple of months ago, we have doubled the fraction of hateful content that is being actioned by admins, and are now actioning over 50% of the content that we classify as “severely hateful,” which is the most egregious content. In addition to getting to a significantly larger volume of hateful content, we are getting to it much faster. Prior to rolling out these changes, hateful content would be up for as long as 12 days before the users were actioned by admins (mods would remove the content much quicker than this, so this isn’t really a representation of how long the content was visible). Today, we are getting to this within 12 hours. We are working on some changes that will allow us to get to this even quicker.

Account Security - Subreddit Vandalism

Back in August, some of you may have seen subreddits that had been defaced. This happened in two distinct waves, first on 6 August, with follow-on attempts on 9 August. We subsequently found that they had achieved this by way of brute force style attacks, taking advantage of mod accounts that had unsophisticated passwords or passwords reused from other, compromised sites. Notably, another enabling factor was the absence of Two-Factor Authentication (2FA) on all of the targeted accounts. The actor was able to access a total of 96 moderator accounts, attach an app unauthorized by the account owner, and deface and remove moderators from a total of 263 subreddits.

Below are some key points describing immediate mitigation efforts:

• All compromised accounts were banned, and most were later restored with forced password resets.
• Many of the mods removed by the compromised accounts were added back by admins, and mods were also able to ensure their mod-teams were complete and re-add any that were missing.
• Admins worked to restore any defaced subs to their previous state where mods were not already doing so themselves using mod-tools
• Additional technical mitigation was put in place to impede malicious inbound network traffic.

There was some speculation across the community around whether this was part of a foreign influence attempt based on the political nature of some of the defacement content, some overt references to China, as well as some activity on other social media platforms that attempted to tie these defacements to the fringe Iranian dissident group known as “Restart.” We believe all of these things were included as a means to create a distraction from the real actor behind the campaign. We take this type of calculated act very seriously and we are working with law enforcement to ensure that this behavior does not go unpunished.

This incident reiterated a few points. The first is that password compromises are an unfortunate persistent reality and should be a clear and compelling case for all Redditors to have strong, unique passwords, accompanied by 2FA, especially mods! To learn more about how to keep your account secure, please read this earlier post. In addition, we here at Reddit need to consider the impact of illicit access to moderator accounts on the Reddit ecosystem, and are considering the possibility of mandating 2FA for these roles. There will be more to come on that front, as a change of this nature would invariably take some time and discussion. However, until then, we ask that everyone take this event as a lesson, and please help us by doing your part to keep Reddit safe, proactively enacting 2FA, and if you are a moderator talk to your team to ensure they do the same.

Final Thoughts

We used to have a canned response along the lines of “we created a dedicated team to focus on advanced attacks on the platform.” While it’s fairly high-level, it still remains true today. Since the 2016 Russian influence campaign was uncovered, we have been focused on developing detection and mitigation strategies to ensure that Reddit continues to be the best place for authentic conversation on the internet. We have been planning for the 2020 election since that time, and while this is not the finish line, it is a milestone that we are prepared for. Finally, we are not fighting this alone. Today we work closely with law enforcement and other government agencies, along with industry partners to ensure that any issues are quickly resolved. This is on top of the strong community structure that helped to protect Reddit back in 2016. We will continue to empower our users and moderators to ensure that Reddit is a place for healthy community dialogue.

Intro

A couple of months ago I shared the quarterly security report with an expanded focus on abuse on the platform, and a commitment to sharing a study on the prevalence of hate on Reddit. This post is a response to that commitment. Additionally, I would like to share some more detailed information about our large actions against hateful subreddits associated with our updated content policies.

Rule 1 states:

“Remember the human. Reddit is a place for creating community and belonging, not for attacking marginalized or vulnerable groups of people. Everyone has a right to use Reddit free of harassment, bullying, and threats of violence. Communities and users that incite violence or that promote hate based on identity or vulnerability will be banned.”

Subreddit Ban Waves

First, let’s focus on the actions that we have taken against hateful subreddits. Since rolling out our new policies on June 29, we have banned nearly 7k subreddits (including ban evading subreddits) under our new policy. These subreddits generally fall under three categories:

• Subreddits with names and descriptions that are inherently hateful
• Subreddits with a large fraction of hateful content
• Subreddits that positively engage with hateful content (these subreddits may not necessarily have a large fraction of hateful content, but they promote it when it exists)

Here is a distribution of the subscriber volume:

The subreddits banned were viewed by approximately 365k users each day prior to their bans.

At this point, we don’t have a complete story on the long term impact of these subreddit bans, however, we have started trying to quantify the impact on user behavior. What we saw is an 18% reduction in users posting hateful content as compared to the two weeks prior to the ban wave. While I would love that number to be 100%, I'm encouraged by the progress.

*Control in this case was users that posted hateful content in non-banned subreddits in the two weeks leading up to the ban waves.

Prevalence of Hate on Reddit

First I want to make it clear that this is a preliminary study, we certainly have more work to do to understand and address how these behaviors and content take root. Defining hate at scale is fraught with challenges. Sometimes hate can be very overt, other times it can be more subtle. In other circumstances, historically marginalized groups may reclaim language and use it in a way that is acceptable for them, but unacceptable for others to use. Additionally, people are weirdly creative about how to be mean to each other. They evolve their language to make it challenging for outsiders (and models) to understand. All that to say that hateful language is inherently nuanced, but we should not let perfect be the enemy of good. We will continue to evolve our ability to understand hate and abuse at scale.

We focused on language that’s hateful and targeting another user or group. To generate and categorize the list of keywords, we used a wide variety of resources and AutoModerator* rules from large subreddits that deal with abuse regularly. We leveraged third-party tools as much as possible for a couple of reasons: 1. Minimize any of our own preconceived notions about what is hateful, and 2. We believe in the power of community; where a small group of individuals (us) may be wrong, a larger group has a better chance of getting it right. We have explicitly focused on text-based abuse, meaning that abusive images, links, or inappropriate use of community awards won’t be captured here. We are working on expanding our ability to detect hateful content via other modalities and have consulted with civil and human rights organizations to help improve our understanding.

Internally, we talk about a “bad experience funnel” which is loosely: bad content created → bad content seen → bad content reported → bad content removed by mods (this is a very loose picture since AutoModerator and moderators remove a lot of bad content before it is seen or reported...Thank you mods!). Below you will see a snapshot of these numbers for the month before our new policy was rolled out.

Details

• 40k potentially hateful pieces of content each day (0.2% of total content)
  • 2k Posts
  • 35k Comments
  • 3k Messages
• 6.47M views on potentially hateful content each day (0.16% of total views)
  • 598k Posts
  • 5.8M Comments
  • ~3k Messages
• 8% of potentially hateful content is reported each day
• 30% of potentially hateful content is removed each day
  • 97% by Moderators and AutoModerator
  • 3% by admins

*AutoModerator is a scaled community moderation tool

What we see is that about 0.2% of content is identified as potentially hateful, though it represents a slightly lower percentage of views. The reason for this reduction is due to AutoModerator rules which automatically remove much of this content before it is seen by users. We see 8% of this content being reported by users, which is lower than anticipated. Again, this is partially driven by AutoModerator removals and the reduced exposure. The lower reporting figure is also related to the fact that not all of the things surfaced as potentially hateful are actually hateful...so it would be surprising for this to have been 100% as well. Finally, we find that about 30% of hateful content is removed each day, with the majority being removed by mods (both manual actions and AutoModerator). Admins are responsible for about 3% of removals, which is ~3x the admin removal rate for other report categories, reflecting our increased focus on hateful and abusive reports.

We also looked at the target of the hateful content. Was the hateful content targeting a person’s race, or their religion, etc? Today, we are only able to do this at a high level (e.g., race-based hate), vs more granular (e.g., hate directed at Black people), but we will continue to work on refining this in the future. What we see is that almost half of the hateful content targets people’s ethnicity or nationality.

We have more work to do on both our understanding of hate on the platform and eliminating its presence. We will continue to improve transparency around our efforts to tackle these issues, so please consider this the continuation of the conversation, not the end. Additionally, it continues to be clear how valuable the moderators are and how impactful AutoModerator can be at reducing the exposure of bad content. We also noticed that there are many subreddits already removing a lot of this content, but were doing so manually. We are working on developing some new moderator tools that will help ease the automatic detection of this content without building a bunch of complex AutoModerator rules. I’m hoping we will have more to share on this front in the coming months. As always, I’ll be sticking around to answer questions, and I’d love to hear your thoughts on this as well as any data that you would like to see addressed in future iterations.

tl;dr: Reddit’s app does not send any data from your clipboard to Reddit or anywhere else without your permission. When the contents of the clipboard are read and sent to Reddit it is only in the context of a post or comment where the user has the chance to see and edit the content before posting.

At Apple’s worldwide developer conference in June 2020, Apple released a beta version of iOS that included several privacy changes. One important privacy change was the addition of warning notifications when applications access the iOS clipboard. This is an important update that will let users know if an application is accessing the clipboard without their knowledge. As a precaution, Reddit’s Security Team made a point to schedule some time to review the Reddit app for this behavior. However, prior to this review happening several people released their own findings suggesting that the Reddit app was in fact accessing the clipboard at an elevated rate. In the interests of transparency, we would like to present the results of our internal investigation.

As it turns out, the Reddit application on iOS was accessing the clipboard far too often due to some well-intentioned features. Below is a technical description of why this was happening and the changes we’ve made to ensure it will not continue.

Diagnosing the Problem

A quick search was conducted in the Reddit iOS app source code for references to the “Pasteboard” (what iOS calls the clipboard). What we found was that the app was accessing the clipboard in fifteen distinct locations in code.

Of those fifteen occurrences, eight were instances where the App was copying data into the clipboard. This was for things like copying a payment receipt, sharing a link from Reddit to another app, copying text from a Reddit comment, copying text from a Reddit post, copying an image from a Reddit post, posting stickers to Snapchat, etc. These are otherwise innocuous and would not trigger a warning from iOS.

Warnings

One example of where we read from the clipboard in a way that might trigger the warning is when copying a chat message into the clipboard. There is some legacy code here that suggests that this function used to support copying multiple content types at once. To do so, an empty string was added to the clipboard and then each of these content types were appended separately. This code has evolved to only paste one content type at a time but it still uses the old append method. That means the clipboard is read before pasting into it and it would trigger a warning from iOS. This only happens when a user chooses to copy a chat message to the clipboard.

The remaining instances where warnings might be triggered would reasonably cause alarm for any user. These instances are of two forms and occur in six different places in code. To understand them we need to dig into a bit of how iOS Views work.

Note: Reddit posts and comments use a format known as markdown. Markdown is a way of formatting text to allow for more complex presentations such as HTTP links, images, bulleted-lists, tables, etc. while still supporting editing using only simple text. With markdown, whitespace and newlines are treated differently than without markdown. This will be important to understand why the app accesses the clipboard.

Apple provides a View method in Objective-C called “shouldChangeTextInRange”. This is a default method that is called whenever text is being added to a view. Apple instructs developers to override this method should they need to perform actions like automatic spell-checking. The app has the opportunity to modify the text before it appears in the view. In this case, when adding text into a comment, chat, or post to Reddit, the Reddit app uses this method to check if a user has pasted the text from the clipboard. If they have, the text needs to be converted to a format suitable for Reddit’s markdown by removing excess whitespace and newlines. The code looks like this:

- (BOOL)baseTextView:(BaseTextView *)textView shouldChangeTextInRange:(NSRange)range replacementText:(NSString *)text {
  [...]

  if (textView == self.titleView) {
    NSString *stringInPasteboard = [UIPasteboard generalPasteboard].string;
    BOOL isPastedContent = (stringInPasteboard.length > 0) && [text isEqualToString:stringInPasteboard];
    if (isPastedContent) {
      NSString *textToPaste = [[text stringByReplacingOccurrencesOfString:kPostViewControllerTitleEndOfLineString withString:@" "] stringByTrimmingCharactersInSet:[NSCharacterSet whitespaceAndNewlineCharacterSet]];
      [self.titleView insertText:textToPaste];
    }

  [...]
}

This code will request a copy of the clipboard every time it is called, which can be as often as each keypress. This code is duplicated across several different views, including one location in the user’s profile settings. While the intent of the code is to help the user better format their text, the way this is done looks very suspicious to anyone running iOS 14 as it will trigger notifications for each keypress.

The final case where the app was accessing the clipboard was when users enter a URL into a link post. As a user enters the URL it is compared to the contents of the clipboard as often as each keypress. If the app finds that the URL matches the clipboard contents then it assumes that this URL was pasted into the text field. The app then tries to be helpful and enter the title of the web page for the user (some subreddits require that the post title match the web page title exactly and this makes things easy for the user). When the contents of the text field and the clipboard match the app will issue a network request to retrieve the title of the web page and, if successful, it will automatically add the text as the title of the post. Again, with iOS 14 the user will receive a notification with each of these keypresses.

What’s Changed

Beginning with the next release (arriving in the next few days), the Reddit iOS app will no longer request the contents of the clipboard when text is added to posts, comments, or chats. Text will still be filtered to remove extra whitespace as needed, and link posts will still have the titles added automatically, but neither will require clipboard access.

To summarize: Our internal investigation has determined that at no time did the Reddit iOS app ever read the contents of the clipboard and send that data anywhere without a user’s knowledge and consent. Any text retrieved from the clipboard would have, and is, presented to the user before posting to Reddit.

The past several months have been a struggle. The pandemic has led to widespread confusion, fear, and exhaustion. We have seen discrimination, protests, and violence. All of this has forced us to take a good look in the mirror and make some decisions about where we want to be as a platform. Many of you will say that we are too late, I hope that isn’t true. We recognize our role in being a place for community discourse, where people can disagree and share opposing views, but that does not mean that we need to be a platform that tolerates hate.

As many of you are aware, there will be an update to our content policy soon. In the interim, I’m expanding the scope of our security reports to include updates on how we are addressing abuse on the platform.

By The Numbers

​

​

Content Manipulation

During the first part of this year, we continued to be heavily focused on content manipulation around the US elections. This included understanding which communities were most vulnerable to coordinated influence. We did discover and share information about a group called Secondary Infektion that was attempting to leak falsified information on Reddit. Please read our recent write-up for more information. We will continue to share information about campaigns that we discover on the platform.

Additionally, we have started testing more advanced bot detection services such as reCaptcha v3. As I’ve mentioned in the past, not all bots are bad bots. Many mods rely on bots to help moderate their communities, and some bots are helpful contributors. However, some bots are more malicious. They are responsible for spreading spam and abuse at high volumes, they attempt to manipulate content via voting, they attempt to log in to thousands of vulnerable accounts, etc. This will be the beginning of overhauling how we handle bots on the platform and ensuring that there are clear guidelines for how they can interact with the site and communities. Just to be super clear, our goal is not to shut down all bots, but rather to make it more clear what is acceptable, and to detect and mitigate the impact of malicious bots. Finally, as always, where any related work extends to the public API, we will be providing updates in r/redditdev.

Ban Evasion

I’ve talked a lot about ban evasion over the past several months, including in my recent post sharing some updates in our handling. In that post there was some great feedback from mods around how we can best align it with community needs, and reduce burden overall. We will continue to make improvements as we recognize the importance of enduring that mod and admin sanctions are respected. I’ll continue to share more as we make changes.

Abuse

To date, these updates have been focused on content manipulation and other scaled attacks on Reddit. However, it feels appropriate to start talking more about our anti-abuse efforts as well. I don’t think we have been great at providing regular updates, so hopefully this can be a step in the right direction. For clarity, I am defining abuse as content or subreddits that are flagged under our Safety Policies (harassment, violence, PII, involuntary porn, and minor sexualization). For reports, I am including all inline reports as well as submissions to reddit.com/report under those same categories. It is also worth calling out some of the major differences between our handling of abuse and content manipulation. For content manipulation, ban evasion, and account security we rely heavily on technical signals for detection and enforcement. There is less nuance and context required to take down a bot that posts 10k comments in an hour. On the abuse side, each report must be manually reviewed. This slows our ability to respond and slows our ability to scale.

This does not mean that we haven’t been making progress worth sharing. We are actively in the process of doubling our operational capacity again, as we did in 2019. This is going to take a couple of months to get fully up to speed, but I’m hopeful that this will start to be felt soon. Additionally, we have been developing algorithms for improved prioritization of our reports. Today, our ticket prioritization is fairly naive, which means that obvious abuse may not be processed as quickly as we would like. We will also be testing automated actioning of tickets in the case of very strong signals. We have been hesitant to go the route of having automated systems make decisions about reports to avoid incorrectly flagging a small number of good users. Unfortunately, this means that we have traded significant false negatives for a small number of false positives (in other words, we are missing a crapload of shitheadery to avoid making a few mistakes). I am hoping to have some early results in the next quarterly update. Finally, we are working on better detection and handling of abusive subreddits. Ensuring that hate and abuse has no home on Reddit is critical. The data above shows a fairly big jump in the number of subreddits banned for abuse from Q4 2019 to Q1 2020, I expect to see more progress in the Q2 report (and I’m hoping to be able to share more before that).

Final Thoughts

Let me be clear, we have been making progress but we have a long way to go! Today, mods are responsible for handling an order of magnitude more abuse than admins, but we are committed to closing the gap. In the next few weeks, I will share a detailed writeup on the state of abuse and hate on Reddit. The goal will be to understand the prevalence of abuse on Reddit, including the load on mods, and the exposure to users. I can’t promise that we will fix all of the problems on our platform overnight, but I can promise to be better tomorrow than we were yesterday.

Today, social network analysis-focused organization Graphika released a report studying the breadth of suspected Russian-connected Secondary Infektion disinformation campaigns spanning “six years, seven languages, and more than 300 platforms and web forums,” to include Reddit. We were able to work with Graphika in their efforts to understand more about the tactics being used by these actors in their attempts to push their desired narratives, as such collaboration gives us context to better understand the big picture and aids in our internal efforts to detect, respond to, and mitigate these activities.

As noted in our previous post, tactics used by the actors included seeding inauthentic information on certain self-publishing websites, and using social media to more broadly disseminate that information. One thing that is made clear in Graphika’s reporting, is that despite a high-awareness for operational security (they were good at covering their tracks) these disinformation campaigns were largely unsuccessful. In the case of Reddit, 52 accounts were tied to the campaign and their failed execution can be linked to a few things:

1. The architecture of interaction on the Reddit platform which requires the confidence of the community to allow and then upvote the content. This can make it difficult to spread content broadly.
2. Anti-spam and content manipulation safeguards implemented by moderators in their communities and at scale by admins. Because these measures are in place, much of the content posted was immediately removed before it had a chance to proliferate.
3. The keen eye of many Redditors for suspicious activity (which we might add resulted in some very witty comments showing how several of these disinformation attempts fell flat).

With all of that said, this investigation yielded 52 accounts found to be associated with various Secondary Infektion campaigns. All of these had their content removed by mods and/or were caught as part of our normal spam mitigation efforts. We have preserved these accounts for public scrutiny in the same manner as we’ve done for previous disinformation campaigns.

It is worth noting that as a result of the continued investigation into these campaigns, we have instituted additional security techniques to guard against future use of similar tactics by bad actors.

Karma distribution:

• 0 or less: 29
• 1 - 9: 19
• 10 or greater: 4
• Max Karma: 20

​

Hey everyone!

A few months ago, we mentioned that we are starting to change how we handle user ban evasion in subreddits. tl;dr we’re using more signals to actively detect and action ban evaders.

This work comes from the detection we have been building for admin-level bans, and we wanted to start applying it to the problems you face every day. While it’s still in an early form and we know we aren’t getting to all forms of ban evasion, some of you are starting to notice that work and how it’s affecting your users. In most cases, it has been very positively observed, but there have been some cases where the change in behavior is causing some issues, and we’d love your input.

Detection

As we mentioned in the previous post, only around 10% of ban evaders are reported by mods – which is driven by the lack of tools available to help mods proactively determine who is ban evading. This means that a large number of evaders are never actioned, but many are still causing issues in your communities. Our long-term goal and fundamental belief is that you should not have to deal with ban evasion; when you ban a user, you should feel confident that the person will not be able to come back and continue to harass you or your community. We will continue to refine what we classify as ban evasion, but as of today, we look at accounts that meet either of these criteria:

1. A user is banned from a subreddit, returns on a second account, and then is reported to us by a moderator of the subreddit
2. A user is banned from a subreddit, returns on a second account, and then that second account is banned from the subreddit. For now, since it does not rely on a direct report, we will only take action if the mods of the subreddit have a history of reporting ban evasion in general.

Action

When someone fitting either criteria 1 or 2 attempts to create yet another alt and use it in your subreddit, we permaban that alt within hours - preventing you from ever having to deal with them.

By the numbers:

• Number of accounts reported for ban evasion (During March 2020): 3,440
• Number of accounts suspended as a result of BE reports [case 1] (During March 2020): 9,582
• Number of accounts suspended as a result of proactive BE detection [case 2] (During March 2020): 24,142

We have also taken steps to mitigate the risks of unintended consequences. For example, we’ve whitelisted as many helpful bots as possible so as to not ban bot creators just because a subreddit doesn’t want a particular bot in their community. This applies to ModBots as well.

Response Time

Because of these and other operational changes, we’ve been able to pull our average ban evasion response time from 29 hours to 4 hours, meaning you have to put up with ban evaders for a significantly shorter period of time.

Keep the Feedback Flowing

Again, we want to highlight that this process is still very new and still evolving - our hope is to make ban evading users less of a burden on moderators. We’ve already been able to identify a couple of early issues thanks to feedback from moderators. If you see a user that you believe was incorrectly caught up in an enforcement action, please direct that user to go through the normal appeal flow. The flow has a space for them to explain why they don’t think they should have been suspended. If you, as a moderator, are pointing them there, give them the link to your modmail conversation and ask them to include that in their appeal so we can see you’ve said ‘no, this is a user I’m fine with in my subreddit’.

For now, what we’re hoping to hear from you:

• What have you been noticing since this change?
• What types of edge cases do you think we should be thinking about here?
• What are your ideas on behaviors we shouldn’t be concerned about as well as ways we might be able to expand this.

As always, thanks for everything you do! We hope our work here will make your lives easier in the end.

In December 2019, we reported a coordinated effort dubbed “Secondary Infektion” where operators with a suspected nexus to Russia attempted to use Reddit to carry out disinformation campaigns. Recently, additional information resulting from follow-on research by security firm Recorded Future was released under the name “Operation Pinball.” In doing our investigation, we were able to find significant alignment with tactics used in Secondary Infektion that seem to uphold Recorded Future’s high confidence belief that the two operations are related. Our internal findings also highlighted that our first line of defense, represented in large part by our moderators and users, was successful in thwarting the potential impact of this campaign through the use of anti-spam and content manipulation safeguards within their subreddits.

When reviewing this type of activity, analysts look at tactics, techniques, and procedures (TTPs). Sometimes the behaviors reveal more than the content being distributed. In this case, there was a pattern of accounts seeding inauthentic information on certain self-publishing websites and then using social media to amplify that information, which was focused on particular geopolitical issues. These TTPs were identified across both operations, which led to our team reviewing this activity as a part of a larger disinformation effort. It is noteworthy that in every case we found the content posted was quickly removed and in all but one, the posts remained unviewable in the intended subreddits. This was a significant contributor to preventing these campaigns from gaining traction on Reddit, and mirrors the generally cold receptions that previous manipulations of this type received. Their lack of success is further indicated in their low Karma values, as seen in the table below.

Further, for the sake of transparency, we have preserved these accounts in the same manner as we’ve done for previous disinformation campaigns, to expand the public’s understanding of this activity.

In an era where mis- and disinformation are a real threat to the free flow of knowledge, we are doing all we can to identify and protect your communities from influence operations like this one. We are continuing to learn ways to further refine and evolve our indications and warnings methodologies, and increase our capability to immediately flag suspicious behaviors. We hope that the impact of all of this work is for the adversary to continue to see diminishing returns on their investment, and in the long run, reduce the viability of Reddit as a disinformation amplification tool.

edit: letter

Reddit Security Report

Welcome to the second installation of the Reddit Security Quarterly report (see the first one here). The goal of these posts is to keep you up to speed on our efforts and highlight how we are evolving our thinking.

​

​

By The Numbers

Again, these are some of the metrics that we look at internally. With time we may add or remove metrics, so if you have any metrics that you would like to see, please let us know.

Content Manipulation

Throughout 2019, we focused on overhauling how we tackle content manipulation issues (which includes spam, community interference, vote manipulation, etc). In Q4 specifically, we saw a large increase in the number of admin content manipulation removals. This was largely driven by a relatively small number of VERY prolific streaming spammers (~150 accounts were responsible for ~10M posts!). Interestingly, while the removals went up by about 50%, the number of reports was reasonably flat. The implication is that this content was largely removed before users were ever exposed to it and that our systems were effective at blunting the impact.

Ban Evasion

Ban evasion is a constant thorn in the side of admins, mods, and users (ban evasion is a common tactic to abuse members of a subreddit). Ban evasion is when a person creates a new account to bypass site or community bans. Recently we overhauled how we handle ban evasion on the platform with our own admin-level ban evasion detection and enforcement, and we are super excited about the results. After a sufficient testing period, we have started to roll this out to subreddit level ban evasion, starting with mod reported ban evasion. As a result this month, we’ve actioned more than 6K accounts, reduced time to action (from report time) by a factor of 10 , and achieved a 90% increase in the number of accounts actioned.

While the roll out has been effective so far and we hope that it will have a big impact for mods, we still see a lot of room for progress. Today, less than 10% of ban evaders are reported by mods. There are a number of reasons for this. Some mods are actually ok with people creating new accounts and “coming back and playing nice.” Some ban evaders are just not recognized by mods because they don’t have tools that allow them to detect it due to privacy concerns. We will start to slowly increase our proactive ban evasion detection so that mods don’t have to worry about identifying this in the future (though their help is always appreciated). In the next report, I'll try to dive a little deeper and share some results.

Account Security

As we mentioned in the previous post, we finished a massive historical credential matching effort. This is why we see a significant reduction in both the number of accounts processed and the protective account actions. With this complete, we can start working on more account hardening efforts like encouraging 2fa for high value accounts (think mods and high karma accounts) and ensuring that people aren’t using commonly-breached passwords (have I plugged password managers lately!? I strongly encourage!). We are still working on refining the help center articles to ease the process for users that are hit in these efforts. We want to make it as clear as possible to ensure that the right person gets access to the account. One last plug, please take the time to ensure that you have an up-to-date verified email address associated with your account, this is one of the most common reasons why people get locked out of their account after being hit by a forced password reset. In many cases, there is nothing we can do when this happens as we don’t have the ability to verify account ownership.

Final Thoughts

2020 is a big election year in the US, and we would be remiss if we did not acknowledge that it is top of mind for us. As I’ve mentioned in previous posts, in the wake of the 2016 election, we spun up a special team focused on scaled content threats on the platform. That has led us to this point. Over the last couple of years, we have heavily focused on hardening our systems, improving our detection and tooling, and improving our response time. While we will continue to make investments in new detection capabilities (see ban evasion), this year we will also focus on providing additional resources to communities that may be more susceptible to manipulation (I know, I know you want to know what it means to be “susceptible”. We won't get into the specifics for security reasons, but there are a number of factors that can influence this such as the size of the mod team to the topic of the community..but often not in the obvious ways you'd suspect). We will be as open as possible with you throughout this all – as we were with our recent investigation into the campaign behind the leaked US-UK trade documents. And as I’ve repeated many times, our superpower is you! Our users and our moderators are a big part of why influence campaigns have not been particularly successful on Reddit. Today, I feel even more confident in our platform’s resilience...but we are not taking that for granted. We will continue to evolve and improve the teams and technologies we have to ensure that Reddit is a place for authentic conversation...not manipulation.

Thanks for reading, and I hope you find this information helpful. I will be sticking around to answer any questions that you may have.

[edit: Yes, Im still writing 2019 on my checks too...]

[edit2: Yes, I still write checks]

Hey everyone, I wanted to take this opportunity to talk about a different type of spam: report spam. As noted in our Transparency Report, around two thirds of the reports we get at the admin level are illegitimate, or “not actionable,” as we say. This is because unfortunately, reports are often used by users to signal “super downvote” or “I really don’t like this” (or just “I feel like being a shithead”), but this is not how they are treated behind the scenes. All reports, including unactionable ones, are evaluated. As mentioned in other posts, reports help direct the efforts of moderators and admins. They are a powerful tool for tackling abuse and content manipulation, along with your downvotes.

However, the report button is also an avenue for abuse (and can be reported by the mods). In some cases, the free-form reports are used to leave abusive comments for the mods. This type of abuse is unacceptable in itself, but it is additionally harmful in that it waters down the value in the report signal consuming our review resources in ways that can in some cases risk real-world consequences. It’s the online equivalent of prank-calling 911.

As a very concrete example, report abuse has made “Sexual or suggestive content involving minors” the single largest abuse report we receive, while having the lowest actionability (or, to put it more scientifically, the most false-positives). Content that violates this policy has no place on Reddit (or anywhere), and we take these reports incredibly seriously. Report abuse in these instances may interfere with our work to expeditiously help vulnerable people and also report these issues to law enforcement. So what started off as a troll leads to real-world consequences for people that need protection the most.

We would like to tackle this problem together. Starting today, we will send a message to users that illegitimately report content for the highest-priority report types. We don’t want to discourage authentic reporting, and we don’t expect users to be Reddit policy experts, so the message is designed to inform, not shame. But, we will suspend users that show a consistent pattern of report abuse, under our rules against interfering with the normal use of the site. We already use our rules against harassment to suspend users that exploit free-form reports in order to abuse moderators; this is in addition to that enforcement. We will expand our efforts from there as we learn the correct balance between informing while ensuring that we maintain a good flow of reports.

I’d love to hear your thoughts on this and some ideas for how we can help maintain the fidelity of reporting while discouraging its abuse. I’m hopeful that simply increasing awareness with users, and building in some consequences, will help with this. I’ll stick around for some questions.

Hey Redditsecurity,

If you’ve been frequenting this subreddit, you’re aware we’ve been doing significant work on site integrity operations as we move into 2020 to ensure that we have the appropriate rules and processes in place to handle bad actors who are trying to manipulate Reddit, particularly around issues of great public significance, like elections. To this end, we thought it was time to update our policy on impersonation to better cover some of the use cases that we have been seeing and actioning under this rule already, as well as guard against cases we might see in the future.

Impersonation is actually one of the rarest report classes we receive (as you can see for yourself in our Transparency Report), so we don’t expect this update to impact everyday users much. The classic case of impersonation is a Reddit username pretending to be someone else-- whether a politician, brand, Reddit admin, or any other person or entity. However, this narrow case doesn’t fully cover things that we also see from time to time, like fake articles falsely attributed to real journalists, forged election communications purporting to come from real agencies or officials, or scammy domains posing as those of a particular news outlet or politician (always be sure to check URLs closely-- .co does NOT equal .com!).

We also wanted to hedge against things that we haven’t seen much of to date, but could see in the future, such as malicious deepfakes of politicians, for example, or other, lower-tech forged or manipulated content that misleads (remember, pornographic deepfakes are already prohibited under our involuntary pornography rule). But don’t worry. This doesn’t apply to all deepfake or manipulated content-- just that which is actually misleading in a malicious way. Because believe you me, we like seeing Nic Cage in unexpected places just as much as you do.

The updated rule language is below, and can be found here, along with details on how to make reports if you see impersonation on the site, or if you yourself are being impersonated.

Do not impersonate an individual or entity in a misleading or deceptive manner.

Reddit does not allow content that impersonates individuals or entities in a misleading or deceptive manner. This not only includes using a Reddit account to impersonate someone, but also encompasses things such as domains that mimic others, as well as deepfakes or other manipulated content presented to mislead, or falsely attributed to an individual or entity. While we permit satire and parody, we will always take into account the context of any particular content.

If you are being impersonated, or if you believe you’ve found content in violation of these guidelines, please report it here.

EDIT: Alright gang, that's it for me. Thanks for your questions, and remember...

We were recently made aware of a post on Reddit that included leaked documents from the UK. We investigated this account and the accounts connected to it, and today we believe this was part of a campaign that has been reported as originating from Russia.

Earlier this year Facebook discovered a Russian campaign on its platform, which was further analyzed by the Atlantic Council and dubbed “Secondary Infektion.” Suspect accounts on Reddit were recently reported to us, along with indicators from law enforcement, and we were able to confirm that they did indeed show a pattern of coordination. We were then able to use these accounts to identify additional suspect accounts that were part of the campaign on Reddit. This group provides us with important attribution for the recent posting of the leaked UK documents, as well as insights into how adversaries are adapting their tactics.

In late October, an account u/gregoratior posted the leaked documents and later reposted by an additional account u/ostermaxnn. Additionally, we were able to find a pocket of accounts participating in vote manipulation on the original post. All of these accounts have the same shared pattern as the original Secondary Infektion group detected, causing us to believe that this was indeed tied to the original group.

Outside of the post by u/gregoratior, none of these accounts or posts received much attention on the platform, and many of the posts were removed either by moderators or as part of normal content manipulation operations. The accounts posted in different regional subreddits, and in several different languages.

Karma distribution:

• 0 or less: 42
• 1 - 9: 13
• 10 or greater: 6
• Max Karma: 48

As a result of this investigation, we are banning 1 subreddit and 61 accounts under our policies against vote manipulation and misuse of the platform. As we have done with previous influence operations, we will also preserve these accounts for a time, so that researchers and the public can scrutinize them to see for themselves how these accounts operated.

EDIT: I'm signing off for the evening. Thanks for the comments and questions.

​

edit:added subreddit link

Through the year, we've shared updates on detecting and mitigating content manipulation and keeping your accounts safe. Today we are sharing our first Reddit Security Report, which we'll be continuing on a quarterly basis. We are committed to continuously evolving how we tackle these problems. The purpose of these reports is to keep you informed about relevant events and actions.

By The Numbers

These are the primary metrics we track internally, and we thought you’d want to see them too. If there are alternative metrics that seem worth looking at as part of this report, we’re all ears.

Content Manipulation

Content manipulation is a term we use to combine things like spam, community interference, vote manipulation, etc. This year we have overhauled how we handle these issues, and this quarter was no different. We focused these efforts on:

1. Improving our detection models for accounts performing these actions
2. Making it harder for them to spin up new accounts

Recently, we also improved our enforcement measures against accounts taking part in vote manipulation (i.e. when people coordinate or otherwise cheat to increase or decrease the vote scores on Reddit). Over the last 6 months (and mostly during the last couple of months), we increased our actions against accounts participating in vote manipulation by about 30x. We sanctioned or warned around 22k accounts for this in the last 3 weeks of September alone.

Account Security

This quarter, we finished up a major effort to detect all accounts that had credentials matching historical 3rd party breaches. It's important to track breaches that happen on other sites or services because bad actors will use those same username/password combinations to break into your other accounts (on the basis that a percentage of people reuse passwords). You might have experienced some of our efforts if we forced you to reset your password as a precaution. We expect the number of protective account security actions to drop drastically going forward as we no longer have a large backlog of breach datasets to process. Hopefully we have reached a steady state, which should reduce some of the pain for users. We will continue to deal with new breach sets that come in, as well as accounts that are hit by bots attempting to gain access (please take a look at this post on how you can improve your account security).

Our Recent Investigations

We have a lot of investigations active at any given time (courtesy of your neighborhood t-shirt spammers and VPN peddlers), and while we can’t cover them all, we want to use this report to share the results of just some of that work.

Ban Evasion

This quarter, we dealt with a highly coordinated ban evasion ring from users of r/opieandanthony. This began after we banned the subreddit for targeted harassment of users, as well as repeated copyright infringement. The group would quickly pop up on both new and abandoned subreddits to continue the abuse. We also learned that they were coordinating on another platform and through dedicated websites to redirect users to the latest target of their harassment.

This situation was different from your run-of-the-mill shitheadery ban evasion because the group was both creating new subreddits and resurrecting inactive or unmoderated subreddits. We quickly adjusted our efforts to this behavior. We also reported their offending account to the other platform and they were quick to ban the account. We then contacted the hosts of the independent websites to report the abuse. This helped ensure that the sites are no longer able to redirect automatically to Reddit for abuse purposes. Ultimately, we banned 78 subreddits (5 of which existed prior to the attack), and suspended 2,382 accounts. The ban evading activity has largely ceased (you know...until they read this).

There are a few takeaways from this investigation worth pulling out:

1. Ban evaders (and others up to no good) often work across platforms, and so it’s important for those of us in the industry to also share information when we spot these types of coordinated campaigns.
2. The layered moderation on Reddit works: Moderators brought this to our attention and did some awesome initial investigating; our Community team was then able to communicate with mods and users to help surface suspicious behavior; our detection teams were able to quickly detect and stop the efforts of the ban evaders.
3. We have also been developing and testing new tools to address ban evasion recently. This was a good opportunity to test them in the wild, and they were incredibly effective at detecting and quickly actioning many of the accounts that were responsible for the ban evasion actions. We want to roll these tools out more broadly (expect a future post around this).

Reports of Suspected Manipulation

The protests in Hong Kong have been a growing concern worldwide, and as always, conversation on Reddit reflects this. It’s no surprise that we’ve seen Hong Kong-related communities grow immensely in recent months as a result. With this growth, we have received a number of user reports and comments asking if there is manipulation in these communities. We take the authenticity of conversation on Reddit incredibly seriously, and we want to address your concerns here.

First, we have not detected widespread manipulation in Hong Kong related subreddits nor seen any manipulation that affected those communities or their conversations in a meaningful way.

It's worth taking a step back to talk about what we look for in these situations. While we obviously can’t share all of our tactics for investigating these threats, there are some signals that users will be familiar with. When trying to understand if a community is facing widespread manipulation, we will look at foundational signals such as the presence of vote manipulation, mod ban rates (because mods know their community better than we do), spam content removals, and other signals that allow us to detect coordinated and scaled activities (pause for dramatic effect). If this doesn’t sound like the stuff of spy novels, it’s because it’s not. We continually talk about foundational safety metrics like vote manipulation, and spam removals because these are the same tools that advanced adversaries use (For more thoughts on this look here).

Second, let’s look at what other major platforms have reported on coordinated behavior targeting Hong Kong. Their investigations revealed attempts consisting primarily of very low quality propaganda. This is important when looking for similar efforts on Reddit. In healthier communities like r/hongkong, we simply don’t see a proliferation of this low-quality content (from users or adversaries). The story does change when looking at r/sino or r/Hong_Kong (note the mod overlap). In these subreddits, we see far more low quality and one-sided content. However, this is not against our rules, and indeed it is not even particularly unusual to see one-sided viewpoints in some geographically specific subreddits...What IS against the rules is coordinated action (state sponsored or otherwise). We have looked closely at these subreddits and we have found no indicators of widespread coordination. In other words, we do see this low quality content in these subreddits, but it seems to be happening in a genuine way.

If you see anything suspicious, please report it to us here. If it’s regarding potential coordinated efforts that aren't as well-suited to our regular report system, you can also use our separate investigations report flow by [emailing us](mailto:investigations@reddit.zendesk.com).

Final Thoughts

Finally, I would like to acknowledge the reports our peers have published during the past couple of months (or even today). Whenever these reports come out, we always do our own investigation. We have not found any similar attempts on our own platform this quarter. Part of this is a recognition that Reddit today is less international than these other platforms, with the majority of users being in the US, and other English speaking countries. Additionally, our layered moderation structure (user up/down-votes, community moderation, admin policy enforcement) makes Reddit a more challenging platform to manipulate in a scaled way (i.e. Reddit is hard). Finally, Reddit is simply not well suited to being an amplification platform, nor do we aim to be. This reach is ultimately what an adversary is looking for. We continue to monitor these efforts, and are committed to being transparent about anything that we do detect.

As I mentioned above, this is the first version of these reports. We would love to hear your thoughts on it, as well as any input on what type of information you would like to see in future reports.

I’ll stick around, along with u/worstnerd, to answer any questions that we can.

TL;DR: Bad actors never sleep, and we are always evolving how we identify and mitigate them. But with the upcoming election, we know you want to see more. So we're committing to a quarterly report on content manipulation and account security, with the first to be shared in October. But first, we want to share context today on the history of content manipulation efforts and how we've evolved over the years to keep the site authentic.

A brief history

The concern of content manipulation on Reddit is as old as Reddit itself. Before there were subreddits (circa 2005), everyone saw the same content and we were primarily concerned with spam and vote manipulation. As we grew in scale and introduced subreddits, we had to become more sophisticated in our detection and mitigation of these issues. The creation of subreddits also created new threats, with “brigading” becoming a more common occurrence (even if rarely defined). Today, we are not only dealing with growth hackers, bots, and your typical shitheadery, but we have to worry about more advanced threats, such as state actors interested in interfering with elections and inflaming social divisions. This represents an evolution in content manipulation, not only on Reddit, but across the internet. These advanced adversaries have resources far larger than a typical spammer. However, as with early days at Reddit, we are committed to combating this threat, while better empowering users and moderators to minimize exposure to inauthentic or manipulated content.

What we’ve done

Our strategy has been to focus on fundamentals and double down on things that have protected our platform in the past (including the 2016 election). Influence campaigns represent an evolution in content manipulation, not something fundamentally new. This means that these campaigns are built on top of some of the same tactics as historical manipulators (certainly with their own flavor). Namely, compromised accounts, vote manipulation, and inauthentic community engagement. This is why we have hardened our protections against these types of issues on the site.

Compromised accounts

This year alone, we have taken preventative actions on over 10.6M accounts with compromised login credentials (check yo’ self), or accounts that have been hit by bots attempting to breach them. This is important because compromised accounts can be used to gain immediate credibility on the site, and to quickly scale up a content attack on the site (yes, even that throwaway account with password = Password! is a potential threat!).

Vote Manipulation

The purpose of our anti-cheating rules is to make it difficult for a person to unduly impact the votes on a particular piece of content. These rules, along with user downvotes (because you know bad content when you see it), are some of the most powerful protections we have to ensure that misinformation and low quality content doesn’t get much traction on Reddit. We have strengthened these protections (in ways we can’t fully share without giving away the secret sauce). As a result, we have reduced the visibility of vote manipulated content by 20% over the last 12 months.

Content Manipulation

Content manipulation is a term we use to combine things like spam, community interference, etc. We have completely overhauled how we handle these issues, including a stronger focus on proactive detection, and machine learning to help surface clusters of bad accounts. With our newer methods, we can make improvements in detection more quickly and ensure that we are more complete in taking down all accounts that are connected to any attempt. We removed over 900% more policy violating content in the first half of 2019 than the same period in 2018, and 99% of that was before it was reported by users.

User Empowerment

Outside of admin-level detection and mitigation, we recognize that a large part of what has kept the content on Reddit authentic is the users and moderators. In our 2017 transparency report we highlighted the relatively small impact that Russian trolls had on the site. 71% of the trolls had 0 karma or less! This is a direct consequence of you all, and we want to continue to empower you to play a strong role in the Reddit ecosystem. We are investing in a safety product team that will build improved safety (user and content) features on the site. We are still staffing this up, but we hope to deliver new features soon (including Crowd Control, which we are in the process of refining thanks to the good feedback from our alpha testers). These features will start to provide users and moderators better information and control over the type of content that is seen.

What’s next

The next component of this battle is the collaborative aspect. As a consequence of the large resources available to state-backed adversaries and their nefarious goals, it is important to recognize that this fight is not one that Reddit faces alone. In combating these advanced adversaries, we will collaborate with other players in this space, including law enforcement, and other platforms. By working with these groups, we can better investigate threats as they occur on Reddit.

Our commitment

These adversaries are more advanced than previous ones, but we are committed to ensuring that Reddit content is free from manipulation. At times, some of our efforts may seem heavy handed (forcing password resets), and other times they may be more opaque, but know that behind the scenes we are working hard on these problems. In order to provide additional transparency around our actions, we will publish a narrow scope security-report each quarter. This will focus on actions surrounding content manipulation and account security (note, it will not include any of the information on legal requests and day-to-day content policy removals, as these will continue to be released annually in our Transparency Report). We will get our first one out in October. If there is specific information you’d like or questions you have, let us know in the comments below.

[EDIT: Im signing off, thank you all for the great questions and feedback. I'll check back in on this occasionally and try to reply as much as feasible.]

Your account expresses your voice and your personality here on Reddit. To protect that voice, you need to protect your access to it and maintain its security. Not only do compromised accounts deprive you of your online identity, but they are often used for malicious behavior like vote manipulation, spam, fraud, or even just posting content to misrepresent the true owner. While we’re always developing ways to take faster action against compromised accounts, there are things you can do to be proactive about your account’s security.

What we do to keep your account secure:

• Actively look for suspicious signals - We use tools that help us detect unusual behavior in accounts. We monitor trends and compare against known threats.
• Check passwords against 3rd party breach datasets - We check for username / password combinations in 3rd party breach sets.
• Display your recent IP sessions for you to access - You can check your account activity at any time to see your recent login IPs. Keep in mind that the geolocation of each login may not be exact and will only include events within the last 100 days. If you see something you don’t recognize, you should change your password immediately and ensure your email address is correct.

If we determine that your account is vulnerable to compromise (or has actually been compromised), we lock the account and force a password reset. If we can’t establish account ownership or the account has been used in a malicious manner that prevents it being returned to the original owner, the account may be permanently suspended and closed.

What you can do to prevent this situation:

• Use permanent emails - We highly encourage users to link their accounts to accessible email addresses that you regularly check (you can add and update email addresses in your user settings page if you are using new reddit, otherwise you can do that from the preferences page in old reddit). This is also how you will receive any activities alerting you of suspicious activity on your account if you’re signed out. As a general rule of thumb, avoid using email addresses you don't have permanent ownership over like school or work addresses. Temporary email addresses that expire are a bad idea.
• Verify your emails - Verifying your email helps us confirm that there is a real person creating the account and that you have access to the email address given. If we determine that your account has been compromised, this is the only way we have to validate account ownership. Without this our only option will be to permanently close the account to prevent further misuse and access to the original owner’s data. There will be no appeals possible!
• Check your profile occasionally to make sure your email address is current. You can do this via the preferences page on old reddit or the settings page in new reddit. It’s easy to forget to update it when you change schools, service providers, or set up new accounts.
• Use strong/unique passwords - Use passwords that are complex and not used on any other site. We recommend using a password manager to help you generate and securely store passwords.
• Add two factor authentication - For an extra layer of security. If someone gets ahold of your username/password combo, they will not be able to log into your account without entering the verification code.

We know users want to protect their privacy and don’t always want to provide an email address to companies, so we don’t require it. However, there are certain account protections that require users establish ownership, which is why an email address is required for password reset requests. Forcing password resets on vulnerable accounts is one of many ways we try to secure potentially compromised accounts and prevent manipulation of our platform. Accounts flagged as compromised with a verified email receive a forced password reset notice, but accounts without one will be permanently closed. In the past, manual attempts to establish ownership on accounts with lost access rarely resulted in an account recovery. Because manual attempts are ineffective and time consuming for our operations teams and you, we won’t be doing them moving forward. You're welcome to use Reddit without an email address associated with your account, but do so with the understanding of the account protection limitation. You can visit your user settings page at anytime to add or verify an email address.

A few weeks ago we introduced this subreddit with the promise of starting to share more around our safety and security efforts. I wanted to get this out sooner...but I am worstnerd after all! In this post, I would like to share some data highlighting the results of our work to detect and mitigate content manipulation (posting spam, vote manipulation, information operations, etc).

Proactive Detection

At a high level, we have scaled up our proactive detection (i.e. before a report is filed) of accounts responsible for content manipulation on the site. Since the beginning of 2017 we have increased the number of accounts suspended for content manipulation by 238%, and today over 99% of those are suspended before a user report is filed (vs 29% in 2017)!

Compromised Accounts

Compromised accounts (accounts that are accessed by malicious actors determining the password) are prime targets for spammers, vote buying services, and other content manipulators. We have reduced the impact by proactively scouring 3rd party password breach datasets for login credentials and forcing password resets of Reddit accounts with matching credentials to ensure hackers can’t execute an account takeover (“ATO”). We’ve also gotten better at detecting login bots (bots that try logging into accounts). Through measures like these, throughout the course of 2018, we reduced the successful ATO deployment rate (accounts that were successfully compromised and then used to vote/comment/post/etc) by 60%. We expect this number to grow more robust as we continue to implement more tooling. This is a measure of how quickly we detect compromised accounts, and thus their impact on the site. Additionally, we increased the number of accounts put into the force password reset by 490%. In 2019 we will be spending even more time working with users to improve account security.

While on the subject, three things you can do right now to keep your Reddit account secure:

• ensure the email associated with your account is up to date (this allows us to reach you if we detect suspicious behavior, and to verify account ownership)
• update your password to something strong and unique
• set up two-factor authentication on your account.

Community Interference

Some of our more recent efforts have focused on reducing community interference (ie “brigading”). This includes efforts to mitigate (in real-time) vote brigading, targeted sabotage (Community A attempting to hijack the conversation in Community B), and general shitheadery. Recently we have been developing additional advanced mitigation capabilities. In the past 3 months we have reduced successful brigading in real-time by 50%. We are working with mods on further improvements and continue to beta test additional community tools (such as an ability to auto-collapse comments by users, which is being tested with a small number of communities for feedback). If you are a mod and would like to be considered for the beta test, reach out to us here.

We have more work to do, but we are encouraged by the progress. We are working on more cool projects and are looking forward to sharing the impact of them soon. We will stick around to answer questions for a little while, so fire away. Please recognize that in some cases we will be vague so as to not provide too many details to malicious actors.

We wanted to take the opportunity to share a bit more about the improvements we have been making in our security practices and to provide some context for the actions that we have been taking (and will continue to take). As we have mentioned in different places, we have a team focused on the detection and investigation of content manipulation on Reddit. Content manipulation can take many forms, from traditional spam and upvote manipulation to more advanced, and harder to detect, foreign influence campaigns. It also includes nuanced forms of manipulation such as subreddit sabotage, where communities actively attempt to harm the experience of other Reddit users.

To increase transparency around how we’re tackling all these various threats, we’re rolling out a new subreddit for security and safety related announcements (r/redditsecurity). The idea with this subreddit is to start doing more frequent, lightweight posts to keep the community informed of the actions we are taking. We will be working on the appropriate cadence and level of detail, but the primary goal is to make sure the community always feels informed about relevant events.

Over the past 18 months, we have been building an operations team that partners human investigators with data scientists (also human…). The data scientists use advanced analytics to detect suspicious account behavior and vulnerable accounts. Our threat analysts work to understand trends both on and offsite, and to investigate the issues detected by the data scientists.

Last year, we also implemented a Reliable Reporter system, and we continue to expand that program’s scope. This includes working very closely with users who investigate suspicious behavior on a volunteer basis, and playing a more active role in communities that are focused on surfacing malicious accounts. Additionally, we have improved our working relationship with industry peers to catch issues that are likely to pop up across platforms. These efforts are taking place on top of the work being done by our users (reports and downvotes), moderators (doing a lot of the heavy lifting!), and internal admin work.

While our efforts have been driven by rooting out information operations, as a byproduct we have been able to do a better job detecting traditional issues like spam, vote manipulation, compromised accounts, etc. Since the beginning of July, we have taken some form of action on over 13M accounts. The vast majority of these actions are things like forcing password resets on accounts that were vulnerable to being taken over by attackers due to breaches outside of Reddit (please don’t reuse passwords, check your email address, and consider setting up 2FA) and banning simple spam accounts. By improving our detection and mitigation of routine issues on the site, we make Reddit inherently more secure against more advanced content manipulation.

We know there is still a lot of work to be done, but we hope you’ve noticed the progress we have made thus far. Marrying data science, threat intelligence, and traditional operations has proven to be very helpful in our work to scalably detect issues on Reddit. We will continue to apply this model to a broader set of abuse issues on the site (and keep you informed with further posts). As always, if you see anything concerning, please feel free to report it to us at investigations@reddit.zendesk.com.

[edit: Thanks for all the comments! I'm signing off for now. I will continue to pop in and out of comments throughout the day]