Bluesky's Open Source Moderation Tools: LLM-based Event Detection in Python

Bluesky is a decentralized social media application built on top of the AT Protocol. One way that Bluesky supports decentralization, and empowers users in the Atmosphere community to run their own unique AT apps, is through open source moderation tools. We do this through our Jetstream, which lets you stream AT protocol data collections into any other application or interface. The goal is to get records off the network and in front of automated rules and human reviewers, with no glue service in the middle.

For example, our moderation stack consists of Osprey, an event stream decisions engine and analysis UI designed to investigate and take automatic action, and Ozone, a web frontend for making moderation decisions. Osprey is written in Python, and was designed and open sourced in collaboration with Discord. Osprey evaluates events using structured logic, user-defined functions, and external signals to assign labels, verdicts, and actions. Some of these, such as our Toxrank model for toxicity detection, make use of fine-tuned LLMs and other classifiers that expose their own web endpoints to Osprey. We also utilize image OCR and hashing to create actionable moderation metadata, which can be automatically actioned, surfaced to moderators via Ozone, and in turn helps shape our Discover feed algorithms.

Although some of our internal heuristics are private, Osprey and Ozone are designed to be deployed and run by other app hosts with their own custom rules and moderation practices, whether they are reimplementing the Bluesky data model and feature set or running a different kind of Atmosphere app with a different set of content. In this talk, you’ll see a demo of the Jetstream, Osprey, and Ozone, learn about our rules engine and other architectural features, and leave with enough knowledge to integrate our open source moderation tools into your own stack.