# Blocklist Requirements & Governance

:::warning
*This is just an initial braindump, please don't treat as anything other than an early pass
at work in progress*.
:::

> *The second mistake is that you connect the inscrutable thing to the Internet of
> hate, which is oftentimes simply abbreviated as "the Internet."*\
> — James Mickens, [USENIX address](https://www.youtube.com/watch?v=ajGX7odA87k)
> ([transcript](https://www.zachpfeffer.com/single-post/2018/11/27/Transcript-of-James-Mickens-USENIX-Speech))

There's bad stuff on the Internet, and some bad people too. And even some of the stuff that's
not all that bad you might not want to see or to host. And even for some arguably very innocuous
content, you probably want to minimize how much you have to think about the potential legal
liability for storing it.

With big centralized platforms, those decisions are delegated to those platforms' content
moderation, trust & safety, and legal teams. As we (re)decentralize the Internet, however, we
need to make sure that our solutions improve on the status quo in which a small group of
corporations decides what content is acceptable for everyone in the world.

For this to work, at the very least for IPFS-like protocols in which operators distribute the
content of unknown parties, we need to take into account the following aspects, while keeping
in mind that every blocking pathway risks being abused for forms of censorship, state or
corporate:

- CSAM and terrorist content
- Copyright infringement
- Data privacy regulation (for doxxing and data leak dumps)
- Safety interventions (e.g. removing phishing content)
- Hate, violence, toxic, etc. material
- Content that operators may simple decide they don't want to host (e.g. no dog pics on this
node, this is a cats-only place)
- *Perhaps* more generally lists of blocked addresses rather than just content. We are wary
of increasing the scope of this workstream, however there is seemingly *a lot* of overlap
between various blocklist domains even if the blocked identifiers differ. Generalizing to
addresses could support:
- IPFS content blocking, via CIDs
- IPNS content blocking, via domains and keys
- Peer blocking (e.g. because you don't want to even consider connecting with a misbehaving
peer on the DHT), via multiaddresses
- People blocking, via account addresses

In a number of cases, supporting all of these is definitely what operators want (e.g. blocking
both Nazis and Nazi content). If we find that a solution can support the full range (or at least
more than just the first two) without significant overhead and has traction with the relevant
communities, we should consider it.

The result needs to be at the very least (in an operator-centric view):

- Legally usable (notably you cannot block CSAM by distributing a list of CSAM sources or URLs).
- Easy for service operators to use and to deploy *correctly*. The stakes for operators
are high: they could go to prison, be fined, see their businesses seriously affected, etc.
We need to avoid footguns in which an operator *thinks* that they're blocking some type of
content but in fact aren't.
- Distributed and open: no central entity controls blocking, no one is obligated (other than
potentially via legal obligations) to use, or prevented from using, any given blocklist.
- Governable in a way that can be transparent and accountable, and that can support appeals.
- Overridable: an operator should be able to make exceptions to a list.

## Existing Art

The primary blocklist system in the decent world is probably the
**[Bad Bits Denylist](https://badbits.dwebops.pub/)**. It is a single list of hashed CIDs
that covers, inside of one list, "*copyright violations, DMCA, GDPR, Code of Conduct or
malware reported to Protocol Labs.*" Operators can manually update it and rely on it to
avoid serving that content. It works, but a recurring complaint is that it's basically one
big list with unclear governance and no transparency into how it is made or what the appeals
process is. Operators are asking for greater granularity and a path to better governance.

There are two active InterPlanetary Improvement Protocol ([IPIPs](https://github.com/ipfs/specs/blob/main/IPIP_PROCESS.md)) for
IPFS specs that are essentially evolutions of bad bits.
[IPIP-383](https://github.com/ipfs/specs/pull/383/files) is a **compact line-oriented blocklist
format** that has some limited metadata, the ability to block IPNS and paths in addition to CIDs,
and can provide hints to differentiate entry types.
[IPIP-298](https://github.com/ipfs/specs/pull/340/files) is **a DAG-JSON format meant to be
distributed as IPLD content**. It too improves what can be matched against, and it provides
more metadata about individual entries.

Both of these IPIPs are improvements over the currently deployed system, but
both need significant work to make them support interoperable implementations. It seems
plausible that some merger of the two could happen and, along with work to tighten up
the specification, lead to a release of a better bad bits format soon. Interoperability
is a salient concern here: operators rely on blocklists to be safe and failure can mean prison, significant fines, or having one's business in serious trouble. To the extent possible, we must
avoid situations in which one party provides a list that works for them but that fails to
block properly in another implementation. Bugs cannot be avoided, of course, but the
processing algorithms for blocklist content can be specified in ways that are
conducive to strong and highly testable interoperability. At this time that is not the
case with either IPIP.

One issue worth considering is **the usefulness of item-level metadata**. Both IPIPs seem to
assume a world in which operators would primarily subscribe to one big list (as bad bits
is today) and address the laundry list aspect by making it possible to filter out items
one does not wish to apply (e.g. don't respect the blocking of DMCA-flagged content). The
problem with unified massive lists is that the governance of different types of blocks
likely differs a lot and ought to be entrusted to different sources. One thing that
item-level metadata could support, however, is a flag for the type of the block entry
itself.

One hypothesis arising from discussions so far is that **list-level metadata
(and separate concurrent lists by type being produced and consumed) might
balance granularity and scale**. To wit, A model in which we assume that any
given blocklist is focused on a single kind of blocking reason and
implementations collate them locally (possibly based on curated lists of lists)
is probably more nimble, more scalable, and aligned with better and more
decentralized governance.

The application of blocklists beyond content blocking (i.e. **blocklists for
individual actors or sets/networks of actors**) brings in interesting
considerations. People are encouraged to read this [detailed and helpful
description of how blocking works on ATProto
(Bluesky)](https://atproto.com/blog/block-implementation). (Their discussion of
[composable moderation](https://blueskyweb.xyz/blog/4-13-2023-moderation) is
also relevant.) It's important to keep in mind that shared blocklists and
blocklist governance could compose as input into such systems irrespective of
the complexities specific to e.g. federated social networking. On its side,
Nostr has an open NIP ([NIP36](https://github.com/nostr-protocol/nips/blob/master/36.md))
about content labelling. Labelling is a
somewhat separate issue — we only mention it because it *could* be supported via
lists. Note: blocking or "defederating" sets of actors or whole networks is
unlikely to solve the problem of operators trying to avoid the distribution of
CSAM, terrorist content, or copyrighted content; its utility is more relevant
to the issue of hateful or toxic content in P2P systems.

While this has not been implemented yet, one idea being considered is to the CSP and
CSP reporting to detect bad behavior at the gateway level and use that to automatically
feed a blocklist. (If you have access to PL repos, you can read [this comment](https://github.com/protocol/bifrost-infra/issues/2587#issuecomment-1578489299) for details.)

## Distribution

A key assumption of this proposal is that an open-ended set of entities will be producing
blocklists. Each blocklist will correspond to a specific policy choice and any policy choice
is acceptable. In additional to other sources publishing perhaps overlapping lists, we could
imagine some "more authoritative" lists emerging, such as:
- a non-profit producing a CSAM blocklist,
- a cybersecurity group distributing a phishing blocklist,
- the MPAA maintaining a confirmed-DMCA-violation blocklist,
- some rando making a blocklist of all CIDs that contain a reference to the color "taupe".

At the consuming end, operators would select which blocklists they subscribe to
(possibly simply using someone's curated list of blocklists). This should be a
simple operation, and it should allow local overrides or processing **per list**
to allow more efficient consumption of unprocessed lists. For example, the following user
story would be supported:
> As an operator, I want to block the color taupe unless except when I know the word is
> instead referring to the French word for mole, *la taupe*.

There is precedent in the fediverse for the distribution of multiple lists sourced in
different ways along with algorithmic ways of selecting and merging sources. See
[Oliphant.Social Mastodon Blocklists](https://writer.oliphant.social/oliphant/the-oliphant-social-blocklist)
for details.

Ideally, blocklists would be distributed over IPFS and updated via IPNS. We should consider
having a list's root CID contain some metadata so that lists could be indexed (perhaps by IPNI?).

Lists also need metadata (the governance section gets into why these are needed):
1. Name
1. Description of the motivation for blocking
1. Point of contact
1. Pointer to the governance and appeals process
1. Last update
1. Expected update frequency

Note that the update frequency is important for operators. We shouldn't expect operators to
want to constantly care about their blocklists unless they want to, this should be largely a
fire-and-forget arrangement that affords them the safety they expect. A concerning failure
mode is that an operator would set up a blocklist for a legally-mandated blocking reason but
the entity providing that list goes out of business or somehow breaks, such that the list
goes stale. We want the implementation to be able to alert the operator that the list hasn't
updated in longer than is reasonable (and possibly to stop serving if the list is staler
than a threshold beyond the expected update).

## Governance

Blocklists are maintained by independent entities and picked up by other
independent entities: as such they might not need much governance, one group
just picks what they want from the other. However, the people whose content is
blocked (or who want to access blocked content) aren't represented in this
interaction, and the blocking system could be abused by malicious actors (e.g.
to block legitimate content they don't like). Therefore we should build enough
**accountability** hooks into the system that some governance can emerge beyond
just a laissez-faire market of blocklists.

Blocklists need sufficient **provenance** (ideally, authenticated) that
operators can make informed decisions about which ones to trust. For instance, I
may be comfortable taking a content-I-don't-like blocklist from a random person
on the Internet whose values seem to align with mine, but if I'm looking to
block doxx dumps in order to comply with the GDPR I'm going to want a reliable
source. This is aimed at providing some degree of transparency and hooks for
accountability (such that an influential list provider could be investigated by
reporters to check that they really are producing what they claim to be). Using
the provenance information to link to a human-readable description to how the
blocklist "sausage" is made should be encouraged.

Blocklists should also be expected to document their **appeals** process. In the
late 90s and early 00s, before email was captured (in large part by economies of
scale required to combat spam effectively), there were blocklists of
misconfigured email servers distributed to help prevent spam. Getting your
server mistakenly added to one of those lists could easily mean being unable to
operate that server for several weeks because the appeals process was often slow
and poorly thought-through. Improperly including content on a popular blocklist
is an efficient censorship vector and in a world in which IPFS is a dominant
content distribution mechanism, we should expect state actors and industrial saboteurs to explore this
option. While it would be overkill in the first iteration, if the blocklist
system is popular we should expect to build standardized appeals processes to be
adopted by lists (including a "no appeals" process — it's legitimate so long as
subscribers to and replicators of that list know what they're getting).

Finally, it is worth noting that the system makes it possible to build a
**reputation** layer for nodes and servers. Self-attestations ("these are the
lists I inherit and enforce") might be less useful here than certification or
spot-checking; to that end, any independent party could permissionlessly spot-
check whether a short set of known (non-hashed) CIDs are being blocked by this
or that gateway, or being made available to the DHT. (In the case of requests,
this approach extends neatly to hashed blocks.) This can generate explicit or
implicit reputation, on the basis of which any node could decide to refuse to
connect with another node that fails to block certain types of content that it
is strictly opposed to propagating or serving.

## Community Outreach

The working group would like to seek feedback (on the draft proposal but also on the problemset generally) from, at the very least:

- Operators
- Ofcom
- #pl-content-policy
- Alex Feerst & Murmuration
- Blockparty
- Bluesky
- Stanford IO
- Cloudflare Research

Please reach out to the WG chairs if you have an intro to someone with deep/inside knowledge in those places who might want to come to a meeting or vet documents/repos.

## Parking Lot

> [name=Brooklyn @expede] Private Set Intersection (PSI) and/or Oblivious Transfer (OT) as part of keeping blocklists private?

> [name=Boris Mann @bmann] I am looking at some thoughts about brigading. I think there is something to be said for the "liveness" of block lists, rather than just static lists. I'm not sure where that fits in. Maybe related: muting/silencing vs blocking? This might be network specific metadata.

> [name=bumblefudge @bumblefudge] - [I blabbered on fil slack about tiered tagging and/or handling: absolute/coarse/fine ](https://filecoinproject.slack.com/archives/C03TUQZ48R1/p1688458235734319?thread_ts=1687941179.333939&cid=C03TUQZ48R1)

> [name=Robin @robin-berjon] worked on [this](https://open.nytimes.com/to-apply-machine-learning-responsibly-we-use-it-in-moderation-d001f49e0644) so he can answer questions about the annotation systems that Perspective and other major MLs use. also couldn't hurt to look into the docs of that ML bsky is using to "moderate" until there's $$ for hoomans?

## References/biblio

- [yoel roth](https://www.yoyoel.com/) of twitter T&S fame has been convening/coordinating lots of moderation topics on this [atlantic council task force](https://www.atlanticcouncil.org/programs/digital-forensic-research-lab/democracy-and-tech-initiative/future-web/). Note that last month they published some documents, including a little 10-pager on [today's moderation tooling and approaches](https://www.atlanticcouncil.org/wp-content/uploads/2023/06/scaling-trust-on-the-web_annex2.pdf) and [a 5-pager specifically on moderation across federated architectures](https://www.atlanticcouncil.org/wp-content/uploads/2023/06/scaling-trust-on-the-web_annex5.pdf). extremely relevant starting points!!!
- see also the [2019 ABCs of disinformation](https://www.ivir.nl/twg/press-release-disinformation-abc/) [report](https://www.ivir.nl/publicaties/download/Co-chairs_report_2019-07-22_Santa_Monica.pdf) by the [transatlantic wg](https://www.ivir.nl/twg/)
- [**openNYT** on ML moderation (mentions annotation data model)](https://open.nytimes.com/to-apply-machine-learning-responsibly-we-use-it-in-moderation-d001f49e0644)
- FediVerse has some ideas but not they're not aligned across implementations yet
- Boris found [this excellent example of a thoughtful user- and instance-blocklist sharing system](https://codeberg.org/oliphant/blocklists)
- **Matrix** has done [some work going back to 2020 on reputation](https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix-without-backdoors/)
- the [long](https://www.reuters.com/article/ctech-us-whatsapp-india-fakenews-idCAKCN1L60R9-OCATC), [sordid](https://thediplomat.com/2019/05/manufacturing-islamophobia-on-whatsapp-in-india/), [litigious](https://thenextweb.com/news/indias-new-idea-for-tracing-whatsapp-messages-is-deeply-flawed) history of **whatsapp** chain letter detection/flagging/throttling approaches
- note content-hash versus actor-traceability tradeoffs
- **bluesky** has a lot of relevant prior art here.
- [mike masnick on **bsky**'s "composable moderation" ideas apr20 2023](https://www.techdirt.com/2023/04/20/bluesky-plans-decentralized-composable-moderation/) is exceptionally relevant beyond bsky
- they've been [accepting end-user input on a finite blocklist of english-language slurs for use in new user registration filtering](https://news.ycombinator.com/item?id=36706856) and [debating ML approaches to same](https://github.com/bluesky-social/atproto/discussions/1329).
- **lib genesis** (and presumably other, smaller at-scale operations to sidestep copyright-related moderation) has very much [discovered the ipfs ecosystem](https://www.reddit.com/r/libgen/comments/ug63sq/libgen_on_ipfs_updated_32_million_books_now/#) and creates a problem at terabyte scale for some operators.
- DSA/DMA
- Daphne Keller's [biting critique](https://cyberlaw.stanford.edu/blog/2023/07/rushing-launch-eus-platform-database-experiment) of a rushed [v1.0 of the DMA content-moderation API](https://github.com/digital-services-act/transparency-database)