HackMD
Privacy for VR
tags: devlog
Privacy = Freedom
Privacy is a fundamental human right and a public good. Studies prove that humans behave differently while they are under watch. What happens to a society constantly under watch and analyzed by AIs?
“Under observation, we act less free, which means we effectively are less free.” - Edward Snowden
The world is adopting new technology at an accelerating rate. When a decent pair of glasses come out wearable computing will likely take off like a rocket, merging humans and machines closer in the process.
VR technology is built on tracking with computer devices that act as direct interfaces into fooling our brains to believe that we are somewhere else. We need to figure out solutions sooner than later before mass adoption arrives.
We can not allow so much power to be in the hands of one company, especially with a medium like VR/AR which hoovers up more data about our surroundings, actions, and reactions to sensory information than any other technology before it. Source: Motivation
We can shape the culture and consumer habits now before it gels and becomes harder to undo, like the case in the tech duopoly of the mobile ecosystem or top social media companies owning all our data.
Avatar Culture
Privacy culture will be a driving force for technological adoption and change. You can't make people care about it on technologic merit alone, it has to be cool. Luckily, this has always been thecase with early internet and online gaming culture.
"Anonymity was one of the major perks of the OASIS. Inside the simulation, no one knew who you really were, unless you wanted them to. Much of the OASIS’s popularity and culture were built around this fact." - Ready Player One
Vtubers
VTubers (virtual youtubers) have been rising in popularity lately, propelling privacy culture with it. Streamers now have an option to create completely new pseudonymous characters in place of themselves to make content with.
There exists a number of free software for creators to animate a custom avatar using just a webcam.
VTuber Software
- https://virtualyoutuber.fandom.com/wiki/List_of_VTuber-related_software_and_resources
- https://gist.github.com/emilianavt/cbf4d6de6f7fb01a42d4cce922795794
- https://www.moguravr.com/vtuber-tools/
VSeeFace
Link: https://www.vseeface.icu/
Github: https://github.com/emilianavt/OpenSeeFace
VMagicMirror
Link: https://malaybaku.github.io/VMagicMirror/en/index
FaceVTuber
Link: https://facevtuber.com/
More Links
Avatar providers
Minecraft / Roblox
Most users of these major platforms are young and pseunonymous. People become known for the things they make and the way they interact on these platforms.
Crypto
Crypto anons are the future. Similar to Minecraft and Roblox, crypto is a window into the future of pseudonymity. People are judged solely on merits. Crypto also allows people to earn money through their handles without needing to trust a middleman.
The Pseudonymous Economy
"Anonymity was one of the major perks of the OASIS. Inside the simulation, no one knew who you really were, unless you wanted them to. Much of the OASIS’s popularity and culture were built around this fact." - Ready Player One
More links about pseudonymous economy:
One might consider VR already a pseudonymous economy since tons of creators are currently making / selling assets across virtual worlds.
Voice Changers
Avatars are inherently a privacy enhancing technology, but our voices can be fingerprinted and can compromise privacy.
With voice changers and text to speech technology, one can almost completely disassociate their real identity with their virtual one. This is especially true if one is in desktop mode with preset animations since gait tracking won't be a thing then.
Another option is a hardware solution. The live music industry has a variety of offerings for vocalists offered as "effects processors" available for purchase. With a few adapters they can be adapted for use on any consumer device, even TRS or Lightning for phonecalls.
While significantly more expensive than a software solution, they have the benefit of existing outside the system. Thus they can be used easily even on closed systems and with less risk of accidental broadcast.
The Boss vocal performer VE-5, a portable battery powered effects processor
Conclusion
There's plenty of room for innovation when it comes to voice changers. Seems like an interesting time to explore it since more voice only social platforms are coming online.
Distribution
Why does distribution matter when it comes to privacy?
Good question.
How do people currently share and discover virtual experiences?
Currently most VR experiences are downloaded from the app store such as the Oculus Store or Steam. It's a long process to get your content approved for distribution through these app stores.
There is also a place for more experimental content like Sidequest and the Oculus App Lab which have more relaxed rules for what content gets approved. However, your content can be pulled from such a place if it includes banned imagery.
Protocols
- WebXR
- Distribute VR/AR apps over the browser, no gatekeepers
- https://immersive-web.github.io/webxr/
- https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API
- https://github.com/immersive-web/webxr/blob/master/explainer.md
- https://immersiveweb.dev/
- Tor
- Anonymize the traffic between endpoints
- https://www.torproject.org/
- https://github.com/micahflee/onionshare
- ENS
- Decentralized naming for wallets, websites, etc
- https://ens.domains/
- IPFS
- Decentralized hypermedia protocol for file storage
- https://ipfs.io/
- Namebase
- TLD name registrar that operates on the Handshake blockchain
- https://www.namebase.io/
Infrastructure
Links: http://nethood.org/links/
Trust, but verify.
Who owns the servers that virtual experiences are hosted on? Being closer to the metal is better for having true ownership over your privacy and data.
Presence
Where are the packets being sent through for movement, text, and speech when you're in a multiplayer experience?
Very few virtual world platforms open source their networking services which makes auditing hard. If you can't self-host the services to enable networking then your privacy is as reasonable as being in a public coffee shop.
Here's a list of open source presence servers you can run on your own hardware:
- https://github.com/janusvr/janus-server
- https://github.com/networked-aframe/networked-aframe
- https://github.com/vircadia/vircadia#how-to-deploy-a-server
- https://github.com/mozilla/janus-plugin-sfu
- https://github.com/mozilla/dialog
- https://github.com/decentraland/catalyst
Honorable mention
The Unity store has a plugin called Photon which is powerful but it's not open source: https://www.photonengine.com/PUN
There's documentation on how to set it up with MRTK https://docs.microsoft.com/en-us/windows/mixed-reality/develop/unity/tutorials/mr-learning-sharing-01
Hardware
VR headsets started out as desktop peripheral devices, like a monitor or keyboard. Then mobile VR became a thing where users would insert their phone into a headset case to power and see the virtual world.
Popular standalone VR headsets like the Oculus Quest 2, are self-contained mobile computers. This is the direction that the biggest VR company, FB, is moving towards since they are discontinuing their desktop VR headset line.
Setting up the Quest 2 requires a phone with location and bluetooth enabled plus a Facebook account in good standing. Your access to Oculus Quest content can be disabled by means of Facebook kicking your account if you break a rule.
It would be more private than the cloud if everything to run the virtual world and connect to friends could be self-hosted on your own hardware at home.
AVALON
DIY Networks: http://nethood.org/links/
Link: https://hackaday.io/project/11279-avalon
Imagine a piece of hardware that doesn't require an internet connection for your own virtual private island. This space is built with open source hardware, software, guaranteed private, and can connect to other islands p2p using mesh networking.
AVALON stands for Anonymous Virtual Augmented LOcal Networks. It's a fork of Piratebox (discontinued) tailored for distributing WebXR content to local devices within range of the WiFi signal.
It can be described as a combination of:
- Dead drop
- WiFi portal
- Mesh net
- Pirate radio
The software runs on Raspberry pi or a router that runs OpenWRT. It essentially acts like a WiFi portal into a WebVR space without needing access to the Internet because it has its own in a box.
Cryptocurrencies
Cashless societies are surveillance societies. Since virtual worlds are cashless and transparent by default, it makes them vulnerable to financial surveillance.
There's a number of technology projects working on solving the privacy issue for cryptocurrencies. Below are some examples that is currently on our radar.
- The image file may be corrupted
- The server hosting the image is unavailable
- The image path is incorrect
- The image format is not supported
Tornado Cash
Link: https://tornado.cash/
Github: https://github.com/tornadocash
A fully decentralized protocol for private transactions on Ethereum. The Tornado Cash initial developers have no control over it and are not running any servers.
Tornado Cash basically lets you move ETH into a "clean wallet" that has 0 transaction history, similar to Bitcoin tumbler services.
Aztec.network
Link: https://aztec.network/
Github: https://github.com/AztecProtocol
High-speed privacy network on Ethereum, aiming to offer:
- Balance privacy — hiding transaction amounts
- User privacy — hiding ‘spender’ and ‘receiver’ info
- Code privacy — hiding asset/code being spent/run
Recently launched https://zk.money/, enabling efficient, completely private transactions on the Ethereum blockchain. Read more here: https://medium.com/aztec-protocol/aztecs-zk-zk-rollup-looking-behind-the-cryptocurtain-2b8af1fca619
ZCash
Link: https://z.cash/
Github: https://github.com/zcash
If Bitcoin were HTTP, ZCash would be HTTPS. ZCash could be a useful currency in virtual worlds where you don't want the other party or your competition snooping on your financial history.
Did an experiment by adding a ZCash address on a QR code to my virtual business card.
Horizen
Link: https://www.horizen.io/
Github: https://github.com/HorizenOfficial
Private sidechains
Secret.Network
Link: https://scrt.network/
Github: https://github.com/SecretFoundation
Secret Network enables privacy-preserving smart contracts by default.
SNIP-721: Private NFTs
Privacy AMMs: Private DEX
Kent Bye
When discussing XR privacy research it is worth bringing up the valiant effort of Kent Bye whom has published 40+ interviews on VR privacy since 2016.
Here's a digested view of the landscape of ethical & moral dilemmas of XR that Kent Bye has organized into an infograph:
Biometric data from XR is not currently defined or protected by existing privacy laws, which focus on identity.
I needed to map the relational dynamics of how XR companies could undermine our mental privacy after talking w/ @brittanheller.
Oculus' ToS/Privacy Policy currently doesn't prevent Facebook from recording data or extrapolating biometric psychographic info: http://voicesofvr.com/988-defining-biometric-psychography-to-fill-gaps-in-privacy-law-to-cover-xr-data-brittan-hellers-human-rights-perspectives/
https://twitter.com/kentbye/status/1381342561734782980
White Paper
Kent Bye is heading up a 6-month effort coordinating the XR Ethics White Paper for the IEEE's Global Initiative on the Ethics of Extended Reality that officially launched on Friday, Feb 5th.
This podcast has more details on how to get more involved: https://voicesofvr.com/976-xr-ethics-white-paper-invitation-from-launch-of-the-ieee-global-initiative-of-the-ethics-of-extended-reality/
With #augmentedreality glasses meant to be worn 24/7, the influence of Big Tech on our life will increase.
It's worth listening carefully to what Sander is presenting in this video.
https://www.youtube.com/watch?v=05njvzxf63k
Pervasive Eye Tracking
Future of Privacy Forum
XR technologies rely upon the collection of huge amounts of information and the processing of sensitive data, including users’ biometric data, unique device identifiers, location, information about the interior of homes and businesses, and more.
Without this data, XR technologies cannot function safely and effectively. At the same time, these sensitive data categories have become flashpoints in ongoing data privacy controversies, prompting vigorous debate, regulation, and proposed limits on the ways organizations may collect, use, and share data.2 As XR technologies become more popular, developers and XR platforms will gain access to rich new sources of information about individuals.
The report contains recommendations for how platforms, manufacturers, developers, experience providers, researchers, and policymakers should implement XR responsibly, such as:
- Policymakers should carefully consider how existing or proposed data protection laws can provide consumers with meaningful rights and companies with clear obligations regarding XR data;
- Hardware makers should consider how XR data collection, use, and sharing can be performed in ways that are transparent to users, bystanders, and other stakeholders;
- XR developers should consider the extent to which sensitive personal data can be processed locally and kept on-device;
- XR developers should ensure that sensitive personal data is encrypted in transit and at rest;
Zuckerberg
Future of AR and VR involves neural interfaces, custom silicon and operating systems, AI assistants.
More timestamps from https://anchor.fm/the-informations-411/episodes/BONUS-EPISODE–Mark-Zuckerberg-on-the-Future-of-AR-and-VR-ervthr
The achilles heal for all of these vertically integrated ecosystems between Microsoft, Facebook, and Apple seems to be the web browser. The web is the Switzerland of the Metaverse.
Artificial Intelligence
Mark Zuckerberg and other tech companies are currently exploring the role AI will play in the future of AR/VR for things like AI assistants.
Over time, its not hard to see how these devices along with Siri and Cortana will eventually have digital avatars within virtual worlds.
If we're going to be connected to computers all day that are constantly learning from us as we are metaversing, then it's worth some privacy preserving alternatives to the current AI assistant botnets. One open source alternative is called Almond: https://almond.stanford.edu/
Almond is the open virtual assistant that you can access through your phone and the web. It helps you configure your devices, accounts, and public services, retrieve data from them, and execute actions on them. It also lets you share your data with your friends and families easily. Oh, by the way, it's open source and it protects your privacy!
Github: https://github.com/stanford-oval
Layers of the Metaverse
Layer 3: Global
3D Internet
- content discovery
- virtual economy
The first face, you show to the world.
The global layer is where the virtual economy takes place. It is the city states of the Metaverse.
Expectations of privacy are lower, like being in a public area, and in general the platforms set the moderation policy standards.
Layer 2: Shared
Interconnected Small Worlds
- Friends
- Group chats
The second face, you show to your close friends and your family.
The shared layer are akin to group chats and discord servers. It is the neighborhood layer to the Metaverse. There is a reasonable expectation of privacy at this layer, like being at home.
Layer 1: Local
3D Desktop
- Hard drives
- Operating system
- Offline
The third face, you never show anyone. It is the truest reflection of who you are.
The local layer is the one closest to the metal. It's the operating system you boot into when you turn on your computer. It's the files and folders you locally create and save from the internet.
There's an extremely high level of trust and expectation of privacy between a user and their computer desktop / local hard drive. This layer only talks between the machine and the user.
This is the layer many productivity apps would happen such as word processors and editors, though more often we see modern productivity apps benefiting from collaboration features.
Quotes
They want to punish you for your speech because they can’t punish you for your thoughts. - Naval
“Ultimately, saying that you don't care about privacy because you have nothing to hide is no different from saying you don't care about freedom of speech because you have nothing to say.” - Snowden
"Privacy is the power to selectively reveal oneself to the world" - Cypherpunk Manifesto
“Psychologists, sociologists, philosophers, novelists, and technologists have all written about the effects of constant surveillance. Studies show that we are less healthy, both physically and emotionally. We have feelings of low self-esteem, depression, and anxiety. Surveillance strips us of our dignity. It threatens our very selves as individuals. It's a dehumanizing tactic employed in prisons and detention camps around the world.” - Bruce Schneier
Catalysts
What events would cause a culture shift to move people towards more private solutions for handling their data?
- something happening to an influencer
- data leak with repercussions
- dramatic movies and film
- really cool piece of local-first hardware
- AR/VR pet that lives in a tamagotchi like wifi node
- custom raspberry pi OS for virtual private island
- desktop visualization software (anarchy arcade inspiration)
- Scare people with a webxr experience using Firefox Lightbeam data
Resources
- The Eternal Value of Privacy by Bruce Schneier
- [1]Hawthorne Effect
- https://secure.oculus.com/my/privacy/
- https://www.eff.org/deeplinks/2020/08/if-privacy-dies-vr-it-dies-real-life
- https://twitter.com/kentbye/status/1191786583529271299
- https://opensource.com/article/20/12/virtual-reality-server
- Security and Privacy Approaches in Mixed Reality: A Literature Survey
- Cyber Security Threats and Challenges in Collaborative Mixed-Reality
https://nymtech.net/nym-whitepaper.pdf
https://twitter.com/naval/status/1343678212648255488
https://twitter.com/DavidSacks/status/1350574248046309378
https://i.imgur.com/etZdCKI.png
