# Ceramic core-devs call - Biweekly Meetings

[📅 Meetings calendar](https://calendar.google.com/calendar/u/0/embed?src=c_[e-mail]&ctz=Europe/Lisbon)

<!-- Template
# YYYY-MM-DD
### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor

### Attendees

### Notes

---
-->

# 2024-10-17

### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Seeing increases and spikes in traffic.
- Particularly in number of active projects in a day
- New status page 99.8% uptime since September
- Plugins and Studio are seeing adoption
- Devs using Orbis with Lit Protocol
- Roadmap:
- Simplify and flexibility around plugins
- Improve DX around Studio
- How to purge old inactive instances
- Question:
- How does migrating to/from self-hosted to cloud-hosted work? Can it leverage Ceramic sync?
- Index
- In building mode
- New architecture is 100x cheaper to operate
- Subscribe to Telegram groups using the app
- Considering switching to OrbisDB for the stable GraphQL endpoint
- Hirenodes
- 3Box Labs
- Ceramic-one release 0.38
- Event Validation - Ceramic-one now can validate integrity of events, i.e. CACOAs and time event proofs, and signatures
- Self-Anchoring - Ceramic-one can now submit events to CAS to be anchored.
- Architecture is moving away from CAS so that it the centralized service is not required.
- Ceramic-one does its own merkle tree building for events.
- More robust compared to js-ceramic as it has simpler control flow
- OLAP Aggregator is ready as an MVP
-

* Open floor
- Need to archive old notes somewhere as we hit the 10K note limit.
-

### Attendees

### Notes

# 2024-09-19

## Notes

### OrbisDB

- Shout out to Seref from Index team form helping with implementing Python libraries for generating streams on Orbis.
- More granual usage analytics with OrbisDB is available to see general behaviour (e.g. number of streams, etc)
- The team will update the Cerascan to use Recon (to index models and streams)
- Working on minor fixes and stability. Preparing a roadmap for the next month. Lots of DX improvements - new events to install plugins, etc.

### Index Network
- Converting js libraries to Python, and are planning to migrate other libraries to Python. Looking for some feedback on it from the community.
- Working on the first implementation of the contextual privacy. First will most likely be a simple mobile app. Exploring using LLMs as privacy gateways. First stages will include early testing within a small group of early adopters and will be opened up for developers later.
- Would like to get some feedback from the team on the early implementation.

### 3Box Labs
- Building an aggregator which is not yet available, but in a week or two will be available for testing.
- Working on some new integrations.
- Lots of progress on self-achoring for ceramic-one.
- Discussions starting around the standalone Ceramic and the future of the Data Feed API.

### Growth
- Creating new educational content
- Messaging and DX updates to Ceramic docs and website
- Index team did an [interview with BlueYard](https://www.youtube.com/watch?v=BUOv-jkWSWE)

# 2024-09-05

## Notes

### OrbisDB

- New developers, some tried to use model created outside Orbis on a Recon node, some issues bc the model wasn't indexed, they are working on that
- Actively supporting new developers and answering questions
- Working on improving the analytics, how many streams being created, active context being used by developers, updating cerscan to gather the context metadata that comes from orbis
- Migrating public studio to use recon instead of legacy node
- environment id right now is the users did but going to migrate to a uuid
- 70 developers who went thru the studio steps but a bad plugin could restart the shared instance
- actively working on Hoku studio, the main interface for developers to explore the work they are doing with hoku
- on Cerscan - still discovering via pubsub, but need to include recon streams - Nathaniel may help
- more than 150k streams being generated thru the orbis studio thanks to the plugins (we saw these!)
- Rohhan asked to see demo of all the projects to share with team- next dev call might showcase them

### Index Network

- Continue to implement evaluation metrics, RAG metrics for truthfulness of results generated by index
- Recently collaborated with developer starting a social listening feature, campaign management
- On twitter shared contextual privacy demo, https://x.com/indexnetwork_/status/1828847341001924833
(based on contextual integrity theory - the author of the theory invited them to present at privacy symposium - similar to token gating)
- Next will be exploring Autonomous privacy
- Started implementing a Metamask snap, started collaborating with Verax, if you are sending a token to another wallet you can see the reputation as a metamask snap
- Supporting other developers using Index Network
- had idea for a Ceramic LLM, working on a prompt using json schema, this prompt will generate postgreql queries on the fly, but needs a lot of work to be production ready
- thinking of implementing as a privacy endpoint that will accept context
- in hackathon people want to discover each other

### 3box Labs

- Integration leading to Ceramic being a more general aggregation pipeline
- Goal is to build a system to get at the data you've written into Ceramic in a pseudo-standard way so you can do analysis and operations over your data - building standalone binary that will talk to a rust-ceramic node, aggregate and write to basin or any s3-like store as parquet files = data lake any tool can talk to
So then any tool that can talk to data lakes can run over the data - orbisdb might want to use it?
Does all the ceramic stream aggregation for you. Like state store but in explicitly open manner.
- If building on DataFeed API this is the more performant future of it
- Self-anchoring - working on getting v1 to be standalone so just the c1 binary can anchor without js-ceramic/composedb. Only one request to cas/hr with the root of the merkle tree - or not. Is in develop branch.

### Growth

- Content and changes to documentation so developers can start after announcement
- Changing the ceramic discord, will communicate with the community members
- Question - about docs when devs set up Recon and then combine with OrbisDB
- Few developers who want to integrate with Ceramic thru Python esp in AI area, there were attempts at python adapter - is that a thing? seref had one that just called the compiled js binary - https://pypi.org/project/indexnetwork-sdk/ (should be easier when rust-ceramic is standalone)
- Devcon coming up with some small sponsorships
- Question about migrations - Aaron said we're working on resumability for big migrations

# 2024-08-22

## Notes

### Index Network

- Improving contextual pubsub topic
- "Private context" is a new concept Index is working on to manage private Data. Tring to find standarized ways for storing and making private data discoverable while being encrypted.

### OrbisDB

- Several stability improvements led by Donat
- New Developer Experience improvements: new PLugins framework to install plugins without having them directly in the code base (potentially using npm packages).

### 3box Labs
2 major focus on 3Box side (mostly on rust Ceramic):
- Self-anchoring improvement - allow rust ceramic node to build trees locally and send trees to CAS directly
- introducing "Delegating Anchoring"
- David from 3box labs is leading efforts on event validation part of rust Ceramic

### Community Events
- 3Box Labs is planning to join Token2049 and participate to several side events.

# 2024-08-08

### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor

### Attendees

### Notes

- Orbis update
- Focusing efforts around the stability of OrbisDB Studio
- Finalized OrbisDB documentation, including a full doc on building OrbisDB plugins!
- Soliciting feedback from folks looking to customize OrbisDB
- Index.network
- Did some writing on [contextual privacy](https://paragraph.xyz/@indexnetwork/can-something-be-private-yet-discoverable)
- Experimenting with these concepts for a while, with some exciting developments in the last month
- Please share feedback!
- Finalized agent-first architecture
- Improves the experience for users by allowing them to express abstract intents that agents can infer against
- Different agents will have different capabilities, e.g. understanding relationships/social graphs, summarization, sentiment analysis, etc.
- Agents will predict which queries they should run and will be evaluated accordingly
- 3Box Labs
- In a few weeks we'll have more information on a new set of APIs that make it easier to consume data from ceramic-one
- Generally part of a broader effort to make ceramic-one standa lone
- A few requisite projects include Event Validation within ceramic-one and self-anchoring
- First phase of self-anchoring is building merkle trees locally within the ceramic-one node and only sending that Merkle root to the CAS, giving CAS an exponential jump in the capacity it can handle
- Second phase will be eliminating CAS altogether and making it possible for ceramic-one nodes to anchor to an EVM contract directly
- Community Events
- 3Box
- Planning for Devcon in November! We'll be sending some teammates and are starting to plan side-events, etc.
- Collaborating with the Orbis team on a bounty for their BASE plugin

# 2024-08-03
### Agenda
* CIP discussions
-
* Ecosystem updates
- Oamo
- Still integrating rust-ceramic
- Finished the OrbisDB plugin! Can now use the studio to navigate Oamo's data sets. Releasing today or tomorrow.
- Orbis
- Major focus the past few weeks has been improving the stabililty of OrbisDB in general, in particular of the public studio.
- Almost 100 slots running right now on a set of shared instances!
- Been working hard to make studio slots better multi-tenant citizens, isolating them from one another.
- Enhancing Cerscan to pick up Recon streams. Some solutions in mind but not yet implemented.
- Creating more educational content on how to use OrbisDB and the studio.
- Index
- Built a mobile app, currently pending approval from Apple!
- Been working on new language to describe "contextual privacy" as "intent-based discovery" instead. DeFi was revolutionized by intents—they helped create a unified interface for transacting in DeFi. We see intent-based discovery as the same for information discovery.
- Working to integrate Index directly into a wallet. Will generate personalized news based off of a user's holdings.
- Hirenodes
- 3Box Labs
* Open floor

### Notes

# 2024-07-25

* Ecosystem updates
- Oamo
- Launched the platorm interface which allows uploading data to Recon. Currently issuing a few million credentials a day. Working on directly talking to Recon using an API.
- Need some help running the Cartonne package that allows dealing with CAR files.
- Action item: to sync with the Ceramic team to get unblocked and resolve the issues.
- Over the next few weeks the team will be focusing on the implementation that allows sending credentials directly to Recon, bypassing the js-ceramic file.

- Orbis
- Focused on improving the docs
- Released the [Dune Analytics plugin](https://x.com/useOrbis/status/1816156528325583242) - plugin allows to connect to Dune Analytics account and push all of your Ceramic data that is indexed on OrbisDB to then analyse it on Dune.
- Getting closer to finalising a plugin for listening on-chain events. This allows to position Orbis as a tool to aggregate the data coming from different data sources.
- Working on improving the stability of the Orbis Studio based on the feedabck provided by the early users.
- Collaborating with the Ceramic team on a bounty for using the Base plugin. Planning to announce it next week on Farcaster.
- Talking with the Index Network team about exploring how Orbis could be used as a vector database. Currently in an early discussion phase.
- For the next two weeks the focus will be on the EVM events listener mentioned above.

- Index Network
- Updates on the [documentation and architeture diagram](https://docs.index.network/docs/overview/architecture). It now shows how agents are orchestrated, incentivised.
- Private conversation graphs are ready. Now the team is working on private intents graph. Starting to work on private network embeddings.
- Finished the Base onchain summer project. Got [a really great submission](https://farlensflow.vercel.app/) from a developer who used all of the Index Network features just by following the documentation.
- Will meet with the Oamo team to share how they are building on Ceramic.
- 3Box Labs
- Built a PoC for private data that is working. There is a video and will be a blogpost release explaining for how everything works. There are still some technical hurdles to get over to get closer to prod implementation. Will be collecting feedback from the users on a demo and would love to hear from teams who would like to use it. Currently the PoC allows to store the event and retrieve the event.
- Kicked off the general milestane called ceramic-one-standalone. Today devs have to run `ceramic-one` and `js-ceramic`, but the team wants to enable users to run only `ceramic-one` without the extra dependency of `js-ceramic`.
- Self-anchoring project is another ongoing work. It's the approach to getting rid off the currently centralised anchoring operated by 3Box Labs. The team is still exploring how this should look exactly and would like to have discussions with the teams who are interested in providing feedback on the implementation. This will enable running multiple nodes in paralell which as a result will also improve the scalability of the network.

# 2024-07-11

* Ecosystem Update
* Orbis
* Released ...
* Write Ceramic stream based on events from smart contracts
* Working closely with Molecule to index blockchain data and metadata
* OrbisDB can index this on chain data allowing for filtering on against the indexed data
* Private data impact on indexing services like OrbisDB?
* Find time to meet with Joel to discuss
* Q: When can we start sharing the studio docs with devs?
* Onboard devs one by one before opening up to everyone
* Its not private but expected to have an intro before using
* 3Box
* Ceramic with ceramic-one (i.e. Rust based IPFS store) GA pre-releases are live, watch for announcement early next week
* Private data demo POC is close to being ready to share around for collecting feedback
* Q: Does writing and reading private data have the same performance as before?
* Haven't tested explicitly however architecturely only effects read side because of some extra predicates on reads.
* We should investigate directly
* Ceramic-sdk moving along, goal is that Orbis and other can leverage the skd to consume ceramic-one directly without the need for js-ceramic.
* Q: Python clients?
* Work with community to get it out there
* Sync up on generating clients for ceramic-one
* Clients need event signing logic

* Events and Community
* EthCC
* Lots of AI for X discussions and strategy

# 2024-06-27
### Agenda
* CIP discussions
- Not a CIP, but!
- Exploring what it would look like for Ceramic to not be *dependent* on anchoring.
- Would reduce complexity in the protocol.
- What does anchoring do now? Are there better ways to achieve this goal?
- Two main uses, revocation of expired keys, providing developer with trustless timestamps in the protocol
- Eager to talk to more devs to see if/how folks are using timestamping to see if there are any ways we could evolve that in the future.
- [Ongoing conversation in the forum](https://forum.ceramic.network/t/ceramic-without-anchoring/1581/1)
- What are security implications of how session keys currently work / how that might change?
- How might removing timestamping impact conflict resolution?
- What are alternative mechanisms of timestamping / ordering?

* Ecosystem updates
- Index
- Seref: Exposing a GraphQL endpoint. Adding capabilities for natural language query strings. ComposeDB already provides generic querying capabilities.
- With four lines of code, developers can index their data via new OrbisDB plugin
- Experience building a plugin for OrbisDB has been perfect—no issues encountered!
- *Demo of new website*
- Orbis
- Focus on engineering side, adding support for both GraphQL and relations within OrbisDB
- Will be integrated into the UI as well, looking to finish that work tomorrow
- Finalizing official version of docs with Donat, targeting July 1 ready date
- Starting in Notion, will be migrated to public site
- Gitcoin Passport plugin released!
- Index plugin coming soon
- Joel: Is the GraphQL API compatible with ComposeDB?
- Baptiste: OrbisDB will be a GraphQL server. There will be some differences in terms of generated names, etc, but functionality wise it should be roughly on parity. It will also have access to plugins, etc.
- Joel: What would a migration from ComposeDB look like
- Baptiste: Start by reindexing content into OrbisDB. There would be query/entity/field name changes.
- Fairly straightforward change for existing CDB users
- Mutations will not happen via GraphQL, but rather via the OrbisDB SDK for the time being
- Joel: Potential to get ComposeDB client library compatbile w/ OrbisDB graphql
- Baptiste: Could have some translation layer that interprets mutation requests into OrbisDB equivalent
- Oamo
- Yannick: Recon appears to be CPU bound in the load they've been putting through.
- 12M credentials issued via Recon so far.
- Looking to bypass js-ceramic & anchoring for the time being.
- When CPU / memory hit limits, performance degrades to the order of 2 tps.
- Focusing on BD to get credentials integrated into their platform upon issuance.
- Working on OrbisDB plugin as well!
- 3Box Labs
- Joel: Still on track for rust-ceramic GA release early next month.
- Proof of concept for private data ahead of schedule, looking forward to sharing that out and getting some feedback in.
- Cole: CAS has seen steady performance improvements. Sustaining 250-280tps with bursts up to 500tps.
- **Events**
- Chris J: Will be hosting a proof of data event at EthCC, including talk from CoinFund CEO
- If anyone is hosting or attending other events, please get in touch and let us know!
- Chris J: We'll be looking into presence at Devcon as well
* Open floor
- Mark K: Hoping to publish a demo app and corresponding video today, would love help circulating it.

### Notes

# 2024-06-13
## Agenda
* Important announcements
* Ecosystem updates
* Protocol updates
* Upcoming events

## Notes

### Ecosystem Updates

#### Orbis
* Big focus on documentation over the past 2 weeks
* Improvements both to the UI and SDK (setting up instances, etc)
* Another big focus on setting up recon node - currently running and working fine
* Facing some issues with datafeed API (triggering transactions in bulk) - works fine when switching to regular node but occurs when using recon
* Meeting with Mark next week to go over onboarding and getting started improvements
* Planning underway for plugin announcements - Charles has been actively discussing with ecosystem teams to coordinate these
* First announcements starting end of June with Passport - use case around bringing reputation to your database
* Next with Index
* Plugin for Oamo (searching credentials)
* Also plugin for Base - still in discovery mode
* Last one for this round will be Dune (working directly with Dune team - observability of Oamo credentials)
* Quick note Chris/Justina/Adam actively working on use case ideas for on-chain summer as well
* Devs will not be forced for launch to use recon (OrbisDB can be configured to talk to both)

#### Index
* Verax plugin is a priority
* Started storing encrypted chat history on Ceramic
* Currently encrypting conversation and titles of convo
* Cannot currently encrypt ownership
* About to publish farcaster integration
* Will be listening to casts in real time
* Just announced multi-agent integration
* Will be listening to realtime events based on context
* Just wrote tests for the SDK - getting ready for hackathon (on-chain summer)
* Chat history for agents will also be available/ready for the hackathon's scope
* Looking for inference provider to host an embedding model

#### Oamo
* Realized missing some of the data for token holdings credential
* Fix will take over 200+ million credentials
* Estimated at around 10-15 already published
* Still issuing credentials in the meantime
* Added 10 defi-related credentials for base
* Doing migration to google cloud
* Help w issuing more credentials at a faster rate
* Focusing on the Orbis plugin - looking forward to nice visualization with Dune
* Doing 800k/min w current Recon setup
* Nature of the data requires faster publish speed given they expire

#### 3box
* Collaborating w Index and Orbis team for Base on-chain summer
* Developing bounties that will be released soon
* Paul - still working on aggregation lib - still in progress

### Protocol/Product Updates
* Working on scaling anchor service
* Getting very close to releasing the lead version of Rust
* Most likely will be within the next 2 weeks
* Started working on a prototype for better privacy on Ceramic - DM Joel if desire to participate and give early feedback
* Looking at how we can provide privacy through physical access control (beyond just encryption)
* Will likely affect AC over data feed api
* Also AC over recon protocol itself
* Index - would help enable search and filtering functionality (had originally considered Fluence for that)

### Upcoming Events
* EthCC
* Working w Textile on proof of data
* Cap of 60
* Reaching out to prev proof of data attendees
* Now is the time to register - reach out to Chris
* Joel going to PL founder's summit right before EthCC

### General
* Justina/Mark - looking for collaboration ideas around helpful content from ecosystem teams
* Prepping for on-chain summer
* If anyone else would like to collaborate on bounties, intiatives, or other ideas, reach out to Justina

# 2024-05-30
## Agenda
* Important announcements
* Ecosystem updates
* Protocol updates
* Upcoming events

## Notes

### Ecosystem Updates

#### Orbis
* Working on official documentation to run orbisDB instances, deal with plugins. #1 priority right now. Improvements to devstudio based on feedback.
* Starting to use recon, some difficulties. Cole to follow up with Steph. Hirenodes is running the underlying Ceramic nodes, Orbis runs Orbis proper.
* Dev studio improvements, second step after reading documentation.

#### Oamo
* Officially issuing credentials through recon. Bottlenecked on node size.
* Lots of revamp on credential model in the last 2 weeks. 127 million credentials through recon, 14 million wallets.
* Looking for recon optimization advice, following up with 3box in the afternoon. Hoping for an Orbis plugin to make credentials easier to access.
* Recon already much much more efficient.
* Solana credentials coming down the pipeline.

#### 3box
* New typescript SDK to interact with rust ceramic coming at some point. Very early, discovery phase; will lead to eventual deprecation of JS-ceramic.

### Protocol Updates

* Performance improvements landed or will be landing in the next few weeks (internal batching, reduced lock contention, reduction-tree).
* CAS throughput steadily increasing.

### Upcoming Events

* EthCC Brussels

# 2024-05-16
## Agenda
* Important announcements
* Ecosystem updates
* Protocol updates
* Upcoming events

## Notes

### Important Annoucements
- Joel from Ceramic is back! :)
- Ceramic core team is full spead ahead on performance and stability. More concrete beta release is scheduled for next week.

### Ecosystem Updates

#### Orbis
- Focused on the shared instance concept as a way to onboard developers on OrbisDB and Ceramic. Starting to test a big shared instance that anyone can use using their wallets and interact with OrbisDB - use plugins, create data using contexts.
- Addressing the feedback on OrbisDB developer experience. Working on improvements on UI with respective plugins, guiding developers through the dev journey better.
- Continued work on OrbisDB Studio not shared publicly just yet, being shared internally for feedback and testing.
- Thinking of some new plugins for the Index team that should help them to onboard some new users through Ceramic. More updates on that in the upcoming days.
- Looking into testing Recon for OrbisDB during the next two weeks.
- Talking with several teams on which plugins need to be built. The team is also looking into ways to scale who can build the plugins. It should be faster and easier when official docs are available.

#### Index Network
- Just launched in private beta!
- Started experimenting with Recon, just updated to the latest version.
- Using OrbisDB as a nature of index and will be integrating with other protocols like Verax. Will be building a plugin for subgraphs which will make it easier to integrate on-chain data on ComposeDB.
- OrbisDB will be used to index Farcaster channels.
- Polishing the SDKs - Python SDK, JavaScript SDK, LangChain integration. They will be shared with developers soon.

#### Hirenodes
- Soon, will receive some hardware to move some of the instances from DigitalOcean to run them locally instead to offset some storage costs.
- Planning updates to SimpleDeploy - making them easier to use and getting devs up and running faster.

#### Oamo
- Got all the data required for Arbitrum and Polygon credentials. Now refactoring the credential models and focusing on reissuing data credentials using Recon. Next week will have 30 million credentials to shoot on Ceramic.
- Docs are pretty much ready, just need to add the how-to guides.
- Points system built by Oamo is ready to go. Will be issued alongside the credentials.
- Talking with OrbisDB regarding the plugin that would make the points and credentials easy to interact with via OrbisDB.

#### 3Box Labs
- Points library and example app is out with written and video tutorials.

### Protocol Updates
- Heads down on performance and stability for rust-ceramic.
- Delivered an updated Data Feed API. It allows subscribing to all creation events and updates to records stored in model instance documents. This update provides the ability to resume the subscription, ensuring that events are processed no more than once. This update completes the production readiness for the Data Feed API.
- Rust-ceramic recently moved to the last performance testing phase, adding end-to-end testing (including rust-ceramic, js-ceramic, and CAS). The team has been working on a number of projects that improved how js-ceramic interacts with CAS, resulting in significant performance improvements.
- Recon is planned to be released in beta next week (likely Monday, the 20th of May).
- Kicking off the design work expanding Ceramic's functionality from documents and models and moving in the direction of streaming systems—being able to publish immutable events with arbitrary separator recon keys. The Ceramic team is looking for use cases and teams who are interested in using Ceramic beyond models and model instance documents.
- Private data on Ceramic is not really on the near-term roadmap, but the team has started looking into the implementation of access control.

### Upcoming Events
- GenAI (end of May). The Ceramic + Index team will be hosting a private dinner for ecosystem partners.
- The Ceramic team is planning for EthCC and will share updates on that in the next week or so.
- Berlin Blockchain Week - Joel from Ceramic will be there, speaking at DappCon (May 21-23), DeSci Berlin (May 23-24), and AIxHope (May 24).
- The Index team will be at NYC Tech Week (June 3-9) as well as EthCC in Brussels in July.

# 2024-05-02
## Agenda

* Important announcements
* Ecosystem updates
* Protocol updates
* Upcoming events

## Notes

### Initial Announcements
None to speak of today.

### Ecosystem Updates

#### Orbis
- Main focus is finalizing migration from existing solutions to the new OrbisDB setup
- Main lift is recreating plugins from Orbis Social in OrbisDB
- Working on a concept of "presets" in OrbisDB that essentially pre-specify sets of models, plugins, and views to pre-load on a node. Will be an easy way for users of existing products to get a familiar experience up and running ASAP.
- Estimating another 2-3 weeks of work.
- Could imagine many such presets. They're currently living in the OrbisDB repo, contributors welcome.
- Presets are somewhat analogous to ComposeDB composites and could be a means of "distributing" configurations for participants in an application or ecosystem.

#### Points SDK (3Box Labs)
- [SDK blog](https://blog.ceramic.network/web3-points-library/) distributed 5/2
- A follow-up tutorial (walk-through of one of the example apps that uses the SDK) is already drafted and will go live alongside a YouTube video, likely week of 5/5
- New [example app](https://github.com/ceramicstudio/solutions-sdk/tree/main/demo/fullstack-app) added to SDK (this is what the tutorial and video mentioned above will walk through)
- The [ComposeDB Sandbox](https://developers.ceramic.network/docs/composedb/sandbox) has also integrated usage of the points library

#### Hirenodes
- Upped monitoring on node storage to prevet any future disk issues.
- Generally quiet few weeks!

#### Index Network
- Will be demonstrating a langchain library today. We have a Python SDK today!
- Olas will be using Index as an agent-first search engine.

The team showed a live demo of their LangChain SDK, which exposes Index as a source for RAG, and their JavaScript SDK, which exposes simple programmatic access to search via Index Network.

### Protocol Updates

#### 3Box Labs

Two major updates:

- rust-ceramic
- Been tracking down a handful of stability and performance issues.
- Performance issues now largely isolated to CAS, we are working on a handful of improvements on CAS.
- In-order delivery is the last remaining feature to be completed.
- rust-ceramic beta landing May 20.
- Data Feeds API
- Resumability lands tomorrow, May 3.
- rust-ceramic users will not be able to rely on it until we ship in-order delivery for rust-ceramic (May 20.)
- js-ceramic users can use it starting May 3.

#### CIPs

No updates for now!

### Upcoming Events

#### EthCC

3Box is currently working on finalizing its plans for EthCC, more on that to come soon! We'll be hosting a dinner at EthCC.

#### GenAI

Conference in SF, we'll be hosting a dinner there, inviting key stakeholders from AI companies. Details and invite list information to come later this month. This will be the first in a series of dinners we'll be hosting in the coming months!

#### 3Box Labs x Lit Hackathon

Currently coordinating a small hackathon with the Lit team end of May. More info to come.

# 2024-04-18
## Agenda

* Important announcements
* Ecosystem updates
* Protocol updates
* Upcoming events

## Notes

### Ecosystem Updates

* **Points SDK** (3Box Labs)
* Aggregator + Allocation primitives which use a shared hosted node (for test purposes)
* [Express server demo](https://github.com/ceramicstudio/solutions-sdk/tree/main/demo/server)
* Available as [npm package](https://www.npmjs.com/package/@ceramic-solutions/points?activeTab=versions); Link to [GitHub repo](https://github.com/ceramicstudio/solutions-sdk)
* Will be integrated into ComposeDB Sandbox on docs
* If time permits, additional fullstack example app will be included as an example as well that consumes the library
* Initial customers want to allocate points for research contributions
* If supports DataFeed API, OrbisDB can provide shared points instance (check with Golda)
* **Index** (Index Network)
* Released TS/JS SDK, to create index wrapping ComposeDB to expose nat lang interactions with CDB models
* Going to release Python SDK
* Langchain search coming soon
* Already 2 teams using SDK today
* Sometimes getting errors on Data Feed API, but resolve once CDB node restarts; maybe K8S doesn't like long lived connections networking between pods? (From Hirenodes)
* Working on more configurable embeddings and developer experience
* **Orbis**
* Working on shared instances, making it easy to onboard new devs; already 2 developers using them; BD for additional users
* Onboard DAOSpace to DB, want to create 300K streams from scraped data from governance platforms (Optimism 20K streams, ...)
* Might be using ODB CSV Uploader plugin
* Testing new plugin for ENS domain names - using ENS name plugin to reverse lookup address - making it easier to issue credentials to users
* Going forward: Autmoatically create preset views that everyone can query. Specific views for specific use cases -- e.g. get posts for social use cases
* Need to migrate from old tile documents by indexing that content into new shared
* **Hirenodes**
* Continuing to run nodes; everyone on v5.5.0
* Got serverside events running over nginx

### Protocol Updates

* Rust-Ceramic (3Box Labs)
* Almost production ready
* Shorter runs hit targets, longer runs hit some issues
* Looking to sustain much higher loads for much longer periods of time; 100tps for a week is the first benchmark
* If you have a use case or scenario that stresses Ceramic with performance or correctness - please get in touch with team
* Orbis was trying to dump 1-2M solana txs per day into ceramic; 3Box Labs to connect with team because this use case is within bounds

* DataFeed API (3Box labs)
* Looking to clean APIs and library abstractions by untangling js-ceramic + composedb into rust-ceramic + some library to create documents from events that DBs/indexers can built on top of
* First, aiming to make DF API stable and production ready, then will look to what comes next
* 3BL Team will connect with DF API users in order to design the next version

* CIPs
* Multi-Prev and After for stream ordering, nothing net new, but feeling more confident they can solve problems for users in the future
* Conflict Resolution Change: Changing the rules when updates happen so time event isn't interfering with stream updates. No changes for devs, just less likely to get a broken/corrupt stream.

### Upcoming Events

* ETH Boston | Boston, MA
* Next weekend 27th and 28th
* Spencer giving Ceramic talk and will have a Ceramic booth

* GenAI Summit | San Francisco, CA
* Index Network is going

* FarCon | Los Angeles, CA
* As of now, no one is going

# 2024-03-21
### Agenda
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor
*

### Attendees

- Michael Sena (3BL)
- Justina Petraityte (3BL)
- Joel Thorstensson (3BL)
- Aaron Goldman (3BL)
- David Estes (3BL)
- Paul Le Cam (3BL)
- Radek Sienkiewicz (3BL)
- Golda Velez (3BL)
- Julissa Dantes (3BL)
- Spencer Brody (3BL)
- Rohan Kochhar (3BL)
- Sam Doyle (3BL)
- Mohsin Zaidi (3BL)
- Nick Kammerdiener (Hirenodes)
- Donat (Orbis)
- Charles Levecque (Orbis)

# 2024-04-04
### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Features
- Shared instances
- Developers can open up their node to other developers
- Orbis will also stand up a default Orbis node
- Depending on your context, you will create different data
- All connecting to same Orbis db default server, but data partitioned by context
- Plugins: CSV uploader to apply large amount of data to Ceramic node
- two types of orbisdb instance
- 1) you run your own orbisdb + ceramic node yourself (e.g., molecule wants their own ceramic node and orbis instance). you run a non shared version where you are the only user. only one db with a set of admins. issues = everything needs to be set up yourself
- 2) diff settings configuration. shared instance. still one ceramic node, one orbis instance, but multiple slots in the orbis db instance, each slot has its own database. could have multiple slots and credentials for each database.
- are there scalability concerns? can one project degrade performance of another?
- initially, orbis will probably be the only one opening up slots.
- could rate limit projects
- Hirenodes
- Running recon.
- More requests for testnet nodes.
- mostly small projects kicking the tiers.
- Proxy issue
- Problem connected to data feed api behind proxy
- TODO: hirenodes to follow up with data feed api lead on this
- 3Box Labs
- looking into longevity issue on rust-ceramic, issue with database access. needs to be resolved before vprime is released. should be resolved within 2 weeks
- building out event validation on rust ceramic side (currently happens on js-ceramic side)
- Request for comments on forum posts
- After: I am interested in events only after x date
- Discussion of ways to modify CAS so you dont have to send all your events to CAS
* Open floor

### Attendees
Aaron D Goldman(3box)
Joel Thorstensson(3box)

### Notes

# 2024-03-21
### Agenda
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor
*

### Attendees

- Michael Sena (3BL)
- Justina Petraityte (3BL)
- Joel Thorstensson (3BL)
- Aaron Goldman (3BL)
- David Estes (3BL)
- Paul Le Cam (3BL)
- Radek Sienkiewicz (3BL)
- Golda Velez (3BL)
- Julissa Dantes (3BL)
- Spencer Brody (3BL)
- Rohan Kochhar (3BL)
- Sam Doyle (3BL)
- Mohsin Zaidi (3BL)
- Nick Kammerdiener (Hirenodes)
- Donat (Orbis)
- Charles Levecque (Orbis)

### Notes

No CIP updates to discuss

* Orbis updates from Charles
- want to start with recon
- avoiding migration is important
- moving orbis social to use MIDs from tile docs
- 1-1.5 million streams per day broke ceramic (another reason to use recon).
- using postgres for time being
- orca, drift, jupiter transactions (https://sol.party/)
- interpreted solana tx data offchain was idea
- sign in with solana => **required for WS1 event validation**
- questions about how to best migrate old tile doc streams to MIDs (new streams are using it)
- possibly plan to import them as read only
- aware that recon will not support tiledocs or caip10 streams

* Nick from Hirenodes
- gitcoin is running well
- some interest after ethDenver (indirect marketing?)
- spin up demo nodes => easy

* 3box updates (Joel)
- new season kicking off. primary focus on making rust-ceramic (ceramic-one) stable
- verifying signatures, time events
- self anchoring
- option to submit tx to blockchain without relying on 3BL anchoring service
- exploring submitting a merkle root hash rather than all events to reduce bandwidth
- getting rust-ceramic production ready
- alpha, beta releases.
- beta target to hit benchmarks of 300 transactions per second sync from node A to B (not network tps) for prolonged periods (i.e. 1 million writes/hour)

* Open floor: "interest in decentralized CAS"
- not a lot of clear interest by builders (unclear how well known it is)
- too much else to worry about
- node runners (nick) are more interested, particularly if CAS issues
- people might want it but "where's the wallet" might deter actually doing it

---

# 2024-02-29
### Agenda
* CIP discussions
- CIP-137 updates
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor

### Attendees
- Cole Brown (3BL)
- Justina Petraityte (3BL)
- Joel Thorstensson (3BL)
- Aaron Goldman (3BL)
- Nathaniel Cook (3BL)
- Paul Le Cam (3BL)
- Mohsin Zaidi (3BL)
- Baptiste Greve (Orbis)
- Liang Qiao (US3R)

### Notes
🦘 🐇 happy leap day 🐸 🦌

#### rust-ceramic/recon API (CIP-137):
Had discussion yesterday to finalize http api
API currently exposes EventID, plan is to hide that as an internal detail
Need to update the CIP with the finalized API design

Right now Recon feed only returns CID of the root of the event to apply, the additional blocks need to be fetched one at a time with multiple round trips. In the future the feed will return a CAR file with all the blocks needed to apply an event.

Want to update CIP by next core-devs call so we can share it with OrbisDB and others. Will make a forum post.

#### OrbisDB Beta Launch

Officially announced!!!
Website updated, branding updated, public annoucement.
Repo is open source
6 or 7 alpha testers, running their own instances
Charles is in discussions with other teams about running in production and/or building plugins.

Two focus points in next month:
- Public shared OrbisDB gateway
- Migrating existing Orbis Social users onto OrbisDB

Old version of Orbis had a centralized API that everyone used. Bad for centralization, but good for developer quickstart. We want some kind of public gateway that devs can use to start playing with OrbisDB. Will be some sort of variation of OrbisDB b/c main OrbisDB assumes only one app is in control.
OrbisDB gateway as a service. In the future others can run shared OrbisDB nodes as well.
Working with Hirenodes to run OrbisDB as a service.

Next want to migrate Orbis Social users onto OrbisDB.
17k profiles
100k posts
on Orbis social today

Right now OrbisDB stack is kubo, postgres, js-ceramic, orbisdb
Want to simplify this stack

Data feed resumability:
easiest path uses the CDB database
We want to test if we can use the same Postgres instance for both ComposeDB and OrbisDB database.

Might be time to start thinking about a minimal aggregation library. Will require cleaning up the relationship between the state store and the index db.

#### US3R:
OrbisDB vs ComposeDB:
Shared modeling layer
Can they serve different use cases?
ComposeDB might offer finer controls

Thinking about S3 hosting OrbisDB as well as ComposeDB

Main focuses:
- driving adoption
- releasing the S3 development experience to end users

S3 more suitable to advanced users who are already familiar with Ceramic and ComposeDB

Talking with Michael about integrating docs

Building our own social protocols using ComposeDB
S3 is dev stack
U3 is social protocol

U3 already integrated with Farcaster and Lens
Can build whole social protocol (social graph, profiles, etc) on ComposeDB

- hoping to use interoperable data models with Orbis social

Thinking about Points use cases

Lens vs Farcaster vs Bluesky vs ActivityPub
- ActivityPub and bluesky have wider adoption
- but lens and farcaster are more crypto-native. Much more likely to have a wallet set up.

#### 3BoxLabs team update

ComposeDB v0.7 is released, with support for SET account relation, field locking, and unindex.

In final week of season 8, getting ready for season 9
Next season primary focus is:
- launching rust-ceramic and Recon in prod
- scaling and performance work
- CAS performance
- Breaking ground on designing what a future decentralized CAS looks like

#### Protocol minimalization

For now aggregation will stay out of the rust codebase, might change in the future.

rust-ceramic is just about event distribution, signature verification, and timestamping.

indexers have to do aggregation (possibly with the help of a common aggregation library they can use)

wen CRDT?
Once CIP-137 is finalized and signature verfication pushed into rust-ceramic, then new types of data structures in Ceramic become possible.

# 2024-02-15
### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor
* points
*

### Attendees
- Michael Sena
- Cole Brown
- Justina Petraityte
- Joel Thorstensson
- Baptiste Greve
- Seref Yarar
- Aaron Goldman
- Sam Doyle
- Donat
- Rohhan Kochhar
- Serhat
- Karahan Sahin
- Adam Corrado
- Radek Sienkiewicz
- Danny Zuckerman
- Mohsin Zaidi

### Notes

- 3Box Labs
- Two major projects concluding with our current season
- rust-ceramic (without anchoring functionality) reaching production readiness
- Data Feed (fka Firehose) API reaching production readiness (with resumability support)
- Orbis
- OrbisDB plugin ecosystem is growing
- Official launch of OrbisDB coming Feb 26, accompanied by a new website to make sure developers have a clear on ramp and don't get conflicting information
- New branding!
- OrbisDB SDK will be open source, separate package from the "instance" which is the indexing service itself
- Aiming to make the SDK as simple to use as their current tooling
- Index
- Vector embeddings will be stored in a queryable index
- Looking at ComposeDB / Ceramic as an agent-orchestration mechanism
- Agents will be using NLP to process and trigger off of events
- Aiming to deliver composability of indexes by ETHDenver
- Community discussion points
- Michael: Points Working Group
- Not trying to pick a winner with regards to schema or other implementation details. We believe these are emergent properties. Winners will be determined by *distribution*, not by technical decisions like this.
- If you're interested in taking part, comment on the [thread in the forum](https://forum.ceramic.network/t/storing-points-on-ceramic/1470) or in the Points WG Telegram group.
- Joel: Protocol minimization
- Conversation happening in the [Protocol Minimization](https://forum.ceramic.network/t/protocol-minimization/1483) forum post.
-
- Joel: CAS Scaling
- Not as controversial, but more a dump of our insights.
- Find it [here](https://forum.ceramic.network/t/proposal-for-cas-scaling/1507).
- Joel: Proposal for flat Recon
- Conversation happening [here](https://forum.ceramic.network/t/proposal-for-flat-recon/1503).
- Current approach relies on a sortable key, but the problem is it's difficult to choose hierarchy when structuring these keys. There are tradeoffs to every configuration. Do we care about grouping by models? Controllers? etc. It's always wrong from one approach.
- Flat Recon rejects this entirely—let's stop thinking of the space as a ring, but rather as a potentially many-dimensional landscape.
- Recon / Ceramic itself would flatten this hierarchy and remove any intrinsic information from the keyspace
- Nodes are then given option to either linearly scan and filter this keyspace when reconciling with other nodes
- OR, pre-index the keyspace based on their preferences
- One could imagine certain indexers (e.g. OrbisDB, ComposeDB) preferring separate indexes over the event keyspace to optimize their specific dataflows.
- We see logical use cases for these indexes, but don't have any community members really asking for them right now. So it is hard to determine whether now is the time to implement.
- Don't want to add too much compute overhead on Ceramic nodes by requiring them to compute arbitrary indexes.
- When do we feel it is important to prioritize this?
- Answer is not immediately obvious
- If anyone feels strongly about sharding servers across fields beyond the model, share with us and that can help us prioritize.

---

# 2024-02-01
### Agenda
* CIP discussions
-
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor
- Protocol minimization

### Attendees

- Michael S (3boxlabs)
- Joel T (3boxlabs)
- Liang Q (U3er Network)
- Yunhe Z (Us3r network)
- Baptiste G (Orbis)
- Donat (Orbis)
- Charles L (Orbis)
- Paul C (3box labs)
- Sam D (3box labs)
- Cole B (3box labs)
- James P (3box labs)
- Mark K (3box labs)
- Rohhan (3box labs)
- Nathaniel C (3box labs)
- Spencer (3box labs)
- Mohsin Z (3box labs)

### Notes
**CIP updates**
Joel
* Followup ticket for Recon to update the CIP

Nathaniel
* Discrepancy btwn current status and difference between now:
* Will likely tweak event id
* Spec is also lacking definition of interest id
* Should be updated in 2 weeks

**Ecosystem updates**

**Orbis**

Baptiste
* Opening OrbisDB alpha version to customers next week
* New SDk interacting with OrbisDB is working
* Will come with indexing + querying features
* will be open source
* Public announcement in late March / early April with updated branding
* Waiting on event type

Spencer
* Release with event type should be going out within 1-2 days

**OrbisDb Plugins**
Sena
* Potential Dune plugin

Baptiste
* Could work with OrbisDB

Sena
* Open questions about indexing and exposing the entire network
* What is the level of compute

Baptiste
* How does it work?

Joel
* Everything is indexed with custom views on top

Baptiste
* Hard to anticipate which fields specifically would be interesting for developers

**OrbisDB <> ComposeDB Feature Parity**
Spencer
* Data modeling feature parity with ComposeDb and OrbisDb
* e.g., interfaces, SET account relation, field locking (immutable fields)
* OrbisDb support expectations?

Baptiste
* If the info is all available in the schema of the model, we can eventually get it
* OrbisDB is in early stages right now
* Can eventually sync with the features in the future

Spencer
* Interfaces: Most consequential one will be interfaces. Can impact model design and architecture. Powerful primitive
* Field locking: should be more trivial, especially since it's enforced at the Ceramic level. Just need to expose the feature to OrbisDB users
* Set account relation: Orbis would need some feature development to ensure that users could create new models using this feature.
* Ideally full production launch of OrbisDb would have feature parity, though not needed for Alpha

**User network**
Liang
* Working on workflow for composites
* Reorganized UX to make more sense to new developers
* Adding user guide: Explore, Compose, and Build
* Main product focus is S3
* Also dogfooding own tooling by building a social network complement using Ceramic
* Currently supporting farcaster and lens but both those protocols need new users to pay to use service, so trouble onboarding
* Working on an Guest mode for ceramic. Profile + social graph without paying. Able to bookmark content from farcaster
* Excited about points, working on design spec. SDK to quickly roll out points. Standard use case across products - leaderboards, google analytics like tracking system, then define and map events collected from users to actual points
* All points can/should be built on Ceramic

Michael
* Need a record + some algorithm over the record
* System of recording actions could be streams on Ceramic
* Useful not just for points, but also for audit logs e.g., Discord moderation panel
* Verifiable actions that are recorded on Ceramic and then computed on top of
* Points primitives could be as important as an ERC-20 type token.

**3box labs**
Cole
* Busy week
* Working on getting the Firehose API out, resumability feature is next
* And other features in ComposeDB: Locked fields, Set account relation
* Moving forward on Recon, marching towards goal of js-ceramic being sunsetted. When ready, partners can help test it
* Getting ready to start focusing on CAS scalability
* Potentially a path to unifying js-ceramic/composedb code bases

Sena
* Announced Proof of Data Summit, co-hosting with Tableland
* Should coordinate some EthDenver and growth plans with Orbis
* Ramping up social account activities

**Ongoing forum discussions**
(missed the beginning of this)
Spencer
* Should aggregation be pushed down into rust ceramic?
* Where should state store live?
* Standardization around aggregation is important

Sena
* Data modeling is a lower level primitive than composedb. so composedb and orbis need to implement standards at same time. otherwise ecosystem can become fractured.

Joel
* Standards bodies agree on standards but different implementations
* Different teams should come together and agree on standard rather than being pushed by one team

Spencer
* Someone who wants to build an indexer shouldn't need to have to build their own aggregator
* e.g., OrbisDb shouldnt need to re-implement model instance documents
* Ceramic protocol vs data product could be thought about separately
* Can node js deliver production level quality at scale?

Nathaniel
* Not convinced that indexers can build on top of an opaque aggregator vs instead indexer being some type of aggregator
* Not sure if there are 2 or 3 layers, prefers 2 layers

Sena
* e.g., Orbis building on a document change API is easier than building on an event change api

Nathaniel
* but is that easier than building an aggregation layer
* can you build a stateless aggregation layer? if yes, then build it

Joel
* aggregation layer is like a library

Nathaniel
* Binary or library? Binary can't be a product necessarily

---

# 2024-01-18
### Agenda
* Ecosystem updates
- Us3r network
- Orbis
- Hirenodes
- 3Box Labs
* Open floor

### Attendees

- Aaron D Goldman (3boxlabs)
- Michael Sena (3boxlabs)
- Liang Qiao (U3er Network)
- Yunhe Zhang (Us3r network)
- Baptiste Greve (Orbis)
- Paul le Cam (3box labs)
- Donat (Orbis)
- Sam D (3box labs)
- Spencer (3box labs)
- Cole Brown (3box labs)

### Notes
No CIP updates.

#### Ecosystem updates
**US3R**
Working on node as a service. New developers work better on the composit level then the model level. In order to make it easyer to share a composite we would need a way to share them as an object in the network. There is not curently a gobal registry for composits.
We releast a proof of consept to store a composit as a data model. Requesting feedback in the forum [post](https://forum.ceramic.network/t/rfc-enhanced-composite-management-and-sharing-in-s3-console/1461).
Feedback for this shema to make sure it solves the general case but a minimal viabal it the ferst step.
To test the node as a servise you need to request with an email workflow.

**Orbis**
Working on the SDK portal. This is the way to write and read data. The goal is to be developer frendly. Sharing a demo of the SDK next week. In feb we are leting the first devs on the platform. Probaly five teams at first. We are makeing the UI so that dev don't need to use the CLI to do the inital setup. Hirenodes will work with this SDK to run the DB instance on the same server as the Ceramic node. Distributed as a docker img that can be co-located on the server.
Waiting on 3box for the firehose API. We will be testing the new firehose shortly. Are there any blocking work prerelease.
Context header will be a StreamID byte string. Joel will look at updating the PR for context.

**3box**
Firehose is out v1 with no resume yet.
The recon speed and metric are improving.
It is not clear what the numbers to use are for advertising the limits of ceramic.

How to limit spam in the events?

# 2024-01-04
### Agenda
* CIP discussions
-
* Ecosystem updates
- US3R network
- 3Box Labs
* Open floor

### Attendees
- Aaron D Goldman (3boxlabs)
- Michael Sena (3boxlabs)
- Spencer (3boxlabs)
- Cole Brown (3boxlabs)
- Sam D (3boxlabs)
- Mohsin Zaidi (3boxlabs)
- Joel Thorstensson (3boxlabs)
- mcclurejt (?)
- Charles Levecque (Orbis)
- Baptiste (Orbis)
- Liang Qiao (US3R Network)
- Yunhe Zhang (US3R Network)

### Notes

#### Context
The context is a header that used to seperate streams that are relivent in a spesific context.
For now this might be lmmited to a StreamID so that a context could be fetch to understand it better.

#### Server Sent Events (firehose)
The ability to send events to an listener when they are discovered.

Basic server-side functionality is built. Http-client support needs to be built, and it needs end-to-end testing.

Orbis would like to have this released in 2 weeks.

#### Costs for node as a service (US3R)
What is the cost to run a node?
What is the willingness to pay for customers?

# 2023-12-21
### Agenda
* CIP discussions
- None to discuss today
* Ecosystem updates
- Us3r netowkr
- Orbis
- Hirenodes
- 3Box Labs
* Open floor
- [RFC: Native support for single-write documents with field locking](https://forum.ceramic.network/t/rfc-native-support-for-single-write-documents-with-field-locking/1398)

### Attendees
- Joel Thorstensson
- Aaron Goldman
- Cole Brown (scribe)
- Baptiste Greve
- Charles Levecque
- Michael Sena
- Spencer Brody
- Nathaniel Cook

### Notes
- OrbisDB
- Key feature is the user-friendly interface. Don't have to be a developer to meaningfully interact with orbisDB
- Much more decentralized than their current indexer—every user can run their own Ceramic node and get their own orbisDB dashboard operating over it.
- "Contexts" concept allows users to organize models into various... contexts
- Sub-contexts represent models w/in a context
- Sub-contexts can leverage plugins, e.g. Gitcoin plugin to verify that controller of a stream has a Gitcoin passport
- Other plugins include ability to query an API and add fields from the result
- e.g. Hitting CoinGecko for asset prices and storing them every 10m
- UI has a built in query / view editor that allows you to query orbisDB streams live
- Plugins truly extend the functionality of orbisDB—they can even add their own API routes
- e.g. CSV uploader plugin based on "generate" primitive
- Upload a CSV, parse by row, write streams
- These streams are owned by a did controlled by the plugin itself in the CSV uploader, but the plugins can implement many other behaviors re: stream ownership
- ChatGPT plugin allows users to transform records based on ChatGPT queries, e.g. classifying a book by interest
- Can update a record or create a new one
- Team currently exploring future options for multiwriter streams
- Contexts support participants by mulitple controllers
- They have themselves a controller who updates their metadata
- Their state are stream based
- Discoverability—Compose is discovery by model, Ceramic by stream. OrbisDB provides context as a new content discovery mechanism.
- In discovery w/ customers, migration from standard SQL DBMS to Ceramic is key
- Need to be able to handle the load and have a smooth onramp to support this
- Some users interested in maintaining their centralized db, esp for private data, but willing to export subsets of their data. Good onboarding opportunity.
- All models are now using ModelInstanceDocuments
- MIDs force programmer to choose a model
- Dynamic model creation / evolution
- Could do a search of existing models that match req's
- See this as a separate product almost
- Ideally we help people converge at the same models
- Ceramic
- Ceramic v4 will be breaking compatibility with v3 and earlier
- In part doing this because we are setting extremely high performance standards for rust-ceramic and Recon that we want to hit from day one
- Targeting subset of functionality then adding it all back in as we become more performant.

# 2023-12-07
### Agenda
* CIP discussions
- CIP-124 Recon
- EventId and sorting
* Ecosystem updates
- Us3r netowkr
- Orbis
- Hirenodes
- Intuition
- 3Box Labs
* Open floor

### Attendees
* Aaron D Goldman
* Cole Brown
* Yunhe Zhang
* Nathaniel Cook
* Charles Levecque
* Cuyler
* Baptiste Greve
* Liang Qiao
* Spencer
* Donat
* Oak
* William Luedke

### Notes

#### CIP-124
sorting by (network_id, hash(separator_value), **context_value**, **time**, controller, stream, event)
- aaron: have a PR ron: have a PR ron: have a PR
aaron: explaining the current sort order
"context" value might be relevant
"time" use ethereum block
Joel: position of time? What is time based on?
Aaron: A CIP with a new field for "after" a particular blockhash
Nathaniel: If both times exist how do we pick?
Aaron: the after might be sooner
Nathaniel: if after is defined do we need to follow the chain back?
Aaron: that is more correct, but might make more sense to just use "after"
Joel: UNIX Timestamp vs blockhash?
Aaron: the thing in the after header, if blockhash stronger proof can be generate

Spencer: clarify sep | ctx | time vs sep | time | ctx
Aaron: Context is for further filter beyond just the schema of the data
Instingt is that bc context is developer construted, people are more likely to divide by context
Spencer: I'm still worried about that it's useful for sync purposes, dev wants to know how close they are to catching up
Nathaniel: you can still answer spencers question if time is after context
I expect number of contexts N to be small
Even if N is big you can still do it
Spencer: you still can't do the time thing if you care about all context in the model
How does it work for zero context?

Next step: Aaron to set a PR to the CIP with the updated range

#### Ecosystem updates

**Orbis**:
Making good progress on OrbisDB
Will share a demo with Michael next week
making progress on server and client
Donat is working on clientside iterface
Using serversent events
ETA is Jan
context as part of the metadata
thinking of ways of batching updates / inserts
Aaron: what exactly do you mean by batch update?
Donat: Not exactly sure yet, taking inspiration from traiditional dbs
For now just creating a model for batches
Spencer: an api that acccepts multiple signed events would be easy
an api that takes multiple events with one signature would be more difficult
Donat: single signature is not a requirement here.
Spencer: Have wanted to separate document Id and StreamId fro a long time
Donat: Context and metadata fields?
Spencer: Discussion from CIP-124, support higher leve would come later
Baptise: We actually can't add the field in the Client library
Is rejection on purpose?
Spencer: need to think carefully about how to use it
If you want to set it we can have a discussion
Baptise: adding the field would be the only thing we need now
Nathaniel: Ceramic doesn't need to understand, or don't want it to?
Baptiste: We just need to be able to have data there.

Baptiste: In orbisdb the context is always a streamid
use this as an application field.

**US3R Network**:
Liang: Making good progress on the node as a service initiative
Bunch of PRs from Keramik project
Looking at a demo in the end of the month
One click to provision resources and get a Ceramic url
some of the issues we had we already posted in the forum
visual model editor we already built
Indexing info is specified in the composite and it has to be shared using github or something else
don't want to save indexing infomration in the model
the low level api is there, some issues when trying to invoke those APIs
spencer is probably already aware of this.
Forum post: https://forum.ceramic.network/t/devtools-composite-frommodels-does-not-support-filter/1304

Spencer: Danny has been thinking about this, don't fully understand why it doesn't work
Danny seem to think that the problem is that node is aready indexing the model and you try to re-add it.
Where in the process does it break down?
Liang: we will follow up in the PR discussion.

**3Box Labs**:
Nathaniel: We got rust-ceramic running on Clay testnet, found issues with Kubo interactions it pruned connections
The way we happen to host nodes with AWS was challenging with perf of rust-ceramic
Now, instead of running rust-ceramic as a replacement for kubo we are going to host net new nodes using kubernetes / Keramik
What this means is that there should be new nodes as rust-ceramic comming online
Once this is there means that you could also start using rust-ceramic
biggest advantage will come when recon drops.

# 2023-11-09
### Agenda
* CIP discussions
- CIP-142 Linkfeed - a content feed abstraction standard
- CIP-145 Multi-Prev
* Ecosystem updates
- Us3r netowkr
- Orbis
- Hirenodes
- 3Box Labs
* Open floor

### Attendees
- Aaron Goldman
- Sam Doyle
- Cole Brown
- Michael Sena
- Baptiste Greve
- Liang Qiao
- Julissa Dantes
- Mohsin Zaidi
- Donat
- Paul Le Cam
- Yunhe Zhang
- Nicolas Kammerdiener

### Notes

#### CIP-142 Social Models
- Liang: How to get social model reuse over re-invention.
- Baptiste: Orbis is currently migrating everything to MIDs for OrbisDB, but the migration is not yet complete. Regarding the mobile story, we have made attempts to achieve react-native compatibility, but haven't had any success yet.
- Liang: Team is interested in aligning on a shared model.

#### CIP-145 Multi-Prev
- Aaron: Not much change in the last weeks. One CIP we've been working on is MultiPrev—have folks in the community been running into diverged state? If so, how do they resolve it?
- Liang: My intuition is that conflict resolution should be the responsibility of the application layer. e.g. Farcaster uses a CRDT to sidestep the issue.
- At the moment, the streaming layer resolves conflicts automatically. Data however isn't technically lost, but we don't do anything with that. Do we want to expose this dead data to developers for their own resolution?
- Baptiste: We don't have much motivation for this, since users will more or less self correct data that doesn't update as they'd expect.

#### CIP-124 Recon
- Mohsin: Mostly ready, pipelined behind rust-ceramic work. Should be seeing its first release next season assuming things go well.
- Aaron: From performance benchmarking, a lot of time is spent publishing to and reading from the DHT. This indicates Recon stands to make a significant improvement here.

#### Ecosystem Updates

##### Orbis
- Thinking about UX/DX for OrbisDB, looking at a graphical configuration application to make life easier.
- Been clarifying the mechanism for listening to Ceramic topics.
- Ideally, only subscribe to updates rather than raw events.
- Been in collaboration with 3BL team to discover an ideal API.
- Tentatively calling this API Firehose, it should be agnostic to the backend (Recon or no)
- Developing a plugin architecture to give developers hooks for certain common tasks and make OrbisDB an extensible platform. First four plugins:
- generate: Hook to execute side effects such as creating a Ceramic stream
- validate: Pre-write data validation.
- add_metadata: Attach application or context specific metadata to records.
- post_process: Automatically transform streams on indexing.
- Example they've prototyped: gate collection writes based on the writer's Gitcoin passport.
- [The OrbisDB specification](https://orbisclub.notion.site/Specifications-158aef7c63064b2ca730d6990d390ce4)

##### US3R
- Have started work on keramik, which is a big step towards their work on Nodes as a Service. Hoping for a demo next month.
- Have hit some issues regarding ComposeDB. Recent changes (post 0.5.0) we are no longer able to get index metadata along with model metadata. It seems as though these are now two separate pieces of metadata.
- Seems like the workflow now requires grabbing the indexing and model information separately before indexing a model locally.
- Currently in discussion with Paul on the form [in this post](https://forum.ceramic.network/t/devtools-composite-frommodels-does-not-support-filter/1304).
- Paul: The way indexing was designed, it was meant to be the applications concern, which is why it is separated from model information. Applications can decide on their own indexes for a model. So as a result, they don't exist on the models themselves, only on the model composites.

# 2023-10-26
### Agenda
* CIP discussions
- [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
* Ecosystem updates
- Us3r network
- Orbis
- Intuition
- Hirenodes
- 3Box Labs
* ComposeDB node separation
* Open floor

### Attendees
Aaron Goldman
Joel Thorstensson
Michael Sena
Nathaniel Cook
Spencer Brody
Liang Qiao
Zhang Yunhe
James Pham
Donat
Mohsin Zaidi
Danny Browning
Paul Le Cam
John McClure
Sergey Ukstv

### Auto notes

#### Database Updates and Field Discussions
Joel and Michael organized a meeting with discussions on a variety of topics including updates on their shared databases and the possibility of using an application-specific field in the event ID. The decision was made to trial having notes from the Zoom summary function to make note-taking easier. Aaron discussed the tension between ordering events for efficiency and maintaining time-based order for cost-effectiveness. Spencer and Baptiste introduced the idea of adding an application-specific field to the event ID, which was initially referred to as a 'namespace' or 'context'. Nathaniel pointed out the implications of using a fixed-length byte field for this data, suggesting it might limit the scope of their applications. The team agreed that the recon needs to be fixed with the context and might need to do the hashing themselves to maintain any kind of ordering.
#### Model Complexities and Data Handling Discussed
Liang discussed the complexities of using the same models in different applications and suggested that either a field in a model or an index level namespace could be used to solve this issue. He also mentioned an infrastructure upgrade that improved their data handling. Baptiste and Joel discussed the possibility of comparing data streams and Joel shared about an internal dashboard that shows individual rights of data production. Aaron and Spencer raised questions about the efficiency of data loading.
#### Cows, Shipping, and New Project Discussion
Spencer, Aaron, Baptiste, Liang, Joel, and Michael discussed their work, specifically about the pace of their cows, with Spencer expressing surprise. The team then shifted their focus to a new project that Liang proposed, which involves shipping a ceramic note as a service. This idea aims to improve developer experience by enabling users to spin up their own node for testing or development. Baptiste suggested integrating this service into their UI, providing a new note for the user. The team agreed to finalize the specifications for this new project in early November.
#### Modeling Framework for Applications and Databases
Michael and Baptiste discussed the modeling framework for applications, protocols, and composable data across databases. They emphasized the need for backwards compatibility and the use of instance documents. The team, consisting of Baptiste, Spencer, Aaron, Liang, Michael, Nathaniel, Joel, and John, deliberated on the handling of arrays in documents, developer experience, and the challenges of building databases on top of indexing and query systems. The conversation also touched on composite models and the concept of individual models. Towards the end, John from Intuition introduced himself and highlighted his work on leveraging attestations for subjective data. He identified some issues related to aggregate models and the need for a standardized procedure for ingesting data uniformly.
#### Progress, Deployment, Migration, Newsletter, and Feedback
John discussed the progress of their project, Nathaniel provided an update on the deployment of Restoramic in their QA environment, and outlined their plans to push it to Clay and then to main net. Nathaniel also mentioned that the migration process is working smoothly and can work with anyone who wants to migrate a node. Joel's microphone was not working well. Michael discussed his idea of creating an ecosystem newsletter to highlight the work of all teams, and asked for input on how to gather and share information. James asked for feedback on a presentation about the composed of B mode separation.
#### Developing Custom Indexing Solution for Ceramic
The meeting discussed the development of a custom indexing solution for Ceramic. Sergi explained that the aim is to provide a simplified method for users to build their own solutions on top of Ceramic, without needing access to its internals. He also outlined different layers involved in this process, including a streaming layer, an aggregation layer, and an index solution. Sergi highlighted the importance of providing some level of introspection into what's happening internally, such as exposing a feed of stream states and Ceramic events. Spencer suggested that focusing on the feed of changes to documents might be sufficient for most users for now, and deferring the lower level of actual events to a later stage. Joel asked how this relates to the Ceramic API, and Sergi responded that it is directly related but they are approaching it from a different angle.
#### Document vs Event Usage in System: Debate Continues
Aaron, Spencer, Liang, SU, and Nathaniel discussed the use of events and documents in their system. Liang highlighted that the users primarily care about the documents and may not be aware of events or commits. However, for building a new database, events could be more useful. The group also debated whether to push everything needed for processing from the firehose or just the stream ID or C id. SU explained that the idea is to push the stream state, which includes processed events, to eliminate unnecessary requests. Nathaniel pointed out a trust relationship that changes when events are not sent but documents are, emphasizing the need to verify the document's construction from the events. Spencer countered that this doesn't alter the current trust relationship with Ceramic.
#### Explanation of Ceramic Node System for Document State and Event Logging
Spencer explained how the current system works, where the ceramic node provides the current state of a document when asked. Nathaniel agreed with Spencer's explanation. Spencer emphasized that the system also includes metadata about the log of events that went into computing the state. John found all proposed solutions interesting and expressed familiarity with constructing a database from a change data capture feed. However, he indicated that they would not dive down to the event level to verify the integrity of the document. It was determined that stream state changes would be sufficient for their needs. Spencer also mentioned the need to determine the right API for connecting to the node and accessing the feed, but time ran out for further discussion. John clarified that they only operate in a private networking context and do not run much off-browser.

---

# 2023-10-12
### Agenda & Notes
* Implementation updates
- Rust
- Keramik
* CIP discussions
- [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
* Open floor

### Attendees
Liang Qiao
Nathaniel Cook
David Galindo
Spencer Broody
Joel Thorstensson
Zhang Yunhe

### Notes

#### Impl upates
NC: still finding bugs in rust ceramic
few have been fixed, few more lingering
have upgraded parts of our own infra to use rust-ceramic

Keramik:
LQ: Nicolas from hirenodes have worked on a majority of the issues
Will ping him for an update. Haven't had time work on Keramik yet. Currently focused on the indexer. Need better recovery when the node goes down
AG: indexer using to pubsub in your own code?
LQ: we are subscribing to libp2p events, but rely on js-ceramic to get the data
there is some perf bottleneck when adding more indexing to js-ceramic. Hit bound of queue
SB: Can't index all of ceramic on one node, curious how cerscan does it
LQ: they don't index everything
we are trying to index all ComposeDB models
JT: they maybe only index pubsub?
Invite Orbis team and Nicolas to these calls

NC: small new features being added from 3BL side as we use it
e.g. wait for k8 status

#### CIP updates

##### Recon
NC: Discussing updating the EventID format. The motivation is to make it easier to observe and debug. e.g. "how synchronized am I?" by putting previous event blockheight into the eventId. Also enables syncing without ggoing back to beginning of time. Disadvantage is that you must know the blockheight to syncronize the event, e.g. you need to know the previous event height to calculate the eventId.
AG: There's a CIP for addign an `after` field to headers. Could point to an ethereum block hash.
NC: high level goal is the ability to observe sync progress
Other proposed change is to add a namespace section to the EventID. Allows apps to put arbitrary data into the eventId. Helpful when there is a many to many reationship between models x users.
Both of these suggestions are pending writing things down.
To update recon CIP

AG: right now we don't know when events happens after some time. I think it makes sense to add an `after`field.
SB: Only downside I can think of is that we need to validate it
JT: Very similar to validate time events
AG: one of the debates I was having with mohsin: Should I use latest eth block or latest block that has an anchor.
JT: store all eth block hashes?
Aaron to follow up on CIPs for after and multi-prev CIPs

#### Open floor

##### associate dapps to models
LQ: exploring ways to get dapps x model mapping. CACAOs was suggested. Not really reliable.
Being able to assiclate models to Dapps enables devs to more easily reuse model.
JT: CACAO approach unreliable why?
LQ: data we get from cacao not really clean,
JT: CACAOs are the only programmatic way, otherwise would need to do it manually
LQ: keep using CACAOs, with some manual mappsings

##### Regarding Recon
LQ: we have a clear use case planned, hypothetically what are the use cases?
AG: should work a lot better if you have a big dump of data, bc of the nature of synchronizing. What are all the evens taht one server has and the other doesn't have. E.g. create a brand new model, as an import of other events, should do decently well. Would to a bit less well if there's a mill events and now a new mill events are added.
LQ: how much of data in a mill events?
NC: eventIDs are 72 bytes now
LQ: when it comes to event data ipfs is still bottleneck?

---

# 2023-09-14
### Agenda & Notes
* Implementation updates
- Rust
- Keramik
* CIP Discussions
- [Multiple previous (CIP-145)](https://cips.ceramic.network/CIPs/cip-145)
- [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
- [Ceramic API (CIP-137)](https://cips.ceramic.network/CIPs/cip-137)
* Open floor

### Attendees
Aaron D Goldman
Liang Qiao
Joel Thorstensson
Mohsin Zaidi
Yunhe Zhang
Spencer T Brody

### Notes

#### Implementation updates

Aaron:
rust-ceramic is making progress. js-ceramic runs successfully with rust-ceramic instead of kubo. Seems to give us a small performance improvement as a bonus. Still doing more testing, but getting close to making rust-ceramic the default supported ipfs implementation for ceramic.

Joel:
Uncovered a bug due to the intersection of bitswap and ReCon. Nathaniel put in a fix.

Mohsin:
Built out a test framework for testing with mix of kubo and rust-ceramic backed Ceramic nodes (based on Keramik), to ensure interoperability.

Aaron:
issues to getting Keramic production-ready?
JT: there are issues open and assigned to Nick from Hirenodes to fix.

#### CIP updates

Aaron:
Multi-prev CIP coming along, handles forks in stream history. Includes changes to conflict resolution and branching rules.
**Need community to review and give feedback** (in forum post)
useful for:
- anchors not conflicting with updates
- letting devs/users explicitly resolve write conflicts
- letting users fork away from writes created by a malicious app.

JT: Updates on ReCon or Ceramic API?
no updates

#### open floor

Liang:
Track usage of a Model by apps?
Currently doing this extracting app info from CACAOs.
Is this the best way?
JT: yes - user attests to app in signature
Liang: Some streams are missing this info. Apps that use did:key instead of did:pkh.
JT: how common is this? I thought most data is written with did:pkh
Liang: a significant portion
JT: this is surprising. Would be good to look into which data models tend to have data written with did:key.

Liang:
Reliability of our indexer.
We listen to p2p pubsub, but if indexer is down data is lost.
Aaron: do you use HDS?
Liang: we tried but had issue
JT: worth trying again, HDS has had a lot of fixes lately
Recon will change this a lot and make this a lot easier.

ReCon won't discover data that isn't still hosted by at least one Ceramic node
HDS can find data even if no Ceramic node is hosting it, so long as the underlying IPFS data is still available. HDS can also theoretically detect when data used to exist but is now missing from the network.

---

# 2023-08-31
### Agenda & Notes
- Implementation updates
- Rust
- CIP Discussions
- [Multiple previous (CIP-145)](https://github.com/ceramicnetwork/CIPs/pull/145)
- [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
- [Ceramic API (CIP-137)](https://cips.ceramic.network/CIPs/cip-137)
- Keramik questions
- Open floor

### Attendees
Aaron D Goldman
Nathaniel Cook
Liang Qiao
Joel Thorstensson
Mohsin Zaidi
Yunhe Zhang

### Notes

#### Implementation updates
Nathaniel: Not much has changed in the implementation but we added a lot to the tests
moved the ceramic and Kubo api to a single port.
This lets us have openAPI yaml for api description.

Liang: what is the testing strategy.

Nathanel: use keramic to run tests that look for properties of the network.
Then run the property test agents many network shape.

Joel: Paused the push to recon to make sure rust-ceramic is a solid drop in replacement.

Nathaniel: recon will then be able to ship behind a feature flag.

Liang: Composition is about the nodes in the network not the topology.

Nathaniel: We are starting with the fully connected network, testing other topologies will come later.
This is a side effect of how we connect the network not a explicit limitation.
We use the DHT to find other ceramic nodes.

Liang: so even without recon you only connect to ceramic nodes.

Nathaniel: We connect to other ceramic nodes first.

Joel: IPNI we may want to use this instead of the DHT.

#### Go Ceramic
Liang: We are testing the use of keramic.

#### Ceramic API

Aaron: CIP-145 multiple prev to merge streams.

Joel: we will need a API for getting the many tips

Nathaniel: we may have core/convenience API e.g. get all events

#### Keramic

Liang: the spin up of a network should be independent of the test runner.
We need to be able to spin up and connect to PostgresDB.
The metrics are tied to the test but we need then from monitoring a network.

Nathaniel: We also need to have the metrics outside of the simulation for the network.

Joel: feel free to make the change and we will re-view and merge it in.

Liang: need to be able to connect to outside resources. Ethereum, CAS, ...

---

# 2023-08-17
### Agenda & Notes

- CIP Discussion
- [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
- [Ceramic API (CIP-137)](https://cips.ceramic.network/CIPs/cip-137)

### Attendees

- Cole Brown
- Aaron D Goldman
- Liang Qiao
- Yunhe Zhang

### Notes

- Recent discussion has been focused around APIs for Recon / rust-ceramic
- Initial effort has been roughly cloning kubo's API for maximum compatibility with minimal effort
- Ultimately we'd like to create our own APIs that more directly reflect what Ceramic is doing
- Aaron & Nathaniel are of the opinion that versioned APIs aren't all that helpful
- Preferred alternative is to simply add routes and avoid conflicts while keeping things easier to understand/recall
- Integral version numbers don't hold much semantic weight
- Primary use cases for Ceramic API
- When users don't require aggregation or wish to aggregate their data in different/novel ways, the Ceramic API gives them primitives for dealing exclusively with events in their raw format.
- Right now we don't have good tools for managing divergent streams
- But we have the capabilities to expose this to a stream controller (e.g. a prompting that stream has diverged, present the tips, prompt for resolution)
- It would be nice to add an API to get all known heads of a given stream
- We'd like to see TileDocuments deprecated over time, in particular in favor of the ModelInstance document.
- US3R: We'd like to learn more about keramik
- keramik is suitable for running nodes for testing as is now
- keramik intentionally spawns an isolated network within Kubernetes
- It's trivial to connect keramik nodes to other nodes, so it's quite possible to use it for prod deployments
- We'll need to collaborate to help facilitate production use-cases, as there will be many warts deriving from it being built for testing
- US3R is very interested in using keramik to automate their nodeops, particularly deploying new nodes
- Recon update
- Mostly MVP-ready, looking good in local tests
- Next step is productionize it and get it deployed via rust-ceramic
- rust-ceramic rollout is incremental
- First we will use rust-ceramic to replace everything we're using kubo for (essentially IPFS content distribution sans DHT)
- rust-ceramic will plug in to js-ceramic, which will use it for content persistance and retrieval
- Second step, we will actually enable recon for content discovery

# 2023-08-03
### Agenda & Notes
* Implementation updates
* Rust
* Go
* CIP Discussion
* [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
* Update on Peer discovery (Nathaniel)
* [Ceramic API (CIP-137)](https://cips.ceramic.network/CIPs/cip-137)
* gRPC vs OpenAPI
* A discussion on finalization ([forum post](https://forum.ceramic.network/t/cip-137-ceramic-api/1153/23))
* Open floor

### Attendees
* Aaron Goldman
* Joal Thorstensson
* Oak
* Cole Brown
* David Galindo
* Nathaniel Cook

### Notes
Start: 8:04 AM PT

rust impl:

We are API compatible for the API serfice we call.
We can drop in rust-ceramic in place of kubo.
More testing is still needed.

We have prof of concept for recon syncing thousands of events.
More performance debugging needed.

#### CIP Discussion Ceramic API (CIP-137)
Sencor:
If a project wished to build on top of Ceramic and wants to know when a write is final.
After this moment the event is unlikely to be rolled back.
If anchoring takes to long then the Cacao could expire and invalidate the event.
If a stream forks both forks can live for some time and a late publishing attack
could revert a number of event in stream after the fork.

Do we want to make consumer deal with the divergence of the covered(committed) and uncovered(uncommitted) events.

Joel: How will this efect Recon/API
Aaron: We need to express the time information in the API returned data.
Spencer: We should enrich the API with timestamps.
But this will not solve the data witholding attack.
These writes are anchored but not final.
When you are maintaining a store you would need to have state for tip and anchored tip.
Joel: It is a problem that a new time event can invalidate a outstanding chain.
Spencer: The API need to be able to respond with invalidation events for when a event moved from valid to invalid.
The consumer needs to be able to replay the stream.
Nathaniel: The conflict resolution should not be the ceramic API. Event retrieval and state aggregation are
distinct tasks. Lets not combine them.
Spencer: What dose the API need to expose to be able to aggregation layer.
Nathaniel: By returning events in the order the node learned them rather then the anchor time.
This may be simplified.
Joel: Example. There is a Data event (exp. T1) gets anchored at (T2) the event is expired not valid.
Nathaniel: Can the ceramic API abstract away the need for clients to validate events? probably no, this changes the trust relationship.
Applications need to make their own tradeoffs for eager evaluation.
We can have a logical replay from beginning and then use checkpoint to improve performance.
Spencer: We can keep the anchored state to replay from.
Remember that in the future there will be stream tipes that use curent state not a delta stream.
Nathaniel: two questions
Can I restore state? Can I detect that I am in invalid state? Answer to both is yes.
Spencer: We should be able to operate in ether a trusting or auditing mode.
Joel: It is like the ETH virtual machine. You could pull the blocks and validate yourself but most won't do that.
Nathaniel: This choice dose not force us to take this choice from the client.

Spencer: late publishing means that we can't get away from needing a roleback.

#### GRPC/Open API
Nathaniel:
Both solve a simalar problem.
Both define the API by writing a machine readable spec.
Open API uses YAML
GRPC uses ProtoBuff
you can generate clients for either.

Open API to methods.
GRPC methods to API.

GRPC is built on. http2 and may give useful performance with long lived channels.
GRPC has multiplexing baked in to its world view.

David: Having to build ProtoBuff is not a real problem.

Nathaniel: It is posible to have both if there are use case.
If the first version is easy to adopt later vertions can give a more performant option.

Joel: Should we merge the PR for using Open API?
Nathaniel: yes, but it is not the final form.

# 2023-07-20
### Agenda & Notes
* Implementation updates
* Rust
* Go
* CIP Discussion
* [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
* Open floor

### Attendees
Lian Qiao
Yunhe Zhang
Mohsin Zaidi
Aaron Goldman
Joel Thorstensson

### Notes
#### Rust impl
- Sqlite to persist events for ceramic
- Nathaniel working on first iteration of Ceramic API
#### Go impl
- Made some progress finally, have to implement a new version of kubo ipfs
- Decided to go back to kubo, "kubo++"
- Discussion on go architecture
- Discussion of Recon, eventIds, and ranges
- Is there a protocol interface for Recon?
- More discussion about internals of recon

#### CIP: Recon
##### fencepost
Need to pin down how fencepost are pinned down and also discovery
##### Both sort-key/value in eventId
Aaron is going to update spec and implementations

---

# 2023-07-06

### Agenda & Notes
* Implementation updates
* Rust
* Go
* CIP Discussion
* [Recon (CIP-124)](https://cips.ceramic.network/CIPs/cip-124)
* ~~Update on Peer discovery (Nathaniel)~~
* An approach to js-ceramic compatibility PoC (Zach)
* [Ceramic API (CIP-137)](https://cips.ceramic.network/CIPs/cip-137)
* gRPC vs OpenAPI
* ~~A discussion on finalization ([forum post](https://forum.ceramic.network/t/cip-137-ceramic-api/1153/23))~~

### Attendees
- Joel Thorstensson
- Aaron Goldman
- Zhang Yun He
- Mohsin Zaidi

### Notes

#### Go implementation

- Team is working on the roadmap.
- Will wait for Liang to provide more detailed updates.

#### Rust implementation

- Recon can work on arbitrary byte arrays instead of just strings.
- Specifically event IDs.
- Nathaniel has added peer discovery so it can use the IPFS DHT to find other nodes on Recon.
- When you first start Ceramic it will look up the same key as it uses for the pubsub topic and find other Cerqmic nodes.
- Will retrieve advertisements for "I am a Ceramic node" and see if it supports Recon and which version.

Aaron:
- If we get to a very large number of Ceramic nodes, we can use the DHT to separate nodes with similar interests.
- For now, since the number of nodes is small, we can just find all Ceramic nodes instead of having to make individual requests for interests.

Joel:
- Would lean towards the event ID approach because it gives us the future path.
- Let's follow up with a detailed forum post.

#### CIP Discussion

##### js-ceramic PoC

- Have an initial implementation working with Recon.
- In a position to start benchmarking it.

Joel:
- Is there a way to handle Tile documents?
- Maybe telling the Ceramic node to handle certain ranges.

Zach:
- Makes sense, can use family as separator to do that.

Joel:
- Should work with IDX.
- There are some use cases for subscribing to individual streams.
- Should be able to call Recon API and subscribe to a stream.

Aaron:
- Using JSON?

Zach:
- Trying to keep it similar to the current Ceramic API at least for testing.

##### Ceramic API

Joel:
- Have been having an interesting discussion about this.
- Once Nathaniel has documented his thoughts on this we'll continue the discussion on the forum.
- Spencer had some thoughts about finalization that we'll continue discussing next week.

---