# AI CEO Project Meeting Notes
Date: Tuesday, January 28, 2025

## Core Topics
1. Deep funding and dependency graphs analysis
2. Private vs. public information management for repositories and dependencies
3. MVP development strategy

## Contribution Evaluation Framework

### Types of Contributions

#### Developer Contributions
1. Code metrics
- Lines of code (with consideration for additions/deletions)
- Pull requests (submitted, reviewed, merged)
- Issue management (opened, closed)

#### Social Contributions
1. Community support
- Peer assistance
- Question answering
- Knowledge sharing

2. Content Creation
- Educational materials
- Documentation
- Tutorial videos

3. Tipping System
- Source Crowd implementation
- Peer-to-peer value recognition
- Tip-based contribution validation

### Tools & Platforms
1. Code Repository
- GitHub integration
- Metrics tracking
- Contribution analysis

2. Social Platforms
- Discord
- Telegram
- Communication tracking

## Evaluation Models

### Model Considerations
1. Rule-based Scoring (example)
- Quantitative metrics (e.g., lines of code = 0.5 points)
- PR complexity scoring (simple = 5 points, complex = 15 points)
- Multiple model approach for comparison

2. Voting System
- Model preference voting
- Vote weighting mechanisms
- Community consensus building

### Implementation Details
1. Automated Scripts
- Database integration
- Metrics collection
- Real-time analysis

2. Review System
- Peer review process
- Contribution rating
- Social component evaluation

## Challenges & Solutions

### Key Issues
1. Gaming Prevention
- Cultural alignment
- Anti-spam measures
- Quality control

2. Incentive Alignment
- Retroactive tipping index
- Base plus tip model
- Stake-based participation

### Risk Mitigation
1. Value Protection
- Time-bound decay functions
- Public/private information management
- Smart contract implementation

## Framework Development

### Core Principles
1. Input Standardization
- Agreed metrics
- Data collection methods
- Platform integration

2. Output Consistency
- Token distribution
- Value attribution
- Performance measurement

3. Model Flexibility
- Multiple model support
- Customizable weights
- Community-driven evolution

## Next Steps
1. LLM Integration
- Unstructured to structured data conversion
- Comment analysis
- Feedback processing

2. Evaluation Schedule
- Rop flight on Friday morning
- Evaluation session scheduled for Friday

## Additional Considerations
- Contextual information processing
- Key decision maker identification
- Weight distribution mechanisms
- Compatibility assessment
- Model testing and validation

# *Granola based notes*

### Deep Funding and Contribution Evaluation

- Discussed deep funding / dependency graphs and how private/public information on repos and dependencies work
- Explored MVP (Minimum Viable Product) for evaluating contributions to emerging projects

#### Types of Contributions:

- Developer Contributions:
- Debated weighting factors: number of lines of code, PRs
- thinking: No ideal qualification of dev contribution exists
- Proposed creating different models for community agreement
- Doesn’t require training different models
- Challenge: How to evaluate conversations on PRs
- Rule-based Contributions:
- Example: 1k lines of code added, X lines deleted, 4 PRs created, 4 PRs reviewed
- Suggestion: Use a model to determine weights for different factors

#### Tools and Data Sources:

- GitHub: Code repository
- Discord/Telegram: Communication platforms
- Emphasized need for both code storage and discussion platforms

#### Metrics to Consider:

- Repository Metrics:
- Lines of code (added/deleted)
- PRs (created, reviewed, commented, merged)
- Issues (opened, closed)
- Social Metrics:
- Quantifying contributions in discussions
- Person-to-person help
- General question answering
- Content creation (e.g., educational videos)
- Tipping system to indicate valuable contributions
- Negative Contributions:
- Spam
- Disruptive behavior in social platforms

### Disagreements and Voting

- Acknowledged potential disagreements on point systems
- Proposed using multiple models (e.g., Model 1, Model 2) with different weightings
- Suggested voting system: e.g., 4 votes for Model 1, 8 votes for Model 2
- Debated whether votes should be weighted

### Technical Implementation

- Scripts to listen and pull data into a database
- Reviewers/judges to rate PRs and models
- First pass: Models provide list of contributions
- Second pass: Random people with context rate contributions
- Score models based on alignment with human ratings

### Challenges and Considerations

- Handling private data in evaluation process
- Retroactive vs. forward-looking evaluation
- Balancing incentives and preventing gaming the system
- Emergent project management and roadmap development
- Competing priorities in evaluation

### Proposed Mechanisms

- Index of liquid karitsu with retroactive tipping
- Base plus tip model with pre-commitment
- Options staking with rage quit for cash
- Time-bound decay function for private/public information

### AI Model Considerations

- Discussion on model behavior and preventing manipulation
- Suggestion to model after improv techniques for more natural responses
- Importance of breaking out of repetitive patterns

### Framework Agreement

- Agree on inputs and outputs, disagree on actual model implementation
- Use of GitHub and communication tools as data sources
- Unified framework with flexibility for individual models

### Next Steps

- Load and run the system with API keys
- Implement multi-social network support
- Develop method for key decision makers to provide private feedback
- Test compatibility of envisioned system with various factors
- Explore use of LLMs for unstructured to structured data conversion
- Consider implementation of autonomous investor concept
- Main track TBD, with evaluation on Friday

---

### Potential Gaming Behaviors
1. Message Spam
- Rapid short messages to inflate message count
- Breaking up single thoughts into multiple messages
- Posting low-effort content like emoji reactions
- Copy-pasting content across channels

2. Points/Reactions Manipulation
- Coordinating with other users for mutual reaction farming
- Creating alt accounts to give reactions
- Strategically posting in channels with higher normalization factors
- Excessive self-reporting of helping instances

3. User Activity Gaming
- Creating artificial questions to "help" with
- Inflating "questions asked" metric through minor variations
- Cross-posting same content to maximize exposure
- Time-based gaming (posting during low-activity periods to stand out)

### Suggested Countermeasures

1. Activity Pattern Detection
- Implement rate limiting for messages
- Track message length distribution and flag abnormal patterns
- Monitor message uniqueness/similarity across channels
- Track timing patterns between messages

2. User Behavior Analysis
- Track reaction reciprocity patterns between users
- Monitor help/question ratios for statistical anomalies
- Implement cool-downs between point-earning activities
- Add weights for message quality (length, complexity, uniqueness)

3. Structural Changes
- Randomize or hide normalization factors
- Add manual review for points above certain thresholds
- Implement diminishing returns for rapid-fire activities
- Create "quality scores" based on community engagement

4. Additional Metrics
- Track edit/deletion patterns
- Monitor user interaction networks for suspicious clusters
- Implement sentiment analysis for message quality
- Add time-based decay factors for points

--

{%hackmd @xr/ai-comms %}